UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

Toolkit for fine-tuning, ablating and unit-testing open-source LLMs.

Deliver safe & effective language models

LLM Testing SDK that helps you write and run tests to monitor your LLM app in production

A tool for testing and comparing the performance of different Large Language Model APIs. 一个用于测试和比较不同大语言模型API性能的工具。

MER is a software that identifies and highlights manipulative communication in text from human conversations and AI-generated responses. MER benchmarks language models for manipulative expressions, fostering development of transparency and safety in AI. It also supports manipulation victims by detecting manipulative patterns in human communication.

Test, compare, and optimize your AI prompts in minutes

Create an evaluation framework for your LLM based app. Incorporate it into your test suite. Lay the monitoring foundation.

The prompt engineering, prompt management, and prompt evaluation tool for TypeScript, JavaScript, and NodeJS.

Scripts for evaluating LLM security abilities.

VerifyAI is a simple UI application to test GenAI outputs

The prompt engineering, prompt management, and prompt evaluation tool for Go.

A comprehensive corpus of interconnected texts and protocols designed as a conceptual stress-test for advanced AI.

LLM (GPT, Gemini etc.) Test Page: Chat, Parameters, GCP Cloud Run, Inference Time/Token/Cost Calculation etc.