Salesforce Enterprise Deep Research

Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.

A Business-Driven Real-World Financial Benchmark for Evaluating LLMs

A comprehensive code domain benchmark review of LLM researches.

A comprehensive guide to LLM evaluation methods designed to assist in identifying the most suitable evaluation techniques for various use cases, promote the adoption of best practices in LLM assessment, and critically assess the effectiveness of these evaluation methods.

A benchmark for prompt injection detection systems.

Examining how large language models (LLMs) perform across various synthetic regression tasks when given (input, output) examples in their context, without any parameter update

AlgoTune is a NeurIPS 2025 benchmark made up of 154 math, physics, and computer science problems. The goal is write code that solves each problem, and is faster than existing implementations.

A collection of LLM related papers, thesis, tools, datasets, courses, open source models, benchmarks

A dynamic forecasting benchmark for LLMs

[CVPR 2025] Program synthesis for 3D spatial reasoning

LLM-KG-Bench is a Framework and task collection for automated benchmarking of Large Language Models (LLMs) on Knowledge Graph (KG) related tasks.

(NeurIPS 2025) Official implementation for "MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?"

Take your LLM to the optometrist.

[AAAI 2025] ORQA is a new QA benchmark designed to assess the reasoning capabilities of LLMs in a specialized technical domain of Operations Research. The benchmark evaluates whether LLMs can emulate the knowledge and reasoning skills of OR experts when presented with complex optimization modeling tasks.

[MM 2025] A Multimodal Finance Benchmark for Expert-level Understanding and Reasoning

BizFinBench.v2: A Unified Offline–Online Bilingual Benchmark for Expert-Level Financial Capability Evaluation of LLMs

RTL-Repo: A Benchmark for Evaluating LLMs on Large-Scale RTL Design Projects - IEEE LAD'24

Test your local LLMs on the AIME problems

Benchmark evaluating LLMs on their ability to create and resist disinformation. Includes comprehensive testing across major models (Claude, GPT-4, Gemini, Llama, etc.) with standardized evaluation metrics.

LisanBench is a lightweight benchmark for LLMs that stresses forward planning, vocabulary depth, constraint adherence, attention, and long-context "stamina" all at once.

LLM Locust combines the simplicity of Locust with deep support for LLM-specific benchmarking

An app and set of methodologies designed to evaluate the performance of various Large Language Models (LLMs) on the text-to-SQL task. Our goal is to offer a standardized way to measure how well these models can generate SQL queries from natural language descriptions

FM-Leaderboard-er allows you to create leaderboard to find the best LLM/prompt for your own business use case based on your data, task, prompts

Official Codebase for "ER-Reason: A Benchmark Dataset for LLM-Based Clinical Reasoning in the Emergency Room"

a platform for monitoring the chip situation

Structured Prompting Enables More Robust Evaluation of Language Models

[Paper][EMNLP 2025] SKA-Bench: A Fine-Grained Benchmark for Evaluating Structured Knowledge Understanding of LLMs

A framework to analyze how AGI/ASI might emerge from decentralized, adaptive systems, rather than as the fruit of a single model deployment. It also aims to present orientation as a dynamic and self-evolving Magna Carta, helping to guide the emergence of such phenomena.

We introduce a benchmark for testing how well LLMs can find vulnerabilities in cryptographic protocols. By combining LLMs with symbolic reasoning tools like Tamarin, we aim to improve the efficiency and thoroughness of protocol analysis, paving the way for future AI-powered cybersecurity defenses.