Benchmark Router
BenchmarksDocs
Sign in

Benchmarks

Request a benchmarkRun with your harness
All 25OCR 1Knowledge 2Coding 10Agentic 3Cyber 3Legal 2Finance 1Health 1Science 1Vision 1
  • Humanity's Last Exam

    2,500 expert-written closed-ended questions across dozens of subjects (multiple-choice and exact-match, ~14% with images), graded by an LLM judge. Official CAIS harness.

    Knowledgeqa8 subsets2,500 samplesvisionAny API
  • Humanity's Last Exam: Diamond

    1,000 refined HLE questions, 500 reasoning and 500 knowledge, graded by an LLM judge. Official CAIS harness, no tools.

    Knowledgeqa2 subsets1,000 samplesvisionAny API