Benchmark Router
BenchmarksDocs
Sign in

Benchmarks

Request a benchmarkRun with your harness
All 25OCR 1Knowledge 2Coding 10Agentic 3Cyber 3Legal 2Finance 1Health 1Science 1Vision 1
  • Harvey LAB (Legal Agent Benchmark)

    Harvey's Legal Agent Benchmark: 2,010 public long-horizon legal tasks (24 practice areas plus contracts, diligence and firm-knowledge) producing memos, redlines, spreadsheets and decks from a document set; every rubric criterion graded by two LLM judges (Claude Sonnet 4.6 + GPT-5.5), a task passes only if all criteria pass. Official LAB harness and evaluator. Public set only: the Vals leaderboard uses Harvey's private held-out set, so scores are not directly comparable.

    Legaltool27 subsets2,010 samplesCustom adapter
  • RedlineBench (Crosby)

    Contract-negotiation redlining: 140 tasks across 3 MSA negotiations and 4 turns, where the agent, as in-house counsel, produces a Word redline with tracked changes and comments. Graded against attorney-authored weighted rubrics by a 3-judge LLM panel (GPT-5.4 mini, Claude Haiku 4.5, Gemini 3.1 Flash-Lite; majority vote). Official Harbor tasks on Modal; Claude Code for Anthropic models, Codex for OpenAI models, Terminus-2 otherwise.

    Legalharbor3 subsets140 samplesAgent CLI