Benchmark Router
BenchmarksDocs
Sign in

Benchmarks

Request a benchmarkRun with your harness
All 25OCR 1Knowledge 2Coding 10Agentic 3Cyber 3Legal 2Finance 1Health 1Science 1Vision 1
  • AutomationBench

    Zapier's business-workflow benchmark: 600 public tasks (Sales, Marketing, Operations, Support, Finance, HR; 100 each) on 47 simulated SaaS apps, graded by deterministic end-state assertions (strict pass rate). Official auto-bench harness, API toolset, 50 steps. Public set only: the zapier.com leaderboard uses a private, harder held-out set, so scores are not directly comparable.

    Agentictool6 subsets600 samplesCustom adapter
  • EnterpriseOps-Gym

    649 public enterprise tasks across Teams, CSM, Email, ITSM, Calendar, HR, Drive and cross-domain Hybrid workflows: the agent uses stateful MCP tools over a seeded database while following internal policies, and hidden SQL checks the final state (a task passes only if every check does). Official harness and ReAct agent in oracle mode; task success rate as on the leaderboard.

    Agentictool8 subsets649 samplesCustom adapter
  • ITSMBench

    89 IT service-management tickets (access, offboarding, incidents, security response) worked against mock ServiceNow, Okta, Slack and Google Workspace APIs, each checked by a hidden verifier. Official Harbor tasks with the Pi agent for every model, as on the ITSMBench leaderboard.

    Agenticharbor1 subset89 samplesAgent CLI