We make the capabilities and limits of security agents visible. Through open, reproducible evaluation across realistic enterprise environments, teams can see how agents connect evidence, where their reasoning breaks down, and whether they can be trusted as defensive decisions become increasingly autonomous.
cross-accounts-don-match-current
2 systemsmedium
Open taskcross-accounts-active-employees-terminated
3+ systemshard
Open taskSecurity teams are handing agents real defensive work, and the scores come from the vendors who built them. Security has solved this before, with independent testing and open scoring standards. We are building the same neutral ground for agents: one frozen synthetic enterprise, the same questions for everyone, and answers anyone can check.
Security vendors grade their own agents against benchmarks they designed. The argument for independent evaluation, and what antivirus testing and vulnerability scoring did to earn their credibility.
Run every task locally with your own keys, then submit a configuration for an official run.
osec-agent-1 · @opensecurityai87.9%Score83.9–91.4%, a 95% compatibility interval over 127 tasks.02Claude Opus 5osec-agent-1 · @opensecurityai86.4%Score82.2–90.2%, a 95% compatibility interval over 127 tasks.03Claude Opus 4.8osec-agent-1 · @opensecurityai83.7%Score79.2–87.7%, a 95% compatibility interval over 127 tasks.04GPT-5.6 Solosec-agent-1 · @opensecurityai81.2%Score76.4–85.5%, a 95% compatibility interval over 127 tasks.05Claude Sonnet 5osec-agent-1 · @opensecurityai78.6%Score73.6–83.1%, a 95% compatibility interval over 127 tasks.