Skip to content
open.securityopen.securityBeta
Skip documentation navigation
Browse documentation

Agent integration

Integrate Codex

Drive your installed Codex CLI headlessly over ACP. OSB uses the CLI’s own login and configuration while the harness records the model, arguments, and context that define the candidate.
Agent kind
codex
Harness id
codex
Benchmark
ISPM-Enterprise-SQL@v1
Bundled dataset
ispm-crossvendor-v1

Prerequisites

Have the runtime and its authentication ready before you author the candidate.
  • OSB and the matching osec-agent-1 binary on PATH.
  • The codex CLI installed and authenticated through your local Codex configuration.
  • Provider credentials and routes for the benchmark’s judge panel.
  • Enough workspace permissions for Codex’s native workspace-write sandbox.

Configuration

Codex authentication and provider routing come from your own local Codex install. OSB does not copy those credentials into the harness.

Create harness/codex/manifest.toml:

harness/codex/manifest.toml
type = "codex"
 
# Optional: omit to use the CLI default.
# model = "gpt-5.6-terra"
 
# Optional Codex config overrides.
# args = ["-c", "model_reasoning_effort=high"]

Context and system prompt

Put candidate context in harness/codex/AGENTS.md. To add a separate prompt file, use append mode:

manifest.toml · optional block
[system_prompt]
mode = "append"
file = "system.md"

What stays outside the manifest

  • Your Codex login under the local config directory.
  • OPENAI_API_KEY and optional OPENAI_BASE_URL environment values.
  • Every judge-panel provider credential.

Preflight

Check the agent runtime first, then let OSB check the full host and benchmark configuration.
shell
$ codex --version
$ osb doctor

Doctor must find Codex and the judge routes. A successful version check does not prove login state, so confirm a small Codex command works before starting a paid benchmark task.

One-task validation

Validate aws-active-access-keys-older before you spend on full coverage.
shell
$ osb run ISPM-Enterprise-SQL@v1 codex \
  --tasks aws-active-access-keys-older

The harness id is the directory name, not an agent flag. OSB resolves the directory, validates the manifest and dataset identity, pings the required models, and only then starts the candidate.

Expected output

The smoke test is complete when the runtime and evidence path are both visible.
  • The setup view identifies the harness type as codex.
  • Codex starts non-interactively in the task workspace.
  • The benchmark MCP server exposes get_schema and run_query to the session.
  • The task reaches scored and OSB prints the saved run directory.
shell
$ osb inspect

Troubleshooting

Fix the first failing boundary before changing the model or prompt.
codex is not found
Install the CLI and make sure codex --version succeeds in the same shell that starts OSB.
Codex is not authenticated
Complete the local Codex login or provider configuration first. OSB deliberately does not manage the CLI’s account.
Manifest rejects system_prompt override
Change the block to mode = "append". Codex has no override channel.
Writes fail inside the task
Check the workspace path and your local Codex sandbox settings. OSB invokes Codex with workspace-write isolation; do not solve the failure by granting unrelated host access.
Provider preflight fails
Fix the provider named by osb doctor. Codex login and judge credentials are separate requirements.

Next

Move from a working smoke test to a stable, complete candidate.

Compare

Choose another agent

Return to the integration hub and compare runtime boundaries.
Compare agents

Run

Complete coverage

Run the full benchmark against the bundled dataset and inspect the result.
Review the run loop

Publish

Submit the candidate

Check eligibility, quota, visibility, authentication, and platform status.
Prepare a submission