Architecture

Runs wherever it already runs. HB-Eval does not host it, proxy its model calls, or sit between it and its tools. Its control flow is unchanged.

This is where measurement happens. The five metrics are computed locally, in your process, from running tallies — about 0.002 ms per step, flat from a hundred steps to fifty thousand.

Safe Halt is decided here too. That matters: if the network is down, your policy still fires. A guard that needs a round trip is a guard that fails exactly when infrastructure is already struggling.

Python and TypeScript. Or nothing at all — one import derives signals from OpenTelemetry spans you already emit.

Telemetry uses the same encrypted, signed envelope as evaluation — live monitoring is not a lower-security path. Batches leave on a separate thread, so network latency never enters your agent’s execution time, and a failed send is dropped rather than raised: instrumentation is not permitted to be the reason a run breaks.

The exception worth knowing: on the evaluation path you submit a task deliberately, so its text does reach the Gateway to be scored. It is encrypted in transit and not retained after scoring.

Authenticates, verifies the signature, rejects replayed requests, enforces quota, and scores evaluation runs against the fault battery. Dispatches alerts to Slack, PagerDuty or your webhook.

Holds the passport signing key — which is why passports are signed here and nowhere else. A key present in two deployments is a key with two chances of leaking.

Sessions, metric history, halt decisions with the policy that caused each one, alert delivery records including the failures.

Agent secrets are encrypted at rest; the API key is stored only as a hash. Step-level snapshots are removed after 90 days by a job that actually runs — the last successful sweep is published on the status page.