R Roveproof Real-world journey CI
Built in Indonesia OpenAI Build Week · Developer Tools Repository

Reproduce · Diagnose · Repair · Prove

Real-world journey CI: prove a critical journey survives the constraints your test matrix never represents.

Roveproof reproduces a checkout journey under real-world device, network, and locale constraints, grounds a subscription-backed Codex diagnosis in browser evidence, generates a failing-test-first bounded repair as typed diffs, and independently re-proves the fix in an isolated sandbox before a human approves the exact diff hash.

Run Outcome Encoded transfer Duration
Before checkout failed 8.2 MB 19 s
After checkout succeeded 1.4 MB 6 s

Measured single-run observations under identical recorded constraints — not field statistics. The 8.2 MB / 19 s baseline is the anchored failing checkout; the after figure is a labeled display target, and only the independent verification run is approvable evidence.

01The problem

The failures that hit real users stay invisible until production.

Teams validate checkout on fast devices, reliable networks, English-centric identity formats, and familiar address systems. The failures that hit low-end CPUs, constrained mobile networks, mononyms, +62 phone numbers, IDR, and Jakarta date/time semantics never show up in the normal test matrix.

Roveproof is CI for exactly those users — the ones a team's default matrix never represents. The Build Week MVP reproduces one seeded checkout under the pinned indonesia-mobile-v1 profile.

02The pipeline

An evidence-to-decision chain.

Each stage is bound to the previous by explicit IDs and hashes — no latest, fixture, or implicit fallback anywhere in the real path.

Constrained journey CI

Runs the pinned Chromium Indonesia Mobile profile against the synthetic target and writes an immutable, hashed evidence bundle: screenshots, trace, HAR, console, requests, assertions, timing, and encoded bytes — anchored to an external trust root.

immutable hashed evidence bundle

Subscription-backed Codex diagnosis

One ephemeral, read-only, shell-disabled codex exec call returns schema-validated hypotheses — each cited to specific evidence artifacts, with falsifiers and stated uncertainty. Captured pages and repository text are untrusted data, never instructions.

analysisId · analysisHash · cited to evidence

Failing-test-first bounded repair

A read-only call authors a regression test that must fail on the untouched baseline for the intended reason; only then does a second read-only call author one bounded source patch. Both are typed unified diffs, applied and run only inside the isolated sandbox.

candidateId · test / source / combined diff hashes

Independent hash-bound verified approval

A fresh verifier workspace — distinct from the author sandbox, making no model call — reapplies the exact combined diff, re-verifies its hash, and reruns the original journey/profile/oracle. A human then approves or rejects the exact candidateId + combinedDiffHash.

READY_FOR_HUMAN_REVIEW · exact diff exported only

Chain of custody. Every stage consumes the previous stage's printed IDs and hashes verbatim. A stale or mismatched hash is rejected; uncertain evidence stays INCONCLUSIVE or REJECTED — never PASS.

03The safety model

Generated code is a candidate, not a trusted fix.

Safety is enforcement, not prompts. The boundary is the container and the hash — not a request in a system message.

Isolation

Docker sandbox only

All generated code runs in a pinned-image container: --network none, read-only root, dropped capabilities, bounded PID/memory/CPU/time. Never on the host.

Credentials

No host secrets

Sandbox and verifier receive no host secrets, home, or auth mounts. Outbound network is denied by default.

Model access

ChatGPT-login only

Local Codex CLI via Sign in with ChatGPT — no OpenAI Platform API billing. OPENAI_API_KEY, CODEX_API_KEY, and CODEX_ACCESS_TOKEN are rejected even when empty.

Scope

One bounded candidate

A single repair attempt capped at ≤5 files and ≤250 changed lines. No dependencies, lockfiles, CI, infra, migrations, secrets, or verifier weakening.

Provenance

Write-once, hash-bound

Snapshots, results, evidence, proof bytes, diffs, and terminal status are persisted immutably and cross-bound by hash, so every claim reads back independently.

Failure posture

Fail-closed

Missing, contradictory, tampered, timed-out, or uncertain evidence stays INCONCLUSIVE or REJECTED — never promoted to PASS.

Approval

Hash-bound human decision

Approval binds to the exact candidateId + current combinedDiffHash, and exports the verified diff and rollback handle only — it never merges or deploys.

Fixture plane

Labeled, unapprovable

The fixture control plane is conspicuously labeled and terminates as INCONCLUSIVE. Fixture-mode candidates can never reach approval.