How to Verify the Exact Code Your Agent Will Ship
Published 24 September 2026 · 7 min read
By Cong Le · Pocket Grove
Original research brief: . Sources reviewed for publication.
Quick answer: Before shipping an agent's changes, identify the exact source snapshot, attach each required check to that snapshot, and record the artifact selected for deployment. Then verify the result from the system that will consume it. A passing run supports a particular input and environment; it does not automatically cover later edits, a different merge result, or an older deployment package.
Imagine two green CI runs. One checked commit A yesterday. Another checked commit B this morning. Your coding agent says the release is ready, but the deployment job has selected the package built from A.
Both runs may be accurate. The missing fact is which evidence supports the release you intend to make.
Coding agents make this question easy to overlook because they can move quickly between editing, testing, merging, and reporting completion. A small verification receipt makes those transitions reviewable. It connects four things:
Source snapshot → required checks → selected artifact → receiving system
The examples below are illustrative designs, not results from an executed experiment. The receipt is a proposed starting point for your workflow, not an industry standard or a complete release gate.
Identify the source that the checks actually read
Record the repository, working directory, and full commit identifier inside the environment running the checks. These read-only commands provide a useful starting point:
git rev-parse --show-toplevel
git rev-parse --verify 'HEAD^{commit}'
git status --porcelain=v1 --untracked-files=all --ignore-submodules=none
Git documents rev-parse as a way to resolve and verify object identifiers. HEAD identifies a commit; it does not identify subsequent working-directory edits. Git revision documentation.
If an agent changes a file after committing A, a test can run against A plus that edit while the receipt still says A. Staged changes and untracked input files matter too. Git's status output helps expose these differences, although ignored files require separate attention. Git status documentation.
For a simple release process, require a clean candidate and prevent concurrent writers while checks run. Capture source identity before and after the run. Those observations alone cannot rule out a temporary edit that was later undone; controlling writes closes that gap.
If dirty working directories are part of your process, preserve an explicit snapshot of every relevant input. A commit plus a tracked-file diff may omit untracked fixtures, generated assets, or ignored configuration. Also record dependency versions, toolchain versions, build settings, and external inputs that affect the result. An empty Git status is not a complete environment inventory.
Record what the candidate will join
A feature branch name is a moving label. Record the target branch and its resolved revision, along with the candidate revision. If the agent worked in a separate worktree, record its starting revision as well.
Do not assume that a worker inherited the orchestrator's current branch. For example, Claude Code documents a configurable worktree base: the default fresh behavior uses the remote default branch, while head selects local HEAD. Choose the behavior your task requires and inspect the resulting revision. Claude Code worktree documentation.
The next question is what CI checked. GitHub's workflow-run API supports filtering by head_sha, which helps find runs associated with a candidate. Still inspect the workflow, attempt, jobs, and conclusion. GitHub workflow-run API.
There is a subtle distinction here: for a GitHub pull_request workflow, GITHUB_SHA normally refers to the pull request's merge commit, and the default checkout uses that merge result. The pull request's head commit is available separately. Record the actual checkout revision alongside the run's associated head revision. GitHub pull request event documentation.
If a rebase, conflict resolution, or newer target branch changes the integration result, reassess the evidence. For a small workflow, rerunning the required checks on the final candidate is usually easier to explain than maintaining rules for when old results remain sufficient.
Keep a receipt that starts incomplete
Copy this JSON into a release record, or download the receipt template, and adapt the check names to your repository. Every null is missing evidence. The single check entry shows the structure; add one entry for each required check.
{
"schema_version": 1,
"repository": "OWNER/REPOSITORY",
"source": {
"checkout_path": null,
"worker_base_commit": null,
"candidate_commit": null,
"checked_commit": null,
"working_tree_clean": null,
"input_snapshot_digest": null
},
"target": {
"branch": "main",
"resolved_commit": null,
"environment": "staging"
},
"environment": {
"toolchain": null,
"dependency_manifest_digest": null,
"build_configuration": null
},
"required_checks": ["unit", "build"],
"checks": [
{
"name": "unit",
"checked_commit": null,
"command": null,
"workflow_revision": null,
"run_url": null,
"attempt": null,
"conclusion": "not_run",
"evidence_location": null
}
],
"artifact": {
"build_run_url": null,
"immutable_id": null,
"sha256": null
},
"delivery": {
"deployment_id": null,
"observed_artifact_id": null,
"consumer_check": null,
"conclusion": "not_checked"
}
}
Define the required checks before assessing the candidate. A receipt with a successful unit check and no build result remains incomplete. Preserve logs or reports so a reviewer can see what ran, including skipped jobs, reduced test selection, and tolerated failures.
Store the completed receipt with the CI run or release artifacts. Include timestamps and relevant configuration identities as your process needs them. Record secret references or configuration versions rather than copying credentials into the receipt.
A JSON file is a record, not an enforcement mechanism. Someone must compare its contents with the actual candidate and release target. Automation should reject missing required fields, mismatched inputs, and absent checks rather than trusting an agent's summary.
Follow the evidence into the artifact
A source check and a package check answer different questions. Testing commit B does not establish that the deploy job selected B's package. Record the build run and an immutable artifact identifier or digest, then compare that identity at the deployment boundary.
For stronger build provenance, GitHub artifact attestations can associate an artifact with information about its originating workflow and repository. Verification still requires an explicit trust policy for the expected producer. GitHub artifact attestation documentation.
A matching digest establishes byte identity. It does not establish functional correctness, test coverage, or whether the build environment was trustworthy. Rebuilding from the same commit also need not produce identical bytes when dependencies, timestamps, or other inputs differ.
These hypothetical cases show where the receipt should stop a release claim:
| Evidence available | Intended release | Decision |
|---|---|---|
| Required checks passed on clean A | The recorded artifact built from A | Source and artifact evidence match; delivery remains to check |
| Required checks passed on A | Candidate B | Obtain evidence for B |
| Checks ran on A plus an unrecorded edit | Clean A | The tested input is insufficiently identified |
| Checks and build passed on B | Deployment selects A's package | Correct the artifact selection |
| Source matches; a relevant runtime setting changed | Changed runtime environment | Assess and check the affected behavior |
Verify from the receiving system
A successful local read-back establishes what that process can see. Another process may have a different view.
Microsoft's MSIX documentation provides a concrete example: virtualized AppData files can live in a private location, appear in the packaged app's merged view, and remain invisible to other apps. This documents an operating-system mechanism; it does not establish the behavior of any particular coding-agent package. Microsoft flexible virtualization documentation.
Apply the same question at deployment: what evidence can the intended consumer provide? Inspect the deployment's selected artifact and exercise an appropriate behavior through the receiving application or service. If only a version label is observable, say so; that is narrower evidence than comparing the deployed bytes.
Before declaring completion, check:
- Does the receipt identify the actual source inputs and integration target?
- Did every required check run on those inputs with retained evidence?
- Did anything relevant change afterward?
- Is the selected artifact the one the receipt describes?
- Has the receiving system confirmed the expected result?
Scoped agent knowledge helps an agent find the right procedure, and instruction evaluations help assess that procedure. The release receipt records which procedure and evidence support this particular delivery. Keep it close enough to the release decision that “ready” can be traced to the code, artifact, and outcome being approved.
Apps from the studio
All appsThese practices come from shipping Pocket Grove's active apps. If you came here looking for something to install, start with one of these.
Related guides
What to Check After You Stop a Coding Agent
Recover after stopping a coding agent: inspect completed writes, handle unknown outcomes, check undo coverage, and decide when a retry is safe.
Jev, Open Source Experiments, and a Different Kind of AI Decision
A look at Jev's hosted decision API, open-source examples, and a hands-on local model demo in your browser.
Write Tool Contracts That Prevent Avoidable Retries
Design AI agent tool schemas with visible limits, useful errors, and server validation. A practical MCP and function-calling contract checklist.