Northline · Part 3. Anonymized composite (Northline B2B)—several implementations, not one audited company. The bands below are illustrative ranges carried from Part 2. Handle time there is the assisted queue, where a person still sent the reply. Shadow percent is how much of that queue the draft ran on. Neither figure is a new measurement for this renewal.
Renewal quarter
The quote arrived while that assisted queue was still on the Part 2 pattern: drafts on most of the queue, a person still sending, pass rate in the same band. The sponsor wanted the host for another year. The account team added a governance module that would hold policy packs, eval cases, and the audit export in the vendor tenant.
The offer was concrete. One console, operated by the vendor, would have spared Legal the join between a git pin and a log row. The question they actually had to answer was narrower. Could they still withhold a production pin without opening a ticket?
What the module could not show
In the demo, rejecting a prompt change opened a vendor ticket. The export of the deprecated-refund near-miss returned the model text and a timestamp. It had no policy_pack_version, so the row could not be tied to a pack. The module missed their requirement to block a pin themselves. That is a miss against their bar. It is not evidence that every governance console fails this way.
Keeping the bar had a cost they were already paying. IT runs smoke on prompt and context pull requests. Support ops runs the full case set each week. They still own the log schema. Moving those into the module would have handed that calendar to the vendor. They declined the line item. The host stayed on the quote.
What blocks the pin
Git and the audit log are different stores.
Git holds the policy pack, the prompt pin, and the eval file. The audit log holds the run. A row carries workflow id, pack version, prompt pin, actor, and UTC time. Reading the row reconstructs what was live. It does not run the model again.
The host executes the production pin they have promoted. The block is the smoke job in evaluation hooks: ten cases, all must pass, on pull requests that touch prompt or context files. A failed smoke stops that merge. The process owner withholds promotion. If a pin has already shipped, IT rolls it back. The YAML is the case file the job runs. A file in a repository does not stop a vendor scheduler.
The split they kept
The exhibit is the hybrid from When to Buy vs Build an Agent Stack:
| Layer | Vendor host | Their side |
|---|---|---|
| Runtime | Renewed | — |
| Connectors already live | Stay | Permission policy stays with the workflow |
| Policy packs and prompt pins | — | Git; Legal bumps the pack |
| Eval file | — | Git; the smoke job stops the merge |
| Run record | Export when the row is complete | Audit log, pointing at pack version and pin |
Human send on customer replies stayed. The renewal added no second workflow.
One near-miss, two checks
They used the deprecated-refund run from the Part 2 window, caught before send.
Reconstructing the run meant reading the log row and checking that policy_pack_version named a pack still in git. The module export could have matched two packs, because the version was missing. Their row named one.
Re-running the case is the other check. The next prompt change had to pass smoke before the pin moved. The forum withheld that pin for about a week. Treat the week like the ranges above: an illustration of the hold inside the composite, not a clock from one contract. The host renewal went through. Handle time on the assisted queue stayed in the Part 2 band. That is the result. Control survived the quote. The queue did not get a new gain.
On a renewal call
- Write the split: what the vendor runs, and which pin you promote.
- Reconstruct one known-bad run from the log row. Require
policy_pack_version. - Confirm a failed smoke job stops the merge that would move the pin, with no vendor ticket on that path.
- Renew the host after those two checks exist.
Copy-paste gates for the job: AI Workflow Eval Checklist.
