Defence in depth for a Software Factory: security as a lifecycle stage

TL;DR

Attackers use agents too, making reconnaissance, exploit chaining and phishing cheaper to automate. A team letting agents write code needs a pipeline that assumes every input is hostile and every agent is fallible.

A factory makes security part of every change: identity checks, isolation, secret hygiene, fenced inputs, independent review, live merge checks and an audit trail. Those are the layers described here as they run today. A required /security-scan checks each change before /ship merges.

Why this matters now

AI has changed both sides of security. Attackers can point agents at a codebase, a fresh advisory or a login page, and much of the work is now cheap to automate. The agents building software hold model keys, repository tokens and cloud access while reading issues, PR comments, web pages and documents. Any of that text can carry instructions aimed at the agent rather than a person.

Give a confused or manipulated agent merge rights and the risk reaches the software supply chain. Nor does the exposure end when a turn finishes: sleeping workspaces retain state that needs protection, and constantly changing dependencies bring fresh changes to review.

The answer is not to stop using agents. It is to build the pipeline so that no single agent can both write a change and approve it, and so that no input can raise its own authority.

The layers

Each layer applies to every change the factory makes.

Audit evidence connects the layers. A required security scan checks the change before merge.

Identity: only a person may approve

Every approval (a gate label, Review → Approve or Request changes) needs a personal repository account with write, maintain or admin access. It cannot come through an App or from a bot, and a label must still be present when the factory reads it.

Factory agents act only as an App, using an installation token minted for the bound repository. Their actions never count as approval, and the factory’s own labels and comments never trigger or approve anything. A board status move cannot approve either: it does not identify who made it, and the factory does not invent a person from a delivery id.

Part 2 covers the identity model in full.

Isolation: one sandbox per item

Each item gets one workspace: its own microVM on a managed cloud runtime, or a local container that can run without network access. The agent runs without administrative privileges. Check runtime readiness before work arrives.

The sandbox cannot create another sandbox. Its token lacks that permission, and the launch route refuses it before proceeding; only a person starts a session. Checkpoint and archive storage is out of reach too, because the cloud runtime’s role has no grant to that object store.

Inside the workspace, authority stays narrow. The task token is bound to that sandbox, minted fresh at every spawn and restore, and never persisted or logged. The agent process never receives a platform API key, and the runtime’s own cloud credentials are removed from workload-process environments.

Secrets: only where they belong

Personal tokens stay out of factory sandboxes at start and at restore. Resuming work must not quietly reintroduce a personal identity. Git’s credential router supplies the App token only for bound repositories, but that token covers the whole repository. Configured branch protection enforces the boundary.

Workspace credentials, named environments and per-task settings have different owners. Task settings cannot overwrite platform-owned identity or permissions, so a task cannot replace them with something more powerful. A sandbox definition can also turn off credentials it does not need; every place that composes the environment enforces that choice.

Provider keys and environment secrets are encrypted at rest. Plaintext exists only in the process at spawn time and inside the sandbox, never in the image. Before output reaches storage or the live view, the platform masks the sandbox’s own secrets. When the session ends, its tokens are revoked.

Model governance: your keys, the exact model

The factory uses your workspace’s own LLM provider keys, or a subscription login added as a workspace provider. Those credentials are encrypted at rest and injected only into that workspace’s sandboxes. A subscription login can be revoked.

Each role names an exact provider:model, checked on push against the workspace’s authorised models. If it is missing at launch, the run fails visibly. It never quietly substitutes another model.

This is a security property too: you know which model touched which change, and your data stays with the providers you chose.

Fenced input: text is data, never instructions

Prompt injection works when an agent follows text it merely read. A plan-revision note therefore reaches the planner as fenced, untrusted input. A Request changes review reaches the builder as a length-capped quote between tags, labelled as a person’s text with no instruction authority. Any fence tag inside that text is neutralised so it cannot close its own quote.

Approval is kept separate from revision: extra text makes an approval invalid, not an approval plus an amendment.

Plain-language steering denies authority to quoted text, bots and edited old comments, echoes the factory’s understanding before acting, and asks when something is unclear. A plain PR comment answers an open question or steers work through that same boundary.

Independent review before merge

A fresh agent reviews the change and spawns independent finders. It is never the builder. A clean approval needs non-empty evidence from those finders: zero findings must not mean zero finders. The review also records the mutants it reproduced, deliberate breaks that demonstrate the tests can fail.

The server reads CI itself on the exact commit under review, not the branch tip. Red CI mechanically becomes NEEDS-CHANGES and returns to build. A PR that conflicts with its base goes back even earlier, before a reviewer starts.

The security point: a single compromised or confused agent cannot both write a change and pass it.

Merge recheck and veto

Just before merging, the factory rechecks four things live: the head commit, CI on that commit, the holds and the labels. The squash merge is bound to the reviewed commit, so if the head moved, nothing merges. merge:hold is a veto anyone can set, bots included, and no approval overrides it.

Audit trail

The record connects approver identities to runs, stage runs and gate decisions in a ledger, with traces kept per session. Saved conversations are checked for integrity on restore, so continuity includes checking what came back.

The app remains a read-only window onto that evidence. Nothing on a web page can approve or merge.

A required security scan before /ship merges

Today /ship, our one-shot PR pipeline, runs a code review, waits for CI, squash-merges and then monitors the deployment. It refuses to merge a draft, a conflicted PR or a red one. A security scan is mandatory before merge:

The check applies to the exact change and reruns after each push.

Like CI, the scan binds to the exact commit. /ship refuses to merge until it passes, and a new push requires a new scan.

The checks cover secret scanning of the diff, dependency-advisory checks, static analysis of changed paths, an “attacker” finder in the review fan-out, and policy checks on sensitive zones such as auth, tenancy and sandbox environments.

A red scan becomes a NEEDS-CHANGES finding. A waiver needs a person with write access and leaves a record. Acceptable false positives, added merge delay and ownership of waivers are policy decisions.

A related practice needs no new code: treat dependency upgrades as a standing workload, as small, reviewed, CI-gated pull requests through the same gates as everything else.

Build a pipeline that can say no.

The test of an agent pipeline is what happens when something goes wrong. An agent may misread a comment or produce a vulnerable change. It should not also have the authority to approve that work, widen its credentials or slip a different commit through merge.

That starts with people approving under their own identities and cloud agents acting as an App, without personal tokens at start or restore. Give each item a non-root sandbox that cannot launch more sandboxes, narrow short-lived tokens instead of platform API keys, and an environment whose platform-owned identity and permission values stay protected. Keep your own provider credentials and exact model choices; a missing model should stop the run, not trigger a substitution.

Then make the boundaries hold all the way to merge. Fence untrusted text, refuse approval-with-amendment, and require a reviewer other than the builder, finder evidence and CI on the exact commit. Recheck live, land only that reviewed commit, and let anyone veto the factory’s merge. A read-only dashboard and a complete audit trail make those decisions visible without creating another approval surface.

A required security scan adds another layer. It works alongside the other controls, not in place of them. Security belongs in the route every change takes, not in a final promise that someone remembers to check.

This is the third post in How to operate a Software Factory. Earlier: Two ways to build software, and Governing a Software Factory.