AI-native SDLC · Part 2
One methodology, three projects, three different endings
auth9 scored close to full marks, orchestrator got archived, deck squeezed the method down to almost nothing. Six months and three projects later, I understand that a methodology's value isn't a constant. It depends on the conditions.
This is the second half of Can AI polish software the way people do?. The first half was written during auth9 and answered “can it?”. This one, written six months later, answers “and then what?”.
Between January and September 2026 I used the same AI-native development method on three projects:
| Project | When | Size | Now |
|---|---|---|---|
| auth9 | Jan–Jun | ~215k lines, 491 commits | experiment finished, method near full marks |
| orchestrator | Feb–Aug | ~200k lines of Rust, 1,606 commits | archived |
| deck | from Aug 27 | ~34k lines, v0.5.15 in 10 days | in daily use |
Three completely different endings. The point of this post is one sentence: a methodology’s value is not a constant, it is a function of its conditions. And to spell out what those conditions are.
auth9: when the conditions are right
The skeleton of the method is in the first half; here are just the bones:
- 16 skills that turn “how to fix a ticket”, “how to write a test”, “how to manage coverage” into procedures an agent can execute directly, mirrored to Claude, Gemini and Cursor;
- 224 test documents with SQL verification, a manifest as the index, a lint script to stop them rotting;
- every test finishes in a second or two, no testcontainers, no real database;
- mechanical gates the agent cannot route around: domain boundary checks,
clippy -D warnings, OpenAPI sync tests, pre-commit secret scanning; - my own role reduced to four things: plan, review, steer, architect. QA execution and ticket fixing were fully autonomous; commit titles like “Resolve 27 QA tickets: fix 9 bugs, close 14 false positives, defer 4 feature gaps” are the pipeline’s own statistics.
Five months, one person plus a set of agents (copilot-swe-agent made about 5% of the commits), and out came an IAM platform with OIDC conformance tests, SCIM, WebAuthn, a V8 action engine, and fully automated K8s plus GitOps deployment. As a methodology experiment, I was happy with it.
But there was a condition I hadn’t thought through: auth9’s intent came for free.
What counts as correct? The RFCs and the OIDC conformance suite decide. Which features to build? Auth0’s feature list decides. How far to take security? Threat-modelling methodology decides. From start to finish, the method only ever had to answer one question, “does the implementation match the intent?”. It never had to answer “what should the intent be?”.
In a domain where correctness is the product, this method has maximum leverage. That explains the high score. It also carries a corollary I only felt later: a pilot in the most favourable domain gives you a score you cannot extrapolate.
orchestrator: when the conditions are wrong
The idea behind orchestrator was natural enough. If this pipeline can get agents to polish an IAM platform, make the pipeline itself the product: a control plane that gives agents workflows, sandboxes and audit, K8s-style manifests, a plan → implement → test → review → fix loop that runs on its own for days.
Looking back, orchestrator was simply the auth9 development process turned into a product. Its workflow loop is a generalisation of auth9’s QA ticket batching; most of its 30 skills descend from auth9’s 16 (one skill’s directory candidates still list auth9-core); even its assumed user persona, the “operator”, is the role I played myself on auth9.
The method did evolve in this phase, and a few of the inventions I still think are worth keeping:
- Mutation verification. The most dangerous failure mode of AI-generated tests is “looks green, verifies nothing”. The record shows several first-draft tests failing mutation checks, being ruled invalid, and being rewritten.
- Typed discipline. Human teams pass conventions on through code review. Agents have no memory; every session is a new hire. So conventions have to sink down to the level of the compiler. Secret redaction became a type: forget to redact and it doesn’t compile.
- A two-sided coverage ratchet. A “coverage may only go up” ratchet quietly loses its ability to detect regressions. With actual coverage at 52.86% and the baseline stuck at 35.49%, a 17-point drop would have kept the gate green. So rising too far without updating the baseline also counts as a failure.
- A concept budget, deliberately not made into a gate. Adding a top-level concept requires arguing “why this is not just a field”. The rule was explicitly refused as a CI gate, because a gate can only check that “some justification text exists”, and using the existence of text as a proxy for review quality is worse than no gate at all.
The problem wasn’t these inventions. The problem was two bills.
Bill one: institutional cost is fixed; the return scales with users. 101 CI gates, a 250-kilobyte CHANGELOG where every entry carries measured numbers, freshness records, negative-fixture meta-verification of the gates themselves. All of that is designed for a platform maintained by many people over a long time. The team was two people, and the users rounded to zero. In the last month before archiving there wasn’t a single new feature in the commit log; it was all the governance system converging on itself, compressing its own records, tiering its own gates for performance. When the governance system needs governing, the account is already overdrawn. This method measured tests, coverage, gate freshness, dependency vulnerabilities. The one thing it never measured was whether it was worth its own price.
Bill two: the method can verify that an implementation matches the intent, but it cannot generate intent. auth9’s intent came from standards. orchestrator’s came from my personal preferences. A K8s-style mental model is the taste of someone who builds infrastructure, not of an individual developer managing a handful of coding agents. The kubectl metaphor assumes the user is an operator, but the real user wants to look at their own terminal, not apply a manifest and wait for the system to converge. A mental model isn’t just an interaction style; it is an assertion about who the user is. That assertion was wrong from day one, and all the rigour of the method sat on top of an assertion nobody had tested. Taste can be a hypothesis. It cannot be an axiom. orchestrator’s hypothesis went seven months without touching anything that could falsify it.
One date worth writing down: orchestrator’s first commit came 11 days after auth9’s. The expansion began before the validation had finished. Piloting in the most favourable domain and then generalising, and expanding before the experiment has reported, turn out to be two faces of the same mistake.
The final self-diagnosis is in a July design document: the production workflow had swollen to 443 lines of YAML, about 70% of it coordination overhead paid for a dumb-agent contract. The exact words were “The dumber the agent contract, the smarter the YAML must be”. The fix at the time was to let agents express intent directly through typed tools. But follow that logic to the end and you walk out of the project: the smarter the agent, the thinner the reason for a harness around it, and “agents getting smarter” is something the model vendors are driving. At the end of August, I archived orchestrator.
deck: what’s left after you squeeze the method down
deck started the day after orchestrator was archived. It’s a native macOS board where every card is a real tmux session, and the board only tells you which agent is producing output, which has gone quiet, which has exited. Where to look is up to you. 34k lines, 8 settings, 0 CLI subcommands.
On deck the method is compressed almost past recognition: no skill library, no FR registry, no meta-verification of gates. But compressed isn’t discarded. Compare it with what “a normal ten-day project” looks like and the skeleton is visible: 312 Rust tests plus 121 JS tests, a hard 70% coverage floor, a real WKWebView smoke test, every CI action pinned to a commit SHA, privacy tests that must write real files (the comment says “never stub the writer and call it proven”). Those muscles were built on the two earlier projects. They don’t need an institution to sustain them, because they have become habits.
What actually survived is three things:
- Machine-verifiable trust, taken as needed rather than laid down in advance. deck’s status display started as a heuristic: no output for 15 seconds, the card turns amber. A few days of use showed it couldn’t tell “quiet because working” from “quiet because waiting for me”, and only then did I add agent hooks that report a closed set of status words. That is a miniature of the core idea behind orchestrator’s streaming-runner pivot, a typed contract replacing guesses about text. It’s the first time the method flowed back in the form “one verified need issues one mechanism”.
- Refusing false greens, extended from tests to the product itself. After a scheduler crash, deck would rather mark a delivery ambiguous than claim success. The status copy says outright that quiet does not mean ready. The first half said verification is the bottleneck; this half adds a line: honesty is what verification looks like when it becomes a product.
- Institutional memory changed containers. From a 250-kilobyte CHANGELOG of argued entries to a casebook: every pitfall tagged “each cost a real bug”, every withdrawn design with its reason. For a one-person, short-cycle project, a casebook is the right shape for an institution.
What deck genuinely added isn’t engineering. It’s where intent comes from. deck tests a product-first AI-native SDLC: first there’s something I use every day, needs surface from the use itself, and the method comes in as required. The question the previous generation couldn’t answer, where does intent come from, this generation answers with observation. Intent comes from real use, not from external specs and not from taste.
Five things I’ll keep
For anyone else trying to write code at scale with agents:
- Ask where the intent comes from first. The method can verify “implementation matches intent” to the limit, but it can’t produce correct intent. Domains with external specs (protocol implementations, standards compliance, migrations) are its home ground. Where the spec has to be discovered from use, solve discovery first, then bring in the pipeline.
- Give the governance system a budget line. The size of the institution should be a function of team size times user count. While you’re measuring everything, measure the institution’s own cost. When you start compressing your governance records and performance-tuning your gates, the account is already overdrawn.
- Sink conventions down to something mechanical, because agents have no memory. Type systems, compile-time checks, CI gates all beat conventions written in a document. But a gate can only check what it can see. Rules that need judgement should stay with people; don’t proxy them with “some text exists”.
- Treat false greens as the number-one enemy. Mutation verification, real-file tests, two-sided ratchets are all the same move: refuse to trust the “pass” signal itself, and demand it prove it can still fail.
- Don’t extrapolate a pilot’s score. A high score in the most favourable domain shows fit, not universality. Before expanding, let the method sit one more exam in a domain where intent doesn’t come for free, and wait for the result even if the wait is only a few months.
Now
The method hasn’t retired. Its core became habit and is in use on deck every day. The institutional layer sits in an archived repository as an option, not a plan.
What deck will need next, I don’t know. It’s still in daily use; which problems will surface and which deserve a mechanism will only be clear after it has been used long enough. This lineage already contains one lesson about “design first, validate later”, and I’m not repeating it. The only thing I’m sure of is the order: needs surface from use first, then decide which practice to import.
The first half asked whether AI can polish software the way people do. Yes. This half asked what happens next. The answer: the method didn’t fail, it just matured before its product did. The next answers come from deck.