Bad Boy Software All articles
Opinion

Your AI Can't Be On Call at 3 A.M.

Bad Boy Software
Your AI Can't Be On Call at 3 A.M.

Photo: DFID - UK Department for International Development, CC BY 2.0, via Wikimedia Commons

It's 2:14 a.m. on a Tuesday. Your checkout service is returning 500s, the error tracker is lit up like a Vegas strip, and the on-call engineer is staring at a function nobody remembers writing. The tests passed. The pipeline was green. The pull request had plenty of approvals. And yet here we are: a production incident with no obvious author, no clear intent, and no human being who can explain why the code does what it does.

Welcome to the accountability gap. AI-assisted development has sprinted ahead of the structures we use to govern code, and a lot of teams are now shipping diffs that nobody truly owns. The model wrote it, the assistant suggested it, the autocomplete accepted it — and somewhere in that chain, responsibility got optional.

That's not a tooling problem. It's an ownership problem. And it gets expensive at exactly the wrong time.

The Diff Doesn't Care Who Wrote It

Git doesn't know whether a human or a model produced a line of code. It records a commit, an author string, and a timestamp. If your process stops there, you've created a very efficient machine for generating orphaned code.

In 2026, it's normal for a sizable chunk of a feature branch to start as model output. That's fine. Speed is the point. But speed without a signature is borrowed time. The moment something breaks — or the moment a security review asks why a dependency was added, why an endpoint skips validation, or why a retry loop has no ceiling — "the AI wrote it" is not an answer. It's an abdication.

Code has always needed a human owner. Not because humans are infallible, but because ownership is how organizations route questions, decisions, and blame. Remove the owner and you remove the routing. Everything becomes a group project, which is another way of saying nobody is responsible.

"It Passed the Tests" Is Not a Review

This is the trap. The model produces a diff. The tests pass. CI goes green. Someone clicks approve because the dashboard is happy. Repeat until the codebase is a haunted house.

Tests are necessary. They are not a review strategy. A test suite checks the behavior you thought to assert, not the behavior you failed to imagine. Model-generated code is especially good at satisfying the shape of a test while missing the point of the requirement. It can pass a unit test and still introduce a race condition, a bad default, a noisy log line that costs real money, or an authorization check that works for the happy path and nobody else.

A review is a human being asking: What is this code trying to do? What happens when it fails? Who owns it after merge? Which assumptions are baked in, and which ones are mine to verify? If your review process can't answer those questions, it's not a review process. It's a rubber stamp with a Jira integration.

Authorship Is a Verb, Not a Prompt

There's a lazy version of this conversation that goes: "AI wrote the code, so AI is the author." No. A model is a tool. It has no liability, no on-call rotation, and no ability to sit in a postmortem and explain itself. Authorship in software is not about who typed the characters. It's about who stands behind the result.

That means the human who accepts a generated diff is the author of that diff in every way that matters. They own the design choice, the edge cases, the operational cost, and the rollback. If that feels heavy, good. It should. The alternative is a codebase where the most consequential lines have the least accountability.

This isn't an argument against AI-assisted development. It's an argument for treating it like any other high-leverage tool: useful, fast, and capable of doing real damage if nobody is watching.

The Ownership Checklist Your Team Actually Needs

You don't need a 40-page AI policy to fix this. You need a few non-negotiable habits that make ownership visible.

1. Put a human name on every pull request

Not a team name. Not a bot. A person. That person is the owner of record for the change and the first call when it misbehaves. If nobody wants to put their name on it, that's a signal — not a formality.

2. Keep a provenance trail

Record what was generated, what model or assistant was used, and what a human changed before merge. You don't need to archive every prompt, but you do need enough context that a future engineer can reconstruct intent. A short note in the PR description beats a forensic investigation later.

3. Review for failure, not just correctness

Ask the uncomfortable questions: What happens under load? What happens with bad input? What happens when the third-party API times out? What data does this touch, and who can see it? If the reviewer can't answer, the PR isn't ready.

4. Require a rollback plan before merge

Every change that can affect production needs a clear undo. Feature flag, revert path, migration rollback — something. "We'll figure it out if it breaks" is not a plan. It's a prayer with a deploy button.

5. Define what "AI-assisted" means on your team

Is it autocomplete? Wholesale function generation? Agentic changes that span multiple files? The risk profile changes with the scope. Write it down. Make the expectations boring and explicit so nobody has to guess at 2 a.m.

Speed Without a Signature Is Borrowed Time

The pitch for AI-assisted development is real: faster drafts, less boilerplate, more time for the hard parts. But the teams that win with it won't be the ones generating the most code. They'll be the ones who kept a human signature on the result.

A model can write a diff. It can't be on call. It can't sit in the incident review. It can't tell your largest customer why their data was briefly exposed. That job still belongs to a person — and if your process doesn't say which person, you've already borrowed against a debt you'll pay back at the worst possible moment.

Put a name on the pull request. Keep the trail. Own the outcome. That's not bureaucracy. That's the bare minimum for shipping code that talks back.

All Articles

Related Articles

Nobody Says 'Simple' When the Invoice Arrives: Decoding the Real Price of 'Easy' Features

Nobody Says 'Simple' When the Invoice Arrives: Decoding the Real Price of 'Easy' Features

Your Beautiful Abstraction Doesn't Pay Rent

Your Beautiful Abstraction Doesn't Pay Rent

Committee Engineering Is Why Your Competitor Beat You to Market

Committee Engineering Is Why Your Competitor Beat You to Market