The Vulnerability That Passed Every Test

An Ordinary Thursday

Marcus liked Thursday afternoons. The week’s standups were behind him, the demo was tomorrow, and the calendar – for once – was clear. He worked from a small office in his attic, two monitors angled inward, a half-finished cup of tea cooling on the desk.

He was a senior backend engineer at a healthcare SaaS company that served regional hospital networks across three countries. Scheduling, lab orders, patient communication. Boring on the outside, regulated to the bone underneath. He had been there four years and reviewed roughly one in three pull requests that touched auth. Auth went past Marcus.

Over the past year, the team had started using AI assistance heavily, and the throughput numbers had moved in the right way without any of the usual signs of corner-cutting. Velocity up. Defect rate flat. Engineers happier. It looked, from where he sat, like they had got the balance right.

The Pull Request

The notification slid into view in the corner of his screen.

Vamika Shetty requested your review on PR #4118: Refactor token validation and refresh flow.

Vamika was eight months into her first software engineering job. Smart, careful, the sort of junior who asked the right questions in code review and remembered the answers. The auth refactor had been on her plate for two sprints. Marcus had told her to take her time.

He opened the diff. It was bigger than he expected – eleven files, around six hundred lines net – but the structure was clean. Each commit told a story. The PR description walked through the old flow, the new flow, the migration path, the rollout, the rollback. CI was green. Marcus opened the first file and started reading.

The First Pass

The code was, frankly, better than what he would have written.

The token-validation helper had been pulled out of a sprawling middleware file into its own small module. Function names precise. Error types specific. There was a clock-skew tolerance on token expiry that he had been meaning to add for two years, and a thoughtful retry policy on the upstream identity provider — jitter, circuit breaker, sensible defaults.

The org-scoping logic, previously spread across a dozen call sites, was now consolidated behind a single helper that read the org_id claim from the JWT and used it to scope every database query. The token was signed; the claim could be trusted; one helper meant one place to audit.

He paused, briefly. The polish was a grade higher than Vamika’s other work — mature, where her recent PRs had been competent but inexperienced. He let the thought go. Pairing with the AI assistant had been good for several juniors on the team.

He scrolled on.

The Tests

The test file was the part that settled him.

Eight-hundred-and-forty-seven-unit tests in the auth module, all green, with a hundred and twelve of them new. He skimmed the names. Expired tokens. Malformed tokens. Replay attacks. Signature mismatch. Tokens missing the org claim. Tokens with an org claim for an org the user did not belong to.

He nodded at the last one. That was the test he had been about to ask for.

He ran through his usual mental checklist — boundary conditions, negative paths, error propagation, rate limits. The test names matched it almost line for line.

He felt the small, familiar lift of a review going well. The kind of lift that, on a different day, might have made him ask why his confidence was rising at exactly the speed the diff was getting larger. But he was forty minutes in on a Thursday afternoon, with the demo tomorrow, and the lift felt earned.

He moved on to the refresh endpoint.

Approval

The refresh endpoint was small — a single new file, eighty lines, well-commented. It validated the original token’s signature, confirmed the user existed, then issued a fresh JWT. A header, X-Active-Org, supported org switching: customers in the network often had access to several hospitals and had asked for a faster way to move between them. The endpoint read the header, set the new token’s org_id accordingly, and returned it. The org-switch test passed. The tokens-are-signed-correctly test passed.

Marcus left a single comment.

The PR auto-merged. The pipeline rolled to staging, then to production. He closed the tab, refilled his tea, and pulled the next ticket.

It was a small, ordinary Thursday.

Three Weeks Later

The email came in at 8:11 on a Monday morning. It was from the security inbox, forwarded by the CISO, with a single character in the body. “?”

The attachment was a quarterly penetration-test report. The first finding was flagged in red.

Marcus read the description twice before he understood what he was looking at.

The Vulnerability

The pen tester had created two test accounts in two unrelated hospital networks — call them North and South. No shared users. No shared data. Separate tenants on the same platform.

She had logged in as her North user and obtained a normal, validly-signed JWT scoped to North. Then she had hit the new refresh endpoint with that token in the Authorization header and X-Active-Org: south in the headers. The endpoint had returned a fresh, validly-signed JWT with org_id set to South.

Every downstream service trusted that claim — exactly as the consolidated org-scoping helper assumed it could. The pen tester used the new token to query patient lists, lab orders, and appointment data belonging to South. The audit log recorded the requests as legitimate access by a South-scoped user. The system did not flinch.

The flaw was almost invisible because it was almost reasonable. The previous endpoint had handled org switching through a separate flow that verified the user belonged to the requested org before issuing a new token. The refactor had not removed that check by accident. It had re-implemented refresh as a small, self-contained module that did not know the check existed. The model had reasoned, plausibly, that a valid signed token was sufficient evidence of the bearer’s identity, and that X-Active-Org was a benign user-preference signal. It had written the code that pattern implied, and the tests that pattern implied. Both were internally consistent. Together, they were wrong.

The Realisation

Marcus pulled up the test file. He read it slowly this time.

Every test passed because every test asked the question the implementation was prepared to answer. Does refresh produce a validly-signed token? Yes. Does the new token contain the requested org? Yes. Is the caller authenticated? Yes. None of them asked whether the user was permitted to switch into that org, because nothing in the implementation suggested the path needed guarding. The assumption — that the bearer of a signed token could legitimately request a token for any tenant — sat underneath both the code and the tests, invisible to both because both had been written from the same starting point.

He scrolled back to the test he had nodded at three weeks earlier. Tokens with an org claim for an org the user did not belong to. He read it more carefully. It tested the case where a maliciously-modified token was presented to the validator and confirmed that signature verification rejected it. It did not test the case where the system itself issued a perfectly valid token for an org the user did not belong to. The names had matched. The tests had not.

The model had not been malicious. It had been confidently incomplete. It had produced an implementation that made sense if you accepted its premise, and a test suite that proved the implementation matched the premise. The premise itself had never surfaced. It had only been assumed, in two places at once, by the same source.

POV: The CISO

By mid-morning, the CISO had read the report twice and was on a call with the Head of Engineering. Her questions were not technical. They were structural.

The Head of Engineering said the things that were true. AI assistance was widely used. It was not banned. It was not, exactly, instrumented. They could ask developers to recall. They could mine commit messages for hints. They had nothing that would survive a deposition.

The CISO did not raise her voice. She rarely did. She made a note in her own handwriting and underlined it twice.

POV: Vamika

Vamika found out from a Slack message asking her to join a call. By the time it started, she had reopened the PR and was reading it again, slowly, with the cold dread of someone who had done everything right.

She had prompted carefully. She had read the output. She had run the tests. She had asked Marcus, who reviewed all auth changes, who had reviewed hers and approved it. She had trusted the seniority in the room, which was what juniors were taught to do.

Marcus had trusted the polish in the diff, which was what senior reviewers had always done.

Neither of them had been negligent. They had both been operating inside a review process designed for a different kind of risk than the one that had just shipped.

What the Review Couldn’t See

Code review evolved to catch human mistakes. Typos. Missed edge cases. Sloppy naming. A reviewer’s instincts, sharpened over years, look for the small inconsistencies that betray a tired engineer at the end of a long day.

AI-generated code rarely fails on those axes. Its variables are consistent. Its comments are confident. Its tests are comprehensive in shape if not in coverage. It fails on coherence — on the gap between what the model assumed and what the system actually requires. The signals senior engineers learned to trust as proxies for care — idiomatic style, broad test coverage, thoughtful comments — are precisely the signals the model is best at producing.

The risk surface has moved. The review surface has not. A reviewer reading a polished, well-tested PR is reading a mirror of the assumptions that produced it. If those assumptions were complete, the review will be reassuring and correct. If they were incomplete, it will be reassuring and wrong — and the wrongness will not surface until something comes at the system from a direction the assumptions did not anticipate.

How Teams Prevent This

Some organisations are starting to close the gap, not by retreating from AI assistance but by making it visible. They treat provenance as a first-class signal: which lines came from a model, which from a human, which tests share authorship with the code they cover. They scan continuously, at snippet level, for the patterns human review is least equipped to catch — implementations and tests that rest on the same unstated assumption, refactors that quietly drop a check, helpers that confidently trust a claim that was never verified.

They do not slow AI down. They make it observable. And when the governance question arrives — what did we know, and when did we know it? — they have an answer that does not depend on developer memory.