TL;DR
- Most painful authentication incidents aren't caused by broken code. They're caused by a real-world state (a pending invite, an SSO-only account, a CRM record) that nobody wrote a rule for.
- Fixing the exception that showed up in the stack trace isn't the same as fixing the lifecycle that produced it. If the same error class keeps returning, the code isn't the problem.
- Authentication, membership, and authorization are three separate questions. Collapsing them into one check is how a "correct" login still exposes the wrong resource.
- Bot rules and edge security sit inside the login flow, not around it. Left undocumented, they'll block your own API calls and health checks along with real threats.
- AI can implement a rule that's already been written down. It can't invent the product decision behind an unwritten one, and a plausible-looking guess is riskier than a crash because it gets merged.
- The lasting fix is the written rule, not just the patch. Document what each system assumed, what was wrong, and what should happen next time.
It is a calm morning. No incidents on the board. Then a Slack message: users can't log in.
You check the login page. You check the logs. Everything is green.
Then you look at the user's record. They were invited to join an organization days ago, so as far as they know, they have an account. But they never clicked the invite link to set a password. The system knows this, but the login form only returns "invalid credentials," so it doesn't reveal whether the account exists. The user tries a password reset. That fails too, because there's no password to reset.
Nothing is broken. The invite flow works. Login validation works. The vague security message is doing exactly its job. The only missing piece is a product decision nobody made: what should happen when someone with a pending invite lands on the login page instead of their email?
That pattern sits behind almost every painful authentication incident. It is rarely a bug. It is a state nobody documented.
Fixing the exception isn't fixing the lifecycle
Ordinary code defects were still the largest category we reviewed: an uninitialized provider, a null reference, a bad click handler. Tests, review, and AI are the right tools for that kind of problem.
The mistake was treating a resolved bug as a finished login flow. What initially appeared to be a series of isolated Google provider-initialization errors eventually revealed a recurring timing assumption in the login flow. Each occurrence looked like a one-off: a different user, a different device. The durable fix required us to address the lifecycle of the integration, not just each visible exception. What was never written down was the rule underneath all of it: nothing stopped the interface from starting authentication before the provider had finished loading.
When the same error class returns, don't open another ticket for the stack trace. Write down the answer instead:
- When is the handler allowed to run?
- What does "ready" mean for Google, Apple, or SSO?
- What should the user see if they click too early?
Until that's on paper, review won't catch it and AI will just wrap the crash.
Logging in isn't the same as being known
Once a product has invitations, company membership, and a CRM that also claims to know who a user is, someone can be valid in one system and unknown in another. Each piece is doing what it was told. What's missing is a rule for what happens when they disagree.
Before anyone touches authentication, the team should be able to answer:
- Identity is proved. Do we create a user, attach a pending invite, or reject the login?
- We have an internal user. Which email rules apply, and do they match everywhere we store users?
- There's an invitation. Which states exist, and what should the login page do in each one?
- We know the company. Who owns account status: the app, the identity provider, or the CRM?
- They want a resource. Does this identity have membership and permission, or only one of those?
If two systems can disagree about the same person and there's no written rule for it, that's not an authentication design. It's a login screen and a pile of future incidents.
Three checks, not one
The easy shortcut is to write one check that stands in for three: the user is logged in, so let them in. That's fast to build, and it's how a login can be "correct" while a resource is still exposed to the wrong person.
Keep them separate:
- Authentication. Have they proved who they are, with a method we accept?
- Membership. Does that person belong to this company, in a state that counts?
- Authorization. May this member do this specific action on this specific resource?
We mixed these up in both directions. An SSO-only account tried to log in with a password, got the same "invalid credentials" message we use for accounts that don't exist, and hit a dead end. Separately, a candidate-sharing feature checked that an email matched a known user or pending invite, and stopped there. Knowing who opened the link isn't the same as confirming they belong to the company that owns what's behind it.
Collapse the three checks into one middleware and a test showing a 200 for a logged-in user will call it done, even when it shouldn't.
Security rules live inside the flow, not around it
Bot rules and edge challenges sit on the same path as real users. Some of what looked like attacks in our review were our own systems, blocked by our own rule: an API call returned an HTML challenge page instead of JSON, and an uptime monitor got flagged as a bot because its health check started redirecting to the login page.
Name three traffic classes before shipping a challenge rule:
- Real people, who should get friction they can complete.
- Real automated clients (APIs, health checks, internal jobs), who need a documented path that skips the challenge and still returns the format they expect.
- Suspicious automation, which should be challenged, throttled, or blocked, with a reason on-call can see in the logs.
If a rule can't tell those apart, the next "outage" won't be an outage. It'll be your own traffic hitting a rule built for bots.
AI speeds up a written rule. It can't invent one
AI is genuinely useful on the first kind of problem: the null path, the click before the provider is ready, the missing test. Ask it to fix an unwritten product rule, though, and it will guess, and a guess that compiles and passes review is worse than a crash, because it gets merged.
A CRM blocking an invite because a person exists as both a Lead and a Contact isn't a coding question. It's a decision about which record owns the invite, and nobody had written that down. A model asked to "improve" a deliberately vague error message might expose exactly the information that message was built to hide. Where the rule exists, AI accelerates the fix. Where it doesn't, AI multiplies the ambiguity.
What actually prevents the next one
After an incident, the instinct is to ship the patch and close the ticket. But code only shows what happens, never what was decided, so the next engineer, or the next AI agent, inherits the fix with none of the reasoning behind it.
We now write both: the code change and the report. What the user saw, what each system assumed, which assumption was wrong, and the rule the patch is meant to enforce.
When two systems disagree about who a person is, code can't settle that. Only the people who own the product can, and only if they write it down. The patch stops today's incident. The written rule stops the next one.
FAQ
What's the difference between authentication, membership, and authorization? Authentication confirms who someone is, through a method the system accepts, like a password or SSO. Membership confirms that identity belongs to the company or organization that owns the resource being requested. Authorization confirms that this specific member is allowed to take this specific action on this specific resource. Treating any one of these as proof of the other two is where most access-control incidents start.
Why do login forms return a vague message like "invalid credentials" instead of something specific? It's a deliberate security decision, meant to avoid confirming whether an email address has an account at all. That vagueness protects against account enumeration, but it comes at a cost: on-call engineers investigating a real failure get the same generic message as someone hitting expected behavior, so incidents that aren't really incidents keep getting re-investigated from scratch.
Where does AI actually help with authentication bugs? AI is strong on defects with a clear, already-understood rule behind them: null references, uninitialized providers, missing regression tests. It's not a substitute for a product decision that hasn't been made yet. Asking a model to "fix" an unwritten rule produces a guess that looks reasonable and often gets merged, which is riskier than a crash because nobody catches it in review.
What should we document before changing anything in the authentication flow? At minimum: which routes are public, private, or machine-only; which account and invite states exist and what the login page does in each one; how each identity provider (Google, Apple, SSO, password) behaves; how bot rules distinguish people from legitimate automated clients; and what a user is allowed to see in an error message versus what on-call needs in the logs.
Is this an argument against writing tests or reviewing code? No. Ordinary code defects were still the largest category of incidents in our review, and tests, review, and AI remain the right tools for catching them. The point is narrower: once those tools have done their job, the harder incidents come from states and assumptions that were never written down anywhere for a test to check against.
