"Just retry it" is the most expensive fix available for a flaky test, because it doesn't tell you why the test failed — it just hides the failure until it eventually costs you a real production incident. Before adding a retry, it's worth spending five minutes checking which of four mechanisms is actually responsible. In practice, nearly every flaky test falls into one of them.
The test asserts on something that hasn't finished yet: an async callback, a debounced UI update, a background job, an animation. Signature: fails more often on a loaded CI runner than on a fast laptop, and adding a fixed sleep() "fixes" it (a strong tell — you papered over a race, you didn't remove it).
Check: does the assertion wait on a specific event/promise, or on wall-clock time? Replace any fixed sleep with a wait-for-condition that polls the real signal (a promise resolving, a DOM mutation, a queue draining).
Two tests mutate the same database row, global variable, temp file, or in-memory cache, and the outcome depends on which one runs first — or on both running at once under a parallel test runner. Signature: passes reliably alone (--filter to just that test), fails intermittently in the full suite.
Check: run the suspect test in isolation 20 times, then in the full suite 20 times. A gap between those two failure rates points straight at shared state, not the test's own logic.
A close cousin of shared state, but the dependency is on setup a previous test left behind (a fixture, a migration, a seeded record) rather than concurrent mutation. Signature: fails when run alone or in a random order, passes when run in the suite's usual fixed order.
Check: run the suite with randomized test order (most runners have a --random or seed flag). If failures appear only under randomization, the test has an undeclared dependency on execution order — fix by making setup explicit in that test, not by pinning order.
The test talks to a real network call, a third-party API, the system clock, or anything else outside the process under test. Signature: fails in bursts correlated with an outage, rate limit, or DST/leap-second boundary, not with load or order.
Check: does the test make any call that isn't mocked — DNS, HTTP, disk with real latency, Date.now() without a fixed clock? If so, the flake is often not in your code at all; it's an untested boundary that needs a fake or a contract test instead.
Check timing first because it's the easiest to misdiagnose as something else (a flaky assertion after a sleep looks identical to shared state until you isolate it). Check external dependencies last because it's tempting to blame the network immediately — most "the API was probably down" flakes turn out to be one of the first three once actually measured.
None of this requires an LLM to run — it's four hypotheses and two commands (isolate, randomize) to test them. Where a model does help is on a large flaky-history log: correlating dozens of past failures against commit timing, PR authors, and CI runner load to point at the specific mechanism faster than reading logs by hand — which is what the flaky-test-detector skill in the Claude Code Power Pack does. If you're also shipping schema changes, see the companion guide on safe database migrations; if a dependency bump is in the mix, see the one on dependency upgrade risk; if the flakiness already caused an incident, see the one on blameless postmortems; if you're chasing the underlying bug itself once it's isolated, see the one on root-cause tracing.