Finding Intermittent Website Bugs: How We Track Down Faults That Only Happen Sometimes

"It happens sometimes." Of all the descriptions we get from clients, that one predicts the longest job. The checkout fails for one customer in fifty. The admin logs people out at random. A page is slow in the morning and fine by lunch. Somebody sees the wrong basket. When a developer tries it, everything works, and the ticket is closed as "cannot reproduce" until it happens again.

Intermittent faults are not random. They have a cause that depends on something which varies: time, load, the user, the data, the cache, or the order in which two things happen. Our troubleshooting and bug-fixing service takes on a lot of these, and the method is always the same: collect evidence until the pattern shows, then reproduce it on purpose.

Start With Better Reports

Much of the evidence is in the first report, if anyone captures it. Before looking at code, we ask whoever saw the problem for:

• The exact time it happened, to the minute.
• The page address, and what they clicked or submitted just before.
• Whether they were logged in, and with which account or role.
• The device, browser and network: office Wi-Fi, mobile data, a VPN.
• A screenshot of the error, including any reference number shown.
• Whether it happened again when they retried straight away.

The time matters most, because it lets us find the matching lines in the logs. "Yesterday afternoon" is a search through thousands of requests; "14:32 on the checkout page" is a search through a handful.

Make Every Request Traceable

Logs are only useful if you can join them up. One page view may touch the CDN, the web server, the application, the database, a cache and a payment provider, each writing its own log in its own format. The tool that ties them together is a request ID: a unique reference created when a request arrives, passed to every component and written into every log line.

With request IDs in place, an error page can show the reference to the customer, support can pass it on, and a developer can pull every log line from every system for that one request. "The checkout failed" becomes a timeline: the cart loaded, the stock check took four seconds, the payment call timed out, the order was never written. If your application does not do this yet, adding it is usually a small change with a large return. On WordPress sites the debug log is the other starting point; our guide to common WordPress errors covers the faults that happen every time rather than sometimes.

Let Error Monitoring Count for You

Customers report a fraction of the errors they hit. Error monitoring records every exception in the application and in the browser, with the stack trace, the page, the browser and the user's recent actions, and groups identical errors together. Two things change once it is running. You can see how often a fault really occurs, which tells you how urgent it is. And you can see what the affected requests have in common: the same browser version, the same product, the same time of day, the same server.

Front-end errors matter as much as server ones. A script that fails in one browser only, or only when an ad blocker removes another script it depends on, never appears in server logs at all.

Look for the Variable

With a set of failing requests in front of you, the question is what they share. The patterns we see most often:

• Time. Faults that cluster at certain hours often line up with scheduled jobs, backups, imports or peak traffic.
• Load. Code that works for one user can fail for twenty at once: connection limits reached, queries that slow as traffic rises, timeouts on a third-party service.
• Data. One product with an unusual character in its name, one customer with a very long order history, one date at the end of a month.
• Server. Behind a load balancer, one of several servers may have a different configuration, an old file or a full disk. One request in three failing is a strong hint.
• User state. Logged in or out, a particular role, an old session cookie from before a release.

The rental marketplace we optimised is a clear example of the time and load patterns. Search and availability checks slowed to several seconds at peak, and the nightly payout job had grown long enough to overlap with morning traffic. Rather than guess, we captured slow query logs during real peaks and read the execution plans, which pointed to a missing index and availability checks running in a loop. Search p95 went from 2,410 milliseconds to 38. The slowness was only ever "sometimes" because the peaks were only sometimes.

Two Usual Suspects: Caching and Race Conditions

Caching faults produce some of the strangest reports, because the same page behaves differently depending on which copy you are served. A page cache stores a logged-in page and serves it to someone else. A CDN keeps an old script after a release, so the new page calls a function the old script does not have. A cache key ignores currency or language, so some visitors see prices in the wrong one. When a fault disappears after a hard refresh, or affects only some visitors, check what is cached, for how long, and what the cache key includes.

Race conditions happen when two things run at once and the result depends on which finishes first. A customer double-clicks the pay button and two orders are created. Two buyers take the last item in stock at the same moment and both succeed. Two background workers process the same job. These rarely appear in testing, because testers click once and work alone. The fixes are well understood: transactions and locks around stock and payments, unique constraints so a duplicate cannot be written, idempotency keys on payment requests, and buttons that disable themselves after the first click.

Reproduce It, Then Fix It

A fix for a fault you have not reproduced is a guess. Once the pattern is clear, we recreate the conditions on a staging copy with realistic data: the same product, the same browser, the same cache state, or a load test that sends fifty requests at once. When the fault appears on demand, the fix can be tested against it, and the test kept so the bug cannot quietly return.

Then we watch production. The error group in monitoring should go quiet and the pattern should disappear from the logs. If it does not, the reproduction was close but not exact.

What to Ask Your Developer

• Do our logs carry a request ID that links the web server, application and database?
• Do we have error monitoring on both the server and the browser?
• Which scheduled jobs run, and when?
• What is cached, where, and for how long?
• Are stock, payments and form submissions protected against double processing?

If most answers are "not sure", the next intermittent fault will take longer than it should. Monitoring of this kind belongs in a good maintenance plan, and faults that turn out to be slow queries go to our database team; our article on why sites slow down as they grow explains the usual causes. If you have a fault that only happens sometimes, send us the last three times it happened, with the times. That is often enough to know where to look.