# Silent Failures: What to Log When a Bug Leaves No Error

If your logs can tell you what broke but can't tell you **why something didn't happen for user 4821**, you are logging events, not decisions. That gap is where the slowest incidents live.

Here's a quick self-check. Take any background job in your system and answer this from its last run:

```
selected == done + skipped + failed
```

If you can't fill in those numbers from logs alone, some of your work is leaving the system without a trace. This post is about making that equation answerable, and the per-user version of it too.

## Key Takeaways

- "Nothing in the logs" usually means the code made a **quiet decision**: an early return, a filter, a flag, a dedupe check, a swallowed exception, or a silent default.
- Logging every early return doesn't scale: it floods, gets demoted to DEBUG, and still can't record work that was never reached.
- The fix is **one decision record per attempt**, keyed by the business ID, with an outcome (`done`, `skipped`, `failed`), a reason code, and the inputs that decided it, emitted in `finally`.
- For batch jobs, add a **funnel record per run** whose numbers must add up.
- Research on real failures found most had no relevant log message, and in many the code had already detected the problem and stayed quiet.
- Use the **Why-Not Test** to find silent branches before an incident does.

## Six places code says no without saying anything

| Shape | Example | What the logs show |
|---|---|---|
| Guard clause | `if not user.opted_in: return` | nothing |
| Filter | `WHERE plan = 'pro' AND ...` | nothing (the database decided) |
| Flag / default branch | flag off, `default:`, empty `else` | nothing |
| Dedupe / rate limit | "already sent today", key seen | nothing |
| Caught exception | `except ProviderError: continue` | a DEBUG line, if anything |
| Fallback value | `tz = user.timezone or "UTC"` | "sent" (with the wrong behavior) |

Each one is correct code most of the time. That's the problem. When one of them is wrong, a correct skip and a bug produce **identical silence**.

![Five checks in a digest job, each with a silent exit; only the success path writes a log line](https://cdn-images-1.medium.com/max/1600/1*HwEtq5YI5pkjwO64cWObrA.png)

## Why "just log every early return" breaks down

It's the standard advice, and it helps a little. It fails in three ways:

1. **Volume.** A nightly job over hundreds of thousands of users produces mostly correct skips. Those lines get expensive, get demoted to DEBUG, and disappear from production.
2. **Fragmentation.** "Unsubscribed" in one module, "provider timeout" in another. Reconstructing one user's story means joining lines that were never designed to be joined.
3. **Unreachable work.** If the user was filtered out by the query, or the job never ran, there's no `return` to log. Zero lines means "skipped somewhere unlogged" *or* "never reached," and you can't tell which.

You can't grep for a line that was never written. So the goal isn't "more lines." It's **making absence impossible**.

## The pattern: one decision record per attempt

Stripe described [canonical log lines](https://stripe.com/blog/canonical-log-lines): one structured line per request per service, emitted at the end with all the important fields, and hardened to maximize the chance it's written even when the request fails. Many teams call the same idea "wide events."

Apply it to decisions. The unit of record is **the attempt to do one thing for one thing**: send this digest to this user, sync this order, process this webhook. Every attempt emits exactly one record. "Skipped" becomes an outcome with a reason instead of an absence.

### A minimal helper (Python, structlog-style logger)

```python
import time

class Decision:
    def __init__(self, event, **key):
        self.f = {"event": event, **key, "outcome": "unknown",
                  "reason": None, "inputs": {}, "fallbacks": []}
        self.t0 = time.monotonic()

    def input(self, name, value):  self.f["inputs"][name] = value
    def fallback(self, what):      self.f["fallbacks"].append(what)
    def skip(self, reason):        self._end("skipped", reason)
    def fail(self, reason, **x):   self._end("failed", reason, **x)
    def done(self):                self._end("done", None)

    def _end(self, outcome, reason, **x):
        self.f.update(outcome=outcome, reason=reason, **x)

    def emit(self):
        self.f["duration_ms"] = round((time.monotonic() - self.t0) * 1000)
        log.info(self.f["event"], **self.f)
```

### Using it in a job

```python
def send_digest(user, trigger):
    rec = Decision("digest.attempt", user_id=user.id, trigger=trigger)
    try:
        if not user.opted_in:
            return rec.skip("unsubscribed")
        flag_on = flags.enabled("new_digest", user)
        rec.input("flag.new_digest", flag_on)
        if not flag_on:
            return rec.skip("flag_off")
        rec.input("last_sent_at", user.last_digest_at)
        if sent_today(user):
            return rec.skip("already_sent_today")
        items = fetch_new_items(user)
        rec.input("new_items", len(items))
        if not items:
            return rec.skip("nothing_new")
        mailer.send(user.email, render(items))
        rec.done()
    except ProviderError as e:
        rec.fail("provider_error", error=type(e).__name__)
        raise
    finally:
        rec.emit()  # exactly one record, whatever happened
```

### The same shape in a TypeScript webhook handler

```typescript
// assumes a TypeScript port of the Decision helper above
async function handleWebhook(evt: WebhookEvent): Promise<void> {
  const rec = new Decision("webhook.process", {
    event_id: evt.id,
    account_id: evt.accountId,
  });
  try {
    if (await alreadyProcessed(evt.id)) return rec.skip("duplicate_event");
    if (!SUPPORTED_TYPES.has(evt.type)) return rec.skip("unsupported_type");
    rec.input("event_type", evt.type);
    await applyEvent(evt);
    rec.done();
  } catch (err) {
    rec.fail("apply_failed", { error: (err as Error).name });
    throw err;
  } finally {
    rec.emit();
  }
}
```

Two design choices matter more than they look:

- **`outcome` starts as `"unknown"`.** If a record ever arrives with that value, the code exited by a path you didn't know existed. Your code just filed its own bug report.
- **The exception is re-raised, not swallowed.** Retry, fail loudly, or skip with a reason code: pick one on purpose.

![Anatomy of a decision record: JSON example linked to six rules](https://cdn-images-1.medium.com/max/1600/1*x9AKnKFCFpaV-uRqPIP8HA.png)

### Rules that make the record searchable

- **Key by the business noun** (`user_id`, `order_id`, `invoice_id`), not only the request ID. Support asks about customers; background jobs don't have requests.
- **Reason codes, not sentences.** `already_sent_today` can be grouped, graphed and alerted on. "User is not eligible at this time" can't.
- **Record the deciding inputs.** "flag_off" says what. The flag value and segment say why it applied to this user.
- **List fallbacks.** Every default you substituted is a decision.
- **Identifiers, not personal data.** You'll search these records often; keep them safe to search.

## Batch jobs: the funnel record and the accounting rule

Add one record per run:

```text
digest.run  run_id=2026-11-13  considered=48210  selected=47905
            done=41022  skipped.unsubscribed=5530
            skipped.already_sent_today=1210  skipped.nothing_new=143
            failed.provider_error=0  unknown=0
```

*(Numbers are illustrative.)*

- `selected` must equal `done + skipped + failed`. A mismatch means work left without a record. Alert on it.
- `considered - selected` is the query's decision. A sudden jump means a `WHERE` clause changed its mind. If you've been bitten by [pages that skip rows](https://simplyexplained.hashnode.dev/offset-vs-cursor-pagination-skipped-rows), this is the same class of disappearing work.
- For "why wasn't *she* selected?", write a helper that evaluates each predicate of the selection query separately for one ID.

## What the research says

- **Yuan et al., "Be Conservative," OSDI 2012.** 250 real failures in Apache httpd, Squid, PostgreSQL, Subversion and Coreutils. **57%** had no failure-related log message. Where diagnosis time was measured, failures with logs were fixed about **2.2× faster** on average (report to correct patch). Among the failures without logs, **46** had code that had already checked for the problem and didn't log it.
- **Yuan et al., "Simple Testing Can Prevent Most Critical Failures," OSDI 2014.** 198 failures in Cassandra, HBase, HDFS, MapReduce and Redis. **92%** of the 48 catastrophic failures came from incorrect handling of non-fatal errors the software explicitly signaled, including handlers that were empty or only printed a log line.

Caveat: these are systems projects in C, C++ and Java, studied years ago. The mechanism (detect, decide, stay quiet) is stack-independent.

## Mistakes that bring the silence back

- Logging the no-path at **DEBUG**, which is off in production.
- **Log-and-continue** in catch blocks. Logging isn't handling.
- **Sampling** away skips and failures. Sample `done` if you must; keep run totals and failures complete.
- Recording the verdict but not the **time-sensitive input** (`last_sent_at`), so a clock bug looks like a correct skip.
- Configuration toggled outside a deploy changes decisions with no diff to blame. Record flag values as inputs. ([Related: production breaking with no deploy](https://simplyexplained.hashnode.dev/nobody-deployed-anything-production-broke-anyway).)

## The Why-Not Test

![The Why-Not Test: four questions plus scoring](https://cdn-images-1.medium.com/max/1600/1*LEPzk1SGY6vOfcoK2WytjA.png)

Pick a thing your system should have done yesterday (a reset email, a renewal charge, a sync). Pick a real user at random. From logs alone, in about five minutes, answer:

1. Did we consider them?
2. What did we decide?
3. Why?
4. Based on which inputs?

Any "read the code" or "wait for it to happen again" is a silent branch. Run it on three features; fix the worst one first.

The code-review version is a single question: **"What does this record when it says no?"**

## FAQ

### What should I log in production?

Log one structured record per unit of work (a request, a job item, a webhook) with the business ID, the outcome, a reason code, the inputs that drove the decision, and duration. Add event-level lines only where they add something the record can't hold.

### How do I debug a bug when there's nothing in the logs?

Treat the silence as evidence that the code took an unlogged branch. List the decisions on the path (guards, filters, flags, dedupe, caught exceptions, defaults), check each against the user's data, then add a decision record so the next occurrence is a search, not a code read.

### Should I log every early return?

Not as separate lines. They flood, get demoted to DEBUG, and still miss work that was never reached. Record the early return as the outcome and reason of a single per-attempt record instead.

### What is a canonical log line or wide event?

It's one structured log record per request (or unit of work) per service, emitted at the end with all the important fields attached. Stripe popularized the term "canonical log lines"; "wide events" is the more common name today.

### Won't one record per attempt make logging too expensive?

Usually it replaces several scattered lines with one, which often costs less. If volume is still too high, sample successful records, keep failures at full rate, and keep complete per-run counts so totals always add up.

### Is logging an exception the same as handling it?

No. A catch block that logs and continues has decided to proceed, often in a broken state. Research on catastrophic failures in distributed systems found handlers that only logged among the trivial mistakes behind them. Retry, fail, or skip with an explicit reason.

## Related reading

- [Offset vs Cursor Pagination: Why Your Pages Skip and Repeat Rows](https://simplyexplained.hashnode.dev/offset-vs-cursor-pagination-skipped-rows)
- [Nobody Deployed Anything. Production Broke Anyway.](https://simplyexplained.hashnode.dev/nobody-deployed-anything-production-broke-anyway)
- [Why Your API Charges People Twice (And the One Idea That Fixes It)](https://simplyexplained.hashnode.dev/why-your-api-charges-twice)
- [How to Read a Stack Trace: Finding the Root Cause](https://simplyexplained.hashnode.dev/how-to-read-a-stack-trace-root-cause)
- [Start Here: A Plain-English Map to Understanding AI](https://simplyexplained.hashnode.dev/start-here-understand-ai)

## The bottom line

Errors are loud by design. Decisions are quiet by default. One record per attempt, with "no" as an outcome instead of an absence, changes that default.

> Nothing in the logs should be impossible, not normal.

*Adam Jaber is a software engineer who writes Simply Explained: complex topics, made simple. No jargon, no hype. Get one of these in your inbox each week: [subscribe on Substack](https://simplyexplainedbyaj.substack.com/subscribe).*
