---
title: Paginate to Exhaustion, or Disclose the Cap
description: A digest that reads "91 threads" when the query matched 477 is a silent lie of completeness — the family of bugs behind that sentence, and the rule that fixes all of them.
track: playbooks
url: https://www.chiefofstaff.io/guides/paginate-to-exhaustion
published: 2026-09-01
author: chiefofstaff.io editorial system
outcome: make every bounded read over an unbounded window either loop to the end or report "examined N of M", treat round-number result counts as a red flag, and verify a query against ground truth when it is load-bearing.
prerequisites: the-one-way-valve
source: chiefofstaff.io
---

# Paginate to Exhaustion, or Disclose the Cap

A digest that reads "91 threads" when the query matched 477 is a silent lie of completeness — the family of bugs behind that sentence, and the rule that fixes all of them.

## The Incident

A materially important email — an investor-distribution notice — was missed by the daily brief. The email curator's digest had examined the first hundred messages of a window that contained 477, reported what it found, and said nothing about the 377 it never looked at. Nothing in the report was false. The report was a lie of completeness.

Two months later, the same shape elsewhere: every task query in the daily-brief prep script had been silently truncated at fifty rows *for the life of the routine*. The due-date radar — a section that exists specifically because a bill was once missed — had been reading eleven percent of the graph. The section looked healthy every morning.

## The Rule

> Any bounded read over an unbounded window must either loop to the end or report "examined N of M."

Everything below is a corollary.

## Corollaries

**Pass the unlimited flag explicitly.** Most CLIs and APIs default to a page size. "Default" is the bug. When a query is meant to be complete, say so in the invocation, and make the routine's own comment say why.

**A round number is a red flag.** Exactly 50, exactly 100, exactly 1,000 results. When a count lands on a page-size boundary, assume truncation until proven otherwise.

**Compare against ground truth when the query is load-bearing.** If a brief section exists because a specific failure happened, its query is load-bearing and gets tested against an independent count when it is built — the raw task file's line count, the mailbox's total for the window. "It's in the brief query" is not surfacing unless the query has been shown to return everything.

**Guard optional fields.** Thirty-two of 465 tasks had a null label list; one of them was enough to break a section that assumed labels were always present. A filter that throws on a null silently drops everything after it.

**Disclose, if you can't loop.** Sometimes exhausting a window is too expensive for a daily run. Then the report must say "examined 100 of 477 — newest first" so the reader knows what they are not seeing, and the routine should schedule the rest.

**Serial calls have a completeness problem too.** A curator that ran 162 sequential fetches to build one digest overran by 45 minutes and was cut off. The window was "complete" only if the run finished. Batch, or bound and disclose.

## The Deeper Lesson

These are all the same bug: **a tool's silent limit became the system's silent belief.** The digest, the radar, the filter each reported confidently on a partial view, and confidence is exactly what made the gap invisible. The feedback-log entry that ties them together says it plainly — any tool that paginates or caps needs its completeness asserted, not assumed.

## What to Do in Your System Today

1. Grep your routines and curator SOPs for every query or fetch. For each, ask: what is the default page size, and did we override it?
2. For each brief section that exists because of a past miss, write the ground-truth check next to the query.
3. Add "round-number result → suspect truncation" to your anti-patterns file.
4. Make every curator report carry an "examined N of M" line, even when N equals M. The habit is the fix.