Summarizing Documents and Long Threads

Summarization is the AI use case with the best effort-to-value ratio at work: paste in something long, get back the shape of it in seconds. It is also where a subtle danger lives — a summary that is fluent, confident, and missing the one paragraph that mattered.

What AI summaries are reliably good at

• Getting the gist of something you'd otherwise skip. A 40-page report you were never going to read fully; a 60-message thread you were added to on day three.
• Restructuring by question. "What does this document say about pricing?" is often more useful than a general summary.
• Triage. Deciding whether something deserves your full attention.

Where they quietly fail

A language model summarizes what is typical of the text overall. Things that occupy a small share of words but a large share of importance — a single liability clause in a contract, one dissenting comment in a thread, a date buried in a footnote — can drop out. The summary isn't wrong, exactly. It's lossy in a way that doesn't announce itself.

The two-question habit

After any summary you plan to act on, ask two follow-ups:

1. "What does the document say about [the thing you care about]?" — a targeted question forces the model back into the text for the part that matters to you.
2. "What in this document is surprising, unusual, or easy to miss?" — this surfaces exactly the low-word-count, high-importance items a general summary flattens.

Then — and this is the step people skip — open the original to the section the answer points to and read that part yourself. You've turned reading 40 pages into reading two. That's the real win: AI as an index, not as a replacement for your eyes.

A note on stakes

Match your verification to the consequences. A summary of an industry newsletter needs no checking. A summary of a contract, a policy you'll be held to, or a customer complaint you'll respond to publicly needs the two-question habit every time.

Key takeaway: treat AI summaries as a map, not the territory. Use targeted follow-up questions, and read the original at the spots that matter.