The change already shipped. Nobody wrote down what it replaced.

Look for the ledger somebody kept for another reason

The instinct is to try to reconstruct the metric, which fails, because the metric was never recorded. The move that works is to find a record that was maintained for an unrelated purpose and happens to contain the underlying fact. Operations leaves more residue than people expect, and almost none of it was created by anyone thinking about measurement.

• Timestamps on records that are opened and closed — tickets, orders, invoices, pull requests, cases. Created-to-closed is a duration nobody had to decide to measure.
• Message and mail metadata. Not the content: volume, timing, and how many people were in the thread. Handover cost shows up here clearly.
• Payroll and scheduling records, for who was doing the work and how many of them there were.
• Anything append-only: audit logs, event streams, accounting entries, version history. The strongest sources, because nothing overwrote them.
• Reports circulated at the time. A monthly summary attached to an email is a snapshot of a system state that no longer exists anywhere else.

The pitfall is that none of these was designed for what you are about to use it for. A ticket marked closed does not mean the work finished; it means somebody touched the record. Reconstruction is legitimate and it comes with assumptions, so write them beside the number rather than in a footnote. A reconstructed baseline presented as a measured one gets found later, and being found later costs more than the caveat would have.

What is gone

Four categories, and no technique recovers them. It is worth saying so early, because the effort otherwise goes into the impossible part and runs out before the recoverable part is done.

• Effort. How long something took when nobody timed it is not recoverable, and asking people afterwards produces a number shaped mostly by whether they liked the change.
• Quality, unless something was already grading it. Rework handled in a conversation between two people left no trace anywhere.
• Anything overwritten in place. A record that was edited rather than versioned holds today’s value and no history at all, which is the most common single reason a reconstruction stops.
• The counterfactual. Even a flawless before-and-after cannot separate the change from everything else that happened in the same months — a seasonal pattern, a pricing move, a competitor, two experienced people leaving.

The last one is the one that is consistently underestimated, including by people who are careful about the first three. A clean before-and-after feels like evidence, and it is evidence of a difference rather than of a cause. That distinction is exactly what a skeptical reader will press on, so it is better to have already conceded it.

Designs that still work after the fact

Four of them, in descending order of how much they let you claim. Pick the strongest one the situation allows, and then say which one it was.

A reversal is the strongest. Turn the change off for a defined window, measure, turn it back on. It controls for almost everything a before-and-after cannot, because the comparison is close in time. It is also unavailable where the change is irreversible, where it touches customers, or where the team would reasonably object to doing the old work again for two weeks — which rules it out more often than not, though it is worth asking rather than assuming.

A staggered rollout is next, and it is free if the change has not finished reaching everyone. The teams, regions or product lines still waiting are a comparison group for as long as they are waiting. This is the argument for deliberately slowing a rollout that could go faster: the measurement window closes permanently on the day the last group is switched over.

An interrupted time series works where the measure already exists monthly going back two years, which is more common than teams assume once the search above has been done. The effect is the break in an established trend at a known date. It needs enough points before the change to establish the trend — eight is thin, eighteen is comfortable — and it is only as good as your ability to say that nothing else happened at the same moment.

A matched comparison is the weakest and often the only one available: a team, region or product line the change did not touch, measured over the same window. It controls for seasonality and for anything happening company-wide, and it controls for nothing that is specific to the group you changed. State that limitation rather than letting a reader discover it.

These support different claims and the difference is not pedantic. Throughput rose eighteen percent in the twelve weeks after the change, against a trend flat for six quarters, in a period with no other operational change we are aware of — that sentence survives being questioned, and it tells a reader how much to believe. “The change raised throughput eighteen percent” does not, and asserting it costs credibility on every other number in the same document.

Twenty minutes, next time

The habit that prevents all of this is small enough that its absence is never a resourcing decision. Before the next change goes in, write one page.

• Three numbers and no more. One for speed, one for quality, one for volume. A longer list is how this becomes a project and then does not happen.
• A written definition of each, and the query that produces it. Not a description of the query — the query.
• The value of each for the eight periods before the change, produced by running that query rather than by recollection.
• A stated window and a stated expectation, written before the change ships rather than after the result is known.
• A note of everything else changing in the same window: a pricing move, a reorganization, a seasonal peak, a system migration.

The last item is the one that gets skipped and the one that makes the result defensible later. It takes five minutes at the time and cannot be assembled honestly afterwards, because by then everyone knows which answer would be convenient.

Then one structural thing, which matters more than the page. Keep the event-level records even when nobody is asking for them — the individual rows with their timestamps, not the monthly summary. Retention costs almost nothing at this size and the trade is heavily in your favor: a metric can always be recomputed from rows, including a metric nobody had thought of yet, and rows can never be recovered from a metric. Almost every measurement problem described above is a team that kept the summary and discarded the rows.