Fermi estimation as a scoping discipline

My undergrad was mathematical physics, and the single most transferable habit isn’t calculus or linear algebra — it’s the Fermi question discipline. Before committing to a multi-week data or engineering project, you should be able to answer within an order of magnitude what success looks like numerically. Otherwise you are gambling with calendar time, not doing work.

What a Fermi estimate is actually for

A Fermi estimate is not a forecast. It’s a sanity check on ambition. You’re asking: “if every assumption goes my way, does this project even touch the number I’m trying to move?” If the generous answer is that it moves the metric by 2%, and the metric has 8% natural variance month-to-month, the project is structurally incapable of producing a signal — no matter how good your model is.

Physicists do the same move before asking for telescope time: compute the signal strength, compare to the noise floor, and decide whether the observation can resolve the thing at all. A data project is the same calculation with uglier units.

The shape of the back-of-envelope

Three multiplicative pieces, each within a factor of 3:

1. Who is it going to touch? Affected unit count — users, pages, orders, whatever.
2. How much can each one move? A realistic per-unit lift based on priors, not the best case you can imagine.
3. What does that translate to in the metric you actually report on? LTV, revenue, minutes saved, incidents avoided.

Multiply, compare to the threshold that would make someone proud of this in a review. If the answer is “maybe 2× the threshold on a good day,” kill the project now. You need a 10× cushion to survive the optimism collapse that happens when real data shows up.

Why ML loses this test so often

A quiet reason ML projects fail: the Fermi ballpark said the win was small, and nobody forced the question before the modeling started. Once you’ve invested three weeks, sunk cost carries you into shipping something, and now the team is congratulating a small effect into existence.

The clean rule: if the order-of-magnitude answer is negative, or within a factor of 2 of the threshold, the project is not an ML project. It might be a pricing change, a UI fix, or a conversation with legal — but it’s not a model.

The habit

Do the estimate before touching the tools. Write the three multiplicative pieces down. Show the result to the person who’ll judge the work. If they flinch at the numerator — that’s the conversation you’re about to waste three weeks discovering anyway. Have it now.

The notebook is downstream of this. It always was.