Tips for market-size estimation modeling

In 2013 and in particular in 2018 and 2019 I spent many, many hours creating various models (in Excel, sometimes also with Simulacion 4.0) in order to estimate total market size and competitor size in terms of quantities shipped in various product categories, such as plug-in humidifiers, temperature-control equipment, and—most notably (and most extensively)—ground-penetrating radar for concrete scanning and geotechnical investigations.

Here’s a few lessons and guidelines distilled from that experience, in no particular order:

• “One is none”. Just one model, is no model. Each model is a lens through which you look at a ground truth that you will never know for sure.
• Try bottom-up, top-down, high-level, low-level models, and test them against each other. Use some of the models to cross-check the rest of them.
• Be consistent with your assumptions across all models.
• Comment your models as you go along, as if you were commenting source code. Even better: write it out like a Jupyter notebook, explaining the rationale behind each model step. Make it possible for others to follow your reasoning and poke holes at it.
• Models = software. Version your software and keep a Changelog.
• Model how the product portfolio of a player makes up its revenues. Remember: not all revenues come from product sales.
• Model what the product portfolio of a player reveals about its target segments; cross-check that the outcome isn’t out of whack.
• Use publicly-available data and scale metrics with ratios; such ratios are often consistent within an industry with low business-model diversity.
• Don’t trust off-the-shelf market reports that you can buy from market-report companies. Use such reports as directional and relative input, if at all. If provided with a sample, do not trust anything in that document.
• If you have many inputs and assumptions, run a Monte Carlo. At the very least, examine scenarios. Nominal estimates are garbage.
• Use your gut feeling and ask if the results make sense. Use others’ gut feelings and do the same. Attach a probability to each estimate based on the feedback, and calculate an expected result.
• If possible, generate weighted averages of the predictions of different models (create ensembles). Jitter the weights; see if the ranking of outcomes changes dramatically.
• Check if your results “break reality elsewhere”; ask yourself: “if these results are true, what else must also be true?”

And always keep in mind:

“All models are wrong, but some are useful.” — George Box