When You’re Wrong: Now What?

You completed your analysis and published it—awesome! Then you or someone else notices a mistake. Now what?

I saw an amazing example of how to deal with this recently while I was scrolling through LinkedIn. I nearly scrolled past a post from a third-degree connection, Eric Gitonga, a data analyst in Nairobi. It started with, “I made an analytical mistake publicly…” and I was immediately hooked. I read his post with interest and appreciated several things about it: first, he had the guts to publish his work publicly; second, he acknowledged the mistake; third, he corrected it, in detail, just as publicly as the original post.

Publishing is intimidating because it invites critique. When you publish something and discover an error, it can be disheartening. Before you decide to stop publishing altogether, consider this: by their very nature, some projects require constant or regular upkeep. Quarterly projects must be done… well… quarterly. Other projects require review because something changed. Some examples are:

• Goals or objectives change
• Assumptions or conditions change
• New data is created or becomes available
• Alternative methodologies emerge, or errors in the methodology are discovered
• Errors in the reasoning or logic of the conclusions are identified
• The audience changes

Notice anything? Each of these reasons maps neatly to the UPDATE workflow and, by extension, a change or error at any stage of UPDATE could trigger a review of a project. However, for this article, let’s return to what is possibly the least desirable reason to review and revise a project: an error in the work.

First and foremost, an error generally isn’t the end of the world. We do our best to avoid them, but they happen. And when they do, our intellectual virtues—Inquiry, Critical Thinking, and Discernment—should lead us to quickly identify, resolve, and correct the mistake.

Inquiry

Inquiry leads us to ask questions, which could be about the quality or content of our work, the feedback we receive, the data we included or disregarded, or whether our conclusions were justified. These questions could touch any stage of UPDATE or the elements of critical thinking. We should always be alert to shifts in assumptions and how they interact with reality. Most projections don’t assume a global pandemic or the outbreak of war when they are in the planning stages, but it is appropriate to revisit work that is impacted by significant changes around the globe. The curious analyst asks: Was this oversight? A limitation of available data? Or did the situation change beyond the original assumptions? Asking critical questions is an effective means of separating error from circumstance, but in either case, those questions can help you be more prepared for the next project.

Critical Thinking

Critical Thinking, as always, evaluates the responses to Inquiry’s questions. Yes, our assumptions turned out to be wrong, but would it have been reasonable to assume anything different? What led to the error, and how can it be avoided in the future? Was a crucial step missed? Was the information of sufficient depth and breadth? Did you accurately identify your own bias and assumptions? Perhaps your financial forecast used a 7% return for the stock market, but during the period of your projections, real returns were much lower at 2%. Is it a failure that you didn’t predict that? No. The average historic return on the DOW is 7.07% since 1886. Further, only 10 years in that time have returned between 1% and 4.4%, compared to 17 that returned between 4.4% and 7.7%. It may not have been the only choice, but it was a reasonable and justifiable one. Critical thinking here isn’t about coming up with a complex methodology for forecasting returns; it’s about evaluating the choice you made.

Remember, critical thinking isn’t thinking hard or deeply; it’s evaluating the quality of thought itself.

Discernment

Discernment decides what to do: do you retract, update, or redo everything entirely? Does your core process need revision, or do you just need to tweak a step or two? Once you’ve critically evaluated your analysis, you have more information with which to decide what to do next. Your decision will depend on many things: the nature of the error itself, how significant it is, the scale of the project, the importance of the project and the error, and the timeliness of the discovery.

• Some errors may be OBE—overcome by events. A mistake in an in-depth analysis of what car to buy may not matter once the purchase is made.
• Minor errors in grammar or design can be corrected with a small note that the edit occurred.
• Factual or material errors should be corrected as soon as they are discovered and noted with an explanation.
• Substantial errors, or errors that call into question the validity of the analysis, should be retracted and republished as time and priorities permit the analysis to be redone correctly.

Final Comments

Remember that hindsight is 20/20. Of course it is easy to look back and see mistakes in retrospect. It is less easy to identify when those mistakes were avoidable and when they were legitimate assumptions at the time. A specific type of analysis—the post-mortem—is a good tool and a good habit to develop regardless of whether an analytical project went well or poorly. It’s not only instructive to understand why our analysis went wrong; it’s equally important to understand when and why it went right. Being right by accident isn’t a success at all; it’s failure in disguise.

But don’t be too hard on yourself. Mistakes happen, some more serious than others, and they are part of the learning and development process. Eric not only corrected his error and showed intellectual honesty, he also created a very public teaching moment for everyone who saw the post, me included. It’s hard to overstate the value of that example. I, for one, have come to realize that I should be publishing more analytical work, case studies, and projects too, so expect to see that happen on the blog sometime soon.

Lastly, the three minds of effective analysis have a role here, as they do in every aspect of analytical work.

• The Curious Mind doesn’t fear being wrong. It seeks to understand mistakes as puzzles to be solved.
• The Prepared Mind evaluates mistakes critically. It builds structure to avoid repeating them.
• The Judicious Mind knows when and how to respond to errors and changing conditions.