Why I put a bug bounty on my personal errors to learn cheap. How actively looking for disconfirming evidence is a sustainable scientific method to arrive at a correct mental map in an empirical world.

My Mantra and the Reactions

Following a rather extensive education in applied logic, my thinking has been strongly influenced and shaped by my reading list over the past years. As an actionable consequence, I’ve made my personal and professional mantra to be

“Prove me wrong and I’ll buy you a pint!”

Whenever I freely quote my seemingly too-good-to-be-true-for-you mantra, I usually get one of the following two reactions: Laughter or disbelief.

The Laughters

Laughter is the most common one. It’s usually followed by a slight look of pity for probably being ripped off on what must have been a seemingly endless supply of pints to hand out over the years.

After all, there’s a chap giving away free pints and all you have to do is point out that he’s wrong on something in order to bag one? That sounds too good to be true. Or ludicrous. What a madman!

That’s usually also the point where this type of audience gets busy on the proof. Of proving me wrong. On Something. After all, there’s a pint in that!

The Disbeliefs

The disbelievers tend to dispute my mantra as some sort of snake oil. Too good to be true. Or mad. Or both.

Anyways, the man must be bonkers to hand out free pints for simply being proven wrong on something. Every time. Surely, he must be loved by bartenders up and down the country. What a madman!

Except that the method behind the mantra isn’t madness.

The Rare Scientist

Hardly ever do I get a nod or a smile from someone who understands the scientific method behind the mantra that’s so easy to grasp — and even easier to remember. Sometimes, it really is all about the marketing.

So, “Prove me wrong and I’ll buy you a pint” isn’t madness or too-good-to-be-true-but-this-time-for-you.

It’s actually science at work in real life. Allow me to explain.

The Mental Map in Our Head

From the day we start our journey on this planet to the very last breath we take, we start building, enhancing, and operating on a mental map of the world.

What we believe to be good, bad, true, false, and so on.

The Role of the Mental Map

The mental map is what it says on the tin. It’s a map we tend to pull out every time we look for directions on something. From which drink to buy (or pass on) to which person to marry (or to rather just “stay in touch” with) and everything else in between.

There hardly is a single decision when we are not guided by our mental map. And when we do, it can feel like it wasn’t us making the decision after all.

To a certain extent, our mental map makes us who we are. How we think. How we feel. How we act. Hence, getting our mental map right really matters.

Adding and Keeping the Right Things

We weren’t given a mental map at birth (at least I wasn’t — let me know if you were, as I would be delighted to talk to you; I’ll even open a tap for you!), so we start from scratch and add to it all life long. Both consciously and subconsciously with the majority of additons happening while we’re on autopilot.

Like a finite storage room that tends to fills itself over time and needs frequent decluttering in order to keep the right things, our mental map also needs a systematic method of what to keep and what to add.

In the case of the storage room, we are dealing with physical objects. In the case of the mental map, we are dealing with information. Note that in both cases, we only want to keep what’s of later use to us. For our mental map, that later use is knowledge.

When adding or maintaining our mental map, we are confronted with information which can lead to knowledge. Note that it doesn’t have to. But it can. More often than enough, it may lead nowhere, really.

So, cutting through the noise and keeping, as well as adding “the right things” to our mental map is the real challenge at hand.

But what are “the right things”?!

Let’s Get Logical!

As a thought experiment, let’s assume the world is one hundred percent mathematical. Things are either right, wrong, or unknown. Nothing in between.

If something is right or wrong then there’s a clear logical argument that is independently verifiable and that everybody can agree on.

Great times for mathematicians! Everything is logical. Hooray!

Identifying the Right Things

In this version of the world, knowledge can finally be obtained by applying standard mathematical methods. Starting with an idea (also called a conjecture), we can now set off to thoroughly investigate it.

If the mathematical tools reveal that the conjecture is true at all times then we have arrived at what we can safely consider “right knowledge”.

Also, if we find a counter example — something that fits all preconditions but does not lead to the desired outcome — we can safely discard the conjecture as “wrong knowledge”.

Note that even though we’ve classified the original conjecture as “wrong knowledge”, the counter example still represents knowledge on it owns as the knowledge “the conjecture is wrong” itself is true. Math can sometimes be tricky. And loads of fun too!

In case our mathematical tools reveal no definitive answer to the correctness of the conjecture and we are also unable to find a counter example, we classify the conjecture as “unknown”. We simply have no idea. Someone help, please?!

An Accurate Mental Map

With the above outcomes, it’s fairly straightforward to build an accurate mental map of the world that contains “the right things”.

We add all conjectures that we were able to conclude as true. We also add all the facts about the conjectures that are not true as we could find counter examples for them.

But most importantly, in the absence of clarity, we resist the temptation to guess and strictly keep all the “unknown” conjectures off our mental map.

Let’s Get Real!

Sounds good?! You’re in?! So am I! Unfortunately, the world isn’t purely mathematical, let along logical. Even at the best of times. And that’s probably a good thing as well.

While we gain life and all it’s beauty, we simultaneously lose a fairly significant amount of confirmed “true” conjectures when applying mathematical tools to real world problems.

So, where are we in terms of our mathematical toolkit from above?

The Boundaries of Proving Correctness

Well, let’s start with the conjectures. We seem to be quite good at them. Except that we refer to them by a slightly differently name. We call them opinions. So, how do we investigate an opinion in real life then?

Let’s start by applying our mathematical tools to the opinion in an attempt to prove it to be correct. But, as we are at the crossroads of mathematics and real life, we then soon hit the limits of those tools for a great majority of times. As Albert Einstein famously said

“As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality.” — Albert Einstein

So, more often than enough, we’re stuck on proving the opinion to be correct. What’s left to do? Well, look for counter examples!

But what happens when both approaches fail to produce a definite outcome and the opinion is still labeled as “unknown”?

Despite mathematics telling us to resist the temptation to guess in the absence of clarity and hence ignore the opinion altogether, we proudly take it on and put it on our mental map. You do it. I do it. We all do it.

So, now we have a potentially wrong opinion lurking around our mental map that we took on despite better knowledge and that now guides all of our future decisions. Yikes!

Disconfirming Evidence to the Rescue!

So, how do we get it off again?

Well, we keep looking for that one counter example. We actively embrace disconfirming evidence as it allows us to erase incorrect opinions from our mental map. With absolute confidence. All it takes is one lousy counter example.

Here, be extra vigilant when tempted to pull out the “special case” argument to counter a counter example (pun intended)! In the face of a counter example, the original conjecture is void. Regardless of how negligible or insignificant it is.

Disconfirming evidence doesn’t simply go away because we choose to hand wave it. Or even ignore it. It should much rather deserve our full and undivided attention.

Why I Happily Hand Out Pints

Looking for counter examples is easier said than done. Decades of research in behavioural psychology have revealed that for our minds,

“What you see is all there is!” — Daniel Kahneman

But then we don’t have to be alone in the search for counter examples. We can ask others to help us out. By setting the right incentives, they will happily jump in and point out all the wrong opinions. This is exactly what “Prove me wrong and I’ll buy you a pint!” is all about.

So, armed with the knowledge that a sustainable scientific method to operate in an empirical world is to actively look for disconfirming evidence, I’ve decided to put a “bug bounty” on my personal errors. In the form of pints.

The wording “Prove me wrong and I’ll buy you a pint!” is merely a memorable marketing wrapper around the fact. Even with London prices of £5+ per pint (as of 2020), this for me still counts as cheap learning.

Yes, I’ve handed out quite a few pints over the years but the benefits of erasing erroneous opinions from my mental map still vastly outweight the financial costs for me.

More importantly, the community assisted hunt for disconfirming evidence constantly challenges my mental map and helps me to keep away from a core fallacy of human thinking for a bit longer as

“What the human being is best at doing is interpreting all new information so that their prior conclusions remain intact.” — Warren Buffett

Don’t get me wrong — It still massively hurts to be proven wrong. I’m human and I like to be right most of the time. The only way to be that is to be proven wrong some of the time.

And I certainly don’t want to end up

“Faced with the choice between changing one’s mind and proving there is no need to do so, almost everyone gets busy on the proof” — John Kenneth Galbraith

Implications in Other Places

Confirming evidence is extremely cheap to come by. Especially when we have strong incentives “to make things work”, we can sometimes focus on confirming evidence rather than keeping an open mind that is actively on the lookout for disconfirming evidence as well.

This tendency gets amplified when we surround ourselves with people or join communities that share your beliefs and opinions. We cannot repeatedly rely on them to supply us with disconfirming evidence. After all, they think like us!

It takes real intellectual work to find disconfirming evidence. And then act on it accordingly. Sometimes this information is best found by leaving the familiar supportive communities and going where it hurts.

Some organisations have even made it a core principle of theirs to look for disconfirming evidence and thereby learn as cheap as possible.

An integral part of that process is accepting, embracing, and celebrating failure. One of the most impressive talks I’ve seen on this subject is by Astro Teller titled “The unexpected benefit of celebrating failure”.

Now, this is what I would consider handing out pints for being proven wrong on an epic scale! Well done!

So, How Do You Learn Cheap?!

While “Prove me wrong and I’ll buy you a pint!” is a memorable summary for a sustainable scientific method in an empirical world that allows me to learn cheap and that Worx for Me!™, yours may differ.

Think this is all rubbish, massively overrated, or heading into the absolutely wrong direction in general?! Feel free to reach out to me on LinkedIn and teach me something new!

As always, prove me wrong and I’ll buy you a pint!