What is Evidence? Part 2: Anecdotal Evidence
Every single day we hear people telling us that “I saw it with my own eyes”, “it worked for me”, “it changed my life” or “I swear by it”. Anecdotes are all around us, and we encounter them every day. It’s the way we humans share information. But there's a problem when it’s presented as evidence for a scientific claim. Then it’s called anecdotal evidence or argument from personal experience, which is also a logical fallacy called...you guessed it...the Anecdote Fallacy. This is the weakest form of evidence anyone can present for any claim they make. So, should we believe them? Let’s dig in, shall we?
DISCLAIMER: I’ve referred to human evolution and its effects on the way we think and our attraction to anecdotal evidence. There is research out there, but most of this is just speculation on why anecdotes are so much more attractive to us based on my understanding of human evolution. I’m not an evolutionary biologist or neurologist. If you are, and I’ve made mistakes here, please contact me and let me know. However, this isn’t central to my argument, so it shouldn’t affect my overall conclusions. Enjoy!
Stories presented as evidence
You’re at a party. You see a friend drinking heavily, and drives home at the end of the night. The next day, you give him a call and ask why he didn’t just take a cab. He tells you, “dude, I’ve been driving home drunk for years! I’ve always been fine. In fact, I drive better when I’m drunk.”
That sounds crazy, right? But there are people all over the world saying the same thing. I have to admit, even I used to at one time. I’m not proud of it. But does that make it right? Of course not! Tens of thousands of drunk driving accidents and scientific studies prove that drunk driving is ridiculously dangerous. The evidence is against your friend, and he needs to start calling Ubers to parties.
Let’s see a more innocuous example. Your favourite aunt tells you she has a sure-shot treatment for your migraines that involves a particular ground herb soaked in milk overnight. You just have to wake up at 5 am and drink it. Your headache will be gone. She says it’s worked for her multiple times and swears it’ll work for you. Should you believe her? What’s the harm?
Check your biases
Every day, our experiences are coloured by our prejudices, our belief systems and biases. Think you don’t have biases? Sorry, buddy. But you’re stuck with them. Even I am biased, and, tbh, they’re the hardest damn things to counteract.
That’s probably how our brains are wired. We haven’t evolved to think rationally and logically, but rather to survive. We are better off accepting what our friends and families believe because it leads to a better social structure than if we started ruffling feathers with unpopular beliefs. Believe me, I know that from personal experience….see what I did there?
In that vein, if we have had bad experiences with doctors, we might be more open to alternative medicine. If we had strange experiences that we couldn’t explain – like sleep paralysis or hearing weird things at night – we might believe in the supernatural like ghosts or spirits.
So, as we go through life, we live through multitudes of experiences. Each of them changes the way we think by just a little bit, and this becomes a part of our identity. Adding to this, our brains build up an incredible number of cognitive biases through our lives based on these experiences. That’s why, if someone tells us we’re wrong about something, our natural instincts kick in, and we dig our heels in because otherwise, we become vulnerable. And vulnerability isn’t suitable for survival. We feel weakened; we think the opponent has won.
Coming back to when I thought the keto diet was ultimate, people laughed at me, ridiculed me and tried talking sense into me. But I was convinced that the science was clear and that they were just behind the times. It had become a part of my identity, even though I couldn’t seem to sustain it. Maybe that was just me being weak. But it had helped me lose weight when I was consistent. Surely that meant I was right…right?
You can imagine my surprise and confusion when I saw bigger, better scientific evidence saying there wasn’t anything special about keto. It showed me why and how it worked. I didn’t want to let it go, but I couldn’t ignore the evidence.
I learned later that I had massive confirmation bias and to some extent was feeling the Dunning Kruger Effect, where I lacked knowledge but was confident about my beliefs. That effect made me dig my heels in and stick to my ideas.
Like this, we all suffer from a multitude of biases that affect how we think, and we make logical fallacies when we try to justify them. And this is where the anecdotal fallacy comes in.
The Anecdotal Fallacy
Let’s go back to what I said about believing what our friends and families tell us, or for that matter, what our social, religious or political leaders tell us. This is usually because we like hearing about people’s personal stories.
If you look at this from a purely evolutionary perspective, it makes sense. Science feels cold and impersonal. I think this is probably because it’s such a new way of thinking, relative to our existence on this planet. For hundreds of thousands of years, we’ve been sharing stories with each other, and sometimes believing them could mean life or death. Personal stories are easy to connect with. That’s one reason I add my experiences to my writing because that just makes it more human and relatable.
We live in a time when it’s become more important than ever to figure out what’s real and what’s not. Just because something is right for you, doesn’t mean it’s objectively true for everyone. This is why the anecdotal fallacy pulls us further away from the truth. Our biases and fallacies are getting a hold of us and leading many people astray like flat-earthers, antivaxxers, climate change deniers, and fans of alternative medicine.
Anecdotes are just not good enough anymore. We have scientific and critical thinking tools that can help us find facts behind any topic. We need to use them to present empirical, falsifiable evidence for our claims, or else we will simply lose the argument.
Blind to Variables
We, humans, are terrible at parsing out variables. If we think a particular alternative medicine works like a dream, not only are our confirmation biases kicking in (not to mention the placebo effect), but we also don’t see the other variables surrounding this treatment that may also be contributing.
Like that aunt who swears by the herbs in the milk. Are the herb actually helping? What chemical in those herb could help? Was it the milk that helped ease the headache? Or was it the waking up at 5 am? Could these all be having some effect or another that we don’t know about, or could they be completely benign and her headache just went away by itself? Headaches have a tendency to do that, you know.
These variables can only be accounted for when each one is taken separately and tested independently, like in blinded randomised controlled trials.
Anecdotes can be useful
I’m not dismissing your experiences and beliefs offhand though. They hold value. They are a part of you and shouldn’t be ignored at a personal level and a scientific one.
After all, when we conduct research, say, like a randomised controlled trial, our first source of information will be the participant’s experiences. But to cancel out the biases and other differences between all the subjects, we need to make sure we include as many people as we can in such studies. That’s why research and surveys need large samples of people to be more reliable. That’s also why well researched scientific findings are also far more reliable than anecdotal evidence.
Conclusion
So, whatever happened to that aunt of yours with the herbal milk potion? Should you believe her? Of course not. There are so many other variables that could have stopped her headaches. Maybe they could have gone away on their own. Plus, she’s probably got a bias towards natural remedies.
Her claims can be checked by scientific methods. If she can find RCTs on that herb and its consumption with milk, she should share them with you. As compelling as anecdotes can be, they shouldn’t be taken as proof of anything. If they are backed up with good science, you can probably believe it, but do your own digging to get to the bottom of it.
Similarly, if you see a YouTuber, an Instagram influencer, a celebrity, a friend or a political or religious leader making big claims based on their experiences, you should be skeptical and try and find evidence for and against it. They are not reliable because they suffer from the same flaws we do. But if they provide evidence for their claims, use it to do your own research to find where the scientific consensus lies and follow those findings.
If it gets turned around on you, provide the evidence. And if you don’t know the answers, it’s perfectly alright to say you don’t know. In the worst-case scenario, if you get slammed with excellent arguments and reliable evidence to prove your position is wrong, have the humility and intellectual honesty to admit you were wrong. It’s when you’re wrong that you have the opportunity to learn something new, something truthful.
Learning new things will make you smarter. We should all strive to be smarter every day.