While this blog is dedicated mostly to weight loss and the science of health and fitness, I sometimes stray from that topic to include some important basics involving any type of science. We’ve all read Tumblr articles that are in direct conflict with each other. For example, how many people have written articles about how important protein is, only to be countered by how protein in high quantities is horrible? If you ever find a new fact and you’re not really sure about it, there’s a lot of ways to fact check.
Let’s break down a theoretical fact.
According to a recent survey, most people prefer cats.
Pretty harmless fact, right? You may personally disagree, or agree, but that’s not the point. How do we break down this fact and see if it’s accurate or not?
1. Where’s it coming from?
Let’s say the person who posted this fact gives us a source and we find that the Cat Lover’s Association did this survey. Right off the bat we can see a conflict of interest. Of course the Cat Lover’s Association is going to find that people prefer cats to dogs. What if the group that funded the survey was Animals of America? Sounds a bit more reasonable, right? Just in case you should search deeper into that organization. Turns out the founder and president of Animals of America is actually a member of the Pretty Persian Club and owns three cat hospitals that are just for felines! Another source of bias. Let’s say the survey was put on by Happy Pet Owners of America. We look into the group and see that it’s run by a mix of cat and dog owners, as well as a fair amount of bird, hedgehog and horse people. Definitely something we can trust more. Apply this to any other fact: who did the study, who are they tied to, and do they have anything to gain/any bias by the results of the survey.
2. How did they do it?
Let’s say this survey was conducted by asking people coming out of a pet store. Seems like a reasonable place to survey people about what kind of pets they like, right? What if the pet store was a pet store that only had cats? What if they asked people coming out of a common grocery store? Take a look at HOW they conducted the survey and see if there’s any bias behind it. Did they ask every person who they saw, or just every other? Women only? Men only? Really break it down and take a look at how they conducted the survey. Surveys that involve polls online can be tricky because not everyone fills them out. This is a huge issue when news stations or TV stations do surveys, because only people who view that station will answer the survey. If Animal Planet does a survey about animals they’ll get different results about certain questions than The Hunting Network.
3. How many people were surveyed?
Let’s say they asked 5 people. 5 people! That’s NOTHING! 3 of those people could have been related and have a lot in common with their preferences. If they asked 5,000 people at 100 stores across the nation you’re going to have a more accurate tally of if people prefer cats or dogs. If surveys have a large amount of people surveyed it’s going to have a wider array of opinions and get a better idea of the “average person.”
4. Where was the survey conducted?
Some surveys are asked in other countries or in certain areas of the country that can give very different answers. For example, in some countries cats are not really “domestic.” People use cats to catch rats, not as a pet. Their opinions on the usefulness of cats may differ from someone who has a lot of birds and sees cats as a threat to the natural environment for the birds. Likewise, people who live near the ocean may answer surveys saying they like seafood best versus people who live very far inland and prefer beef or vegetables. The location of the survey is important.
5. What question was asked?
This is very important! What if the question was, “Do you prefer cats or dogs?” There are only two possible answers to this question. People who prefer anything other than cats or dogs won’t chose the animal they prefer. Here are other questions that could have been asked…
“Are cats your favorite animal?”
“Do you like dogs?”
“Do you think cats are pretty?”
“Do you think cats are useful?”
Sometimes people will ask these questions and come to their own conclusion about it. What does it mean to “prefer” an animal? If people answer that they do, indeed, think cats are useful they may interpret that are “Most people prefer cats.” Others may say, “Most people thought cats were useful” or “People find cats to be able hunters.” Make sure that the question that is asked parallels the conclusion the survey came to.
6. When did the survey take place?
“According to a recent survey…” is very vague. What is recent? 6 months? 1 year? 10 years? A survey that takes place in 1997 may have different results than one taken in the last few weeks, especially when you’re asking questions about technology.
7. Their sources.
Surveys and studies often cite other sources when they submit their papers. Scroll down to the end of the study to see where they got a lot of their research from and ask yourself the above questions.
Now that you know the many ways “facts” can be interpreted, start using this any time you see a new “fact.” This will help you to become informed and allow you to make your own decisions about what you hear or read.