Which questions do you need to ask yourself if you read about a study? This great video gives you a headstart! (HT Tommy Opgenhaffen)
- #1 What was the purpose of the study?
The reason they did the study might influence the answers they found. The data might be biased if it was collected to further the interests of a particular group or organization. For example, a drug company funding a study about a drug they have created.
- #2 Who collected the information?
There’s a difference between your friend collecting data and a scientist. It’s key to think if they have any biases. Check to see if certain ideas are being promoted (for example: religious or political) over others, if assumptions are being made, or if it’s the facts. For example, when trying to find out what type of studying is preferred, you might ask your friends, where a researcher, who studies behaviour of university students, would have the skills and abilities to approach this in a more sound way.
- #3 What information was actually collected?
There are lots of things that we can’t actually measure directly. For example there is no clear way to measure things like happiness or satisfaction. So in this case, we have to use a different way of measuring to figure it out.
For example we might collect information about grades and participation to understand how happy the students felt with respect to the class. This might tell us some things about their happiness but we could also be missing things.
- #4 When was the information collected?
When your data was collected influences how useful it is to your question. Some topics, might require very current information that changes rapidly. For example, when looking at trending social media topics, we’ll want up to date information. This changes really quickly making older data useless
- #5 How was the information obtained?
The tools, number of responses received, how many participants and how the participants were selected can make a big difference in the quality of data. How did they get their information? Consider the tools: Did they do interviews or surveys? If they did surveys, were they online or were they mailed out? Once you know what tool they used. Consider: How many they sent vs. completed (the response rate). How many participants (the sample size). Did they look at 10 people, 100, 1000, 10000? This can make a big difference. How they selected the participants — was it random? It’s important that the researchers let you know how the data was collected. It they don’t let you know, they might be hiding something and it could be unreliable.
- #6 How consistent is the information with other sources?
Try to make sure that more than one source tells you the same thing or comes to similar conclusions. If there are results that tell you something completely different, start questioning.