Category Archives: Myths

Some good news? “Critical thinking instruction in humanities reduces belief in pseudoscience”

Sometimes it’s quite difficult for (educational) mythbusters: what if you only make it worse. There are some studies – also mentioned in our book – who say so. But this new study – although with a very specific group and with a rather small amount of participants – at least suggests there are opportunities to battle pseudoscience…

From the press release:

A recent study by North Carolina State University researchers finds that teaching critical thinking skills in a humanities course significantly reduces student beliefs in “pseudoscience” that is unsupported by facts.

“Given the national discussion of ‘fake news,’ it’s clear that critical thinking – and classes that teach critical thinking – are more important than ever,” says Anne McLaughlin, an associate professor of psychology at NC State and co-author of a paper describing the work.

“Fundamentally, we wanted to assess how intentional you have to be when teaching students critical thinking,” says Alicia McGill, an assistant professor of history at NC State and co-author of the paper. “We also wanted to explore how humanities classes can play a role and whether one can assess the extent to which critical thinking instruction actually results in improved critical thinking by students.

“This may be especially timely, because humanities courses give students tools they can use to assess qualitative data and sort through political rhetoric,” McGill says. “Humanities also offer us historical and cultural perspective that allow us to put current events into context.”

For this study, the researchers worked with 117 students in three different classes. Fifty-nine students were enrolled in a psychology research methods course, which taught statistics and study design, but did not specifically address critical thinking. The other 58 students were enrolled in one of two courses on historical frauds and mysteries – one of which included honors students, many of whom were majors in science, engineering and mathematics disciplines.

The psychology class served as a control group. The two history courses incorporated instruction explicitly designed to cultivate critical thinking skills. For example, students in the history courses were taught how to identify logical fallacies – statements that violate logical arguments, such as non sequiturs.

At the beginning of the semester, students in all three courses took a baseline assessment of their beliefs in pseudoscientific claims. The assessment used a scale from 1 (“I don’t believe at all.”) to 7 (“I strongly believe.”).

Some of the topics in the assessment, such as belief in Atlantis, were later addressed in the “historical frauds” course. Other topics, such as the belief that 9/11 was an “inside job,” were never addressed in the course. This allowed the researchers to determine the extent to which changes in student beliefs stemmed from specific facts discussed in class, versus changes in a student’s critical thinking skills.

At the end of the semester, students took the pseudoscience assessment again.

The control group students did not change their beliefs – but students in both history courses had lower beliefs in pseudoscience by the end of the semester.

Students in the history course for honors students decreased the most in their pseudoscientific beliefs; on average, student beliefs dropped an entire point on the belief scale for topics covered in class, and by 0.5 points on topics not covered in class. There were similar, but less pronounced, changes in the non-honors course.

“The change we see in these students is important, because beliefs are notoriously hard to change,” says McLaughlin. “And seeing students apply critical thinking skills to areas not covered in class is particularly significant and heartening.”

“It’s also important to note that these results stem from taking only one class,” McGill says. “Consistent efforts to teach critical thinking across multiple classes may well have more pronounced effects.

“This drives home the importance of teaching critical thinking, and the essential role that humanities can play in that process,” McGill says. “This is something that NC State is actively promoting as part of a universitywide focus on critical thinking development.”

The paper, “Explicitly teaching critical thinking skills in a history course,” was published March 20 in the journal Science & Education.

Abstract of the study:

Critical thinking skills are often assessed via student beliefs in non-scientific ways of thinking, (e.g, pseudoscience). Courses aimed at reducing such beliefs have been studied in the STEM fields with the most successful focusing on skeptical thinking. However, critical thinking is not unique to the sciences; it is crucial in the humanities and to historical thinking and analysis. We investigated the effects of a history course on epistemically unwarranted beliefs in two class sections. Beliefs were measured pre- and post-semester. Beliefs declined for history students compared to a control class and the effect was strongest for the honors section. This study provides evidence that a humanities education engenders critical thinking. Further, there may be individual differences in ability or preparedness in developing such skills, suggesting different foci for critical thinking coursework.

1 Comment

Filed under Education, Media literacy, Myths, Research

You can now download the preprint of the failed replication (x 3) of mindset research

It has been one of the most often read posts on this blog for the past few weeks: A Mindset “Revolution” Sweeping Britain’s Classrooms May Be Based On Shaky Science

Now you can download the full preprint of the research by Li and Bates (HT Timothy Bates):

Mindset theory asserts that children’s cognitive ability and school grades depend heavily on whether they believe basic ability is malleable. The theory also predicts that praise for intelligence dramatically lowers cognitive performance. Here we test predictions in 3 studies totalling 624 individually-tested 10-12-year-olds. Praise for intelligence failed to harm post- challenge cognitive performance. Children’s mindsets had no relationship to their IQ or to their school grades. Finally believing ability to be malleable had not association with improvement of grades across the year. We conclude that the belief that basic ability is fixed is harmless, and that implicit theories of intelligence play no significant role in development of cognitive ability, response to challenge, or educational attainment.

Leave a comment

Filed under Education, Myths, Research

What causes smart people to resist scientific messages?

This could become an interesting new study – not only because we seem to be living in post truth-times. Psychological researchers are working to understand the cognitive processes, ideologies, cultural demands, and conspiracy beliefs that cause smart people to resist scientific messages. They do this using surveys, experiments, observational studies and meta-analyses, the researchers capture an emerging theoretical frontier with an eye to making science communication efforts smarter and more effective. The only sad thing: I can’t read the actual paper as it’s only an announcement of a paper at a conference past weekend. The only thing we have for now is this press release:

Protecting “Pet Beliefs”

One striking feature of people who hold science-skeptic views is that they are often just as educated, and just as interested in science, as the rest of us. The problem is not about whether they are exposed to information, but about whether the information is processed in a balanced way. It manifests itself in what Matthew Hornsey (University of Queensland) describes as “thinking like a lawyer,” in that people cherry-pick which pieces of information to pay attention to “in order to reach conclusions that they want to be true.”

“We find that people will take a flight from facts to protect all kinds of belief including their religious belief, their political beliefs, and even simple personal beliefs such as whether they are good at choosing a web browser,” says Troy Campbell (University of Oregon).

Dan Kahan (Yale University) agrees, finding in their research that “the deposition is to construe evidence in identity-congruent rather than truth-congruent ways, a state of disorientation that is pretty symmetric across the political spectrum.”

Changing Minds

Merely talking about “evidence” or “data” does not typically change a skeptic’s mind about a particular topic, whether it is climate change, genetically modified organisms, or vaccines. People use science and fact to support their particular opinion and will downplay what they don’t agree with.

“Where there is conflict over societal risks – from climate change to nuclear-power safety to impacts of gun control laws, both sides invoke the mantel of science,” says Kahan.

“In our research, we find that people treat facts as relevant more when the facts tend to support their opinions,” says Campbell. “When the facts are against their opinions, they don’t necessarily deny the facts, but they say the facts are less relevant.”

One approach to deal with science skepticism is to identify the underlying motivations or “attitude roots,” as Hornsey describes in his recent research (American Psychologist, in Press).

“Rather than taking on people’s surface attitudes directly, tailor the message so that it aligns with their motivation. So with climate skeptics, for example, you find out what they can agree on and then frame climate messages to align with these.”

Kahan’s recent research shows that a person’s level of scientific curiosity could help promote more open-minded engagement. They found that people who enjoyed surprising findings, even if it was counter to their political beefs, were more open to the new information. As Kahan and his colleagues note, their findings are preliminary and require more research.

Hornsey, Campbell, Kahan and Robbie Sutton (University of Kent) will present their research at the symposium, Rejection of Science: Fresh Perspectives on the Anti-Enlightenment Movement. The talks take place on Saturday, January 21, 2017, at the SPSP Annual Convention. More than 3000 scientists are in attendance at the conference in San Antonio from January 19-21.

1 Comment

Filed under Education, Myths, Research

Interesting read: A Mindset “Revolution” Sweeping Britain’s Classrooms May Be Based On Shaky Science

Growth and fixed mindset are all the rage, but this article by Tom Chivers shows something else: the research behind it contains worrying errors.

An excerpt from the article:

But the striking effects in Dweck’s findings have surprised psychologists. Timothy Bates, a professor of psychology at the University of Edinburgh, told BuzzFeed News that the “big effects, monstrous effects” that Dweck has found in the 1998 study and others are “strange – it’s an odd one to me”.

Bates told BuzzFeed News that he has been trying to replicate Dweck’s findings in that key mindset study for several years. “We’re running a third study in China now,” he said. “With 200 12-year-olds. And the results are just null.

“People with a growth mindset don’t cope any better with failure. If we give them the mindset intervention, it doesn’t make them behave better. Kids with the growth mindset aren’t getting better grades, either before or after our intervention study

Dweck told BuzzFeed News that attempts to replicate can fail because the scientists haven’t created the right conditions. “Not anyone can do a replication,” she said. “We put so much thought into creating an environment; we spend hours and days on each question, on creating a context in which the phenomenon could plausibly emerge.

“Replication is very important, but they have to be genuine replications and thoughtful replications done by skilled people. Very few studies will replicate done by an amateur in a willy-nilly way.”

Nick Brown, a PhD student in psychology at the University of Groningen in the Netherlands, is sceptical of this: “The question I have is: If your effect is so fragile that it can only be reproduced [under strictly controlled conditions], then why do you think it can be reproduced by schoolteachers?”

And Nick Brown did much more, you can read it here.


Filed under Education, Myths, Research, Review

Lillienfeld on microaggression, still a long way to go for research.

When a researcher who has had a big influence on my personal work – Lilienfeld because of this book – writes something about a theme that has been a worry for me lately – microaggressions, programs against microagrrassion, but also safe zones and the thin line between censorship and free speech- , it would be a mistake not to share this – open access – article on this blog.

If you wonder what’s it all about:

Microaggressions are typically defined as subtle snubs, slights, and insults directed toward minorities, as well as to women and other historically stigmatized groups, that implicitly communicate or at least engender hostility.

The article in itself is not against the idea of microaggression as such, but it does warn that the present scientific evidence is weak.

This quote from the conclusion describes the position of Lilienfeld quite nicely:

I encourage microaggression researchers to continue their scholarly inquiries while substantially tempering their assertions, especially those concerning (a) the causal association between microaggressions and adverse mental health and (b) the presumed effectiveness of microaggression intervention efforts. The MRP has generated a plethora of theoretically and socially significant questions that merit thoughtful examination in coming decades. But it is not close to being ready for widespread real-world application.

This the abstract of the paper:

The microaggression concept has recently galvanized public discussion and spread to numerous college campuses and businesses. I argue that the microaggression research program (MRP) rests on five core premises, namely, that microaggressions (1) are operationalized with sufficient clarity and consensus to afford rigorous scientific investigation; (2) are interpreted negatively by most or all minority group members; (3) reflect implicitly prejudicial and implicitly aggressive motives; (4) can be validly assessed using only respondents’ subjective reports; and (5) exert an adverse impact on recipients’ mental health. A review of the literature reveals negligible support for all five suppositions. More broadly, the MRP has been marked by an absence of connectivity to key domains of psychological science, including psychometrics, social cognition, cognitive-behavioral therapy, behavior genetics, and personality, health, and industrial-organizational psychology. Although the MRP has been fruitful in drawing the field’s attention to subtle forms of prejudice, it is far too underdeveloped on the conceptual and methodological fronts to warrant real-world application. I conclude with 18 suggestions for advancing the scientific status of the MRP, recommend abandonment of the term “microaggression,” and call for a moratorium on microaggression training programs and publicly distributed microaggression lists pending research to address the MRP’s scientific limitations.

1 Comment

Filed under Education, Myths, Research, Review

The “Law of the instrument”

In 1966 Abraham Maslow said, “I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.”, this has become known as the “Law of the instrument“, maybe the least discussed law when talking about education, but imho the most important to warn for in educational discussions.

Not that we only have hammers in education, no, it’s sheer impossible to grasp the almost infinite possibilities, but many people do have their favorite tool.

Some examples:

  • How many times do you read: this tool will change everything in education? We know that it won’t, but most of the time the person expressing this is really convinced of his hammer.
  • In a similar vein let’s teach all children how to code. Most of the time this is said by people who are really convinced of his or her hammer.
  • This morning I woke up to find myself in the midst of a discussing pro and contra PBL, and again, well, you guessed it.

But do you remember the learning pyramid? If that awful piece of educational myth would have been correct, it would mean there is one approach that works best in any given context, for any child, for any given goal. Or that there is only one golden hammer.

The same reason why the learning pyramid isn’t true (for sources, check our book), there is seldom a golden hammer in education. It’s true: as Hattie often claims: most of the things we do in education have effect, it’s to know the impact. But it’s more difficult: it’s also knowing when, why and how something needs to be applied.

When thinking about this law of the instrument the past few days, I found it quite confronting as I did find discover I’ve made this mistake myself too. Let’s make it one for our educational New Year’s resolutions…


Filed under Education, Myths

Sad… how inaccurate are introductory psychology textbooks? #urbanmyths

Long live Twitter as sometimes it can be a source of pretty depressing studies, like this tweet:

It’s good that some myths aren’t covered, but that still many myths persist in introductory textbooks is a call to action.

This study is a nice follow up to the teacher training report earlier this year, do check the abstract:

The introductory psychology class represents the first opportunity for the field to present new students with a comprehensive overview of psychological research. Writing introductory psychology textbooks is challenging given that authors need to cover many areas they themselves may not be intimately familiar with. This challenge is compounded by problems within the scholarly community in which controversial topics may be communicated in ideological terms within scholarly discourse. Psychological science has historically seen concerns raised about the mismatch between claims and data made about certain fields of knowledge, apprehensions that continue in the present “replication crisis.” The concern is that, although acting in good faith, introductory psychology textbook authors may unwittingly communicate information to readers that is factually untrue. Twenty-four leading introductory psychology textbooks were surveyed for their coverage of a number of controversial topics (e.g., media violence, narcissism epidemic, multiple intelligences) and scientific urban legends (e.g., Kitty Genovese, Mozart Effect) for their factual accuracy. Results indicated numerous errors of factual reporting across textbooks, particularly related to failing to inform students of the controversial nature of some research fields and repeating some scientific urban legends as if true. Recommendations are made for improving the accuracy of introductory textbooks.


Filed under Education, Myths, Psychology, Research, Review

A smile on Sunday: Little boxes

Yesterday I was a bit upset, as I saw this tweet by the World Economic Forum:

Sigh. This is Myers Briggs and it has been debunked over and over again.

I received a great reply:

And then I knew this is the smile I want to give you this Sunday morning:

Leave a comment

Filed under Funny, Myths

Dear Andreas Schleicher, a few minutes about your latest opinion

I’m not really into writing open letters, but this time I just couldn’t help myself.

Dear Andreas Schleicher,

We’re getting close to the PISA 2015 release, and I’m really looking forward to it. I know there are many people who have problems with – the power of – PISA, but I do think that this data-collection has its merits as one of the possible sources to discuss educational policy. You tend to end your presentations with the same quote: without data you’re just another person with an opinion.

It’s this quote that makes me write this letter. You wrote a post for Google, and while you’re entitled to have an opinion, I was surprised that you would write something that’s just that.

Can I share some examples?

Today, schools need to prepare students for more rapid economic and social change than ever before, for jobs that haven’t been created, to use technologies that haven’t yet been invented, and to solve social problems that we can’t yet imagine.

Is this true? Do you have data to back this up? Sounds very popular indeed, but how compare the present evolutions with the social changes of the previous century? What about the previous industrial revolution?

Or take this:

In traditional school systems, teachers have been provided an exact prescription for what to teach and then left alone in classrooms. The past was about delivered wisdom, the future is about user-generated wisdom.

Wow, if I ever read a cliché. Maybe it’s a good idea to read Hannah Arendt about the conservative role education also has to make progression. And what does user-generated wisdom mean? Do we need to put kids under an apple tree so they can discover gravity themselves, I have big news for you: they won’t.

Let’s continue:

…you had teachers and content divided by subjects and student destinations; and the past was isolated: schools were designed to keep students inside, and the rest of the world outside. The future needs to be integrated, that means emphasising integration of subjects, integration of students and integration of learning contexts; and it needs to be connected: that means connected with real-world contexts, and also permeable to the rich resources in the community. Instruction in the past was subject-based, instruction in the future will be project based. The past was hierarchical, students were recipients and teachers the dominant resource, the future is co-created, and that means we need to recognise both students and adults as resources for the co-creation of communities, for the design of learning and for the success of students.

You like data, you said? You worry about the existing gap between rich and poor? Then read this new report please. The report is very careful in its conclusion about project based learning, but rather than the correlational data PISA has, it actually did a RTC. If you have read the report, and maybe some other sources on cognitive psychology, maybe you’ll want to rewrite your paragraph.

In the past, different students were taught in similar ways. Now we need to embrace diversity with differentiated pedagogical practices. The past was curriculum-centered, the future is learner centered. The goals of the past were standardisation and compliance, that is, students are educated in batches of age, following the same standard curriculum, all assessed at the same time.

Oh, you mean as in “a whole group of 15-year olds being tested in a similar way to compare countries?” But more important: did you notice that while you now propagate ‘differentiated pedagogical practices’ this is in stark contrast with what you wrote a couple of sentences before and immediately after this very sentence.

Oh and this:

The future is about personalising educational experiences, that is building instruction from student passions and capacities, helping students personalise their learning and assessment in ways that foster engagement and talents.

Does sound good, but I would suggest you also take a look at the work of Gert Biesta with the different roles education has. If education should only be this, then we’ll end up in an extreme neoliberal approach with everybody following their own passion but with no society left.

As said, I appreciate your work at OECD.



Filed under Education, Myths, Review

Howard Gardner on his ‘multiple intelligences’: the theory is no longer current!

Howard Gardner was asked to reply to this question “Reflecting on your life, what has been your greatest accomplishment so far and why?” His reply – a short essay – can be read here.

A first interesting quote:

I termed the resulting categories “intelligences” rather than talents. In so doing I challenged those psychologists who believed that they owned the word “intelligence” and had a monopoly on its definition and measurement. If I had written about human talents, rather than intelligences, I probably would not have been asked to contribute to this volume.

I do understand this reasoning, but as we already discussed on this blog and in our book: the name intelligence includes a predictive element, while MI doesn’t.

Another interesting quote:

Nor, indeed, have I carried out experiments designed to test the theory. This has led some critics to declare that my theory is not empirical. That charge is baloney! The theory is not experimental in the traditional sense (as was my earlier work with brain-damaged patients); but it is strictly empirical, drawing on hundreds of findings from half-a-dozen fields of science.

This quote may raise some discussion, and actually I don’t agree with it neither. It’s not because you have a lot of sources that deliver you input for a theory, that this means you’re theory is tested.

But the most important quote:

At the same time, I readily admit that the theory is no longer current. Several fields of knowledge have advanced significantly since the early 1980s. Any reinvigoration of the theory would require a survey similar to the one that colleagues and I carried out thirty-five years ago. Whether or not I ever carry out such an update, I encourage others to do so.

And that is because I am no longer wedded to the particular list of intelligences that I initially developed. What I – and others, most notably Daniel Goleman and Robert Sternberg -have done is to undermine the hegemony over the concept of intelligence that was maintained for a century by adherents to a Spearman- Binet-Piaget concept of intelligence. I have no idea where the study of intelligence(s) will be a century from now, but I am confident that the field will recognize a plurality of skills, talents, and intelligences.



Filed under Education, Myths, Research, Review