Category Archives: Review

Gender differences in spatial skills probably start in elementary school (meta-analysis)

Is it nature or nurture that men seem to better in spatial skills than women? Well, a new meta-analysis suggests that nurture plays a role and a takeaway idea for parents is that parents it’s a good idea to encourage both their daughters and sons to play with blocks and other construction items that might help in the development of spatial reasoning skills.

From the press release:

It is well-established that, on average, men outperform women on a spatial reasoning task known as mental rotation — imagining multi-dimensional objects from different points of view. Men are not, however, born with this advantage, suggests a major meta-analysis by psychologists at Emory University. Instead, males gain a slight advantage in mental-rotation performance during the first years of formal schooling, and this advantage slowly grows with age, tripling in size by the end of adolescence.

The Psychological Bulletin, a journal of the American Psychological Association, is publishing the findings.

“Some researchers have argued that there is an intrinsic gender difference in spatial reasoning — that boys are naturally better at it than girls,” says lead author Jillian Lauer, who is set to graduate from Emory in May with a PhD in psychology. “While our results don’t exclude any possibility that biological influences contribute to the gender gap, they suggest that other factors may be more important in driving the gender difference in spatial skills during childhood.”

Co-authors of the paper include Stella Lourenco, associate professor of psychology at Emory, whose lab specializes in the development of spatial and numerical cognition. Co-author Eukyung Yhang worked on the paper as an Emory undergraduate, funded by the university’s Institute for Quantitative Theory and Methods. Yhang graduated in 2018 and is now at Yale University School of Medicine.

The meta-analysis included 128 studies of gender differences in spatial reasoning, combining statistics on more than 30,000 children and adolescents aged three to 18 years. The authors found no gender difference in mental-rotation skills among preschoolers, but a small male advantage emerged in children between the ages of six and eight.

While differences in verbal and mathematical abilities between men and women tend to be small or non-existent, twice as many men as women are top performers in mental rotation, making it one of the largest gender differences in cognition.

Mental rotation is considered one of the hallmarks of spatial reasoning. “If you’re packing your suitcase and trying to figure out how each item can fit within that space, or you’re building furniture based on a diagram, you’re likely engaged in mental rotation, imagining how different objects can rotate to fit together,” Lauer explains.

Prior research has also shown that superior spatial skills predict success in male-dominated science, technology engineering and math (STEM) fields, and that the gender difference in spatial reasoning may contribute to the gender disparity in these STEM fields.

“We’re interested in the origins of gender differences in spatial skills because of their potential role in the gender gap we see in math and science fields,” Lauer says. “By determining when the gender difference can first be detected in childhood and how it changes with age, we may be able to develop ways to make educational systems more equitable.”

It takes most of childhood and adolescence for the gender gap in spatial skills to reach the size of the difference seen in adulthood, Lauer says. She adds that the meta-analysis did not address causes for why the gender gap for mental rotation emerges and grows.

Lauer notes that previous research has shown that parents use more spatial language when they talk to preschool sons than daughters. Studies have also found that girls report more anxiety about having to perform spatial tasks than do boys by first grade, and that children are aware of gender stereotypes about spatial intelligence during elementary school.

“Now that we’ve characterized how gender differences in spatial reasoning skills develop in children over time we can start to hone in on the reasons for those differences,” Lauer says.

Meanwhile, she adds, parents may want to be aware to encourage both their daughters and sons to play with blocks and other construction items that might help in the development of spatial reasoning skills, since evidence shows that these skills can be improved with training.

“Giving both girls and boys more opportunities to develop their spatial skills is something that parents and educators have the power to do,” Lauer says.

Abstract of the meta-analysis:

Gender differences in spatial aptitude are well established by adulthood, particularly when measured by tasks that require the mental rotation of objects (Linn & Petersen, 1985; Voyer, Voyer, & Bryden, 1995). Although the male advantage in mental rotation performance represents one of the most robust gender differences in adult cognition, the developmental trajectory of this male advantage remains a topic of considerable debate. To address this debate, we meta-analyzed 303 effect sizes pertaining to gender differences in mental rotation performance among 30,613 children and adolescents. We found significant developmental change in the magnitude of the gender difference: A small male advantage in mental rotation performance first emerged during childhood and then subsequently increased with age, reaching a moderate effect size during adolescence. Procedural factors, including task and stimulus characteristics, also accounted for variability in reported gender differences, even when controlling for the effect of age. These results demonstrate that both age and procedural characteristics moderate the magnitude of the gender difference in mental rotation throughout development.

Leave a comment

Filed under Education, Psychology, Research, Review

Understanding math anxiety (Best Evidence in Brief)

There is a new Best Evidence in Brief with this time many interesting studies, such as this one:

While mathematics is often considered a hard subject, not all difficulties with the subject result from cognitive difficulties. Many children and adults experience feelings of anxiety, apprehension, tension or discomfort when confronted by a math problem. Research conducted by the Centre for Neuroscience in Education at the University of Cambridge examined the math performance of more than 2,700 primary and secondary students in the UK and Italy who were screened for math anxiety and general anxiety. Researchers then worked one-to-one with the children in order to gain deeper understanding of their cognitive abilities and feelings towards math using a series of cognitive tasks, questionnaires, and interviews.
Emma Carey and colleagues found that a general feeling that math was more difficult than other subjects often contributed to feelings of anxiety about the subject, and that teachers and parents may inadvertently play a role. Girls in both primary and secondary school were found to have higher levels of both math anxiety and general anxiety.
Students indicated poor test results, or negative comparisons to peers or siblings, as reasons for feeling anxious. Secondary school students also indicated that the transition from primary to secondary school was a cause of math anxiety, as the work seemed harder and there was greater pressure on tests and increased homework.
The report sets out a series of recommendations, including:
  • Teachers should be aware that math anxiety can affect students’ math performance.
  • Teachers and parents need to be aware that their own math anxiety might influence students’ math anxiety.
  • Teachers and parents also need to be aware that gendered stereotypes about math ability might contribute to the gender gap in math performance.
  • Reducing classroom pressure and using methods like free writing about emotions before a test could help to alleviate math anxiety.

Leave a comment

Filed under Education, Psychology, Research, Review

So just you know: Trials testing new educational methods in schools ‘often fail to produce useful evidence’

Doing research in education is difficult, very difficult. Trust me. So while this review may come as a shock, I’m not that shocked.
I’m performing a RCT myself at the moment and doing it in real life situations is proving to be very hard.

From the press release:

Educational trials aimed at boosting academic achievement in schools are often uninformative, new research suggests.

The new study, published in the journal Educational Researcher, found that 40% of large-scale randomised controlled trials (RCTs) in the UK and the US failed to produce any evidence as to whether an educational intervention helped to boost academic attainment or not.

The researchers evaluated 141 trials involving more than one million students, which tested schemes ranging from whether providing free school breakfasts raises grades in Maths and English, to whether playing chess improves numeracy skills.

The trials, which were carried out by the charitable organisation the Education Endowment Foundation (EEF) in the UK and the National Center for Education Evaluation and Regional Assistance (NCEE) in the US, are expensive – with costs often exceeding £500,000.

The authors of the study argue that more research is urgently needed to understand why RCTs in education are so often uninformative.

Lead author of the research, Dr Hugues Lortie-Forgues, from the Department of Education at the University of York, UK, said: “Just like in medicine, trials of educational interventions are an important way to allow policy makers and teachers to make informed decisions about how to improve education. However, many of these trials are currently not fulfilling their main aim of demonstrating which interventions are effective and which are not.”

“Further research to investigate the reasons for this should be a priority. These organisations are trying to achieve something positive and reform is urgently needed to help them to do so.”

In recent years there have been a growing number of RCTs conducted in education. For example, in the UK, the Education Endowment Foundation (EEF) has commissioned more than 191 trials since 2012.

The researchers cite possible reasons why current trials may be ineffective, including:

  • The interventions being tested may not be suitable for trial in the first place.
  • Interventions may not be being correctly implemented during trials – for example due to inadequate training of teachers in the methods being tested.
  • The trials themselves may be poorly designed

The authors suggest a series of changes that could make the trials more informative, including higher-standards when considering which new initiatives are trialled.

Rigorous Large-Scale Educational RCTs are Often Uninformative: Should We Be Concerned? Is published in the journal Educational Researcher. The study was carried out in collaboration with researchers at Loughborough University.

Abstract of the review:

There are a growing number of large-scale educational randomized controlled trials (RCTs). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale RCTs commissioned by the UK-based Education Endowment Foundation and the U.S.-based National Center for Educational Evaluation and Regional Assistance, which evaluated interventions aimed at improving academic achievement in K–12 (141 RCTs; 1,222,024 students). The mean effect size was 0.06 standard deviations. These sat within relatively large confidence intervals (mean width = 0.30 SDs), which meant that the results were often uninformative (the median Bayes factor was 0.56). We argue that our field needs, as a priority, to understand why educational RCTs often find small and uninformative effects.

1 Comment

Filed under Education, Research, Review

The effects of self-assessment

There is a new Best Evidence in Brief with among others, this study:

An article published in Educational Research Review examines the effects of self-assessment on self-regulated learning (SRL) and self-efficacy in four meta-analyses.
To understand the impact of students’ assessment of their own work, Ernesto Panadero and colleagues from Spain analyzed 19 studies comprised of 2,305 students from primary schools to higher education. The meta-analyses only included studies published in English that contained empirical results of self-assessment interventions in relation to SRL and/or self-efficacy, had at least one control group, and had been peer-reviewed.
The findings indicated that:
  • Self-assessment had a positive effect on SRL strategies serving a positive self-regulatory function for students’ learning, such as meta-cognitive strategies (ES= +0.23)
  • Self-assessment had a negative effect on “Negative SRL,” which is associated with negative emotions and stress and is thought to be adverse to students’ learning (ES =-0.65)
  • Self-assessment was also positively associated with SRL even when SRL was measured qualitatively (ES= +0.43)
  • Self-assessment had a positive effect on self-efficacy (ES= +0.73), the effect being larger for girls
The authors suggest that self-assessment is necessary for productive learning but note that the results have yet to identify the most effective self-assessment components (e.g., monitoring, feedback, and revision) in fostering SRL strategies or self-efficacy.

Leave a comment

Filed under Education, Research, Review

A great presentation by Dorothy Bishop on insights from psychology on lack of reproducibility

1 Comment

Filed under Psychology, Research, Review

New publication: The Science of Early Learning (Deans for Impact)

I saw the announcement yesterday, and you can now download this new publication by Deans for Impact yourself here.

This is the short introduction:

How do young children develop their sense of self? How do they learn to understand what they read, and express their ideas in writing? How do they develop abstract knowledge of mathematical concepts?

These questions and others are explored in Deans for Impact’s publication, The Science of Early Learning. This report summarizes current cognitive-science research related to how young children — from birth to age eight — develop skills across three domains: agency, literacy, and numeracy. This document is intended to serve as a resource to anyone who is interested in our best scientific understanding of how young children develop control of their own behavior and intentions, how they learn to read and write proficiently, and how they develop the ability to think mathematically.

Although The Science of Early Learning is not intended to cover every aspect of early learning and development, it may be considered a starting point for educators, teachers, parents, and caregivers who wish to explore principles of learning science as they relate to young children. Deans for Impact intends to incorporate these principles into the work we do with programs that prepare early-childhood educators, and we hope this resource will be broadly useful to the education community

1 Comment

Filed under Education, Research, Review

25 ways to stimulate learning

Found this via Tim Surma (do follow him!), check the source here.

Leave a comment

Filed under Education, Review

A week in replication, some failed, some successful: Big Five, note taking and stereotype threats

The past week I read several replication studies with mixed results:

First, although direct replications (using methods from Mueller and Oppenheimer 2014) yielded longhand-superiority effects, the effects relevant to the encoding function of note-taking were small and did not reach conventional levels of significance. Such outcomes do not support strong recommendations about whether students should take notes longhand or by laptop in class. Second, based on their evidence, Mueller and Oppenheimer (2014) hypothesized that the higher word count and verbatim overlap for laptop groups were in part responsible for diminished performance for laptop users. However, when combining results across the direct replications, differences in word count and verbatim overlap were large, whereas differences in performance were small (and not statistically significant). Thus, differences in word count and verbatim overlap do not appear sufficient to produce performance differences between longhand and laptop groups.

The effects of gender stereotype threat on mathematical test performance in the classroom have been extensively studied in several cultural contexts. Theory predicts that stereotype threat lowers girls’ performance on mathematics tests, while leaving boys’ math performance unaffected. We conducted a large-scale stereotype threat experiment in Dutch high schools (N = 2064) to study the generalizability of the effect. In this registered report, we set out to replicate the overall effect among female high school students and to study four core theoretical moderators, namely domain identification, gender identification, math anxiety, and test difficulty. Among the girls, we found neither an overall effect of stereotype threat on math performance, nor any moderated stereotype threat effects. Most variance in math performance was explained by gender, domain identification, and math identification. We discuss several theoretical and statistical explanations for these findings. Our results are limited to the studied population (i.e. Dutch high school students, age 13–14) and the studied domain (mathematics).

In both examples of failed replication it’s not the case that we should abolish the theories as such straightaway. It’s rather that these new studies show that it’s more complicated than thought maybe due to regional and age differences (stereotype threat), maybe due to maybe differences in personal experiences (note taking).

1 Comment

Filed under Education, Psychology, Research, Review

Can you train executive functions? A second meta-analysis says…

In 2015 I wrote this blogpost on a meta-analysis on the training of executive functions. The results were a bit depressing. Today I found a new meta-analysis via Jeroen Jansen and the results… are again not that good and it all has to do with one of the key concepts in education: transfer.  The meta-analysis was based on 38 studies with 47 contrasts.

I’ll first share with you the abstract of the study:

In the present meta-analysis we examined the near- and far-transfer effects of training components of children’s executive functions skills: working memory, inhibitory control, and cognitive flexibility. We found a significant near-transfer effect (g+ = 0.44, k = 43, p < .001) showing that the interventions in the primary studies were successful in training the targeted components. However, we found no convincing evidence of far-transfer (g+ = 0.11, k = 17, p = .11). That is, training a component did not have a significant effect on the untrained components. By showing the absence of benefits that generalize beyond the trained components, we question the practical relevance of training specific executive function skills in isolation. Furthermore, the present results might explain the absence of far-transfer effects of working memory training on academic skills (Melby-Lervag & Hulme, 2013; Sala & Gobet, 2017).

So, this study is not that surprising as it states that near transfer can happen more easily, far transfer is far more difficult (pun intended). But the consequences of this finding can be pretty big:

The most important aim of the present meta-analysis was to gain a deeper understanding of whether and to what extent the different components of executive functions are trainable and whether train- ing a specific executive function has an ameliorating effect on other main executive function components. This is of crucial importance because in the long run the main aim of training executive functions skills is to improve children’s everyday functioning; for example, academic and social skills as well as emotion regulation. These complex skills are not supported by one sole executive function but generally rely on the interplay among most of them.

So to be clear the study doesn’t say that executive functions can’t be trained, or…

Overall, we found a significant, medium-sized near-transfer effect. However, no far-transfer effect appeared. More specifically, there were significant near-transfer effects on all three components: a moderate-sized effect on working memory and small-sized effects on inhibition and cognitive flexibility. In contrast, no far-transfer effects were found on working memory, inhibitory control, or flexibility. The finding that there was a significant near-transfer effect excludes the possibility that the interventions in the primary studies were not effective in training the components that they targeted. Instead, performance on the components that were trained did significantly improve, however, these gains did not transfer to the untrained components.

But what does this mean?

The lack of far-transfer effect found in the present meta-analysis even within the set of executive function skills makes it—though logically not impossible—still highly unlikely that training unique executive functions could have measurable ameliorating transfer effect on more distantly related and complex constructs, such as academic and social skills (Blair & Razza, 2007) that rely just as much on the trained executive function component as on the other untrained and largely unaffected components. The results of the present meta-analysis therefore provide a possible explanation for the previously found absence of far-transfer effects of working memory trainings on academic skills.

So what do we need to do?

The present meta-analysis shows that there are limited practical benefits— other than on the trained component— of training single executive function components in childhood. Thus, it might be more advisable, both in the educational and in the clinical practice, to use approaches that target multiple executive function components.

But maybe the most important question is in the end of the meta-analysis:

“The lack of causal evidence for significant relationships among the three core components might contradict accounts of executive functions as a single construct.”

So maybe you can’t train ‘executive functions‘, but you can train an executive function.


Filed under Education, Research, Review

A meta-analysis on computer-supported collaborative learning (Best Evidence in Brief)

There is a new Best Evidence in Brief with among others, this study that might surprise some people and oh, my dear friend and co-author Paul Kirschner was involved:

Juanjuan Chen and colleagues recently performed a meta-analysis on the effects of computer-supported collaborative learning (CSCL).

Using 425 empirical studies (all of which used a controlled experimental or quasi-experimental design) published between 2000 and 2016, researchers found several main characteristics to examine: the effects of the collaboration itself; the effects of computer use during collaboration; the effects of extra technology-related learning tools used in CSCL, such as videoconferencing and sharing visuals with team partners; and strategies such as role assignment and peer feedback.
Collaborative learning itself positively affected:
  • Knowledge gain (+0.42)
  • Skill acquisition (+0.62)
  • Student perceptions of the experience (+0.38)
The use of computers, when combined with collaborative learning, positively affected:
  • Knowledge gain (+0.45)
  • Skill acquisition (+0.53)
  • Student perceptions (+0.51)
  • Group task performance (+0.89)
  • Social interaction (+0.57)
Lastly, extra technology-related learning tools during CSCL positively affected knowledge gain (+0.55), as did the use of strategies (+0.38).


Filed under Education, Research, Review