Randomized controlled trial suggests: web-based teaching can improve science understanding for struggling pupils
This is an interesting, large study on web-based teaching specifically aimed at science understanding with a surprising bigger effect for struggling, less able students.
Learn more from the press release:
Web-based learning tools can help deepen science knowledge among all middle school students, and ease the science literacy gap for underachieving students, according to a three-year study published today in the International Journal of Science Education.
Researchers introduced four interactive online science units, which students and teachers accessed with computers or tablets, into 13 middle school in two US states.
The online units were tested in a randomized, controlled trial with over 2,300 students and 71 teachers. While all participating students improved their science knowledge, the results were particularly notable for less able students.
Students with learning disabilities improved 18 percentage points on assessments of science knowledge from pre-test to post-test, and English language learners increased 15 percentage points. Pupils taught the same content with traditional methods, such as textbooks, showed only 5-point gains.
The results are especially important given that students with learning disabilities and English language learners have been historically marginalized in science, technology, engineering, and mathematics (STEM) fields. Despite recent gains, a wide educational attainment gap remains for these students, making them less likely than Caucasian and Asian pupils to complete science coursework in school and pursue STEM careers.
“These significant findings demonstrate that the online curriculum was effective in improving science knowledge for students who struggle with science,” said Dr Fatima Terrazas Arellanes of the University of Oregon, Principal Investigator of the project.
“Well-designed instructional technology really works to lessen the science literacy gap among diverse groups of learners. Technology offers an engaging and motivating environment for learning, and we are just beginning to understand how we can use it effectively to support students with learning disabilities and English language learners.”
The online units were structured with lessons and activities like textbooks, but the content was much more interactive. Guided by their teachers, students learnt science through watching videos, playing educational games, conducting virtual experiments, and collaborating with their classmates.
The content was especially beneficial to students who struggle thanks to embedded eText supports, such as text-to-speech (hearing online text read aloud), pop-up vocabulary definitions, interactive diagrams, digital note-taking, and captioned videos.
Going forward, the challenge will be for researchers and policymakers to develop more evidence-based online tools that teachers can implement with students in their classrooms. The ultimate goal is to help all students, and especially those who struggle, to increase society’s science literacy and forge careers in STEM fields.
Dr Terrazas Arellanes added: “Our work adds to a growing body of evidence suggesting that instructional technology has a place in the classrooms of today and tomorrow — especially for science and especially for students with learning disabilities. We have shown that these tools are not only effective, but can be easily integrated.”
Abstract of the study:
The purpose of this study was to document the design, classroom implementation, and effectiveness of interactive online units to enhance science learning over 3 years among students with learning disabilities, English learners, and general education students. Results of a randomised controlled trial with 2,303 middle school students and 71 teachers across 13 schools in two states indicated that online units effectively deepened science knowledge across all three student groups. Comparing all treatment and control students on pretest-to-posttest improvement on standards-based content-specific assessments, there were statistically significant mean differences (17% improvement treatment vs. 6% control; p < .001); no significant interactions were found between treatment condition and learning disability or English learner status, indicating that these two groups performed similarly to their peers; students with learning disabilities had significantly lower assessment scores overall. Teachers and students were moderately satisfied with the units.
What is grade inflation? Wikipedia gives two descriptions:
- grading leniency: the awarding of higher grades than students deserve, which yields a higher average grade given to students
- the tendency to award progressively higher academic grades for work that would have received lower grades in the past.
In this study, we need to take the second description into account. Grade inflation in this second sense is something a lot of people think, other people say it’s not the case and this study says… grade inflation at English primary schools can increase the price of surrounding houses by up to £7,000. Do note this is a working paper and not a peer reviewed study!
From the press release:
The study finds that as parents are drawn to areas with what appear to be higher school scores, the demand for housing escalates and poorer residents are driven out. The researchers examined data from more than 23,000 neighbourhoods in England, using results of more than five million students enrolled since 1998.
The study, published as a QMUL School of Economics and Finance Working Paper, looked at the period from 1998 to 2007, when English schools used a process called ‘borderlining’ to regrade exams from students who narrowly missed out on a higher Key Stage result.
Erich Battistin, Professor of Economics at QMUL and lead author of the study says the period provides a “perfect test environment” to interrogate an important policy question: can grade inflation change the composition of neighbourhoods?
Borderlining was abolished in 2007 by the Department of Education, following evidence that the procedure caused grade inflation in primary schools for thousands of students. However, the effects of grade inflation that accumulated over one decade before the abolition of borderlining triggered inequalities across neighbourhoods that are persistent and identifiable through to the present day.
The results of the study, co-authored with Dr Lorenzo Neri from QMUL, show that a three percentage point increase (from a baseline of 26 per cent) in the number of students who perform above expectations at Key Stage 2 increases local house prices by 1.5 per cent.
The effect on prices is more dramatic in areas with more than one good school. According to Dr Neri, this is due to a “hedging effect,” where parents gravitate more strongly to areas that have a number of highly-rated schools. He says that in these areas the combined grade inflation of more than one school can increase house prices by three per cent, or £7,000.
“What our study shows is that even very small levels of grade inflation can make a significant impact on house prices,” says Professor Battistin. “The reason for this is well documented by previous studies: parents respond to even the smallest marginal differences in the performance of local schools. Over time, this has a significant effect on the composition of the local neighbourhood and makes the area less affordable for poorer families.
“It’s not new to show that prices and demography are influenced by quality — but what we show is that they can be affected significantly even by a false perception of quality. It’s not really there, it’s just statistical noise — sometimes generated by the benign intentions of markers to bump up marginal students, not necessarily for accountability purposes.” He adds that the results are relevant in the context of recent cheating scandals, in the UK and elsewhere, which he says need to be understood in terms of policy implications as well as in the context of standards and behaviour.
The researchers also show that the effects spill over to the composition of businesses and demography in local areas. They demonstrate that neighbourhoods in the catchment of schools with more grade inflation experience a more pronounced increase in the number of grocery shops, restaurants and coffeehouses surrounding schools, most likely because local retailers respond to the arrival of richer homeowners.
The researchers compared similar blocks in the catchment of schools which, without borderlining, would have scored the same quality in national performance tables. The underlying assumption is that prices across these blocks would have changed similarly over time had manipulation not occurred.
They found a sharper price change for blocks in the catchment of schools where scores were the most inflated. The research methods included the use of large administrative databases and econometric analysis exploiting micro-level data on students, schools, house transactions and businesses. Results survived to the inclusion of neighbourhoods’ socio-economic characteristics at a very fine level; furthermore, regulation regarding the borderlining practice, coupled with a series of robustness checks, ensured that a clear causal relationship can be established.
Abstract of the working paper:
We show that grading standards for primary school exams in England have triggered an inflation of quality indicators in the national performance tables for almost two decades. The cumulative effects have resulted in significant differences in the quality signaled to parents for otherwise identical schools. These differences are as good as random, with score manipulation resulting from discretion in the grading of randonly assigned external markers. We find large housing price gains from the school quality improvements artificially signaled by manipulation as well as lower deprivation and more businesses catering to families in local neighbourhoods. The design ensures improved external validity for the valuation of school quality with respect to boundary discontinuities and has the potential for replication outside of our specific case study.
Ok, I had to read this title a couple of times, but this seems important taken into account that it’s predominantly a correlation for which people try to find explanations and thus speculate. But that this correlation exists is for me the big news: countries with greater gender equality have a lower percentage of female STEM graduates.
From the press release:
Countries with greater gender equality see a smaller proportion of women taking degrees in science, technology, engineering and mathematics (STEM), a new study has found. Policymakers could use the findings to reconsider initiatives to increase women’s participation in STEM, say the researchers.
Dubbed the ‘gender equality paradox’, the research found that countries such as Albania and Algeria have a greater percentage of women amongst their STEM graduates than countries lauded for their high levels of gender equality, such as Finland, Norway or Sweden.
The researchers, from Leeds Beckett University in the UK and the University of Missouri in the USA, believe this might be because countries with less gender equality often have little welfare support, making the choice of a relatively highly-paid STEM career more attractive.
The study, published in Psychological Science, also looked at what might motivate girls and boys to choose to study STEM subjects, including overall ability, interest or enjoyment in the subject and whether science subjects were a personal academic strength.
Using data on 475,000 adolescents across 67 countries or regions, the researchers found that while boys’ and girls’ achievement in STEM subjects was broadly similar, science was more likely to be boys’ best subject. Girls, even when their ability in science equalled or excelled that of boys, were often likely to be better overall in reading comprehension, which relates to higher ability in non-STEM subjects. Girls also tended to register a lower interest in science subjects. These differences were near-universal across all the countries and regions studied.
This could explain some of the gender disparity in STEM participation, as Gijsbert Stoet, Professor in Psychology from Leeds Beckett University explains:
“The further you get in secondary and then higher education, the more subjects you need to drop until you end with just one. We are inclined to choose what we are best at and also enjoy. This makes sense and matches common school advice.” he said. “So, even though girls can match boys in terms of how well they do at science and mathematics in school, if those aren’t their best subjects and they are less interested in them, then they’re likely to choose to study something else.”
The researchers also looked at how many girls might be expected to choose further study in STEM based on these criteria. They took the number of girls in each country who had the necessary ability in STEM and for whom it was also their best subject and compared this to the number of women graduating in STEM. They found there was a disparity in all countries, but with the gap once again larger in more gender equal countries. In the UK, 29% of STEM graduates are female, whereas 48% of UK girls might be expected to take those subjects based on science ability alone. This drops to 39% when both science ability and interest in the subject are taken into account.
Co-researcher Professor David Geary from the University of Missouri said: “Although countries with greater gender equality tend to be those where women are actively encouraged to participate in STEM, they lose more girls from an academic STEM track who might otherwise choose it, based on their personal academic strengths. Broader economic factors appear to contribute to the higher participation of women in STEM in countries with low gender equality and the lower participation in gender-equal countries.”
Countries with higher gender equality tend also to be welfare states, providing a high level of social security for their citizens, compared to those with lower gender equality which tend to have less secure and more difficult living conditions. Using the UNESCO overall life satisfaction (OLS) figures as a proxy for economic opportunity and hardship, the researchers found that in more gender equal countries, overall life satisfaction was higher.
Professor Stoet said: “STEM careers are generally secure and well-paid but the risks of not following such a path can vary. In more affluent countries where any choice of career feels relatively safe, women may feel able to make choices based on non-economic factors. Conversely, in countries with fewer economic opportunities, or where employment might be precarious, a well-paid and relatively secure STEM career can be more attractive to women.”
Professor Geary adds: “Essentially when you lessen economic concerns, as is the case in gender-equal countries, personal preferences are more strongly expressed. In this situation, sex differences in academic strengths and occupational interests more strongly influence college and career choices, creating the STEM paradox we describe.”
Despite extensive efforts to increase participation of women in STEM, levels have remained broadly stable for decades, but these findings could help target interventions to make them more effective, say the researchers.
“It’s important to take into account that girls are choosing not to study STEM for what they feel are valid reasons, so campaigns that target all girls may be a waste of energy and resources,” said Professor Stoet. “If governments want to increase women’s participation in STEM, a more effective strategy might be to target the girls who are clearly being ‘lost’ from the STEM pathway: those for whom science and maths are their best subjects and who enjoy it but still don’t choose it. If we can understand their motivations, then interventions can be designed to help them change their minds.”
Abstract of the study:
The underrepresentation of girls and women in science, technology, engineering, and mathematics (STEM) fields is a continual concern for social scientists and policymakers. Using an international database on adolescent achievement in science, mathematics, and reading (N = 472,242), we showed that girls performed similarly to or better than boys in science in two of every three countries, and in nearly all countries, more girls appeared capable of college-level STEM study than had enrolled. Paradoxically, the sex differences in the magnitude of relative academic strengths and pursuit of STEM degrees rose with increases in national gender equality. The gap between boys’ science achievement and girls’ reading achievement relative to their mean academic performance was near universal. These sex differences in academic strengths and attitudes toward science correlated with the STEM graduation gap. A mediation analysis suggested that life-quality pressures in less gender-equal countries promote girls’ and women’s engagement with STEM subjects
I found this preprint of a new study via both Stuart Ritchie and Wouter Duyck and it’s quite relevant. Besides some people who still are convinced nurture is almost everything – I’m looking at you, Anders – most of us have experienced that some traits are heritable. But how well are we in guessing this correctly? Well, women with multiply kids are best at it, which seems not so strange to me:
Explanation about the graph:
Comparison of absolute mean difference scores in accuracy of heritability judgments across surveyed human traits for men, women, and full sample for those without children, one child, and two or more children. These estimates represent the magnitude of distance between each participant’s judgment of heritability and the published estimate on a 0 (only environmental factors) to 1 (only genetic factors) scale. Lower mean distance (y-axis) therefore represents better judgment of heritability across all traits. Error bars represent +/- standard error of the mean.
And how did the people predict the different traits?
And this next graphic compares the average prediction with the actual average found in studies:
The fact that genes and environment contribute differentially to variation in human behaviors, traits and attitudes is central to the field of behavior genetics. To the public, perceptions about these differential contributions may affect ideas about human agency. We surveyed two independent samples (N = 301 and N = 740) to assess beliefs about free will, determinism, political orientation, and the relative contribution of genes and environment to 21 human traits. We find that beliefs about the heritability of these traits cluster into four distinct groups, which differentially predict both beliefs about human agency and political orientation. Despite apparent ideological influences on these beliefs, the correspondence between lay judgments of heritabilities and published estimates is large (r = .77). Belief in genetic determinism emerges as a modest predictor of accuracy in these judgments. Additionally, educated mothers with multiple children emerge as particularly accurate in their judgments of the heritabilities of these traits.
I’m still not sure if I think this is amazing or creepy, and I’m guessing it is both: researchers used a magnetic resonance machine to read participants’ minds and find out what song they were listening to. Are we at a dawn of mind reading machines? Well, no. The research is still pretty rudimentary, still: fascinating.
From the press release:
In the experiment, six volunteers heard 40 pieces of classical music, rock, pop, jazz, and others. The neural fingerprint of each song on participants’ brain was captured by the MR machine while a computer was learning to identify the brain patterns elicited by each musical piece. Musical features such as tonality, dynamics, rhythm and timbre were taken in account by the computer.
After that, researchers expected that the computer would be able to do the opposite way: identify which song participants were listening to, based on their brain activity — a technique known as brain decoding. When confronted with two options, the computer showed up to 85% accuracy in identifying the correct song, which is a great performance, comparing to previous studies.
Researchers then pushed the test even harder by providing not two but 10 options (e.g. one correct and nine wrong) to the computer. In this scenario, the computer correctly identified the song in 74% of the decisions.
In the future, studies on brain decoding and machine learning will create possibilities of communication regardless any kind of written or spoken language. “Machines will be able to translate our musical thoughts into songs,” says Sebastian Hoefle, researcher from D’Or Institute and PhD student from Federal University of Rio de Janeiro, Brazil. The study is a result of a collaboration between Brazilian researchers and colleagues from Germany, Finland and India.
According to Hoefle, brain decoding researches provide alternatives to understand neural functioning and interact with it using artificial intelligence. In the future, he expects to find answers for questions like “what musical features make some people love a song while others don’t? Is our brain adapted to prefer a specific kind of music?”
Abstract of the study:
Encoding models can reveal and decode neural representations in the visual and semantic domains. However, a thorough understanding of how distributed information in auditory cortices and temporal evolution of music contribute to model performance is still lacking in the musical domain. We measured fMRI responses during naturalistic music listening and constructed a two-stage approach that first mapped musical features in auditory cortices and then decoded novel musical pieces. We then probed the influence of stimuli duration (number of time points) and spatial extent (number of voxels) on decoding accuracy. Our approach revealed a linear increase in accuracy with duration and a point of optimal model performance for the spatial extent. We further showed that Shannon entropy is a driving factor, boosting accuracy up to 95% for music with highest information content. These findings provide key insights for future decoding and reconstruction algorithms and open new venues for possible clinical applications.
I’m keeping track what schools need to do for Flanders as stated in the media. One that has been popular in the past is preventing childhood obesity. This new British study shows that schools can’t do this in on their own.
From the press release:
The warning comes after one of the largest childhood obesity prevention trials undertaken to date has found that a healthy lifestyle intervention carried out in dozens of schools did not lead to significant changes in pupils’ weight.
Led by the University of Birmingham, the West Midlands ActiVe lifestyle and healthy Eating in School children (WAVES) study was a trial funded by the National Institute for Health Research (NIHR). It aimed to assess the clinical and cost-effectiveness of a programme of activities designed to support children aged six and seven in keeping their weight at a healthy level by promoting healthy eating and physical activity.
Excess weight in childhood is a global problem, affecting around 41 million children under the age of 5 years1. In addition to physical and psychosocial health consequences in these early years, childhood excess weight is an important predictor of obesity in adulthood2, with additional adverse health and economic3 effects. In the UK around a quarter of children have excess weight at school entry (age four to five years)4. The proportion of very overweight children doubles during the subsequent six years (from approximately 9% to 19%)4, highlighting this time period as critical for preventive action.
The 12-month WAVES study intervention included a daily additional 30 minute school-time physical activity opportunity, and a six-week interactive skill based programme in conjunction with a premiership football club. It also included signposting of local family physical activity opportunities through six-monthly mail-outs and termly school led family healthy cooking skills workshops.
Almost 1,500 pupils from 54 state primary schools in the West Midlands took part in the trial. Their measurements — including weight, height, percentage body fat, waist circumference, skinfold thickness, and blood pressure — were taken when they started the trial. They also wore an activity tracker for five days, recorded their dietary intake and took part in assessments to establish their perceived quality of life, social acceptance and body image. These measurements were taken again 15 months and 30 months later and were compared among pupils who were or were not taking part in the intervention.
The results of the randomised controlled trial, published today in The BMJ, found that the intervention did not result in a significant difference in participants’ weight status.
Professor Peymané Adab, of the University of Birmingham’s Institute of Applied Health Research, said: “Our research, combined with wider evidence, suggests that schools cannot lead on the childhood obesity prevention agenda.”
Dr Miranda Pallan, also of the University of Birmingham’s Institute of Applied Health Research, added: “Whilst school is an important setting for influencing children’s health behaviour, and delivery of knowledge and skills to support healthy lifestyles is one of their mandatory functions, widespread policy change and broader influences from the family, community, media and the food industry is also needed.”
The research team added: “although wider implementation of this WAVES study intervention cannot be recommended for obesity prevention, the lower cost components could in the future be considered by schools to fullfil their mandated responsibilities for health and wellbeing education.”
Abstract of the study:
Objective To assess the effectiveness of a school and family based healthy lifestyle programme (WAVES intervention) compared with usual practice, in preventing childhood obesity.
Design Cluster randomised controlled trial.
Setting UK primary schools from the West Midlands.
Participants 200 schools were randomly selected from all state run primary schools within 35 miles of the study centre (n=980), oversampling those with high minority ethnic populations. These schools were randomly ordered and sequentially invited to participate. 144 eligible schools were approached to achieve the target recruitment of 54 schools. After baseline measurements 1467 year 1 pupils aged 5 and 6 years (control: 28 schools, 778 pupils) were randomised, using a blocked balancing algorithm. 53 schools remained in the trial and data on 1287 (87.7%) and 1169 (79.7%) pupils were available at first follow-up (15 month) and second follow-up (30 month), respectively.
Interventions The 12 month intervention encouraged healthy eating and physical activity, including a daily additional 30 minute school time physical activity opportunity, a six week interactive skill based programme in conjunction with Aston Villa football club, signposting of local family physical activity opportunities through mail-outs every six months, and termly school led family workshops on healthy cooking skills.
Main outcome measures The protocol defined primary outcomes, assessed blind to allocation, were between arm difference in body mass index (BMI) z score at 15 and 30 months. Secondary outcomes were further anthropometric, dietary, physical activity, and psychological measurements, and difference in BMI z score at 39 months in a subset.
Results Data for primary outcome analyses were: baseline, 54 schools: 1392 pupils (732 controls); first follow-up (15 months post-baseline), 53 schools: 1249 pupils (675 controls); second follow-up (30 months post-baseline), 53 schools: 1145 pupils (621 controls). The mean BMI z score was non-significantly lower in the intervention arm compared with the control arm at 15 months (mean difference −0.075 (95% confidence interval −0.183 to 0.033, P=0.18) in the baseline adjusted models. At 30 months the mean difference was −0.027 (−0.137 to 0.083, P=0.63). There was no statistically significant difference between groups for other anthropometric, dietary, physical activity, or psychological measurements (including assessment of harm).
Conclusions The primary analyses suggest that this experiential focused intervention had no statistically significant effect on BMI z score or on preventing childhood obesity. Schools are unlikely to impact on the childhood obesity epidemic by incorporating such interventions without wider support across multiple sectors and environments.
This is a claim proposed in a new study published in Frontiers in Psychology and honestly? I have my doubts. I’m not doubting if this claim is true or not, but I’m not sure if the researchers can back their claim by their own research.
What are the claims? (I read the whole study that was heavily influenced by the Selfdeterminationtheory by Deci and Ryan, but I’m taking these quotes from the more accessible press release).
Lead author Dettweiler from the study published in Frontiers in Psychology concludes that outdoor instruction with explorative learning methodology significantly promotes the attitudes of students toward learning, i.e. their intrinsic motivation. ‘Explorative’ means nothing more than simply giving students the freedom to discover the subject matter through independently organized experiments. These outdoor dynamics, which provide a strong boost to more situational interest for science and engagement with the subject, can be evoked in occasional outdoor instruction sessions as well.
The teaching techniques explored and developed for this instructional program should therefore be included as a standard feature of lessons in schools. “Whether it involves rural study centers away from school or forms a part of the science curriculum, or both, this statistical analysis demonstrates that regular outdoor teaching is an appropriate strategy to meet the challenges of the 21st century,” Dettweiler concludes. “Such models might even be suitable to bridge the existing gap between science education and environmental education in the long term.”
What have they done?
Between 2014 and 2016, approximately 300 students participated in the program which is based on the curriculum for science subjects in secondary level I. Students are prepared for the one-week stay in the classroom. This is then continued on site during the research week, culminating in a two-day research expedition with experiments.
Both before and after the course, the students completed a questionnaire on their satisfaction and overall motivation in relation to their autonomy for a study developed at TUM. At the end of the week, the students again shared their experiences during the outdoor class.
Ok, sounds great, pre-test, post-test, but… wait. No control group? And the participants were actually a convenience sample of n = 281 students. And the actual learning wasn’t measured, so the students could be more open to learning, but we can’t tell if they actually have learned more.
And this all was just a one-off experience? I don’t want to shout Hawthorne effect, well, maybe I do want to shout it.
So does outdoor instruction make students more open to learning? Could well be, and this study does suggest it, but it’s hard to tell.