There is a new systematic review study on a topic that is highly relevant to the present situation. High‐quality evidence shows that, on average, school‐based interventions aimed at students who are experiencing, or at risk of, academic difficulties, do improve reading and mathematics outcomes in the short term. This is in line with a previous meta-analysis by Dietrichson et al:
Two instructional methods stand out as being particularly and consistently effective. Both peer‐assisted instruction and small‐group instruction by adults showed the largest (short‐term) improvements in reading and mathematics. Other instructional methods showed smaller improvements however, there is substantial variation in the magnitude of these effects.
Follow‐up outcomes measured more than three months after the end of the intervention pertain almost exclusively to studies examining small‐group instruction and reading. There is evidence of fadeout but positive effects are still reported up to 2 years after the end of intervention. Only five studies measured intervention effects after more than 2 years.
From the press release:
School-based interventions that target students with, or at risk of, academic difficulties in kindergarten to grade 6 have positive effects on reading and mathematics, according to an article published in Campbell Systematic Reviews.
The review analyzed evidence from 205 studies, 186 of which were randomized controlled trials, to examine the effects of targeted school-based interventions on students’ performance on standardized tests in reading and math.
Peer-assisted instruction and small-group instruction by adults were among the most effective interventions. The authors noted that these have substantial potential to boost skills in students experiencing academic difficulties.
“It is exciting to see that there are many interventions with substantial impacts on math and reading skills, especially in these times when many students have not been able to attend school and the number of students who need extra help may be even larger than usual,” said lead author Jens Dietrichson, PhD, of VIVE, the Danish Center for Social Science Research. “It is also interesting that there is large variation: far from all interventions have positive effects, and there are substantial and robust differences between the types of interventions. Thus, schools can boost the skills of students with difficulties by implementing targeted interventions, but it matters greatly how they do it.”
Abstract of the review:
Low levels of numeracy and literacy skills are associated with a range of negative outcomes later in life, such as reduced earnings and health. Obtaining information about effective interventions for children with or at risk of academic difficulties is therefore important.
The main objective was to assess the effectiveness of interventions targeting students with or at risk of academic difficulties in kindergarten to Grade 6.
We searched electronic databases from 1980 to July 2018. We searched multiple international electronic databases (in total 15), seven national repositories, and performed a search of the grey literature using governmental sites, academic clearinghouses and repositories for reports and working papers, and trial registries (10 sources). We hand searched recent volumes of six journals and contacted international experts. Lastly, we used included studies and 23 previously published reviews for citation tracking.
Studies had to meet the following criteria to be included:
– Population: The population eligible for the review included students attending regular schools in kindergarten to Grade 6, who were having academic difficulties, or were at risk of such difficulties.
– Intervention: We included interventions that sought to improve academic skills, were conducted in schools during the regular school year, and were targeted (selected or indicated).
– Comparison: Included studies used an intervention‐control group design or a comparison group design. We included randomised controlled trials (RCT); quasi‐randomised controlled trials (QRCT); and quasi‐experimental studies (QES).
– Outcomes: Included studies used standardised tests in reading or mathematics.
– Setting: Studies carried out in regular schools in an OECD country were included.
Data Collection and Analysis
Descriptive and numerical characteristics of included studies were coded by members of the review team. A review author independently checked coding. We used an extended version of the Cochrane Risk of Bias tool to assess risk of bias. We used random‐effects meta‐analysis and robust‐variance estimation procedures to synthesise effect sizes. We conducted separate meta‐analyses for tests performed within three months of the end of interventions (short‐term effects) and longer follow‐up periods. For short‐term effects, we performed subgroup and moderator analyses focused on instructional methods and content domains. We assessed sensitivity of the results to effect size measurement, outliers, clustered assignment of treatment, risk of bias, missing moderator information, control group progression, and publication bias.
We found in total 24,414 potentially relevant records, screened 4247 of them in full text, and included 607 studies that met the inclusion criteria. We included 205 studies of a wide range of intervention types in at least one meta‐analysis (202 intervention‐control studies and 3 comparison designs). The reasons for excluding studies from the analysis were that they had too high risk of bias (257), compared two alternative interventions (104 studies), lacked necessary information (24 studies), or used overlapping samples (17 studies). The total number of student observations in the analysed studies was 226,745. There were 93% RCTs among the 327 interventions we included in the meta‐analysis of intervention‐control contrasts and 86% were from the United States. The target group consisted of, on average, 45% girls, 65% minority students, and 69% low‐income students. The mean Grade was 2.4. Most studies included in the meta‐analysis had a moderate to high risk of bias.
The overall average effect sizes (ES) for short‐term and follow‐up outcomes were positive and statistically significant (ES = 0.30, 95% confidence interval [CI] = [0.25, 0.34] and ES = 0.27, 95% CI = [0.17, 0.36]), respectively). The effect sizes correspond to around one third to one half of the achievement gap between fourth Grade students with high and low socioeconomic status in the United States and to a 58% chance that a randomly selected score of an intervention group student is greater than the score of a randomly selected control group student.
All measures indicated substantial heterogeneity across short‐term effect sizes. Follow‐up outcomes pertain almost exclusively to studies examining small‐group instruction by adults and effects on reading measures. The follow‐up effect sizes were considerably less heterogeneous than the short‐term effect sizes, although there was still statistically significant heterogeneity.
Two instructional methods, peer‐assisted instruction and small‐group instruction by adults, had large and statistically significant average effect sizes that were robust across specifications in the subgroup analysis of short‐term effects (ES around 0.35–0.45). In meta‐regressions that adjusted for methods, content domains, and other study characteristics, they had significantly larger effect sizes than computer‐assisted instruction, coaching of personnel, incentives, and progress monitoring. Peer‐assisted instruction also had significantly larger effect sizes than medium‐group instruction. Besides peer‐assisted instruction and small‐group instruction, no other methods were consistently significant across the analyses that tried to isolate the association between a specific method and effect sizes. However, most analyses showed statistically significant heterogeneity also within categories of instructional methods.
We found little evidence that effect sizes were larger in some content domains than others. Fractions had significantly higher associations with effect sizes than all other math domains, but there were only six studies of interventions targeting fractions. We found no evidence of adverse effects in the sense that no method or domain had robustly negative associations with effect sizes.
The meta‐regressions revealed few other significant moderators. Interventions in higher Grades tend to have somewhat lower effect sizes, whereas there were no significant differences between QES and RCTs, general tests and tests of subdomains, and math tests and reading tests.
Our results indicate that interventions targeting students with or at risk of academic difficulties from kindergarten to Grade 6 have on average positive and statistically significant short‐term and follow‐up effects on standardised tests in reading and mathematics. Peer‐assisted instruction and small‐group instruction are likely to be effective components of such interventions.
We believe the relatively large effect sizes together with the substantial unexplained heterogeneity imply that schools can reduce the achievement gap between students with or at risk of academic difficulties and not‐at‐risk students by implementing targeted interventions, and that more research into the design of effective interventions is needed.