There is a new Best Evidence in Brief and this time I pick a study that shows that it is also interesting to know if something doesn’t have the expected effects:
A recent study from the Education Endowment Foundation in the UK found disappointing results for a Saturday school designed to improve the reading and math achievement of underachieving and disadvantaged students in Key Stage 2 (the equivalent of 2nd through 5th grade in the U.S.).
Developed by the SHINE Trust and Hallé Orchestra, the intervention provided additional school-based literacy and numeracy lessons, based on musical themes, as well as visits to Hallé rehearsals, performances, and other theme-based activities. Twenty-five Saturday sessions, each lasting five hours, were planned for the intervention over the course of an academic year, delivered by qualified teachers, teaching assistants, peer mentors, and professional musicians.
The evaluation, by Victoria White and colleagues from the University of Durham, consisted of two randomized controlled trials (RCTs)-a pilot trial and a main trial-and a process evaluation. The pilot trial involved 361 Year 5 and 6 students (4th and 5th grade in the U.S.) in 18 schools; the main trial involved 2,306 Year 4, 5, and 6 students in 38 schools.
There was no evidence that the program had an impact on reading or math achievement, or attitudes toward reading, math, music, and school, of the children in the trial.
Attendance of eligible students was often low and considered as a barrier to successful implementation. Reasons for low attendance included students’ lack of availability to attend the Saturday sessions, variable parental engagement with the program, and limited time at the beginning of the program for schools to engage children and parents.
The process evaluation revealed a positive picture of involvement and engagement for those students who attended the Saturday school activities. Evaluators observed good working relationships between the teachers and students, and positive and purposeful learning environments in lessons. All stakeholders felt students were making noticeable improvements in behavior, confidence, and development of social skills.