The largest research study ever conducted on summer learning in urban schools shows mixed results

Elementary school students with high levels of attendance in voluntary summer learning programs – defined as at least 20 days of a five- to six-week program – shows benefits according to new RAND Corporation findings from the largest research study ever conducted on summer learning:

  • There is promising evidence that students with high attendance in one summer benefited in mathematics and that these benefits persisted through the following spring.
  • There is promising evidence that students with high attendance in the second summer benefited in mathematics and language arts and that these effects persisted.
  • There is promising evidence that students with high attendance in the second summer benefited in terms of social-emotional outcomes.
  • There is promising evidence that students with high levels of academic time on task benefited from the programs and that these benefits persisted in both mathematics and language arts.
  • There is promising evidence that students who received high-quality language arts instruction benefited from the programs.

It’s a huge research study, but… it’s hard to tell what cause and what is consequence. The results I just mentioned are correlations, but what are the reasons pupils are having a higher level of attendance?

The study did examine some causal relations too, but the effects give much less hope:

  • Summer programs produced a modest near-term benefit in mathematics that dissipated by the next fall.
  • There was no causal evidence that the programs produced benefits in language arts, social emotional outcomes, or student attendance or grades during the school year.
  • There was no causal evidence that offering two summers of programming provided benefits.

From the press release:

The $50 million National Summer Learning Project, funded by The Wallace Foundation since 2011, seeks to find out whether and how voluntary summer programs can help low-income students succeed in school. Summer is a time when low-income students lose ground relative to their wealthier peers, but it also holds promise as a time to improve outcomes for them by providing additional opportunities for academics and enrichment.

Until this study, little research had been conducted about the effectiveness of voluntary summer learning programs led by urban districts and offered to large numbers of low-income students.

The five school districts in the project are Boston, Dallas, Duval County, Florida, Pittsburgh and Rochester, New York. Each offered five to six weeks of free summer programs that included enrichment activities and instruction in mathematics and English language arts.

The new results are on the impact of programs during summers in 2013 and 2014. School-year outcomes measured included grades, achievement test scores in math and language arts, measures of social and emotional skills, and school-year attendance and suspension rates. The project will track student outcomes through spring 2017.

RAND researchers conducted a randomized controlled trial, comparing the results for students who participated in the program with results for students who did not. Researchers also compared specific groups of students – such as high attenders with at least 20 days of attendance – to the control group. The analysis summarizes the outcomes for 3,192 students accepted into the programs who had completed third grade before the first summer.

After the first summer, students who attended at least 20 days outperformed the control group in math, and the improvements persisted through the school year. After the second summer, these high attenders outperformed the control group of students in math and reading, both in fall 2014 and in the following spring. These findings are correlational, but are controlled for prior achievement and demographics, giving researchers confidence that the benefits are likely due to the programs and meeting the requirements for promising evidence under the federal Every Student Succeeds Act.

The academic advantage for the students with high attendance levels after the second summer translates to between 20 and 25 percent of the typical annual gains in mathematics and reading, the study found.

High-attending students also were rated by teachers as having stronger social and emotional competencies than the control group students. However, researchers have less confidence that this was caused by the programs, given the lack of prior data on these competencies.

Among the students who attended for at least one day each summer, 60 percent reached the 20-day threshold associated with benefits. Average daily attendance was 76 percent, lower than the typical 96 percent during the school year.

“Our study clearly shows the benefits for the students who had high attendance rates or high amounts of academic instruction in the summer learning programs,” said Catherine H. Augustine, the study’s lead author and senior policy researcher at RAND, a nonprofit research organization. “To help as many students as possible reap these benefits, the study suggests that districts run summer programs for at least five weeks, include sufficient time on academics, and focus on the challenge of achieving high attendance rates.”

“Until now, we didn’t know if urban school districts could offer high-quality summer learning programs for low-income students and whether they would make a difference for children,” said Will Miller, president of The Wallace Foundation. “We have learned two important things: That high-quality summer learning programs are capable of helping disadvantaged students succeed in school, and that high attendance is crucial to delivering these benefits.”

Because nearly 80 percent of students with high levels of attendance in 2014 also were high attenders in 2013, it is not clear whether the broader positive outcomes for high attenders after the second summer are the result of cumulative attendance over both summers or program improvements in the second summer, and researchers concluded outcomes likely reflected a combination of both.

The school districts took different approaches to their summer programs, but all provided at least three hours daily of academic instruction in math and reading by certified teachers, along with enrichment activities such as art, music, tennis and swimming.

For students to experience lasting benefits from attending summer programs, the report recommends that districts run programs for at least five weeks, promote consistent attendance, include sufficient instructional time and protect it, invest in instructional quality, and factor in attendance and likely no-show rates when staffing the programs in order to lower per-student costs.

The report, “Learning from Summer: Effects of Voluntary Summer Learning Programs on Low-Income Urban Youth,” along with accompanying research brief and infographic, is available athttp://www.rand.org and http://www.wallacefoundation.org.

Advertisements

2 Comments

Filed under At home, Research

2 responses to “The largest research study ever conducted on summer learning in urban schools shows mixed results

  1. Pingback: Research Digest – September 2016

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s