Yesterday I discovered this new, impressive paper by Barbara Biasi and Song Ma, The Education-Innovation Gap. In this paper they want to answer the question from the title of this post. Or more correct:
…we want to measure the distance between the knowledge content of each course – as described in its syllabus – and the knowledge frontier, represented by top academic articles recently published in the course’s field.
But how can they do this? They use a new metric:
To quantify this distance, we develop a new metric: the education-innovation gap, designed to capture the similarity between the content of a course and older knowledge (contained in articles published decades ago) relative to new, frontier knowledge (contained in recently published articles). For example, a Computer Science course that teaches Visual Basic (an obsolete programming language) in 2018 would have a larger gap than a course that teaches Julia (a recent and updated programming language), because Visual Basic is more frequent among old academic articles and Julia is more frequent among recent articles.1
And what does this metric mean?
On average, the education-innovation gap (which we multiply by 100 for simplicity) is equal to 95, which indicates that courses tend to be more similar to newer than to older research. However, a significant amount of variation exists across syllabi. Simulations where we manually change the content of each syllabus indicate that, in order to move a syllabus from the 25th percentile (92) to the 75th percentile (99) of the gap distribution, we would have to replace approximately 74 percent of its content with “newer” knowledge, i.e., words that are most frequent in recent publications. A variance decomposition exercise indicates that differences across fields explain 4 percent of the total variation in the gap; differences across schools explain an additional 2 percent, differences across courses explain 29 percent, and differences across instructors explain 40 percent of the variation.
What are the results?
Our analyses of student outcomes indicate that a lower education-innovation gap at the school level is associated with improved academic and economic outcomes of the students at each school, such as graduation rates and incomes after graduation. The lack of experimental variation in the gap across schools prevents us from pinning down a causal relationship with certainty. Nevertheless, our results are robust to the inclusion of controls for a large set of school and student characteristics, indicating that these correlations are unlikely to be driven by cross-school differences in spending, selectivity, major composition, or parental background. Thee findings point to a potentially important role for up-to-date instruction on the outcomes of students as they exit college and enter the labor market.