While I read this book, my oldest son was waiting for his grades. We had been noticing that he became steadily more nervous the past days. But what do his grades tell us? After reading the new book by Daniel Koretz, you won’t be that sure if those grades will tell me anything besides what my son did on those tests.
Daniel Koretz has been dedicating a large part of his academic career to the effects of high-stakes testing and standardized testing. While often used as synonyms, they are not. ‘Standardized test’ says something about the test in itself, it is a test that is administered and scored in a consistent, or “standard”, manner. The concept of high stakes testing says something about why the test is being used, e.g. to be allowed to go to the university. A third concept often incorrectly used as a synonym are centralized tests, which puts the focus on who organizes the test, but not about how standardized or how high stake a test is. Standardized tests can be used for any given age, and while with high stake testing people often refer to final exams, they can also exist for any given age.
To make this concrete let’s use these concepts to discuss the French Baccalauréat, one could say it is centralized, as the different tests are the same for all participants. The stakes of the Bac are reasonable high as they have an influence on university-admissions, but the stakes are not as high as e.g. the UK of some of the Asian Countries because not passing your BAC doesn’t necessarily mean the end of the line in higher education. And it can be argued that not all the different subjects in the Baccalauréat are standardized, as marking essays is very hard to standardize.
In his new book Koretz goes to a great length to explain how testing in itself isn’t necessarily a bad thing. It can give both children and teachers information about the learning process. But when stakes get too high, e.g. if the future of the children, the teachers or the schools depend on the results, a lot can go wrong. High stakes for children can mean, e.g. in South Korea, that if you don’t have maximum grades for the central exams, you won’t be able to reach for the SKY, the acronym of the 3 top universities in Korea. Maximum grades mean your ticket to a solid and great future. Not having those grades mean a much lesser chance on success.
High stakes for teachers can mean being sacked if certain results aren’t met or rewards if your pupils do better than expected. Koretz describes how schools in the US sometimes get more money from their state if they perform better than the given targets, targets who are checked by administering standardized tests on pupils from different age groups. This all can seem reasonable if you want pupils, teachers and schools to do better, but Koretz warns in his book that you may end with a lot of stress, even less learning and fraud.
A central theme in the book is Campell’s law. Donald T. Campbell stated that “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” What does this law mean for standardized testing? Let’s pick some examples from the book.
What will happen if teachers know that chapter 5 will not be that important in a centralized math test? This can be for many different reasons, not necessarily because it is not important to real life, but maybe because it is more difficult to assess. There is a big chance that teachers will pay less attention to the content of chapter 5, spending more time on e.g. chapter 4 that will be a major part of the test.
What happens within a subject, e.g. maths, can also happen between subjects. If reading and mathematics are key in high stakes centralized exams, often schools will opt to pay less time to subjects such as history, arts of science until those subjects become important in those tests. In the book Koretz describes rather anecdotally how some schools even skipped recess for their children and closed the art rooms to replace them with math labs to prep their pupils for the tests.
Even worse, this all can lead to grade inflation. Test scores may rise because of the strong focus of students, teachers and schools on those tests, but this doesn’t necessarily mean that they become better in the subject but only in what is being tested. Koretz did interesting research on this in an unnamed state by comparing two tests on the same subject, the official state organized test and a test the state used to use, making it possible to compare. While the new test had shown an increase during the past 3 years, the students didn’t do better on the older test that was used before.
It gets really ugly, when Koretz describes several examples of fraud, often performed by teachers and schools because their job and school depend on it. Koretz describes it with a sense of compassion with the perpetrators, as they often are confronted with unreasonable high targets, impossible to meet in an honest way.
Most – not all – examples in Koretz’ book are taken from the United States, which may mean a hurdle to take for people from outside the US. But what the academic describes is relevant for anybody thinking about educational reform and who is looking at making pupils, teachers and schools more accountable, you may get not what you wished for. I’ve noticed that people who read this book, tend to overlook the possible, positive elements of standardized testing. Koretz also discusses the benefits of regular testing for the learning process. Testing in this sense is a mean to give formative assessment. Testing should also therefore be a part of a larger, more holistic approach to education, without the high stakes that as Campbell noted often can corrupt education.