Recent research by Vanitha Virudachalam, an assistant professor of business administration at Gies College of Business, and her coauthors Sergei Savin (University of Pennsylvania) and Matthew P. Steinberg (Accelerate) reveals that mid-year, or interim, testing of students can have limited utility. They looked specifically at the impact of interim assessments in the presence of merit-based incentives. In that context they found that the interim tests could prove to be a disincentive for teachers.

Student testing has become increasingly important to schools since the passage of No Child Left Behind (NCLB) in the early 2000s. Schools have traditionally utilized two types of assessment:

• Summative assessments – Tests administered at the end of an instruction period and that can be aggregated to evaluate a larger entity (e.g., annual state tests used to evaluate schools).
• Formative assessments – Ongoing assessments to give teachers feedback on student progress (e.g., an impromptu poll or a short quiz).

With the emphasis on high-stakes annual testing that stemmed from NCLB, school district leaders increasingly turned to interim assessments, a newer type lying somewhere in between summative and formative. These can be used to evaluate students against achievement goals with the intent of providing guidance to classroom teaching, district decision-making, or both. The importance of testing is reflected in the growth of the third-party testing industry. According to Virudachalam, spending on assessments grew from $434 million in 2001-2002 to more than $1.63 billion in 2018-2019.

Virudachalam and her coauthors examined these interim assessments in their article “Too Much Information: When Does Additional Testing Benefit Schools?” In particular, given the explosive growth of the testing industry, they were interested in determining whether these assessments were indeed a valuable investment for school districts.

Principal-agent design

Their paper uses a principal-agent model, which examines the relationship between a principal (in this case, the school district) and an agent (the group of teachers at a school). In their model, the district decides whether to implement interim tests to gain more precise information about student progress before an end-of-year assessment, where teachers will receive merit-based bonuses tied to student performance on those same standardized tests.

Using a two-period model – each period representing half of the school year – there were three points of measure for student proficiency: the beginning of the year, halfway through the year (the time of an interim test), and the end of the year (the time of the state standardized test). They allowed for two different starting states in their model. In one, students begin the year on track to achieve end-of-year proficiency; in the other, students start the year already behind where they should be.

Virudachalam explained: “Our model looks at how the state of student proficiency evolves when you have additional information that the district can choose to provide to teachers in the form of an interim assessment – additional, often third-party, tests – and how that interacts with also providing teachers with merit-based rewards, based on student performance on the end of the year state standardized tests. This combination of interim assessments and merit-based rewards is something that some school district utilize, so we wanted to understand whether this was a good use of resources.”

Testing shows mixed results

Virudachalam and her coauthors found that in the presence of a merit-based reward, interim assessments had limited usefulness. This is largely driven by the the fact that accurate information, when paired with merit-based incentives, is a double-edged sword: positive midyear results can make it easier to incentivize teacher effort, but negative results can have a demotivating effect. For low-performing schools – that is, those that began the year behind and which therefore may receive more interventions – they find that the district is most likely to benefit from interim assessments in two cases: (a) when the budget is small and teachers are much more likely to exert effort after an accurate positive midyear result, and (b) when the budget is moderate and teacher effort is less dependent on the merit-based incentives.

Even if the school was on track from the beginning of the year to achieve proficiency, the researchers found that these interim tests provided useful information in fairly limited situations. “It would be natural to assume that more insight into student performance throughout the year and merit-based incentives for teachers are both good things for schools,” Virudachalam said. “However, in our study, we find that the combination of these two programs may lead to unexpected results. For example, highly accurate negative midyear assessment results can be discouraging to teachers, particularly in the absence of additional support to help them effectively address student needs. Additionally, even when interim assessments may lead to better student outcomes, this may strain school budgets if offered in conjunction with merit-based rewards. Our results suggest that districts offering both interim assessments and merit-based incentives should think carefully about the implications of these programs.”

Part of public sector research

This paper is an example of Virudachalam’s interest in studying the public sector and how incentives in general impact those types of operations. That is where this research on education and testing fits in. “Education is an area that is less studied in our field,” she said. “So, this paper is a little bit unusual in terms of that context. We were lucky to have an editorial team that believed in the value of the paper.”