What explains the increase in math scores while teachers use an old method during a pilot program?

The John Henry Effect refers to the phenomenon where individuals increase their performance in response to the awareness that they are being observed or compared to others. In the context of a pilot program where teachers are using an old method, if teachers and students are aware that their progress is being monitored, they might exert more effort to improve their math scores than they typically would. This heightened motivation and effort can lead to an increase in scores, regardless of the instructional method being employed.

This effect can be particularly pronounced in educational settings where students may feel motivated to prove their capabilities or strive for better outcomes, knowing that their performance is being evaluated as part of a study or pilot program. The older teaching method combined with the increased motivation from the John Henry Effect can lead to the surprising result of improved math scores.