by Emily Sholtis & Matthew Pepper, Basis Policy Research
This blog has featured several articles exploring the impacts of chess on school-aged children. The effects have ranged from improved scores of math assessments, increased cognitive abilities, and stronger social skills. However, new chess research out of England suggests chess may not affect student outcomes at all.
In their new article, researcher John Jerrim and his team from the University of London conducted a large, rigorous effectiveness study of an in-school chess intervention and its impact on student performance. The researchers tested whether a year-long, one-hour-per-week chess intervention had a statistically significant impact on students’ academic performance. The study, funded by Chess in Schools and Communities (CSC), randomly assigned an equal number of 100 participating schools to treatment and control groups. These 100 schools represented 11 school districts from all over England and enrolled 4,009 5th grade children that participated in the study.
The treatment intervention consisted of 30 one-hour lessons delivered by an experienced chess tutor, which replaced elective courses such as physical education or music. The intervention was designed so that students were to receive 15 minutes of instruction followed by 45 minutes of play for one hour a week. This intervention was delivered throughout the 2013-2014 school year ending in June of 2014. The research team then collected testing data on their student sample, a full year after the treatment, in June of 2015. They purposely chose to delay data collection in an effort to see if the year of chess lessons had a lasting impact on students’ academic performance. Test scores from the treatment group were compared to test scores from the control group in order to assess the effect of CSC’s chess intervention. The team also included analyses of implementation quality and student participation, both of which were not found to significantly impact the RCT’s results. The final analysis of this intervention found that the program had no statistically significant impact on student performance in any subject area, including math.
Overall, this study and its findings are a valuable contribution to the wider body of literature on the impact of chess on student achievement. The study design and level of rigor are commendable, and future studies should build on its lessons of implementation. Particularly, future work should carefully consider:
-
Dosage –Participating students were supposed to receive one hour of chess instruction per week for thirty weeks. The appropriate scope of chess integration in schools remains an important question, and this study suggests that an hour per week may be too low.
-
Fadeout – The impact of the treatment was measured a full year after the intervention, which might be explained by a fadeout of the impact. Ideally metrics are available to measure the impact at the immediate conclusion of the intervention and a year later.
-
The Training of Coaches & Teachers – Lessons were delivered by tutors, who had varying background and teaching experience. Support was sometimes provided by classroom teachers. Tutors received from zero days of training to up to several weekends of training seminars. The extent of tutor training on the CSC curriculum and their teaching experience varied widely, and survey results suggest that many struggled with classroom management.
-
Opportunity Costs – Schools selected what chess replaced, with many replacing a “topics/humanities” lesson but others replacing a math lesson. It is not clear how replacing a class in order to introduce chess impacts student achievement scores, but future studies should also carefully consider the differential impacts by how chess is incorporated into the school day.
-
Implementation Fidelity – Only three out of the five treatment schools reached the goal of providing all thirty hours of chess instruction. This study also analyzed carefully collected qualitative data, which allows for a richer investigation into implementation challenges. For example, the study found that classes were less effective when the teacher did not actively collaborate with the chess tutor. There are numerous lessons within this study for school-centered chess programs that are considering scaling up.
Overall, the body of chess research still finds a moderate, but positive statistically significant effect for chess interventions. In a meta-analysis of 24 chess studies conducted by Basis Policy Research, we find an average effect size of 0.316 on students’ academic performance and 0.571 on math performance. A similar meta-analysis, also consisting of 24 studies, conducted by Sala finds effect sizes of 0.338 and 0.382 for overall and math scores respectively. This new, high-quality study is an important contribution but also should be viewed in the context of the greater body of chess research.
This study by Jerrim et al. is valuable because of its rigorous design, size, and mixed-methods approach. This represents a huge accomplishment in chess research and is the type of study the CCSCSL has been advocating for. We eagerly await the publication of more high quality research like this evaluation in the future.
