Philosophy of Education
Joyce, K. & Cartwright, N. (2019) Bridging the Gap Between Research and Practice: Predicting What Will Work Locally, American Educational Research Journal 57 (3).
Abstract: This article addresses the gap between what works in research and what works in practice. Currently, research in evidence-based education policy and practice focuses on randomized controlled trials. These can support causal ascriptions (‘‘It worked’’) but provide little basis for local effectiveness predictions (‘‘It will work here’’), which are what matter for practice. We argue that moving from ascription to prediction by way of causal generalization (‘‘It works’’) is unrealistic and urge focusing research efforts directly on how to build local effectiveness predictions. We outline various kinds of information that can improve predictions and encourage using methods better equipped for acquiring that information. We compare our proposal with others advocating a better mix of methods, like implementation science, improvement science, and practice-based evidence.
Joyce, K. (2019) The Key Role of Representativeness in Evidence-based Education Educational Research and Evaluation 25 (3-4).
(Reprinted in The Evidential Basis of Evidence-based Education, Routeledge 2020; ISBN 9780367520335)
Abstract: Within evidence-based education, results from randomised controlled trials (RCTs), and meta-analyses of them, are taken as reliable evidence for effectiveness – they speak to “what works”. Extending RCT results requires establishing that study samples and settings are representative of the intended target. Although widely recognised as important for drawing causal inferences from RCTs, claims regarding representativeness tend to be poorly evidenced. Strategies for demonstrating it typically involve comparing observable characteristics (e.g., race, gender, location) of study samples to those in the population of interest to decision makers. This paper argues that these strategies provide insufficient evidence for establishing representativeness. Characteristics typically used for comparison are unlikely to be causally relevant to all educational interventions. Treating them as evidence that supports extending RCT results without providing evidence demonstrating their relevance undermines the inference. Determining what factors are causally relevant requires studying the causal mechanisms underlying the interventions in question.
Joyce, K. & Cartwright, N. (2018) Meeting Our Standards for Educational Justice: Doing Our Best with the Evidence with Nancy Cartwright in Theory and Research in Education 16 (1).
Abstract: The United States considers educating all students to a threshold of adequate outcomes to be a central goal of educational justice. The No Child Left Behind Act introduced evidence-based policy and accountability protocols to ensure that all students receive an education that enables them to meet adequacy standards. Unfortunately, evidence-based policy has been less effective than expected. This article pinpoints under-examined methodological problems and suggests a more effective way to incorporate educational research findings into local evidence-based policy decisions. It identifies some things educators need to know and do to determine whether available interventions can play the right casual role in their setting to produce desired effects. It examines the value and limits of educational research, especially randomized controlled trials, for this task.
Works in Progress
"Using Evidence of Effectiveness to Narrow Achievement Gaps"
Abstract: The evidence-based approach to school improvement is presented as a strategy for advancing aims of educational justice in the U.S. It’s supposed to narrow achievement gaps while simultaneously raising achievement across all students. That means students at the low end of the gap would have to improve faster than students at the high end. I argue that the current model is unlikely to narrow achievement gaps between racial and socioeconomic groups by facilitating faster gains for disadvantaged groups. The idea that it can do so stems from dubious assumptions about the quality of schools serving disadvantaged students and the relevance of factors outside of school. I suggest that the evidence-based approach could be used to narrow achievement gaps if the research agenda focused on interventions and strategies that address disadvantage or its influence on learning at school.
"Evidence-based Education and Fair Accountability Practices"
Abstract: In addition to the evidence-based approach to school improvement, the No Child Left Behind Act introduced protocols to hold educators accountable for their students’ outcomes. It’s only fair to hold someone accountable for an outcome if they have sufficient control over that outcome. Does access to evidence-based educational interventions give educators sufficient control to justify holding them accountable for their students’ academic outcomes (usually measured by test scores)? The best argument for the fairness of the accountability protocols appears to be valid, but unsound because it relies on a premise that overstates the causal claims warranted by experimental research. However, replacing that premise with a weaker, true, claim seems to render the argument invalid. I consider potential harms that may befall educators and students if we fail to see that this crucial premise is false and explore how we could make accountability practices fair.