Philosophy of Education

My primary research in this area concerns the use of evidence-based policy/practice in education. My four publications on this topic (three coauthored with Nancy Cartwright) focus on methodological and epistemological issues; the papers I'm currently working on focus on ethical issues.

In 2001, the No Child Left Behind Act (NCLB) introduced an evidence-based strategy for school improvement to narrow achievement gaps and raise achievement across all students. Essentially, it seeks to improve the quality of schools nationwide by encouraging them to base their instructional decisions on scientific research that speaks to the causal effectiveness of available interventions (e.g., programs, practices). The U.S. Institute of Education Sciences sponsors rigorous experimental research to identify effective educational interventions. Schools nationwide are encouraged to implement these 'evidence-based' interventions with fidelity to produce similar effects, thereby improving student outcomes and opportunities for learning. Unfortunately, evidence-based education has yet to yield the intended results. This is partly because evidence-based interventions that produce positive effects in research settings often fail to produce similar effects when used in practice. This is known as the ‘research-practice gap.’ Nancy Cartwright and I argue that this gap stems in part from unsupported causal inferences and discuss ways in which the knowledge produced by various research methods can evidence the causal claims relevant to decision-making within education.


Publications

Joyce, K. & Cartwright, N. (forthcoming) "How Should Evidence Inform Education Policy?" in Routledge Handbook of Philosophy of Education.

This chapter explores how evidence from various sources can support education policy decisions. Although policy arguments include some normative premises, we focus on the evidence needed to support their descriptive premises, homing in on predictions about how candidate policies are likely to perform in specific target sites. Although evidence from RCTs is viewed as the gold-standard, we argue that it’s insufficient and unnecessary for predictions about education policies. Trustworthy predictions require information about how the policy operates, the conditions under which it can do so, and the conditions present in the target setting, which comes from a mix of research methods, theory, and local sources. This evidence is also useful for feasibility assessments and implementation planning.

Joyce, K. & Cartwright, N. (2019) Bridging the Gap Between Research and Practice: Predicting What Will Work Locally, American Educational Research Journal 57 (3).

Abstract: This article addresses the gap between what works in research and what works in practice. Currently, research in evidence-based education policy and practice focuses on randomized controlled trials. These can support causal ascriptions (‘‘It worked’’) but provide little basis for local effectiveness predictions (‘‘It will work here’’), which are what matter for practice. We argue that moving from ascription to prediction by way of causal generalization (‘‘It works’’) is unrealistic and urge focusing research efforts directly on how to build local effectiveness predictions. We outline various kinds of information that can improve predictions and encourage using methods better equipped for acquiring that information. We compare our proposal with others advocating a better mix of methods, like implementation science, improvement science, and practice-based evidence.

Joyce, K. (2019) The Key Role of Representativeness in Evidence-based Education Educational Research and Evaluation 25 (3-4).

(Reprinted in The Evidential Basis of Evidence-based Education, Routeledge 2020; ISBN 9780367520335)

Abstract: Within evidence-based education, results from randomised controlled trials (RCTs), and meta-analyses of them, are taken as reliable evidence for effectiveness – they speak to “what works”. Extending RCT results requires establishing that study samples and settings are representative of the intended target. Although widely recognised as important for drawing causal inferences from RCTs, claims regarding representativeness tend to be poorly evidenced. Strategies for demonstrating it typically involve comparing observable characteristics (e.g., race, gender, location) of study samples to those in the population of interest to decision makers. This paper argues that these strategies provide insufficient evidence for establishing representativeness. Characteristics typically used for comparison are unlikely to be causally relevant to all educational interventions. Treating them as evidence that supports extending RCT results without providing evidence demonstrating their relevance undermines the inference. Determining what factors are causally relevant requires studying the causal mechanisms underlying the interventions in question.

Joyce, K. & Cartwright, N. (2018) Meeting Our Standards for Educational Justice: Doing Our Best with the Evidence with Nancy Cartwright in Theory and Research in Education 16 (1).

Abstract: The United States considers educating all students to a threshold of adequate outcomes to be a central goal of educational justice. The No Child Left Behind Act introduced evidence-based policy and accountability protocols to ensure that all students receive an education that enables them to meet adequacy standards. Unfortunately, evidence-based policy has been less effective than expected. This article pinpoints under-examined methodological problems and suggests a more effective way to incorporate educational research findings into local evidence-based policy decisions. It identifies some things educators need to know and do to determine whether available interventions can play the right casual role in their setting to produce desired effects. It examines the value and limits of educational research, especially randomized controlled trials, for this task.

Works in Progress

"Using Evidence of Effectiveness to Narrow Achievement Gaps"

Abstract: The evidence-based approach to school improvement is presented as a strategy for advancing aims of educational justice in the U.S. It’s supposed to narrow achievement gaps while simultaneously raising achievement across all students. That means students at the low end of the gap would have to improve faster than students at the high end. I argue that the current model is unlikely to narrow achievement gaps between racial and socioeconomic groups by facilitating faster gains for disadvantaged groups. The idea that it can do so stems from dubious assumptions about the quality of schools serving disadvantaged students and the relevance of factors outside of school. I suggest that the evidence-based approach could be used to narrow achievement gaps if the research agenda focused on interventions and strategies that address disadvantage or its influence on learning at school.

"Evidence-based Education and Fair Accountability Practices"

Abstract: In addition to the evidence-based approach to school improvement, the No Child Left Behind Act introduced protocols to hold educators accountable for their students’ outcomes. It’s only fair to hold someone accountable for an outcome if they have sufficient control over that outcome. Does access to evidence-based educational interventions give educators sufficient control to justify holding them accountable for their students’ academic outcomes (usually measured by test scores)? The best argument for the fairness of the accountability protocols appears to be valid, but unsound because it relies on a premise that overstates the causal claims warranted by experimental research. However, replacing that premise with a weaker, true, claim seems to render the argument invalid. I consider potential harms that may befall educators and students if we fail to see that this crucial premise is false and explore how we could make accountability practices fair.