Sign up for PRRAC’s biweekly newsletter here.
Excerpted from Poverty & Race, Volume 31, No.2 (Oct-Dec 2022)
By Peter Piazza and James Noonan
When he signed the No Child Left Behind (NCLB) Act into law in January 2002, President George W. Bush perhaps knew that the new federally-mandated testing requirements would not be popular. In the auditorium of an Ohio high school, he insisted, however, that the tests were necessary. “I understand taking tests aren’t fun,” he said. “Too bad. We need to know in America. We need to know whether or not children have got the basic education.” In the time since then, the tests indeed have taught us a lot, though not always in the way that Bush anticipated. Instead, we’ve learned how state accountability systems structure access to schools and communities, and we’ve gathered considerable evidence to evaluate whether the law has lived up to the hopes of the civil rights community.
Notably, the data collection provisions first introduced with NCLB have been lauded by civil rights groups for shining a light on educational inequality (e.g., see statements from the Mexican-American Legal Defense and Educational Fund, 2007 and the NAACP Legal Defense Fund, 2015). Moreover, the widespread, systematic collection and public release of student learning data has supported a boon of research on educational equity and opportunity gaps.
And yet, so many years later, NCLB’s promise of more equitable learning outcomes remains elusive. One way to view its limitations is to consider its reforms in light of what is, by far, the most promising intervention related to educational equity in the last 60 years: the movement for holistic, or real school integration, that moves beyond school-level desegregation.
Research literature is clear that desegregation efforts were highly effective at improving student learning and narrowing the test score gap between Black and White students (Grissmer et al., 1998; Johnson, 2019; reardon et al., 2015). By contrast, the short-term positive effects of NCLB on student learning were generally modest (Dee & Jacob, 2011; Figlio & Loeb, 2011; Lee, 2008). Meanwhile, during the last two decades of educational accountability, schools in the US have continued resegregating. These observations lead us to an important question for researchers and policymakers alike: what, if any, relationship exists between systems of accountability and the persistence of school segregation? And how, if at all, might accountability systems be refined in order to contribute constructively to real integration?
We take up these questions in a new research brief written for the National Coalition on School Diversity (NCSD). In particular, we review the research on the relationship between NCLB-style testing and contemporary school resegregation and offer research-based guidance for revising accountability policy in light of the contemporary struggle for racial integration. We further outline suggestions for how the research community can generate new data to better understand the relationship between school accountability and segregation. In this way, our brief joins a growing public conversation about how a future reauthorization of the Elementary and Secondary Education Act (ESEA) might return the law to its roots in the civil rights movement (see also DeBray et al., 2022). Importantly, we believe that the relationship between accountability and segregation is complex, defying simple explanations and solutions. Thanks in part to these accountability systems, gaps in achievement by students’ race, class, and special education status have been made more transparent. As a result, the range of stakeholders in education systems and in society more broadly have greater insight into the relationship between student test score outcomes and various elements of racially unequal schooling, such as inequity in school funding or access to experienced educators (Cardichon et al., 2020; EdBuild, 2019).
As detailed in our new brief, however, underlying systemic inequity remains unchanged long after NCLB. Not only has inequity persisted, but the most common forms of accountability have created new barriers to racial inclusion in American public education. Narrow and flawed measures have taken on outsized significance in how we measure the quality of our schools and even our neighborhoods. Especially considering the way that test scores mirror school demographics, terms like “good schools” and “bad schools” have not only become commonplace in the educational debate, they have also functioned as proxies for school racial composition (Knoester & Au, 2017; Muñiz & Barragán, 2022; Noonan & Schneider, 2022; Piazza, 2022).
Research that we review in the full brief suggests that NCLB-style accountability may even have accelerated trends toward resegregation. For example, responding to NCLB’s strict requirement that schools make progress toward achievement targets, schools had an incentive to exclude students who tended to score lower on standardized tests (students of color and students experiencing poverty). Indeed, some empirical evidence suggests that public schools operating under high-stakes accountability systems have taken active or tacit steps to boost test scores by managing their student populations: for example, excluding students from testing who were more likely to score lower (Vasquez Heilig & Darling-Hammond, 2008), drawing attendance zone boundaries to exclude students of color and/or low-income students (Wells & Holme, 2005), or “pushing out” students based on disciplinary records (Kho et al., 2022).
Educational accountability systems also appear to exacerbate housing segregation, which has a direct impact on the racial and economic resegregation of schools. In particular, the use of publicly prominent “school report cards,” which pre-date NCLB (Portz & Beauchamp, 2020), may shape public perceptions about school quality. Figlio and Lucas (2004) found that an “A” on a school report card had significant effects on housing price and nearby property values (see also Hasan & Kumar, 2019). Consistent with these findings, reardon and colleagues (2019) determined that geographic variation in racial achievement gaps were largely explained by differences in family income and school segregation.
Our brief offers a more detailed exploration of this research. We argue that high-stakes accountability and school resegregation have become more deeply entwined in recent years. Untangling that relationship requires that we consider potential solutions within and parallel to federal law.
Most notably, the Every Student Succeeds Act (ESSA) – which succeeded NCLB – required states to define and measure school quality using at least one non-academic indicator of student success (known as the so-called “5th indicator”). In 2016, NCSD suggested that states include indicators to measure progress on integration in their ESSA accountability systems (see Hilton, 2016). More recently, NCSD (2020) and its partner organizations have developed a model state policy that would require states to include measures of racial and socioeconomic integration in annual school ratings. The model policy includes clear guidance on how to measure integration via a proportionality score and how to ensure that historically underserved student subgroups have equitable access to school supports associated with high educational achievement. Unfortunately, the 5th indicator remains dramatically underutilized as a vehicle for integration. The majority of states use “chronic absenteeism” as their non-academic indicator and several use measures of student behavior, school discipline or so-called “dropout” rates, all of which mirror the racial bias evident in standardized test scores (Education Commission of the States, 2018; Portz & Beauchamp, 2020).
In 2018, NCSD reported that only one state proposed using measures of real integration for its 5th indicator. New York’s ESSA plan identifies “Integration of Students” as one of several potential indicators in an accountability system that utilizes multiple measures of school quality. Under this plan, which was formally approved in 2018, the state would consider the extent to which students of various racial and socioeconomic subgroups “are in schools and classrooms together,” compared to their presence in the district as a whole. Importantly, this information would factor into a school’s overall accountability rating. To date, however, officials have not made use of this indicator in their approach to accountability.
Even if more states were interested in using the 5th indicator as a vehicle for real integration, there is little guidance about specific measures that might be used. The same is true for the other potential levers for school integration in ESSA, such as the Competitive Grants for State Assessments program and the Innovation Assessment Development Authority provision. We review each in detail in the brief, ultimately arguing that a future iteration of the law could incentivize equity-oriented changes to state assessments by relaxing its most onerous requirements.
Especially given the ways that state accountability and related sanctions can maintain or exacerbate segregation, it is important to look beyond federal law to understand how states or local coalitions might use alternative forms of accountability to weaken the relationship between school measurement and school composition. Professional organizations have offered visions for a broader approach to assessment and accountability, while district and non-profit advocates have begun similar experiments at the local level. Instead of focusing on narrow measures of academic learning, state and/or district efforts can be more holistic, and their low-stakes nature allows these efforts the freedom to experiment.
A recent report from the Beyond Test Scores project at UMass Lowell detailed promising practices from various state and local accountability efforts (Carey & Schneider, 2022). For example, the Massachusetts Consortium for Innovative Education Assessment (MCIEA) measures school quality according to 34 indicators, including students’ perspectives on cultural responsiveness in curriculum. MCIEA’s full data collection portfolio – including its survey scales and administrative data measures – are publicly available as source material for other states, districts, or coalitions pursuing similarly holistic forms of school quality measurement. In addition, as with state accountability systems, MCIEA data are posted publicly on an online dashboard. (See https://mciea-dashboard.herokuapp.com/welcome.)
In addition to providing a broader range of data to drive real integration in schools and districts, alternative forms of accountability also contribute to continuing research on the benefits of school integration. A fuller range of school quality data can generate more evidence about student experiences in schools and challenge the “good” schools versus “bad” schools binary reinforced by test-based measurement. For example, using MCIEA survey responses from over 25,000 students, Schneider and colleagues (2020) compared student experience in “diverse” schools with those in comparatively more segregated schools. They found that White students in diverse schools reported more positive experiences than their White counterparts in more exclusively segregated White schools, including higher levels of physical safety, engagement, sense of belonging, and civic participation. Consistent with recommendations from major school integration advocacy organizations (Potter et al., 2021), research in this vein helps illustrate the benefits of school integration for all students, and it would not be possible without alternative forms of school quality measurement.
Even if parallel accountability systems don’t reach the level of federal or state policy, they provide schools with a broader spectrum of meaningful and relevant data to create more inclusive learning environments, and these data can generate new research that complicates narrow and overly simplistic conceptions of school quality. Along those lines, we conclude the brief by outlining suggestions for how the research community might build on a strong, but nonetheless, emergent research base on school accountability and segregation.
First, because outcomes of accountability systems are in part a reflection of the measurement tools they employ, we urge further research on the development and potential impact of new school quality frameworks and measures. For example, there is a considerable amount of unexplored potential in the development of culturally-relevant performance assessments as well as the use of growth scores and/or student-centered survey data in state accountability systems. Alternative forms of school quality measurement, however, are relatively new and more information is needed to better understand how accountability systems would (or would not) change with the incorporation of new measures.
Second, as grassroots initiatives to broaden school quality measurement and accountability take shape, research is needed to assess the impact of these initiatives and how – if at all – they might complement or even replace local, state, or federal accountability systems. We detail evaluation and empirical research on MCIEA above, but similar efforts are underway with consortia in California, Colorado, New York, and more (see Carey & Schneider, 2022). These are sites for experimentation in school accountability as well as settings for research on alternative forms of accountability. With access to a wider array of school quality data, researchers, for example, can ask questions about how parents use new information in their school choice decisions or about how school or district leaders can develop strategies for responding to settings where survey data indicate that students of color may not feel fully included in the school community. Ultimately, more holistic data emerging from these initiatives can help shape public perceptions of “bad” schools and “good” schools in a way that contributes constructively to the movement for real integration.
Finally, too often research efforts on accountability and on segregation are siloed, but as this research brief helps make clear, these two domains are in fact closely interwoven. While research exists on the relationship between segregation and accountability broadly speaking (e.g., reardon et al., 2022), there are fewer studies exploring the way that accountability systems interact with the many residual impacts of segregation such as funding inequities, teacher shortages, curriculum quality, school closures, or parental decision-making. Studies that trace the downstream effects of accountability within and across schools and communities could lead to new conceptual frameworks and language for talking about racial equity.
Shortly after NCLB was enacted, Gary Orfield and Chungmei Lee (2007) argued that “the basic educational policy model in the post-civil rights generation assumes that we can equalize schools without dealing with segregation through testing and accountability.” Indeed, test-based accountability systems have become embedded in the social structures that shape students’ access to schools and neighborhoods. Over time, the legacies of school segregation and educational accountability – and the effects of each on student learning – have come to mirror one another. Each has been characterized by the firing or voluntary attrition of teachers, the closing or restructuring of schools, and a narrowing of the curriculum. As described in the full brief, research can help us make stronger arguments about the link between school accountability and segregation. Research can also help us look forward to alternatives. By asking new kinds of questions and piloting alternative models for school accountability, the research and advocacy communities can work together to imagine a future for accountability that includes real integration.
Peter Piazza (ptpiazza@gmail.com) is Director of School Quality Measures at the Education Commonwealth Project; James Noonan (jnoonan@salemstate.edu) is Assistant Professor in the McKeown School of Education at Salem State University.
References
Cardichon, J., Darling-Hammond, L., Yang, M., Scott, C., Shields, P. M., & Burns, D. (2020). Inequitable Opportunity to Learn: Student Access to Certified and Experienced Teachers. Learning Policy Institute.
Carey, A., & Schneider, J. (2022). Alternative Educational Accountability Right Now: Examples from the Field. Beyond Test Scores Project, UMass Lowell.
DeBray, E., Finnigan, K.S., George, J., & Scott, J. (2022). A Civil Rights Framework for the Reauthorization of ESEA. National Education Policy Center.
Dee, T. S., & Jacob, B. (2011). The impact of No Child Left Behind on student achievement. Journal of Policy Analysis and management, 30(3), 418-446.
EdBuild. (2019, February). $23 Billion.
Education Commission of the States. (2018, May). Accountability and reporting: ESSA Plans.
Figlio, D. N., & Loeb, S. (2011). School Accountability. In E. A. Hanushek, S. Machin, & L. Woessmann (Eds.), Handbooks in Economics (Vol. 3, pp. 383–421). Elsevier.
Figlio, D. N., & Lucas, M. E. (2004). What’s in a grade? School report cards and the housing market. The American Economic Review, 94(3), 591–604.
Grissmer, D., Flanagan, A., & Williamson, S. (1998). Why Did the Black-White Score Gap Narrow in the 1970s and 1980s? In C. Jencks & M. Phillips (Eds.), The Black-White Test Score Gap (pp. 182–226). Brookings Institution Press.
Hasan, S., & Kumar, A. (2019). Digitization and Divergence: Online School Ratings and Segregation in America [SSRN Scholarly Paper]. Social Science Research Network.
Hilton, M. (2016). Prioritizing School Integration in ESSA State Implementation Plans (Issue Brief No. 6). National Coalition on School Diversity.
Johnson, R. C. (2019). Children of the Dream: Why School Integration Works. Basic Books.
Kho, A., Zimmer, R., & McEachin, A. (2022). A descriptive analysis of cream skimming and pushout in choice versus traditional public schools. Education Finance and Policy, 17(1), 160–187.
Knoester, M., & Au, W. (2017). Standardized testing and school segregation: Like tinder for fire? Race Ethnicity and Education, 20(1), 1–14.
Lee, J. (2008). Is test-driven external accountability effective? Synthesizing the evidence from cross-state causal-comparative and correlational studies. Review of Educational Research, 78(3), 608–644.
Mexican American Legal Defense and Educational Fund. (2007, March 23). No Child Left Behind.
Muñiz, R. & Barragán, S. (2022). Disrupting the Racialized Status Quo in Exam Schools?: Racial Equity and White Backlash in Boston Parent Coalition for Academic Excellence v. The School Committee of the City of Boston, Fordham Urban Law Journal 49(5), 1043-1090.
Noonan, J., & Schneider, J. (2022). Beyond “good” and “bad”: Disrupting dominant narratives about school quality. Phi Delta Kappan. 104(3), 6-11.
Orfield, G., & Lee, C. (2007). Historic reversals, accelerating resegregation, and the need for new integration strategies. UCLA Civil Rights Project/Proyecto Derechos Civiles.
NAACP Legal Defense Fund. (2015, December 2). LDF Statement on the Every Student Succeeds Act.
National Coalition on School Diversity. (2018) State of Integration 2018. Poverty and Race Research Action Council & the National Coalition on School Diversity.
National Coalition on School Diversity. (2020, May). Model State School Integration Policies. (Policy Brief No. 11). Poverty and Race Research Action Council & the National Coalition on School Diversity.
Piazza, P. (2022). Back to the Future: Revising Old Critiques to Find a Culturally Sustaining Form of School Integration. In C. L. Ivery & J. A. Bassett (Eds.), Detroit and the New Political Economy of Integration in Public Education (pp. 127-148). Palgrave Macmillan.
Portz, J., & Beauchamp, N. (2020). Educational accountability and state ESSA plans. Educational Policy, 36(3), 717-747.
Potter, H., Lallinger, S., Burris, M., Kahlenberg, R., & Edwards, A. (2021). School Integration is Popular. We Can Make it More So. The Century Foundation.
reardon, s. f., Robinson-Cimpian, J. P., & Weathers, E. S. (2015). Patterns and Trends in Racial/Ethnic and Socioeconomic Academic Achievement Gaps. In H. Ladd & M. Goertz (Eds.), Handbook of Research in Education Finance and Policy (pp. 491–509). Lawrence Erlbaum.
reardon, s. f., Weathers, E., Fahle, E., Jang, H., & Kalogrides, D. (2022). Is Separate Still Unequal? New Evidence on School Segregation and Racial Academic Achievement Gaps (No. 19–06; CEPA Working Paper). Stanford Center for Education Policy Analysis.
Vasquez Heilig, J., & Darling-Hammond, L. (2008). Accountability Texas-style: The progress and learning of urban minority students in a high-stakes testing context. Educational Evaluation and Policy Analysis, 30(2), 75–110.
Wells, A. S., & Holme, J. J. (2005). No Accountability for Diversity: Standardized Tests and the Demise of Racially Mixed Schools. In G. Orfield & J. C. Boger (Eds.), School Resegregation: Must the South Turn Back? (pp. 187 – 211). The University of North Carolina Press.
