2022 Conference Objects

The Power of Networked Education Systems: Mapping the Flow of PISA-Induced References

Pizmony-Levy, Oren; Kessler, Erika Lyn

The potential for countries and their education systems to learn from each other is one of the central idea(l)s in the Comparative and International Education field (Arnove, 2013). The development of routinized International large-scale assessments (ILSAs) and the test-based accountability movement reflects this idea(l). The OECD, for example, administers the Programme for International Student Assessment (PISA) to 15-year-old students triennially. In the wake of PISA results, media outlets frantically cover PISA by publishing articles featuring country scores, discussing the outcomes and the potential consequences. Indeed, local policymakers worldwide cite the achievement of foreign countries as they initiate and design education reforms. Understanding what factors explain these reference patterns formulates the central question in our study.

ILSAs bring not only transnational anxiety about the “learning crisis” and its implications to economic development, but also a flurry of debate around educational reform and knowledge (Baker & LeTendre, 2005; Sahlberg, 2014; Steiner-Khamsi, 2003; Waldow et al., 2014). Questions persist, however, about whether ILSAs promote new ideas of learning to address our evolving realities. In a recent special issue, Pizmony-Levy & Gan (2021) argue that current test-based accountability (e.g., ILSAs) is preventing schools and teachers from unlocking their potential in promoting sustainability and climate change solutions.

To date, research on public discourse around PISA (and other ILSAs) has mainly looked at individual or small groups of countries (but see: Pizmony-Levy & Torney-Purta, 2018). In large part, this is attributable to the paucity of data availability and languages. Past research has also overlooked the relations between education systems that participate in ILSAs and the social structure that emerges from systems sending and receiving flows of information and discourse about ILSA results. In our pilot study, we analyzed the reference network of 23 countries that were triggered by PISA 2012 results, we found references between countries are based on trade networks and co-membership in international organizations. We argued these findings reflect a stark limitation in the learning process and production of new knowledge (Kessler & Pizmony-Levy, forthcoming).

As a next step, our current study engages Social Network Analysis to examine the discursive network of 35 education systems participating in PISA 2015. We call this structure the PISA-Induced Reference Societies network (PIRS network). Our study has two empirical and theoretical goals. First, we seek to determine what characterizes the PIRS network. Second, we seek to evaluate the nature of the discursive network by examining to what extent does the PIRS network reflects other additional international networks? We aim to examine the flow of references with historical networks (history of colonial ties), cultural networks (formal alliances and treaties, co-membership in international governmental organizations), economic networks (trade import/export, donor activities), and geographic proximity.

Theoretical Framework: Social Network Analysis (SNA)
SNA is a broad research paradigm that includes theory, substance, and methodology. Within the tradition of SNA, a network consists of “a finite set or sets of actors and the relation or relations defined on them” (Wasserman & Faust, 1994, p. 20). A network includes some actors that are connected and other actors that might be disconnected or isolated from each other. These two fundamental components – actors and relationships – are common in most network definitions. The key premise of SNA is that relationships between actors determine in part what happens to a group of actors as a whole. Furthermore, SNA posits that an actor’s position in a network shapes the opportunities and the constraints that the actor will encounter. This perspective is different from traditional social science that focuses on the characteristics of actors as predictors of different outcomes. In traditional social science, we might explain differences in the performance of individuals or groups by certain qualities or characteristics. In contrast, SNA considers the web of relationships in which individuals or groups are embedded.

Using network analysis techniques, we seek to demonstrate the centralized and hierarchical nature of the PIRS network. Previous education network studies have shown that relationships in social networks shape the outcome of educational reforms (Daly, 2011). Identifying the knowledge transfer in news media is crucial for revealing the way in which power in education policy discourse is distributed across countries (Kolleck, 2015) and understanding the relational interdependence of global policy processes (Stone, 2008).

Data and Methods
Data for this study come from a larger study on PISA and education policy discourse (Pizmony-Levy et al., 2018). We constructed social network data drawing from an original dataset of 463 news articles covering the results of PISA 2015 in 35 countries. To collect and code the data, we trained and supervised over 40 research assistants who were enrolled in a graduate-level seminar on the social analysis of ILSAs. Assistants were trained to identify at least 10 news articles from at least two different sources in a country where they fluently spoke the language. Next, we trained assistants on a standardized codebook to analyze the ways in which newspapers cover PISA. One question in the codebook asked research assistants whether the news article referred to results from other countries and to provide the names of those countries.

Significance
This study will contribute to CIES 2022 by exploring one key idea(l) in the Comparative and International Education field. Further, we approach the issue of learning from other countries by applying an innovative methodology (SNA). Our presentation will focus on public discourse as a “window” to the ways in which the public experience transnational policymaking according to relational contexts. While scholars often emphasize the importance of the global and local contexts, our study points to a third context – the extent to which countries are connected to each other. By examining the relational contexts of PISA reference networks, we aim to map how different types of relations between countries (e.g., cultural, historical, economic, or geographic) shape local policy debates and education reforms. In other words, the current study provides an opportunity to explore the role of global processes in education policy that changes what education is like on the ground and therefore shapes idea(l)s of knowing and learning in education policymaking.

This paper was presented at the 2022 Annual Meetings of the Comparative and International Education Society (CIES). The paper was part of Highlighted Session: Educational policy and accountability: Dealing with tensions and influences on education systems.