Save

Recent Discussions on Implementation Research at CERME12

In: Implementation and Replication Studies in Mathematics Education
Authors:
Linda Marie Ahl National Center for Mathematics Education, University of Gothenburg Vera Sandbergs Allé 5A, Gothenburg Sweden

Search for other papers by Linda Marie Ahl in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-1928-7537
,
Mario Sánchez Aguilar Instituto Politécnico Nacional, CICATA Legaria, Programa de Matemática Educativa Calzada Legaria No. 694, Col. Irrigación, C.P. 11500, Alcaldía Miguel Hidalgo, Mexico City Mexico

Search for other papers by Mario Sánchez Aguilar in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-1391-9388
,
Morten Misfeldt Department of Science Education, University of Copenhagen Universitetsparken 1, Copenhagen Denmark

Search for other papers by Morten Misfeldt in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-6481-4121
,
Boris Koichu Department of Science, Weizmann Institute of Science Rehovot Israel

Search for other papers by Boris Koichu in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0001-7298-8175
, and
Uffe Thomas Jankvist Danish School of Education, Aarhus University Tuborgvej 164, Copenhagen Denmark

Search for other papers by Uffe Thomas Jankvist in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0002-4089-0242
Open Access

1 Introduction

The primary concern of Implementation and Replication Studies in Mathematics Education (IRME), is the development of a vibrant environment around research and development in implementation and replication studies in mathematics education. The Thematic Working Group 23 (TWG23) entitled “Implementation of research findings in mathematics education” at the three last Congresses of ERME (European Society for Research in Mathematics Education) has had an important impact on developing implementation research (IR) in mathematics education research (MER). In this editorial, we provide an extended report from TWG23 at CERME12, the twelfth Congress of ERME. TWG23 was led by IRME’s three associate editors, Mario Sánchez Aguilar (chair), Boris Koichu and Morten Misfeldt, along with two younger researchers, Rikke Maagaard Gregersen (until August 2021) and Linda Marie Ahl (from September 2021). Linda Marie Ahl was invited to co-author this editorial, which is an extensive report from this year’s TWG23.

Due to the Covid-19 situation, CERME12 was held virtually. TWG23 consisted of eight sessions involving presentations of papers and posters as well as three thematic discussions among the group’s 21 participants from 12 different countries. This was the third time that TWG23 took place, and the group has now proven somewhat stable in terms of participant number and contributions. We will say more about the evolution of the group in the next section of the editorial, but it is important to be aware of the value of the work of this TWG for the existence and evolution of IRME. (This was already acknowledged by Michèle Artigue (2021) in her overview of theoretical resources for implementation research in mathematics education in the very first issue of IRME.)

In the present editorial, we will first describe the evolution of the work on IR in the context of CERME, and then we will describe the contributions at CERME12’s TWG23. Afterwards we will describe three overarching themes that were discussed at this year’s meeting in TWG23: (1) the role of change and theory of change; (2) matters of scaling; and (3) the conception of stakeholders. These three themes are also used to situate the four papers in this issue of IRME, since they all are extensions of studies presented at this year’s CERME12 (TWG23). The editorial concludes with several observations and considerations about the next steps for IR in mathematics education in general and for the TWG23 at CERME in particular.

2 The Evolution of the Work on IR in the Context of CERME

Implementation has always been a concern in mathematics education, but the idea to focus explicitly on IR was first suggested at CERME10 in Dublin in 2017, where the first TWG dedicated to this was run by Uffe Thomas Jankvist (chair), Jonas Bergman Ärlebäck, Kjersti Wæge and Mario Sánchez Aguilar. The articulated ambition of the new TWG was to address the fact that despite MER having accumulated quite a lot of knowledge in terms of research findings and theoretical frameworks, there is a tendency that this knowledge does not find a way into classrooms and teacher education programs. Hence, the purpose of TWG23 was to provide a forum to discuss this challenge under the heading of “implementation research” (Jankvist et al., 2017). The work at CERME10 attempted to draw attention to IR in MER, and the participants used a number of references to build on, including Rogers’ (1962) description of “diffusion of innovations” and Nilsen’s (2015) description of IR as an area within health science.

At CERME11, two years later, a new stage was reached in the TWG23. Mario Sánchez Aguilar was now leader of the TWG, Kjersti Wæge, Ana Kuzle and Morten Misfeldt were co-leaders. This meeting made progress in defining implementation and moving towards some sort of paradigmatic structure of IR. The result was the first version of a working definition of implementation, and an ambition of scoping the field with a special issue of ZDM — Mathematics Education (Koichu et al., 2021) as well as a starting point for creating a stable academic outlet for IR in MER, namely IRME. These two tasks took up a lot of time and energy in 2019, 2020 and 2021, but the regular TWG activities continued. Rikke Maagaard Gregersen (Denmark) and Boris Koichu were both enrolled as co-leaders, replacing Ana Kuzle and Kjersti Wæge. CERME12 was postponed due to the pandemic, but in February 2021 an intermediate ERME meeting was arranged, with two discussion sessions dedicated to TWG activities. For TWG23, these two sessions attracted both well-established MER scholars as well as a group of young researchers. In order to further accommodate the discussions on IR in MER among young researchers, and to create a space for preparing CERME12 contributions, Rikke Maagaard Gregersen and Andreas Tamborg organized a young researchers’ day, a virtual meeting, in the summer of 2021.

As mentioned, Artigue (2021) reviewed the activities of TWG23 at CERME10 and CERME11 in the inaugurational first issue of IRME. In her review, she noted, with reference to the set of papers presented in TWG23 at CERME10:

[An] important part of theoretical resources concerns the research areas to which the results to be implemented belong. They may concern a particular mathematical content (e.g., numbers, functions, calculus) or more transversal competences (e.g., problem solving, modelling), learning approaches (e.g., collaborative learning, active learning, self-regulated learning), teaching approaches (e.g., explicit teaching), or teacher training approaches.

Artigue, 2021: p. 40.

Artigue (2021) further observed that some theoretical resources dealing explicitly with implementation and the spread of innovations had been used. This included Rogers (1962) that was applied already at CERME10 (e.g., Koichu & Keller, 2017) as well as Nilsen (2015) (e.g., Jankvist et al., 2017). At CERME11 and CERME12, a great number of papers referred to characterizations and conceptualizations of IR in the works by Century and Cassata (2016) and Chen (2012). Furthermore, at CERME12, a number of papers referred to Coburn’s (2003) discussion of deep and lasting change in education. In this sense, there has been a development from local MER theories and frameworks (e.g., about problem solving or algebraic thinking) towards theories and frameworks that deal specifically with IR concerns such as innovation spread (Rogers, 1962), theory of change (Chen, 2012), and scale (Coburn, 2003).

3 Contributions to TWG23 at CERME12

The TWG23 at CERME12 received 18 contributions consisting of 15 papers and three posters. The authors of the contributions came from Austria, Denmark, Finland, Greece, Israel, Italy, Luxembourg, Mexico, Norway, Sweden and the UK. The contributions were organized into the following thematic categories for presentation:

  • Implementation of problem-solving and problem-posing approaches

  • Implementation of teaching models and teachers’ perspectives on implementation

  • Conditions for sustainable implementations

  • Diagnostics tasks, instructional sequences and curriculum design

  • Implementation of programming, computational thinking and other digital technologies

Due to the overarching nature of the poster by Konrad Krainer (2022), this was selected to be presented at the beginning of the first meeting of TWG23. This poster presentation served to set the scene in the group, and thus provided the participants with conceptual categories (technical rationality, reflective rationality, and societal rationality) that were useful for identifying and contrasting different approaches to the implementation and implementability of research.

The first thematic category “Implementation of problem-solving and problem-posing approaches” included contributions providing methodological and theoretical tools for the implementation of mathematical problem-posing and problem-solving approaches. The paper by Nafsika Patsiala and Ioannis Papadopoulos (2022) presented an instrument to record and examine whether students develop — through their experiences with problem posing — the habit of mind named “seeking and using structure”. The paper by Jason Cooper and Boris Koichu (2022) introduced the notion of “problem-solving implementation chain” for analyzing the evolution of a problem-solving activity as it passes from the proponents, to teachers, and eventually to students.

The second thematic category grouped two types of contributions. One type of contribution referred to studies that addressed the implementation of specific teaching approaches. This was the case of the research by Morten Blomhøj and collaborators (2022), who introduced a three-phased didactical model to facilitate the implementation of an inquiry-based approach to mathematics teaching. Another case was the paper presented by Ola Helenius (2022), who offered a theoretical discussion on large-scale implementation of a research-based teaching model for elementary school arithmetic. The other type of contributions within this category included studies with an emphasis on teachers’ actions and perspectives on implementation processes. For instance, Maria Kirstine Østergaard and Uffe Thomas Jankvist (2022) used theoretical constructs from IR to identify elements of a mathematics teacher’s practice and thereby identified factors that seemed to influence the implementation of teaching units aimed at fostering students’ reflections on the nature of mathematics as a discipline. Other papers included in this category were by Åsmund Lillevik Gjære (2022), who examined one Norwegian teacher’s enactment of an innovative system for mathematics teaching called “developmental education in mathematics”; and by Alessandra Boscolo (2022), who reported perspectives of teachers about the implementation of active, bodily experienced mathematics learning activities.

The contributions grouped in the third thematic category brought to the fore the discussion of the sustainability of the implementation of innovations. For instance, the paper by Johan Prytz and colleagues (2022) paid particular attention to the issue of sustainability of an innovation in mathematics education, and the potential role of textbooks in sustaining the innovation. Another contribution within this category was the work presented by Mario Sánchez Aguilar and Apolo Castaneda (2022a), who pointed out the importance of distinguishing between the politics of enactment and the politics of implementation as a first step in integrating the analysis of political sustainability into IR.

In the fourth category was a group of contributions that addressed issues of the implementation of diagnostics tasks, instructional sequences, and curriculum design. One contribution included in this category was the work presented by Morten Elkjær and Jeremy Hodgen (2022a), where they formulated an implementation process model for designing and implementing tasks for formative feedback in an online learning environment. Another contribution contained in this category was the literature review developed by Linda Marie Ahl, Mario Sánchez Aguilar and colleagues (Ahl et al., 2022a). This review dealt with the implementation of instructional sequences aimed to enhance students’ learning of mathematical concepts or competencies. The review revealed which competencies are more and less readily targeted and what characterizes the implementation of the instructional sequences. Included in this category was also the contribution of Ellen Jameson and collaborators (2022), who pointed out challenges to implementing research through the processes and products of curriculum design. Finally, the work reported by Tuula Koljonen and colleagues (2022) examined the feasibility of using the Documentational Approach to Didactics to gauge the fidelity and character of teachers’ implementation of scripted teaching sequences for the teaching of arithmetic in primary school.

The fifth thematic category included contributions focused on the implementation of digital technologies, mainly those related to computer programming. One study presented was by Raimundo Elicer and collaborators (2022), who addressed the role of educational task design in implementation research. In collaboration with a 4th grade school teacher, the authors designed a geometry task from a hypothetical learning trajectory that required students to draw on their knowledge of Programming and Computational Thinking (PCT). Another presentation was based on the paper by Andreas Lindenskov Tamborg and colleagues (2022), who reported on the development of a survey tool to investigate how PCT is implemented in Denmark, Sweden, and England. Within this category was also a poster called “Comparing the implementation of programming and computational thinking in Denmark, Sweden and England” (Misfeldt et al., 2022). This poster was one of three winners of the first ever ERME Poster Award (http://erme.site/awards/). Another poster included in this category was by Ben Pierre Haas and collaborators (2022), where they gave an overview of how technologies such as augmented reality, 3D printing, and tutoring systems could be employed by different users for teaching and learning STEAM-based educational ecosystems.

Furthermore, the program for TWG23 included three thematic discussions. The themes were selected on the basis of previous discussions that had emerged as being central for IR. Namely, they had been identified based on the papers published in the special issue on IR in ZDM — Mathematics Education (Koichu et al., 2021) as well as papers published in the first two issues of IRME (Jankvist et al., 2021a,b).

4 The Role of Change

We now present the first of the three TWG23 thematic discussions, i.e., that on “change”. Next, we present two of this issue’s papers, by Linda Marie Ahl, Ola Helenius and Tuula Koljonen (2022), and Boris Koichu, Jason Cooper and Mirela Widder (2022), respectively, and relate them to the thematic discussion.

4.1 TWG23 Thematic Discussion on “Intended Change”

The first theme was “intended change”. The background was that it has proved difficult to determine to what extent implementation of innovations is, or is not, successful (Helenius, 2021; Karsenty, 2021). For example, it is sometimes difficult to answer such a basic question as the following one: in a given implementation, are we reaching the intended change? To this end, Jankvist, Gregersen and Lauridsen (2021) pointed out the importance of working on calibrating research programs, so that it would become clear which phase of the implementation is being studied and which factors of influence affect that particular phase. Indeed, the overarching goal with implementing innovations based on results from MER is to improve practice and thus students’ learning. Still, while an improvement in students’ outcomes can easily be measured, student achievement is a blunt instrument for evaluating both the whole as well as individual parts of an implementation process. Hence, there is reason to believe that the field of IR needs to deepen its view of program design in relation to program theory and theories of change, when implementation projects are planned and evaluated (Chen, 2012; Krainer, 2015; Cobb & Jackson, 2021; Helenius, 2021; Jankvist et al., 2021c; Karsenty, 2021). The question for the first thematic discussion was:

How can we work with articulating and evaluating intended change? What are the pros and cons in relation to constructions such as program theory (theory of change) and realistic evaluation?

From the discussion in TWG23, we identified four key notions in the need for further conceptualization: Change; Object (of implementation); Stakeholders; and Agency. The participants pointed out that usually there is a distance between the planned/intended change and the actual change achieved after implementation. A tension can be discerned from the discussions. On the one hand, it is important to stay true to the program components in an implementation, but on the other hand, it is necessary to give freedom and agency to different stakeholders, so that they can negotiate the core of the object of the innovation. It is often the case that some core components of the innovation must remain as intended, otherwise the innovation is changed beyond recognition. It would also be very difficult to evaluate an innovation that does not have the same characteristics as the original innovation. But despite these concerns, the TWG discussion concluded that innovations must be adapted into organizations in a flexible manner in order to sustain the particular context and in relation to the interests of different stakeholders. Therefore, stakeholders must be given agency to negotiate the use of the innovation.

4.2 The Documentational Approach to Didactics as a Way to Measure Change

In this issue, using the Documentational Approach to Didactics (DAD), Linda Marie Ahl, Ola Helenius and Tuula Koljonen seek out to theorize the measuring of fidelity in a curricular implementation in their paper: “Gauging fidelity to an implemented teaching model through the lens of the Documentational Approach to Didactics”. The curricular implementation is the Swedish TRR model (Thinking, Reasoning, and Reckoning), a strict six-phase teaching model that makes up the “innovation” in the study. The TRR model has been implemented with more than 1,000 primary school mathematics teachers in 14 Swedish municipalities. The study of the present paper is, however, an in-depth analysis of two teachers’ implementation of the TRR model in their respective classrooms (Ahl et al., 2022b).

The more specific question asked by the authors is how DAD can help gauge the fidelity and adaptation of the determinants of a highly prescriptive curricular resource such as TRR, where “determinants” refer to core components of the TRR. In the analysis of the two teachers making up the case of the study, a distinction is made between “classroom organization determinants” and “content organization determinants”. As an embedded component of DAD, the authors utilize Vergnaud’s notion of scheme in terms of teachers’ “schemes of utilization”. It is found that both teachers made adaptations in their utilization schemes, and created new “documents” in relation to working with the new resource of TRR. Yet, while these documents seemed aligned — exhibiting high fidelity — with the content organization determinants of TRR, the same was not the case for the classroom organization determinants. Hence, the use of DAD appears to offer a promising theoretical basis for researching teachers’ implementation and adaptation of curricular resources, while the construct of teachers’ utilization schemes provide insight into aspects of fidelity as well as teachers’ “change”.

4.3 Implementation Chain as a Matter of Change

The paper by Boris Koichu, Jason Cooper and Mirela Widder (2022) in this issue is entitled “Implementation of problem solving in school: From intended to experienced”. It proposes the concept of Problem Solving Implementation Chain. The paper takes as its departure point that problem solving is widely recognized as an important and desirable activity in a classroom, but that there is a huge gap between intentions for mathematical activities to trigger problem solving and the reality in which students too often experience mathematics as drill and practice. The authors take up the challenge of attempting to characterize and explore this gap.

They suggest a framework that operates with the key elements of the process of making problem solving a feasible activity in a regular classroom, from the design of problem-solving resources to the enactment of the resources in a classroom by teachers, who engage with these resources in communities of inquiry (Jaworski, 2006). The problem-solving implementation chain framework is built as a local theory that capitalizes upon the Documentational Approach to Didactics, a model of problem solving in instructional settings by Koichu (2018) (Shift-and-Choice Model) and on theoretical developments in the field of teacher collaboration (Jaworski, 2006). The framework is illustrated by examples from the Raising the Bar in Mathematics Classroom project in Israel. The authors end with discussing four tensions between intentions and enactments in relation to problem solving, and posit these tensions as learning opportunities for the stakeholders involved in the everlasting pursuit of the educational change associated with the frequent use of problem-solving activities in regular classrooms.

5 Matters of Scaling

We move on to the second TWG23 thematic discussion on “scale” and present this issue’s paper by Morten Elkjær and Jeremy Hodgen (2022b) in relation hereto.

5.1 TWG23 Thematic Discussion on “Scale”

Large-scale implementations are complex endeavors, often with the goal to reach a large number of classrooms. If an idea is proven to work well in a certain setting, we are interested in disseminating it into other settings. There are various strategies for reaching out with ideas on a large scale. While reforms of national curriculum standards are expected to be implemented on a large scale — and to function from day one — many professional development projects are designed on a small scale first before being spread at larger scale (e.g., Krainer, 2015, 2021; Cobb & Jackson, 2021; Karsenty, 2021; Roesken-Winter et al., 2021). As phrased by Artigue (2021): “… implementation research, even if it can take advantage of studies of limited scope, must be able to meet, as a genre, the scaling-up challenge” (Artigue, 2021: p. 22).

In light of these different scaling strategies for implementation projects, one may ask how these affect IR, i.e., “… the systematic inquiry of innovations enacted in controlled settings or in ordinary practice, the factors that influence innovation enactment, and relationships between innovations, influential factors, and outcomes” (Century & Cassata, 2016: p. 181). Therefore, the question guiding the second thematic discussion was:

What are the possible roles of small-scale vs. large-scale studies in implementation research? For example in relation to different phases such as planning, testing, enacting, and evaluating.

The role of small-scale vs. large-scale studies in IR in MER has been discussed on several occasions (e.g., Burkhardt & Schoenfeld, 2021; Koichu et al., 2021; Roesken-Winter et al., 2021). Yet, the TWG23 participants at CERME12 agreed that IR in MER should not be reserved for large-scale studies only. Instead, it was argued that there is a need to combine small-scale studies with large- scale studies.

Generally speaking, an implementation project consists of the three phases of: planning and testing; implementation; and evaluation. In the planning and testing phase, there is a need for small-scale pilot studies to test the outcome of the innovation and to refine its design. The evaluation needs to be done on a large scale in order to have validity and reliability. Nevertheless, the large-scale studies often need to be supplemented with case studies that illustrate in-depth how different challenges in the implementation have been handled. This issue is closely related to the first thematic discussion in which program theory and theories of change were discussed. A large-scale evaluation can give a clear result on whether the implementation has been successful or not. Yet, in order to have a deep understanding of what works and why, and to learn more about specific mechanisms with the aim of improving the implementation process, small-scale studies are often well-suited as a complement to large-scale studies.

The TWG discussion to some extent shifted from addressing studies on implementation projects to addressing implementing projects of various scales. The notion of semi-large scale implementation projects was introduced, referring to projects that do not have the explicit goal to spread to a large number of classrooms. One example can be a group of teachers that works together to implement an innovation in a defined geographical area, e.g., a small municipality. Consequently, the evaluation will be on a semi-large data set, which is why it can be called a semi-large implementation study. Referring to Coburn’s (2003) four dimensions of scaling, small-scale implementation projects can scale up and be evaluated in terms of in-depth, sustainability, and shift in reform ownership. It is only the dimension of spread that is reserved for large-scale implementation projects.

5.2 Scaling Formative Feedback Practices

This issue of IRME includes a study by Morten Elkjær and Jeremy Hodgen (2022b) that addresses the challenge of implementing research-based knowledge on a large scale: “Operationalising Vergnaud’s notion of scheme in task design in online learning environments to support the implementation of formative assessment”. This work proposes and articulates an implementation process model for designing and implementing tasks for formative feedback in an online learning environment for mathematics classrooms using Vergnaud’s notion of scheme. The online learning environment discussed in the paper has been running for over 10 years in Denmark. On a typical day, 45,000 primary school and lower secondary school students use the tasks offered by the site and collectively answer up to 1,500,000 tasks, thus creating a potential source of feedback to teachers about the difficulties that students encounter in learning mathematics.

The study focuses on the intentional, computational and generative aspects of students’ schemes, and draws on research on learners’ errors and difficulties with the concepts of linear equations and the equals sign. It illustrates how a theoretical notion in combination with research results from the field of mathematics education can guide the design of diagnostic tasks for implementation in online learning environments. Due to the massive nature of this type of instructional context, the information generated through the users’ responses provide potentially valuable feedback to teachers about the difficulties that students encounter in their learning of mathematics.

6 The Conception of Stakeholders

We now embark on the TWG23 thematic discussion on “stakeholders” and present this issue’s last paper by Mario Sánchez Aguilar and Apolo Castaneda (2022b).

6.1 TWG23 Thematic Discussion on “Stakeholders”

No large scale implementation can be successful without the scaffolding of stakeholders at different levels (e.g., Cai & Hwang, 2021; Helenius, 2021; Krainer, 2021; Prytz, 2021). Krainer (2021) invited the MER community to address the following questions: “Who are the relevant stakeholders whose voices should be heard when discussing implementation? What is the role of policymakers, administration experts, researchers, and practitioners with regard to defining and solving problems that occur in practice?” (p. 1175). With insight into the absolute necessity of having a strategy for working with stakeholders, the theme for the third thematic discussion was:

How do we work with stakeholders? This includes conceptualizing their roles, designing/framing their participation, and evaluating the impact of their involvement in relation to three levels of: administration/policymakers; researchers; and practitioners.

The TWG discussion showed that there is a lack of consensus on how the concept of stakeholders is understood and can potentially be defined. Some argued that every group involved in an implementation project makes up a “stakeholder”, including also the students. Others argued that it makes no analytical sense to define students as stakeholders as they cannot be held responsible for the outcome of the implementation. In particular, some argued that it only makes sense to define a group as a stakeholder, if it has the power and agency to influence the implementation process. Voices for the position that students should be defined as stakeholders argued that students are indeed stakeholders, since they are the end-users in the implementation process and as such are explicitly affected by its outcome, and also affect it. Indeed, without students, there would be no reason to conduct implementation projects, some pointed out.

The fragmented view above may be traced back to how one views the purpose of defining categories of interest in IR. Those who see the stakeholders as key players for success in the process may find that a perspective of power is more interesting than the actual involvement in the implementation and the dependence on it. Those who find that stakeholders should be all groups affected by a project represent the second position, i.e., that the power and agency perspective is not the only matter which is decisive.

The need for strategies to study stakeholders in different contexts was highlighted in the discussion. Here, parallels were drawn to other research areas such as political science and economics. With the insight that different institutional and environmental contexts can lead to the same implementation approaches succeeding or failing, the question of what we can learn from other fields was posed, as well as the question of specificity of IR in MER.

6.2 The Covert Influence of Critical Stakeholders

The work by Mario Sánchez Aguilar and Apolo Castaneda (2022b) included in this issue of IRME illustrates how different groups of stakeholders — and their own political interests — interact and influence the implementation of research knowledge as part of educational reforms and mathematics textbooks in Mexico. This study sheds light on the political dynamics that takes place out of the public eye and that can influence the implementation of research knowledge and innovations produced in the field of MER. The study is focused on political aspects that underlie the implementation processes. The authors use a conceptual framework that breaks down an implementation process into two phases — called “the politics of enactment” and “the politics of implementation” — as a first step in integrating the analysis of political sustainability into IR. This conceptual framework also makes it possible to identify decision venues for policy design and enactment, and interest-based coalitions active during each phase of an implementation process.

Based on the analysis of in-depth interviews with a key informant — an insider standing between the culture of research in MER and the culture of educational reforms and mathematics textbooks in Mexico — the researchers identify three different political factors that have the potential to influence the implementation of research-based knowledge from the field of mathematics education: (1) the opposition of interest-based coalitions; (2) the distribution of non-explicit guidelines and diffuse information that is subject to varied interpretations by different stakeholders; and (3) the compatibility between the proposed mathematical contents and the current political agenda.

7 Concluding Remarks and the Future of TWG23

The three thematic discussions of TWG23 were centered on “change”, “scale” and “stakeholders”.

In relation to “change”, the TWG23 participants agreed that the tension between “intended change” and “achieved change” in an implementation project is a delicate question of interest for our research field. We thus see a continuing need to discuss the question of how program theory and theory of change can be used to design, understand and evaluate implementations. As for “scale” and scaling, the participants of TWG23 reinforced the need for both small-scale and large-scale studies in IR in MER. What remains to be discussed is strategies needed in order to make decisions about which types of studies can provide the most relevant information for different parts of the implementation. If for example a program theory guides the process of implementation, it can also guide the analysis of the implementation processes, for example, by explaining the mechanisms expected to work in favor of the implementation in each step, and it may eventually be used to evaluate the success of implementation. Therefore, the matters of scale and scaling may benefit from being discussed in close relation to that of “change”, i.e., the role of program theories and theories of change in design and evaluation. In addition, it was clear that there is a need for further discussions on the conception and definition of “stakeholders”. There is also a need for progressing our knowledge on how the concept of stakeholders can be used to refine different types of analysis of implementation projects. For example, should the stakeholder concept be reserved for the groups with power and agency to impact the implementation, or should it denote all groups affected by the implementation? And how do the different definitions affect the concept of stakeholders as a tool in IR? The overall outcome of the TWG23 thematic discussions was that none of the three themes of “change”, “scale” and “stakeholders” can be fully understood and explored in isolation from the others.

In the final discussion at TWG23, it was pointed out, by Michéle Artigue among others, that the TWG and its contributions had undergone a natural evolution in its short time of existence. From mainly consisting of small-scale studies at CERME10 and CERME11 — and in some respects as a consequence of this — to having an intensive discussion about the issue of scale and scaling at the virtual pre-CERME meeting in February of 2021, and in the IR special issue of ZDM — Mathematics Education (Koichu et al., 2021), to have entered a more mature stage at CERME12, where small-scale and large-scale implementation-related studies are seen as serving different purposes from the perspective of IR. The contributions of Krainer (2022) and Helenius (2022) are examples of this. The nuanced TWG discussions of change, also in terms of recognizing the need for a theory of change or a program theory, and how to best go about defining and using the stakeholder term, provide further evidence of this maturing of the IR subfield of MER.

We close this editorial by acknowledging that IR, and not least IR in MER, is a complex endeavor that calls for a holistic perspective of all factors of influence in the implementation process. Yet, to reach such a holistic perspective, we also need to dig deeper into well-focused questions. Surely, the discussion will continue where we left it, when TWG23 meets again at CERME13 in Budapest in 2023. We, as chairs of TWG23 and editors of IRME, are glad to encourage everyone with an interest in implementation and replication studies in mathematics education to partake in future TWG23s at CERME as well as to consider IRME as an outlet for publishing high-quality research which addresses theoretical and practical challenges discussed in this editorial, among other issues pertinent to implementation and replication problematics in MER.

References

  • Aguilar, M. S. & Castaneda, A. (2022a). Political challenges for the implementation of research knowledge as part of educational reforms and mathematics textbooks [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Aguilar, M. S. & Castaneda, A. (2022b). Out of the public eye: Researching political factors that influence the implementation of research knowledge as part of educational reforms and mathematics textbooks. Implementation and Replication Studies in Mathematics Education, 2(1). DOI: 10.1163/26670127-bja10001.

    • Search Google Scholar
    • Export Citation
  • Ahl, L. M., Aguilar, M. S., Jankvist, U. T., Misfeldt, M. & Prytz, J. (2022a). Implementation research on instructional sequences focusing on mathematical concepts and competencies: Results from a review [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, Bozen-Bolzano, Italy.

  • Ahl L. M., Helenius, O. & Koljonen, T. (2022b). Gauging fidelity to an implemented teaching model through the lens of the documentational approach to didactics. Implementation and Replication Studies in Mathematics Education, 2(1). DOI: 10.1163/26670127-bja10003.

    • Search Google Scholar
    • Export Citation
  • Artigue, M. (2021). Implementation studies in mathematics education: What theoretical resources? Implementation and Replication Studies in Mathematics Education, 1(1), 2152. https://doi.org/10.1163/26670127-01010002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blomhøj, M., Haavold, P. Ø. & Pedersen, I. F. (2022). Inquiry-based mathematics teaching in practice: A case of a three-phased didactical model [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Boscolo, A. (2022). Active, bodily experience mathematics learning activities. Looking at implementation from the teachers’ perspective [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Burkhardt, H. & Schoenfeld, A. (2021). Not just “implementation”: The synergy of research and practice in an engineering research approach to educational design and development. ZDM — Mathematics Education, 53(5), 9911005. https://doi.org/10.1007/s11858-020-01208-z.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cai, J. & Hwang, S. (2021). What does it mean to make implementation integral to research? ZDM — Mathematics Education, 53(5), 11491162. https://doi.org/10.1007/s11858-021-01301-x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Century, J. & Cassata, A. (2016). Implementation research: Finding common ground on what, how, why, where, and who. Review of Research in Education, 40(1), 169215. https://doi.org/10.3102/0091732X16665332.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chen, H. (2012). Theory-driven evaluation: Conceptual framework, application and advancement. In R. Strobl, O. Lobermeier & W. Heitmeyer (Eds.), Evaluation von programmen und projekten für eine demokratische kultur (pp. 1740). Springer. https://doi.org/10.1007/978-3-531-19009-9_2.

    • Search Google Scholar
    • Export Citation
  • Cobb, P. & Jackson, K. (2021). An empirically grounded system of supports for improving the quality of mathematics teaching on a large scale. Implementation and Replication Studies in Mathematics Education, 1(1), 77110. https://doi.org/10.1163/26670127-01010004.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 312. https://doi.org/10.3102/0013189X032006003.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Cooper, J. & Koichu, B. (2022). Problem-solving implementation chain: From intended to experienced [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Elicer, R., Tamborg, A. L. & Jankvist, U. T. (2022). Revising a programming task in geometry through the lens of design-based implementation research [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Elkjær, M. & Hodgen, J. (2022a). The components of Vergnaud’s notion of scheme as frame for designing diagnostic tasks in the area of linear equation solving [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Elkjær, M. & Hodgen, J. (2022b). Operationalising Vergnaud’s notion of scheme in task design in online learning environments to support the implementation of formative assessment. Implementation and Replication Studies in Mathematics Education, 2(1). DOI: 10.1163/26670127-bja10002.

    • Search Google Scholar
    • Export Citation
  • Gjære, Å. L. (2022). Depth as a key issue for implementing DEM: The case of a teacher [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Haas, B. P., Kreis, Y., Lavicza, Z., Houghton, T. & Fenyvesi, K. (2022). STEAM-based learning ecosystems involving digital toolkits, tutoring systems, 3D printing and mathematical trails [Poster presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Helenius, O. (2021). A stakeholder analysis of the development of a large scale professional development program. Implementation and Replication Studies in Mathematics Education, 1(2), 227256. https://doi.org/10.1163/26670127-01010009.

    • Search Google Scholar
    • Export Citation
  • Helenius, O. (2022). Designing stakeholder scaffolding for implementing a teaching model at scale [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Jameson, E., Macey, D. & McClure, L. (2022). Implementing research in the practice of curriculum designers: Barriers and an example approach [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Jankvist, U. T., Aguilar, M. S., Ärlebäck, J. B. & Wæge, K. (2017). Introduction to the papers of TWG23: Implementation of research findings in mathematics education. In T. Dooley & G. Gueudet (Eds.), Proceedings of the Tenth Congress of the European Society for Research in Mathematics Education (pp. 3769–3775). DCU Institute of Education, Dublin; ERME.

  • Jankvist, U. T., Aguilar, M. S., Misfeldt, M. & Koichu, B. (2021a). Launching Implementation and Replication Studies in Mathematics Education (IRME) [Editorial]. Implementation and Replication Studies in Mathematics Education, 1(1), 119. https://doi.org/10.1163/26670127-01010001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jankvist, U. T., Aguilar, M. S., Misfeldt, M. & Koichu, B. (2021b). What to replicate? [Editorial]. Implementation and Replication Studies in Mathematics Education, 1(2), 141153. https://doi.org/10.1163/26670127-01010015.

    • Search Google Scholar
    • Export Citation
  • Jankvist, U. T., Gregersen, R. M. & Lauridsen, S. D. (2021c). Illustrating the need for a ‘Theory of Change’ in implementation processes. ZDM — Mathematics Education, 53(5), 10471057. https://doi.org/10.1007/s11858-021-01238-1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jaworski, B. (2006). Theory and practice in mathematics teaching development: Critical inquiry as a mode of learning in teaching. Journal of Mathematics Teacher Education, 9(2), 187211. https://doi.org/10.1007/s10857-005-1223-z.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Karsenty, R. (2021). Implementing professional development programs for mathematics teachers at scale: What counts as success? ZDM — Mathematics Education, 53(5), 10211033. https://doi.org/10.1007/s11858-021-01250-5.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Koichu, B. (2018). Mathematical problem solving in choice-affluent environments. In G. Kaiser, H. Forgasz, M. Graven, A. Kuzniak, E. Simmt & B. Xu (Eds.), Invited lectures from the 13th International Congress on Mathematics Education (pp. 307324). Springer. https://doi.org/10.1007/978-3-319-72170-5_18.

    • Search Google Scholar
    • Export Citation
  • Koichu, B. & Keller, N. (2017). Implementation enterprise through the lens of a theory of diffusion of innovations: A case of online problem-solving forums. In T. Dooley & G. Gueudet (Eds.), Proceedings of the Tenth Congress of the European Society for Research in Mathematics Education (pp. 3841–3848). DCU Institute of Education, Dublin; ERME.

  • Koichu, B., Aguilar, M. S. & Misfeldt, M. (Eds.). (2021). Implementation and implementability of mathematics education research [Special issue]. ZDM — Mathematics Education, 53(5).

    • Search Google Scholar
    • Export Citation
  • Koichu, B., Cooper, J. & Widder, M. (2022). Implementation of problem solving in school: From intended to experienced. Implementation and Replication Studies in Mathematics Education, 2(1). DOI: 10.1163/26670127-bja10004.

    • Search Google Scholar
    • Export Citation
  • Koljonen, T., Helenius, O. & Ahl, L. M. (2022). The documentational approach to didactics as a gauge of fidelity of a teaching model implementation [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Krainer, K. (2015). Reflections on the increasing relevance of large-scale professional development. ZDM — Mathematics Education, 47(1), 143151. https://doi.org/10.1007/s11858-015-0674-7.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krainer, K. (2021). Implementation as interaction of research, practice, and policy. Considerations from the Austrian initiative IMST. ZDM — Mathematics Education, 53(5), 11751187. https://doi.org/10.1007/s11858-021-01300-y.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Krainer, K. (2022). From ‘technical rationality’ and ‘reflective rationality’ to ‘societal rationality’ [Poster presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Misfeldt, M., Jankvist, U. T., Elicer, R., Geraniou, E., Bråting, K., Tamborg, A. L. (2022). Comparing the implementation of programming and computational thinking in Denmark, Sweden and England [Poster presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Nilsen, P. (2015). Making sense of implementation theories, models and frameworks. Implementation Science, 10, Article number 53. https://doi.org/10.1186/s13012-015-0242-0.

    • Search Google Scholar
    • Export Citation
  • Østergaard, M. K. & Jankvist, U. T. (2022). When researcher and teacher talk past each other: An IR analysis [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Patsiala, N. & Papadopoulos, I. (2022). Developing an instrument to connect problem-posing strategies and mathematical habits of mind [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Prytz, J. (2021). When research met policy: A history of innovation and a complicated relationship in three Swedish development projects in mathematics education, 1960–2018. ZDM — Mathematics Education, 53(5), 10351046. https://doi.org/10.1007/s11858-021-01247-0.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Prytz, J., Ahl, L. M., Jankvist, U. T. (2022). What is a successful implementation in mathematics education? On sustainable innovations and the role of textbooks [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

  • Roesken-Winter, B., Stahnke, R., Prediger, S. & Gasteiger, H. (2021). Towards a research base for implementation strategies addressing mathematics teachers and facilitators. ZDM — Mathematics Education, 53(5), 10071019. https://doi.org/10.1007/s11858-021-01220-x.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rogers, E. M. (1962). Diffusion of innovations. Free Press of Glencoe, Glencoe, NY.

  • Tamborg, A. L., Nøhr, L. & Misfeldt, M. (2022). Towards designing a comparative survey for implementing PCT in Danish, Swedish, and English K-9 mathematics education [Paper presentation]. 12th Congress of the European Society for Research in Mathematics Education, 2–5 February 2022, Bozen-Bolzano, Italy.

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 247 247 9
PDF Views & Downloads 375 375 14