Abstract
The indigo Impact Bond Dataset is an open-access dataset that describes a specific form of impact-focused cross-sector partnership adopted worldwide since 2010. These partnerships are data-rich in principle, yet historically, little data is shared and re-used. The dataset is the result of an engaged, collaborative process where different organisations involved in impact bond projects share data with the indigo initiative data stewards so that practitioners and researchers can analyse and learn from these partnerships. This article introduces the dataset in terms of scope, data collection methods, and data model. The authors provide descriptive summaries of the current landscape and demonstrate a practical application of the dataset. In closing, they discuss future avenues for research and dataset development, as well as the limitations of working with a collaborative approach.
- –Related data set “Impact bond dataset” with doi www.doi.org/10.5287/bodleian:6RxneM0xz in repository “Oxford University Research Archive”
- –See the showcase of the data in the Goverment Outcomes Lab: https://golab.bsg.ox.ac.uk/knowledge-bank/indigo/impact-bond-dataset-v2
1. Introduction
Cross-sector partnerships involving the public, private and social sectors are increasingly acknowledged as an important approach for tackling societal issues (Purdy & Gray, 2018). By drawing together a range of resources, innovative approaches, and collaboration between diverse stakeholders the promise is that these partnerships will better tackle the ‘wicked issues’ of our time, including widening social inequalities, climate change and health crises (Stadtler et al., 2024). There are well-known challenges in the coordination of such partnerships, and their proliferation and complexity also pose challenges for research (Hodge et al., 2010). The absence of public, standardised information on partnership arrangements and performance is a profound handicap for empirical work (Palcic et al., 2019; Petersen et al., 2018). The dataset that we describe in this article responds by setting out an open, standardised description of a recently developed contracting arrangement for cross-sector collaboration: social impact bonds1 and development impact bonds, or simply ‘impact bonds’.
Impact bonds (ib s) are a subset of socially motivated cross-sector partnerships. We define ib s as a relationship that includes two core factors: 1) a social outcomes contract, that is, a contract that links payment to social or environmental outcomes or goals, and 2) up-front repayable finance provided by a third party, the repayment of which is (at least partially) conditional on achieving specified outcomes.2 Unlike many cross-sector partnerships where organisational involvement and functions are vague and performance is unclear, data on impact bonds shed light on the contributions of key actors and (at least in theory) bring heightened accountability for the achievement of meaningful social objectives.
The focus on measurable outcomes means that ib accountability structures generate a wealth of data that can be used to guide learning and better practices, yet in practice, data on these partnerships is sparse, partial and dispersed (Carter et al., 2018; Fraser et al., 2018; Lagarde et al., 2013). In 2017, the Government Outcomes Lab (go Lab), a research team focused on cross-sector partnerships based at the University of Oxford, prepared a series of basic datasets and webpages to describe ib s in the UK. In 2019, the go Lab launched the first version of a global, open ib dataset and in 2020 initiated the International Network for Data on Impact and Government Outcomes (indigo, described further below) as an emergent data collaborative. This collaborative has enabled the most comprehensive, open and global impact bond database (Walker et al., 2023).
This article introduces the indigo Impact Bond Dataset (ibd) as a co-created, open learning platform. The intended audience for the dataset includes government policymakers, philanthropists and impact investors, social sector organisations, researchers, and academics. The article is organised as follows. The methods section (section 2) explains how data is collected, quality assured and hosted. Section 3 describes the data model and potential analytical approaches. The conclusion outlines future avenues for research and development, as well as the limitations of working with a collaborative approach.
The framing of a data collaborative is informed by the work of Susha, Janssen and Verhulst who define data collaboratives as “cross-sector (and public-private) collaboration initiatives aimed at data collection, sharing, or processing for the purpose of addressing a societal challenge” (2017, p. 2691). indigo includes a network of Data Stewards (Verhulst et al., 2020) who collect data from different ib projects and host this as open datasets and tools. At the time of writing, there are indigo Data Stewards in Oxford, United Kingdom, Sao Paulo, Brazil and Cape Town, South Africa.
2. Data Collection Method
The ibd is a co-produced dataset. Since no single organisation holds data on all aspects of each ib, a range of organisations voluntarily contribute data. This data is shared with the goal of building a common learning asset. The indigo core team oversees the collection, quality assurance, and hosting of the dataset.
Figure 1 presents the process of data collection. There are several routes through which stakeholders can share data including via a web form, via an email to a Data Steward or through information – such as a press release or evaluation report – being circulated by a member of the indigo policy and practice network.3 Whenever the Data Steward finds publicly available data that is missing from the dataset, the Data Steward will start the process of updating records or creating a new record.
Update process for ibd guided by Data Steward
Citation: Research Data Journal for the Humanities and Social Sciences 2024; 10.1163/24523666-bja10043
prepared by authors and informed by indigo handbook for data stewards (unpublished).There are two different types of quality checks:
- 1)New projects require the creation of a new indigo poj id (a unique project-level identifier) and record. If there is no previous data, the Data Steward ensures that data is consistent and sources are reliable.
- 2)Existing projects require comparison between newly contributed data and extant data. It is possible that some data points are contradictory (e.g., projects report different financial values). In this case, the Data Steward confers with the original source of information, confirms which data is correct, and makes the necessary amendments. To minimise human error, a second Data Steward or indigo Data Analyst checks the data to ensure that the record reads well and that there are no errors.
Although building a collaborative dataset presents accuracy and data completeness challenges, it brings the opportunity to co-create a public learning asset and generate actionable insights with practitioners, for example through ‘hack and learn’ events (Outes Velarde et al., 2022).
3. Data Model and Insights
- –Impact bond dataset deposited at Oxford University Research Archive – doi: www.doi.org/10.5287/bodleian:6RxneM0xz
- –Temporal coverage: 2010–2024, continuing
The variables, definitions and data model that underpin the ibd are developed iteratively through dialogue between academics and practitioners (Bonsaver et al., 2021). What makes ib projects both pioneering and challenging to describe is that they are simultaneously public-private partnerships for social programmes, performance-related ‘outcomes’ contracts, and impact investment opportunities (Heinrich, 2018; see the left side of Figure 2).
Academic concept map (Heinrich, 2018) and Conceptual Entity Relationship Diagram for ibd model
Citation: Research Data Journal for the Humanities and Social Sciences 2024; 10.1163/24523666-bja10043
indigo data standardThe initial variables were developed by academic researchers at the go Lab, informed by public administration theory and prior empirical analysis of outcomes-based contracts (Carter & FitzGerald, 2017). Definitions were then tested with UK government officials, leading intermediaries such as Social Finance and Bridges Fund Management and with international stakeholders such as the Impact Bond Working Group and through consultation with other scholars including the go Lab academic advisory group. In 2019, core definitions were harmonised with the impact bond dataset held by the Brookings Institution.
The ibd is supported by a relational database (see right side of Figure 2). The database collects data not only on ib projects, organisations, and outcome funds but also on the relationships between them. A full set of variables and details on the data model are available in the indigo Data Standard.4
The ib ‘project’ is the foundational unit of analysis and represents ib s as a partnership which aims to improve social outcomes for a specific group of people. Each project can then be described through subcomponents, such as the contributing organisations and their roles, the outcome measures that are used to appraise success, the investment commitment to resource the project, and the results. The database has different tables for these components. Upstream sources of outcome payments (outcome funds) may be identified by linking to the indigo id of an outcome fund.
3.1 Coverage and Inclusion Criteria for Impact Bond Projects
To be included in the dataset the ib project must involve both an outcomes contract and up-front repayable finance (see Introduction) and the project partners must have signed a contract. A separate pipeline dataset is available for ib projects that are under development. Each ib project that begins work under a new contract, with a new target cohort of participants, a distinct geography, and/or with a later start date is counted as a separate project, with a separate unique identifier (indigo poj). The complete list of variables and definitions can be found in the indigo Data Dictionary (indigo, 2023). Regular updates on new projects and a snapshot of the landscape of impact bonds is available through the indigo Impact Bond Insights series (for example, Outes Verlarde et al., 2023).
Data at the project level allows users to describe the landscape of ib projects and key trends. As of March 2024, the ibd identifies 292 projects. These projects have raised more than 764 million usd as upfront capital and have engaged with more than 2.5 million service users. Figure 3 shows the distribution of these projects across policy domains. This data has been used by researchers to identify case studies that are responding to particular challenges. For example, Kabli et al. (2021) identify projects that deploy innovative financing models to tackle the global learning crisis.
Distribution of Impact Bond projects by policy area
Citation: Research Data Journal for the Humanities and Social Sciences 2024; 10.1163/24523666-bja10043
Figure 4 shows the distribution of service delivery locations for ib projects. This map allows users to analyse which countries or regions have more experience with this partnership model.
Distribution of impact bond projects’ service delivery locations
Citation: Research Data Journal for the Humanities and Social Sciences 2024; 10.1163/24523666-bja10043
3.2 Data on Project Components
Each ib project is described through a series of discrete subcomponents (shown on the right of Figure 2). As of March 2024, the data model has further components, such as technical assistance and performance scenarios but these are at the prototype stage and are not described in this article.
The service provision, investment and outcome payment commitment components have links to the organisations responsible for these functions. This linkage enables a better understanding of the configuration of the cross-sector partnerships. Economy et al. (2022) have used the ibd to investigate the variation in the design of Impact Bond projects across time to understand if the model has ‘stretched’ and compare the features of projects across the U.S. and the UK.
The outcome metrics and results components provide data on the pre-agreed objectives for each project and the actual results achieved. Using the outcome metrics and results components enables users to understand the success of these projects in their own terms. Policymakers who are interested in understanding which metrics have been used to assess the achievement of specific outcomes can use the dataset to identify measures that have been embedded in impact bond contracts.
3.3 Data at the Organisation Level
Every organisation that plays a role in an ib project is allocated a unique indigo organisation id (indigo-org-0000). Most are formal organisations, such as government departments, philanthropic ngo s, social investors, or charities. Some listed organisations are not formal organisations, but groups of people that come together to develop an ib project. For instance, the Standing Strong Impact Bond in the Netherlands (indigo-poj-0285) received investment from a large number of people via the crowdsourcing platform ‘One Planet Crowd’.
As of March 2024, the ibd includes 1,422 discrete organisations. The link between unique organisation id s, ib projects and components enables users to generate a range of visualisations. This data structure enables network analysis, for example, to identify nodal organisations in the development of an ib market or ecosystem. For example, Figure 5 illustrates the network of organisations working on ib projects in India. Arrows in Figure 5 indicate which organisations are working for which projects.
Network of organisations working on impact bond projects in India
Citation: Research Data Journal for the Humanities and Social Sciences 2024; 10.1163/24523666-bja10043
visualisation developed with the kumu appThe network shown in Figure 5 indicates that ib projects can involve different combinations of stakeholders. In India, there are some organisations working on multiple ib projects, which makes these organisations more central to the network than others. The same network analysis can be replicated for other countries and regions.
Further research may investigate trends in ib scale or investment or analyse whether there is an association between specific partnership structures and social issue areas. Scholars may use the data to investigate whether specific configurations or project characteristics are associated with better results.
3.4 Limitations
Adopting a collaborative approach enables indigo to improve data availability and increases the viability of detailed, comparative empirical research on cross-sector collaborations. Nonetheless, there are limitations. The dataset does not currently contain microdata that may be of interest to project evaluators and there are challenges in using the raw data for comparative analysis. For example, the measurement approach for project success is highly varied: one project may aim to support 160 people who are unemployed into full-time work and capture this data at a participant level (before and after analysis), another project may use a quasi-experimental impact evaluation to detect whether there has been a statistically significant reduction in reconvictions amongst a group of former prisoners. The dataset describes these varied metrics but considerable cleaning is required to harmonise across a diffuse sample of ib projects. Those wishing to conduct comparative analysis across projects should confirm whether the variables of interest are appropriately standardised.
Since organisations share data on a voluntary basis, without willingness or resources, data will not be represented in the dataset. Data completeness varies significantly across projects. Although basic descriptive variables are well populated, (stage of development, dates, policy sector, involved organisations) variables on outcome metrics, pricing and results are less consistently available. We plan to describe data completeness more fully in a forthcoming report. This is an important area for future work as varied data sharing practices may result in a biased dataset.
4. Concluding Remarks
The indigo ibd is an online, publicly available dataset with accompanying visualisations that provides policy practitioners and researchers with a tool for the study of impact bond projects around the world. Our goal in this article is to describe not just the data and data model, but also the collaborative process through which the data is collected and made available. We hope that, as the use of the dataset becomes widespread, the ibd will be used to better understand different ways in which cross-sector partnerships can come together and tackle social issues in different contexts.
The next steps for the indigo collaborative include linking the ibd to other datasets and indicators, such as the UN sdg s Database, the Open Contracting Data Standard and the 360 Giving datasets. Questions, comments, and collaboration possibilities are most welcome at indigo@bsg.ox.ac.uk.
Acknowledgements
This article is about a dataset that describes impact-focused cross-sector partnerships, but the ibd is itself the product of a cross-sector partnership. The ibd would not exist were it not for the support and contribution of many generous people and organisations. In no particular order, we would like to thank Clare FitzGerald, Tanyah Hameed, Stefan Baconschi, Patricia Silva Castillo, Aigerim Zhaparova, Franziska Rosenbach, Srinithya Nagarajan, Ruth Dixon, Andreea Anastasiu, Reiji Ikeda, Nigel Ball, Aidan Shilson-Thomas, Josh Pritchard, and all the researchers who are part, or were part, of the Government Outcomes Lab and contributed to the efforts toward building the ibd. We would also like to thank the Research Data Journal’s editorial team and anonymous reviewers.
An important part of the ibd has been collected and shared by participants of the indigo Hack and Learn events. We also thank these participants, who volunteered their time and effort to create a more complete dataset. We thank our partners and learning community for their continuous support and particularly thank Children’s Investment Fund Foundation; Social Finance UK; Social Finance Netherlands; Social Finance Israel; Bridges Outcomes Partnership; Social Ventures; Big Issue Invest; Big Society Capital; Acrux Partners; Maze; Instiglio; World Bank; Open Data Services, Cursive, Education Outcomes Fund; British Asian Trust; ubs Optimus Foundation; Department for Culture, Media and Sport; The National Lottery Community Fund; UK Foreign, Commonwealth and Development Office; the US Agency for International Development. This work has been partially supported by ukri grant mr/T040890/1. This is not an exhaustive list of contributors but illustrates how the ibd has grown and developed thanks to the generosity and openness of many people and organisations.
References
Bonsaver, L., Gibson, M., & Ball, N. (2021). Government Outcomes Lab: The first five years. https://golab.bsg.ox.ac.uk/knowledge-bank/resource-library/government-outcomes-lab-the-first-five-years.
Carter, E., & FitzGerald, C. (2017, November 2). Too many promises? Social impact bonds and ‘improved’ social interventions. 2017 APPAM Fall Research Conference. https://appam.confex.com/appam/2017/webprogram/Paper22231.html.
Carter, E., FitzGerald, C., Dixon, R., Economy, C., Hameed, T., & Airoldi, M. (2018). Building the tools for public services to secure better outcomes: Collaboration, prevention, innovation. Government Outcomes Lab, Blavatnik School of Government, University of Oxford. https://golab.bsg.ox.ac.uk/documents/176/BSG-GOLab-EvidenceReport.pdf.
Economy, C., Carter, E., & Airoldi, M. (2022). Have we ‘stretched’ social impact bonds too far? An empirical analysis of sib design in practice. International Public Management Journal, 0(0), 1–24. www.doi.org/10.1080/10967494.2022.2077867.
Fraser, A., Tan, S., Kruithof, K., Sim, M., Disley, E., Giacomantonio, C., Lagarde, M., & Mays, N. (2018). Evaluation of the social impact bond trailblazers in health and social care. Final Report, 158.
Heinrich, C. (2018). “What exactly [is] the public management Emperor now wearing …?” Comparingsib s and outcomes-based approaches across different countries and policy sectors. Social Outcomes Conference, Blavatnik School of Government, University of Oxford.
Hodge, G. A., Greve, C., & Boardman, A. E. (2010). Introduction: The ppp Phenomenon and its evaluation. In International Handbook on Public–Private Partnerships. Edward Elgar Publishing. www.elgaronline.com/display/edcoll/9781848443563/9781848443563.00008.xml.
indigo. (2023). indigodata standard (4.0). GitHub. https://indigo-data-standard.readthedocs.io/en/latest/index.html.
Kabli, A., Rizzello, A., & Trotta, A. (2021). Roadmapping new impact bonds in a post-covid world: Insights from case studies in the education sector. Sustainability, 13(8), Article 8. www.doi.org/10.3390/su13084121.
Lagarde, M., Wright, M., Nossiter, J., & Mays, N. (2013). Challenges of payment-for-performance in health care and other public services – design, implementation and evaluation (p. 36). Policy Innovation Research Unit, London School of Hygiene and Tropical Medicine. https://piru.ac.uk/assets/files/Challenges%20of%20payment%20for%20performance%20in%20healthcare%20and%20other%20public%20services%20final.pdf.
Outes Velarde, J., Nagarajan, S., Bregazzi, H., Ikawa, J., Degirmen, E., Lowe, A., Goomer, I., & Gangwar, A. (2022). Reflections on the 2022 indigo summer hack and learn. The Government Outcomes Lab. https://golab.bsg.ox.ac.uk/community/blogs/reflections-2022-summer-hack-and-learn.
Outes Verlarde, J., Chauhan, M., Grennan, E., Nagarajan, S., & Paul, O. (2023). indigoimpact bond insights report—October 2023. Government Outcomes Lab, University of Oxford. https://golab.bsg.ox.ac.uk/knowledge-bank/resource-library/indigo-impact-bond-insights-october-2023.
Palcic, D., Reeves, E., & Siemiatycki, M. (2019). Performance: The missing ‘P’ in ppp research? Annals of Public and Cooperative Economics, 90(2), 221–226. www.doi.org/10.1111/apce.12245.
Petersen, O. H., Hjelmar, U., & Vrangbaek, K. (2018). Is contracting out of public services still the great panacea? A systematic review of studies on economic and quality effects from 2000 to 2014. Social Policy and Administration, 52(1), 130–157. Social Science Premium Collection. www.doi.org/10.1111/spol.12297.
Purdy, J., & Gray, B. (2018). Collaborating for our Future: Multistakeholder partnerships for solving complex problems. University Press. www.doi.org/10.1093/oso/9780198782841.001.0001.
Stadtler, L., Seitanidi, M. M., Knight, H. H., Leigh, J., Clarke, A., Le Ber, M. J., Bogie, J., Brunese, P., Hustad, O., Krasonikolakis, I., Lioliou, E., MacDonald, A., Pinkse, J., & Sehgal, S. (2024). Cross-sector partnerships to address societal grand challenges: Systematizing differences in scholarly analysis. Journal of Management Studies, n/a(n/a). www.doi.org/10.1111/joms.13053.
Susha, I., Janssen, M., & Verhulst, S. (2017). Data collaboratives as a new frontier of cross-sector partnerships in the age of open data: Taxonomy development. Hawaii International Conference on System Sciences. www.doi.org/10.24251/HICSS.2017.325.
Verhulst, S. G. V., Zahuranec, A., Young, A., & Winowatan, M. (2020). Wanted: Data Stewards – (Re-)Defining the roles and responsibilities of Data Stewards for an age of data collaboration. GovLab. www.doi.org/10.15868/socialsector.40383.
Walker, T., Goubran, S., Karami, M., Dumont-Bergeron, A., Schwartz, T., & Vico, K. (2023). Mainstreaming social impact bonds: A critical analysis. Finance Research Letters, 53, 103546. www.doi.org/10.1016/j.frl.2022.103546.
Impact Bonds are not ‘bonds’ in a conventional sense as repayment to investors is contingent on the successful achievement of outcomes.
More details on the definition of Impact Bond projects and the inclusion criteria for the dataset can be found via the indigo Data Dictionary at https://indigo-data-standard.readthedocs.io/en/latest/data-dictionary/index.html.
The indigo mailbox is indigo@bsg.ox.ac.uk.
The indigo Data Standard is available at https://indigo-data-standard.readthedocs.io/en/stable.
