1 Introduction
The objectives of the EU Framework Programmes for Research and Innovation presented in the first Framework Programme (FP1) in 1984 as well as in subsequent programs (such as enhancement of Europe’s competitiveness, creation of the knowledge-based economy, contribution to the realization of the Single Market, achievement of sustainable development, economic growth, and an inclusive and user-friendly information society) (EPEC, 2011), serve as instruments for the implementation at the EU level of a strategy for the development of R&D activities facing Societal Challenges (European Commission, 2012).
These are described in the Lund Declaration (Lund Declaration, 2009), which states that the European research community “has put much emphasis on the necessity (…) to respond to the Grand Challenges of our times” (Chuberre & Liolis, 2010).
Horizon 2020 (H2020), the biggest EU research and innovation funding program with a budget of nearly €80 billion for the period 2014 to 2020, is structured around Grand or Societal Challenges (European Commission, 2020a), which goes in line with previous suggestions put forward by many researchers and policy-makers (ERA Rationales Group, 2007; Georghiou, 2008; European Commission, 2009), and emphasizes challenges such as:
Health, demographic change and wellbeing;
Food security, sustainable agriculture and forestry, marine and maritime and inland water research, and the bioeconomy;
Secure, clean and efficient energy;
Smart, green and integrated transport;
Climate action, environment, resource efficiency and raw materials;
Europe in a changing world inclusive, innovative and reflective societies;
Secure societies – protecting freedom and security of Europe and its citizens.
As the timeframe of H2020 has already ended, and it can be expected that most of the financed projects can be already evaluated, the purpose of this chapter is to assess H2020 Health, Demographic Change and Wellbeing program effects measured by outputs divided into four groups: economic, academic, health, and social media, using officially available data. The obtained results, which grasp only partly the possible impact of the financed projects, call for a holistic approach, which may potentially deliver more accurate data on the effects of public policy, with Horizon 2000 being one of its tools.
2 Theoretical Background
2.1 Rationale for the Government Intervention
The significance of innovation in enhancing the efficacy of businesses and economic growth of countries is the most compelling argument for government policy to promote innovation (Crépon, Duguet, & Mairesse, 1998; Van Leeuwen & Klomp, 2006). At both the macro- and microeconomic levels, innovation is a critical component of international competitiveness (Brusoni, Cefis, & Orsenigo, 2006; Halpern, 2007), while technological gap theory suggests that innovation is a critical component of international competitiveness at the sector level (Posner, 1961; Soete, 1981).
According to economic theory (Nelson, 1959; Arrow, 1962), a firm will not invest in innovation unless it can capture and take advantage of all of the benefits (Luukkonen, 2000).
Innovation policy, which is a component of the state’s economic policy, is a system of public administration activities (at various levels – national, regional, and local) that promote the development of new solutions, as well as their dissemination and implementation (Weresa, 2014, p. 87). Two premises underpin the rationality of the innovation policy: market failure and system failure (de Jong et al., 2008; de Jong et al., 2010).
Market failure manifests itself in the following elements: limited intellectual property protection, uncertainty associated with a high probability of failure of an innovative project, limited divisibility of a process that requires a relatively smooth and uninterrupted inflow of funds, and information asymmetry (von Hippel, de Monaco, & de Jong, 2014).
The failure of the market justifies the use of novel policy tools such as R&D subsidies, basic research support at universities or research institutes, and the establishment of an intellectual property protection system. In the case of market failure caused by individual innovators’ unwillingness to share innovation with its adapters, it is critical to promote cooperation in innovation, which allows the costs of innovation as well as potential benefits to be shared at an earlier stage of the process (de Jong et al., 2015).
The most frequently mentioned factors influencing system failure and justifying state intervention are: insufficient innovative abilities of innovation system participants – enterprises, research institutes, venture capital availability; insufficient cooperation skills, which are not always self-contained and sometimes need to be stimulated. The final component is the unreliability of the system’s framework, which includes elements such as values and norms, as well as consumer demand.
In this context, innovation policy has to affect an increasing number of areas of business activity, as well as new groups of enterprises (OECD, 2005). This, in turn, leads to an increase in the number of influence tools used by decision-makers, which can be classified into four categories: regulations (legal regulations, norms, standards, prohibitions, and limits); systemic instruments (statutory financial incentives); government programs and projects (including public procurement); and instruments supporting organizations intermediating in innovation processes (Jasiński, 2010; Jasiński, 2013; Jasiński, 2014, p. 76).
There is a general assumption among decision-makers that increased public support for R&D activities leads to an increase in R&D expenditure in an organization and, as a result, an increase in its level of innovativeness. It is especially important, as according to studies on innovation, one of the greatest impediment to its introduction is a lack of financial resources within the enterprise (Guijarro-Madrid, Garcia, & Van Auken, 2009; Watkins & Paff, 2009; Lewandowska, 2012; Madeira, 2017; Moura et al. 2019).
The government can help businesses via a variety of tools, including grants, loans, subsidies, preferential loans, loan guarantees, tax reductions, and tax deferral.
Grants do not come without drawbacks, such as asymmetry of knowledge between an innovator and a government agency, costly procedures, corruption, and, in many cases, political pressure (Czarnitzki, Hamel, & Rosa, 2011; Czarnitzki & Lopes-Bento, 2014; Hünermund & Czarnitzki, 2019).
Incentives and tax credits (deferrals in paying taxes and tax credits for R&D, enabling the reduction of burden on remuneration related to R&D activity, preferential rates on royalties and other income related to knowledge resources) can be used as market tools to reduce the marginal costs of R&D activities. As it does not need arbitrary decisions about the distribution of support to individual sectors of the economy, industries, and firms, this method of solving the challenge of financing innovation may be more effective than direct support for R&D operations. As a result, more businesses are motivated to innovate (OECD, 2012; Gande et al., 2020; Gaessler, Hall, & Harhoff, 2021; Ivus, Jose, & Sharma, 2021). Unfortunately, in the Covid pandemic era, traditional tools to deal with market failures seem to be no longer adequate, and in order to progress the economy, a new concept, called the “Entrepreneurial State” (Mazzucato, 2013), must be put in place, which means that governments are engaged in the process of bringing new products and services to market and creating a market rather than merely holding the current market in place (Mazzucato, 2018; 2019; Mazzucato, 2021).
Thus, a challenge-based approach, creation of markets, and integration of supply and demand-side policies are the three central ideas of mission-oriented policies (Georghiou, 2008). Orienting policy toward a specific mission necessitates the adoption of two factors (European Commission, 2017a). The first and foremost requirement is accountability. Whatever the mission, the institution that has been “mandated” to carry it out should be held accountable for the decisions made, the processes followed, and the outcomes obtained. Measurability is a second, related component. Keeping track of whether the task is being completed, especially if goals have been defined, provides for more precise and accurate assignment of responsibility. This calls for designing a holistic, new approach of evaluation of the results and impact of H2020 financing.
2.2 Horizon 2020: An Overview
The Europe 2020 strategy, which defines the development paths of European Union member states, identifies three mutually reinforcing priorities: smart, sustainable, and inclusive growth. The Horizon 2020 Framework Programme for Research and Innovation (2014–2020), established on 11 December 2013 by Regulation No. 1291/2013 of the European Parliament and of the Council, is a major program for financing research and innovation in the European Union and is part of this strategy.
The goal of H2020 is to develop European innovations of global significance and to build a competitive advantage for the European economy based on innovations in line with the Europe 2020 strategy and the Innovation Union initiative.
The Horizon 2020 combines three previously separate research support programs:
the 7th Framework Programme for Research, Technological Development, and Demonstration Activities;
an innovation-focused component of the Framework Programme for Competitiveness and Innovation (CIP) for 2007–2013; and
the work of the European Institute of Innovation and Technology.
It combines research and innovation with an emphasis on three key areas: “Excellent Science,” “Industrial Leadership,” “Societal Challenges,” and two additional priorities: Access to Risk Finance and Innovation in SME (European Commission, 2017). These key pillars are supplemented by specific objectives such as “Excellence and Broadening Participation,” “Science with and for Society,” and the work of the Joint Research Centre and the European Institute of Innovation and Technology.
2.3 Indicator-Based Approach as a Tool to Assess Results of Innovation Policy
An indicator is defined as “a parameter, or a value derived from parameters, which points to, provides information about, describes the state of a phenomenon/environment/area with a significance extending beyond that directly associated with a parameter value” (OECD, 1993). Indicators have two purposes: they summarize information and can be used to explain complicated phenomena to many stakeholders in a simplified form. There are various types of indicators that can be used to monitor and evaluate the performance of H2020 participants.
Generally, there are six different sorts of indicators: inputs, activities, throughput/outputs, immediate outcomes, intermediate outcomes, and ultimate outcomes. These indicators track the progression of the results chain.
The inputs, activities, and throughput/outputs of an investment address the “how” of an investment, whereas the varied outcomes represent the actual “changes” that occur: the development results. Financial, human, material, and information resources can all be used as inputs. Activities are actions taken in order to mobilize inputs and produce outcomes.
Throughputs and outputs are the indirect and direct results of an initiative’s activities. Immediate results (short-term outcomes) are changes that can be instantly attributed to an initiative’s outputs. Intermediate outcomes (medium-term outcomes) are improvements that are typically reached toward the end of a project and typically involve a beneficial behavior/practice change. The ultimate outcome (the purpose for an initiative) is the maximum level of change that can be legitimately assigned to the initiative in a casual manner, and it is the result of one or more intermediate results (European Commission, 2015). Indicators of efficiency represent the ratio of inputs required per unit of output produced. Indicators of effectiveness demonstrate the ratio of outputs required to generate one unit of outcome, or the extent to which outputs influence outcomes. The persistence of outcomes across time is measured by sustainability indicators. The evaluation of effectiveness and efficiency levels depends on the organization’s strategy and the aim to achieve (Etzkowitz & Leydesdorff, 2000; Laliene & Sakalas, 2014).
3 Assessment of Horizon 2020 Health, Demographic Change and Wellbeing Projects
3.1 Aims and Scope of Horizon 2020 Health, Demographic Change and Wellbeing
Nowadays Europe is confronted with four major healthcare challenges: (i) the increase in chronic diseases combined with an aging population and increasing societal demands; (ii) the influence of external environmental factors such as climate change; (iii) inequalities in healthcare access and (iii) the risk of losing the ability to protect the populations against the threats of infectious diseases, such as the Covid pandemic (European Commission, 2020b).
Thus, the H2020 Health component aims to keep older people active and independent for longer and support the development of new, safer, and more effective interventions. It also contributes to the sustainability of health and care systems.
The obstacles to achieving these goals include decreases in the number of people employed, population, and labor productivity, which increase public spending (European Commission, 2017).
H2020 Health, Demographic Change and Wellbeing was divided into three Work Programmes: 2014–2015; 2016–2107; 2018–2020.
For the years 2014 and 2015, the Horizon 2020 societal challenge of “Health, demographic change, and wellbeing” included 34 topics in the “personalizing health and care” focus area call and 16 topics in the “coordination activities” call. Eight additional actions designed to support the ’implementation of the challenge were also included, which were not subject to competitive calls for proposals.
For the Health, Demographic Change and Wellbeing Work Programme 2016–2017, the overall strategic focus was on the promotion of healthy aging and personalized healthcare. Research priorities included “personalized medicine, rare diseases, human bio-monitoring, mental health, comparative effectiveness research, advanced technologies, e/m-health, robotics, patient empowerment, active and healthy ageing, data security, big data, valorization, anti-microbial resistance, infectious diseases including vaccines, maternal and child health and the silver economy.” By aligning organizational priorities with evidence-based policies based on scientific research data, ICT solutions, and best practices in interventions, a faster development of evidence-based health and care policies is expected (European Commission, 2017).
The last Work Programme 2019–2020 incorporated numerous broad recommendations made in the Horizon 2020 interim evaluation, such as increasing social involvement and impact.
3.2 An Overview of Horizon 2020 Health, Demographic Change and Wellbeing Projects Based on Financial Data
According to financial data obtained from European Union Contact Points, until December 2020 there were 26,629 projects accepted under the whole Horizon 2000 program, with 138,875 participants. The total financing amounted to € 1,032,697,991,384, with the total project budget of €1,487,187,308,269. The average financing per participant was €7,436,169. These data are for both completed and still ongoing projects.
It is worth understanding the meaning of the term “participant.” One or more applying institutions submit a proposal to the European Commission to finance a project. If the proposal is accepted, it becomes a project that is carried out by one or more participants. A participant may be involved in more than one project, which is why the concept of “participations” exists (European Commission, 2014). Thus, “the number of participations,” or the number of grants awarded, does not directly translate into the number of organizations receiving co-financing because an organization can apply for co-financing multiple times.
The projects dealing with Health constitute 4% (1,045 in numbers) of the total number of H2020 projects, the share of participants was 7% (10,219 in numbers), the total financing was 9% of all H2020 projects (€98,103,433,963), with the total project budget of 10% (€150,007,185,986).
The average financing per participant in H2020 projects was €7,436,169 whereas for Health projects it was higher at €9,600,101. Further details are presented in Table 6.1.
Horizon 2020 and H2020 health, demographic change and wellbeing projects in numbers – data for period ended December 2020
Source: Author’s elaboration based on data from European Union Contact Points
As Figure 6.1 shows, the biggest beneficiaries of the H2020 Health funds, are the United Kingdom (€13,834,966,846), Germany (€13,423,309,892), France (€11,512,294,925), Spain (€9,703,927,696), Italy (€9,276,718,354), the Netherlands (€9,168,576,261), and Belgium (€6,757,710,382). There is a visible and striking difference between these old member states and the new ones, which are strongly lagging behind.
H2020 health, demographic change and wellbeing projects, by European Union country, completed and ongoing, data for period ended December 2020
Source: Author’s elaboration based on financial data from EU Contact Points
3.3 Key Performance Indicators for Horizon 2020 Societal Challenges Set by European Union
Horizon 2020 marks a move toward the use of indicators to track outcomes and impacts. There has traditionally been a focus on examining participant characteristics, R&D inputs, and EU-funded project outputs in evaluating the success of the Framework Programmes for Research. In Horizon 2020. more attention will be paid to measuring the program’s effects and their economic and social impact on Europe, particularly in the fields of science and technology (Horizon 2020 Indicators, 2015).
Key Performance Indicators were identified prior to the start of the Framework Programme, providing a solid foundation for the monitoring and evaluation of Horizon 2020, as well as a focus on measuring the results and impacts of the program.
For all Societal Challenges, including Health projects, the key performance indicators are as follows:
Patent applications and patents awarded in the area of the different Societal Challenges
Publications in peer-reviewed high impact journals in the area of the different Societal Challenges
Number of prototypes and testing activities
Number of joint public-private publications
New products, processes, and methods launched into the market.
The European Union set specific targets for two of these five indicators.
For patents, the target is 2 per €10 million funding (2014–2020) and for publications, it is set at 20 per €10 million funding (for all Societal Challenges).
For the three remaining indicators, the target was expected “to be developed on the basis of first Horizon 2020 results,” but there is no source available with these targets set to date. For details see Table 6.2.
Key performance indicators for H2020 societal challenges set by the European Union
Source: Based on European Commission (2015)
3.4 Preliminary Evaluation of H2020 Health, Demographic Change and Wellbeing Projects
The pilot evaluation of the H2020 Health, Demographic Change and Wellbeing projects is based on the input-output method of analysis. The efficiency analysis is also provided for selected indicators.
The input data, which are mainly financial, are obtained from EU Contact Points at the project level, whereas data on the indicators are retrieved from CORDIS (Community Research and Development Information Service for Science), where details of all EU-funded research projects and their outcomes are made publicly available. The database encompasses 100,000 project cases that stretch all the way back to the very first Framework Programme.
Table 6.3 presents the methodology of data collection for H2020 Health, Demographic Change and Wellbeing projects completed by December 2020. The starting point for the research was the list of N = 480 Health, Demographic Change and Wellbeing projects completed by December 2020 obtained from EU Contact Point. Out of this list, N = 314 projects were extracted, where the information about the throughput/output indicators in the Cordis database was fully available.
Methodology of data collection for the assessment of H2020 health projects completed by December 2020
Source: Author’s elaboration based on data obtained from European Union contact points matched with data from the Cordis database
This number constitutes 30% of all Health, Demographic Change and Wellbeing projects, that is, EU funding of €606,357,424, with total project budget of €735,209,023.
The throughput/output results of these N = 314 Health, Demographic Change and Wellbeing projects were as follows:
economic: 4 patent fillings, 135 demonstrators, pilots and prototypes;
academic: 1680 articles, 68 book chapters, 5 monographic books, 20 theses/ dissertations;
health: 1517 documents/reports, 104 websites, platforms, portals, 115 datasets via the OpenAIRE repository, 901 conference proceedings; and
media: 105 videos produced.
There were 164 projects where the only output was periodic reporting. For details see Figure 6.2.
Input and output analysis of H2020 health, demographic change and wellbeing projects, data for period ended December 2020
Source: Author’s elaboration based on data obtained from European Union contact points matched with data from Cordis
Out of these 11 identified indicators, only two of them: economic (patent applications) and academic (articles) have their targets, which means that the European Union sets the exact level of spending that has to produce a certain number of patent or articles.
For patents, there should be an average of 2 per €10 million funding (2014–2020), which means that the “cost” of one patent for the European Union as the “investor” is €5,000,000.
For publications, there should be an average of 20 per €10 million funding (for all Societal Challenges), which means, that the “price” of one publication that is “paid” by European Union is €500,000. It is important to note, that such a publication should appear in peer-reviewed high impact journals in the area of the different Societal Challenges.
Unfortunately, as regards the number of prototypes and testing activities; the number of joint public-private publications; new products, processes, and methods launched into the market, even though it is said that the target is “to be developed on the basis of first Horizon 2020 results,” there is no available publication with these data officially accessible. That is why any further investigation is complicated, as the only measurable outcomes are those for patents and publications.
In the list of N = 314 projects, there were N = 3 projects where patents were registered. The lowest “price” for a patent was €1,130,053 and the highest €4,234,330, which was still below the target set by the European Union at €5,000,000.
Out of the list of N = 314 projects completed by December 2020, in N = 113 selected H2020 Health, Demographic Change and Wellbeing projects with the financing of €487,585,975, there were 1,680 articles published at the average “cost” of €290,230, which seems to be low given the target set by EU at €500,000. One has to remember, however, that such a target is set for “publications in peer-reviewed high impact journals in the area of the different Societal Challenges” and in our research we took into account all the published articles, regardless of their quality and impact of journals.
There are striking difference in the number of articles published as the outcome of H2020 Health projects. There was one project with funding of €5,917,266, where 214 articles were published (which means the “average cost” of €27,658). It is hard to believe that all of them were published in high ranked journals. On the other hand there was one project with EU funding of €15,153,216 which “produced” only two articles, 5 documents and one website, which puts the “cost” of one publication at €7,576,608.
The details concerning the exact numbers are presented in Table 6.4.
Efficiency indicators for H2020 health, demographic change and wellbeing projects, data for period ended December 2020
Source: Author’s elaboration based on data obtained from European Union Contact Points matched with data from Cordis
4 Holistic Approach to Impact Assessment of H2020 Health Projects
In our opinion, the results obtained based on the input-output method and efficiency indicators do not provide sufficient information about attained objectives of the H2020 Health, Demographic Change and Wellbeing projects; what is more, they do not deliver data on the impact of the projects. We do believe that only a holistic approach to these issues, where mixed evaluation methods are used, would bring expected results.
The Interim Evaluation of Horizon 2020 (European Commission, 2017) presents a highly advanced model with 18 in-depth methods covering: expert groups, case studies, surveys, interviews, text mining, statistical analysis, documentary reviews, internal assessments, bibliometric analysis, patent analysis and social network analysis, which is far more advanced than the one presented here. Its weakness is that it needs extensive surveys, which is a costly exercise and requires a huge number of people to be involved. The presented approach is based solely on publicly available data.
The logic of the author’s proposed assessment methodology is presented in Figure 6.3.
Holistic assessment of H2020 health, demographic change and wellbeing societal challenge
Source: Author’s Elaboration
In Step 1, the H2020 Health, Demographic Change and Wellbeing rationale should be presented. The main source of knowledge here is data from H2020 web pages, related documents, and the Health Work Programmes 2014–2016, 2016–2017, and 2018–2020. This is similar to what is presented in this chapter, although, using a big data analysis, mapping of the goals and scopes of the program can be put forward. Content data from calls has to be gathered here as well.
In Step 2, input data should be investigated. This is what has also been done in this chapter but only regarding the financial data. The input data obtained from EU Contact Points are in fact mainly financial data covering issues such as: EU funds per project; funds per project per entity; total sum of the project; call type; but also project duration; type of entities, number of entities involved; coordinator. What can be done here is the ranking of the entities involved, for example based on university rankings, which may provide some qualitative assessment on the potential leverage effect, based on financial data, with some references to non-financial data.
In Step 3, throughput and output data should be gathered and analyzed. They should be categorized, similarly to what is presented here, into four groups: economic (patents, prototypes, etc.); academic (publications, dissertations), health (new drugs, new healthcare solutions, final reports, conferences), and media (press releases). Data (number of outcomes) should be collected on the project basis (one by one) from Cordis and OpenAIRE, and matched with the financial data gathered in Step 2. This allows the proportion of the invested funds to the measurable outcomes to be measured against the expected outcomes set by the European Union. A qualitative analysis should be performed especially for publications in order to investigate whether the publications meet the targets set by the European Union. Big data analytics should be applied here in order to capture the areas covered by the publications under study.
In Step 4, impact areas should be investigated using mixed methods. In the Economy and Academia part, the starting point are financial data (EU funds) for Private for Profit (PRC), Public Bodies (PUB) and Others (OTH) entities broken down between the countries identified in Step 1 and accompanied by qualitative characteristics of all the entities based on rankings, financial data, and so on. Using the input-output analysis based on funding data for particular entities from different countries representing specific qualitative characteristics, the impact of funding would be calculated for each EU country.
A similar analysis will be carried out for academia entities – Research Institutes (REC) and Higher Education (HES).
In the Health part, big data analytics would be performed for the content (text) of goals/objectives, final reports and other available information gathered in Step 3. The results obtained would be plotted on maps constructed based on the calls in Step 1. This would show to what extent the completed projects meet the scope and aims of H2020 Health, Demographic Change and Wellbeing.
In the Social Media part, a similar approach would be implemented. Based on the data gathered from the internet, the area of interest would be described (another big data approach) and plotted with the data on press release content from Step 3.
The operationalization of data and the method applied are explained in Table 6.5.
Description of the steps to be followed in holistic assessment, operationalization of the indicators, method applied
Source for Figure 6.3 and Table 6.5: Author’s Elaboration
The proposed methodology is universal and can be used not only to assess Health projects, but also any other projects funded under Horizon 2020.
5 Conclusions
The goal of this chapter was to provide a comprehensive overview of the Horizon 2020 Health, Demographic Change, and Wellbeing projects, as well as to deliver a preliminary assessment of how effective European Union investments are in terms of measurable outcomes in accordance with projected goals.
Health, Demographic Change, and Wellbeing projects account for 9% of the whole H2020 financing with 10,219 participants involved (by December 2020).
Based on the financial data of the completed as well as ongoing projects, it was shown that there is a striking discrepancy in the allocation of the EU Health, Demographic Change and Wellbeing funds between Western European and Central and East European countries. In order to cope with this inequality, a policy aimed at active cooperation between research organizations from the EU15 and the EU13 should be implemented. This should lead to stronger involvement of EU13 participants. Otherwise, the gap in innovation ability between the EU MS will rise.
In a further step of the research, conducted on N = 314 projects completed by December 2020, it was proved, that with the total funding of €606,357,424, there were only 4 patent applications, whereas the number of prototypes reached 135. This shows a potential which is not ultimately converted into a finalized output that has a commercial (marketable) value.
For academic outputs, the situation looks better, as there were 1,680 publications reported, but it was not further investigated if those were works of the highest quality according to the target set by European Union.
For health and media outputs, a massive number was produced: 1,517 documents and reports, 104 website platforms, 115 data sets; 901 conference proceedings, 105 video movies, but in order to assess their quality, a more detailed research is needed.
The lack of qualitative measurement is the serious limitation of this research and the input-output analysis itself, which has been conducted here. One of the solutions may be to base the “value” of a publication on the number of citations, but this requires a follow-up analysis to be introduced.
Such ex-post evaluation should be conducted two or three years after the funding program ends, as was the case with the evaluation of FP7 (Interim Evaluation of Horizon 2020, 2017).
In order to overcome these serious obstacles, a more comprehensive methodology, based mainly on big data (text) mining is proposed and explained. It is universal and can be implemented for other Horizon 2020 projects.
The implementation of such a methodology calls for a more open policy to be embarked on by the European Union, where data would be available at the project level and accessible in an easier and more user-friendly way. Such change in data availability as well as assessment methodology is needed now but also for the future, as many researchers and policy-makers underline the imperative to shift the focus from R&D inputs to the whole impact of complex systemic interactions involving basic and applied research, development, innovation, diffusion, and all the associated spill-overs, and as a result the implementation of a mission-oriented R&D policy (European Commission, 2017a, p. 8).
References
Arrow, K. J. (1962). Economic welfare and the allocation of resources for invention. In R. R. Nelson (Ed.), The rate and direction of inventive activity: Economic and social factors. National Bureau of Economic Research, Conference Series (pp. 609–625). Princeton University Press.
Brusoni, S., Cefis, E., & Orsenigo, L. (2006). Innovate or die? A critical review of the literature on innovation and performance, Working Paper No. 179. CESPRI, Universita Commerciale Luigi Bocconi.
Chuberre, N., & Liolis, K. (2010). Contribution to Grand Societal Challenges. Integral SatCom Initiative. Retrieved January 7, 2021, from http://ec.europa.eu/invest-in-research/pdf/download_en/isi_contribution.pdf.
Crepon, B., Duguet, E., & Mairesse, J. (1998). Research, Innovation and Productivity: An Econometric Analysis at the Firm Level. Economics of Innovation and New Technology, 7, 115–158.
Czarnitzki, D. & Lopes-Bento, C. (2014). Innovation subsidies: Does the funding source matter for innovation intensity and performance? Empirical evidence from Germany. Industry & Innovation, 21(5), 380–409.
Czarnitzki, D., Hanel, P., & Rosa, J. M. (2011). Evaluating the impact of R&D tax credits on innovation: A microeconometric study on Canadian firms. Research Policy, 40(2), 217–229.
de Jong, J. P. J., Kalvet, T., & Vanhaverbeke, V. (2010). Exploring a theoretical framework to structure the public policy implications of open innovation. Technology Analysis & Strategic Management, 22(8), 877–896.
de Jong, J. P. J., Vanhaverbeke, V., Kalvet, T., & Chesbrough, H. (2008). Policies for open innovation: Theory, framework and cases. VISION Era-Net.
de Jong, J. P. J., von Hippel, E., Gault, F., Kuusisto, J., & Raasch, Ch. (2015). Market failure in the diffusion of consumer-developed innovations: Patterns in Finland. Research Policy, 44(10), 1856–1865.
EPEC. (2011).Understanding the long term impact of the Framework Programme. Evaluating the long term impact of Framework Programme research – Final Report. Retrieved June 6, 2021, from http://ec.europa.eu/research/evaluations/pdf/archive/other_reports_studies_and_documents/long_term_impact_of_the_fp.pdf.
ERA Rationale Group. (2007). Challenging Europe’s research: Rationales for the European Research Area (ERA). Report of the ERA Expert Group. Retrieved August 12, 2021, from https://ec.europa.eu/research/era/pdf/eg7-era-rationales-final-report_en.pdf.
Etzkowitz, H., & Leydesdorff, L. (2000). The dynamics of innovation: From National Systems and “Mode 2” to a Triple Helix of university–industry–government relations. Research Policy, 29, 109–123.
EuroAccess. (2022). Programme: Health, Demographic Change and Wellbeing. Retrieved March 11, 2022, from https://www.euro-access.eu/programm/health_demographic_change_and_wellbeing.
European Commission. (2015). Horizon 2020 indicators. Assessing the results and impact of Horizon 2020. Retrieved May 17, 2021, from https://op.europa.eu/en/publication-detail/-/publication/68686e76-8f53-11e5-983e-01aa75ed71a1.
European Commission. (2017). Interim evaluation of Horizon 2020. Retrieved August 11, 2021, from https://op.europa.eu/en/publication-detail/-/publication/fad8c173-7e42-11e7-b5c6-01aa75ed71a1/language-en/format-PDF/source-77918455.
European Commission. (2020a). Horizon 2020. Retrieved January 7, 2021, from https://ec.europa.eu/programmes/horizon2020/en/h2020-section/societal-challenges.
European Commission. (2009). The Role of community research policy in the knowledge- based economy. Expert Group report. Retrieved August 10, 2021, from https://op.europa.eu/en/publication-detail/-/publication/8edc4431-c8e7-49ba-a98a-ba0dc911ca13/language-en.
European Commission. (2012). The Grand Challenge. The design and societal impact of Horizon 2020. Retrieved June 7, 2021, from https://ec.europa.eu/programmes/horizon2020/en/news/grand-challenge-design-and-societal-impact-horizon-2020.
European Commission. (2013). Horizon 2020. Work Programme 2014–2015. Retrieved May 1, 2021, from https://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/main/h2020-wp1415-health_en.pdf.
European Commission. (2017a), Towards a mission-oriented research and innovation policy in the European Union. An ESIR Memorandum. Retrieved August 12, 2021, from https://op.europa.eu/en/publication-detail/-/publication/4177ae56-2284-11e8-ac73-01aa75ed71a1/language-en.
European Commission. (2017b). Work Programme 2016–2017 Retrieved May 1, 2021, from https://ec.europa.eu/research/participants/data/ref/h2020/wp/2016_2017/main/h2020-wp1617-health_en.pdf.
European Commission. (2020b). Work Programme 2018–2020. Retrieved May 1, 2021, from https://ec.europa.eu/research/participants/data/ref/h2020/wp/2018-2020/main/h2020-wp1820-health_en.pdf.
Gaessler, F., Hall, B. H., & Harhoff, D. (2021). Should there be lower taxes on patent income? Research Policy, 50(1). https://doi.org/10.1016/j.respol.2020.104129.
Gande, A., John, K., Nair, V. B., & Senbet, L. W. (2020). Taxes, institutions, and innovation: Theory and international evidence. Journal of International Business Studies, 51, 1413–1442. https://doi-org/10.1057/s41267-020-00375-1.
Georghiou, L. (2008). Europe’s research system must change. Nature, 452, 935–936 https://www.nature.com/articles/452935a.
Georghiou, L., Tataj, D., Celio, J., Gianni, S., Pavalkis, D., Vegranti, R., & Renda, A. (2018). Mission-oriented research and innovation policy: A RISE perspective. European Commission. https://doi.org/10.2777/426921.
Guijarro-Madrid, A., Garcia, D., & Van Auken, H. (2009). Barriers to innovation among Spanish manufacturing SMEs. Journal of Small Business Management, 47(4), 465–488.
Halpern, L. (2007). Literature survey on the links between innovation, competition, competitiveness, entry & exit, firm survival and growth. Working Paper no. 02/07, MICRO-DYN, EU Sixth Framework Programme. http://www.se2009.eu/polopoly_fs/1.8460!menu/standard/file/lund_declaration_final_version_9_july.pdf.
Hünermund, P., & Czarnitzki, D. (2019). Estimating the causal effect of R&D subsidies in a pan-European program. Research Policy, 48(1), 115–124.
Ivus, O., Jose, M., & Sharma, R. (2021). R&D tax credit and innovation: Evidence from private firms in India. Research Policy, 50(1). https://doi.org/10.1016/j.respol.2020.104128.
Jasiński, A. H. (2010). Innowacyjność polskiej gospodarki w okresie transformacji. Wybrane aspekty. Wydawnictwo Naukowe Wydziału Zarządzania Uniwersytetu Warszawskiego.
Jasiński, A. H. (2013). Innowacyjność w gospodarce Polski. Modele, bariery, instrumenty wsparcia. Wydawnictwo Naukowe Wydziału Zarządzania Uniwersytetu Warszawskiego.
Jasiński, A. H. (2014). Innowacyjność w gospodarce Polski. Modele, bariery, instrumenty wsparcia. Wydawnictwo Naukowe Wydziału Zarządzania Uniwersytetu Warszawskiego.
Laliene, R., & Sakalas, A. (2014). Conceptual structure of R&D productivity assessment in public research organisations. Economics and Management, 19(1), 25–25.
Lewandowska, M. S. (2012). Bariery finansowe a sprawność innowacyjna polskich przedsiębiorstw. In T. Rynarzewski & E. Mińska-Struzik (Eds.), Znaczenie innowacji dla konkurencyjności międzynarodowej gospodarki (pp. 350–362). Zeszyty Naukowe nr 246. Uniwersytet Ekonomiczny w Poznaniu.
Lund Declaration. (2009). Europe must focus on the grand challenges of our time. Swedish EU Presidency. Retrieved August 8, 2021, from http://www.se2009.eu/polopoly_fs/1.8460!menu/standard/file/lund_declaration_final_version_9_july.pdf.
Luukkonen, T. (2000). Additionality in EU Framework Programmes. Research Policy, 29(6), 711–724.
Madeira, M. J., Carvalho, J., Moreira, J. R. M., Duarte, F. A. P., & de Sao Pedro Filho, F. (2017). Barriers to innovation and the innovative performance of Portuguese firms. Journal of Business, 9(1), 2–22.
Mazzucato, M. (2013). The Entrepreneurial State: Debunking public vs. private sector myths. Anthem Press.
Mazzucato, M. (2018). Mission-oriented research & innovation in the European Union. Technical report, Directorate-General for Research and Innovation (European Commission).
Mazzucato, M. (2019). Governing missions in the European Union. Technical report, Directorate-General for Research and Innovation (European Commission).
Mazzucato, M. (2021). Mission Economy. Penguin, Allen Lane-Penguin.
Moura, D. C., Madeira, M. J., Duarte, F. A., Carvalho, J., & Kahilana, O. (2019). Absorptive capacity and cooperation evidence in innovation from public policies for innovation. International Journal of Innovation Science, 11(1), 2–19.
Nelson, R. R. (1959). The simple economics of basic scientific research. Journal of Political Economy, 67(3), 297–306.
OECD. (1993). OECD core set of indicators for environmental performance reviews. http://www.oecd.org/env/indicators-modelling-outlooks/24993546.pdf.
OECD. (2005). Governance of innovation systems: Volume 2. Case studies in innovation policy. OECD Publishing. https://doi.org/10.1787/9789264013452-en.
OECD. (2012). Tax incentives for research and development. OECD Science Technology and Industry Outlook. OECD Publishing.
Posner, M. V. (1961). International trade and technical change. Oxford Economic Papers, 13(3), 323–341.
Regulation (EU) No 1291/2013 of the European Parliament and of the Council of 11 December 2013 establishing Horizon 2020 – the Framework Programme for Research and Innovation (2014–2020) and repealing Decision No 1982/2006/EC. OJ L347/104.
Soete L. (1981). A general test of technology gap trade theory. Weltwirtschaftliches Archiv, 117(4), 638–660.
Van Leeuwen, G. & Klomp, L. (2006). On the contribution of innovation to multi-factor productivity. Economics of Innovation and New Technologies, 15(4/5), 367–390.
Von Hippel, E., DeMonaco, H. J., & de Jong, J. P. J. (2014). Market failure in the diffusion of clinician-developed innovations: The case of off-label drug discoveries. https://doi.org/10.1093/scipol/scw042.
Watkins, T. & Paff, L. A. (2009). Absorptive capacity and R&D tax policy: Are in-house and external contract R&D substitutes or complements? Small Business Economics, 33(2), 207–227.
Weresa, M. A. (2014). Polityka innowacyjna. PWN.