1 Introduction
Vector-borne diseases (VBD s) account for more than 17% of all infectious diseases and, according to the World Health Organization (WHO), cause more than 700,000 deaths each year, as well as substantial global economic burden (Sarma et al. 2019; Shepard et al. 2016). Climate change, international travel and trade, and rapid urbanisation, are shifting patterns in the transmission and burden of VBD s (Ryan et al. 2019). Dengue cases have increased substantially in recent decades with an 8-fold increase in the number of cases reported to the WHO between 2000 to 2019, alongside an over 4-fold increase in the number of deaths reported between 2000 and 2015. Following initial outbreaks in the Pacific, the emergence of chikungunya virus in the Americas in 2013 and Zika virus in 2015 resulted in major outbreaks in many countries leading to substantial morbidity and mortality, and, in some cases, overwhelming health systems (Puntasecca et al. 2021). Despite significant elimination efforts, malaria continues to be responsible for the majority of the global burden of VBD s, with nearly half of the world’s population thought to be at risk of malaria in 2020 (World Malaria Report, 2021).
Climate-informed early warning systems (EWS) integrate climate data to predict the risk of an infectious disease outbreak. These systems offer the potential for advanced warning of disease outbreaks, with longer lead times for public health action than decision-support systems based solely on outbreak detection through epidemiological surveillance. The timescale of forecasts produced by a climate-driven EWS typically range from short-term predictions (e.g. days or weeks ahead) to sub-seasonal or seasonal predictions from several weeks to several months ahead. As such, a climate-informed EWS can improve the timeliness and impact of public health response to outbreaks, and will become increasingly crucial as the climate crisis leads to more frequent climatic extremes. While traditional infectious disease monitoring systems focus on detecting and predicting infectious disease risk, it is important that an EWS is also linked to appropriate response mechanisms such that timely action can be taken to prevent an outbreak from occurring or mitigate potential disease (World Health Organization 2021). Depending on stakeholder needs and capacity, a climate-informed EWS could provide support for public-health decision making on when or where to begin vector-control activities, enhance surveillance, or increase hospital capacity. Climate-informed EWS are an example of a climate service, defined by the World Meteorological Organization (WMO) as a decision aide derived from climate information that assists individuals and organisations in society to make improved ex-ante decision-making (World Meteorological Organization 2015). Despite the established use of EWS in other fields, such as for the prediction of natural hazards for disaster risk reduction or prediction of food security risk in the humanitarian sector, the integration of climate-informed EWS within the health sector remains limited to date. A recent review of software tools for predicting outbreaks of climate-sensitive infectious diseases found few operationalised tools, and suggested that research in this field is rarely translated into usable decision-support tools for early-warning (Ryan et al. 2023).
End-to-end overview of the stages involved in the co-creation of an early warning system for vector-borne disease
In this chapter, we provide an end-to-end overview of the stages involved in the development of an EWS for vector-borne diseases, drawing on insights from the epidemic forecasting community and those from researchers and institutions working on the application of climate services for health. We will focus on the following stages in the development of an EWS: (1) engagement, (2) conducting a feasibility study, (3) tool development, (4) implementation and monitoring, and (5) evaluation (Figure 1). While we hope this series of steps provides a useful overview of the process of co-design and development of an EWS for vector-borne diseases, it is not intended to be prescriptive. In fact, the process of EWS development and implementation is likely to evolve as research and implementation leads to new learning, improved communication, and public health action.
1.1 Engagement
Developing an EWS which can be integrated into public health planning and lead to an improved public health response to outbreaks is a complex transdisciplinary process, and building partnerships between key stakeholders is essential. Co-creation should be central to the development of an EWS and engagement between different partners, including decision-makers and tool developers, should happen at all stages of EWS development. Once these partnerships have been established, the co-creation process can begin with partners jointly identifying the public health needs and priorities and setting out the broad aims for an EWS tool.
1.1.1 Build partnerships
The co-creation of an EWS requires well-coordinated interaction from transdisciplinary stakeholders spanning governmental institutions, academia, the private sector, and civil society organisations to ensure the end-to-end functionality of the system (UNDP 2018) (Figure 1).
Building strong partnerships at the outset of EWS development is important to establish trust (both in the EWS and between partners), improve confidence in the EWS, create a sense of ownership of the final product, promote institutionalisation of the EWS, and, importantly, ensure warnings are used to prevent an outbreak (where possible) or to attempt to reduce the rate of transmission and mitigate the impact of an outbreak (Githeko et al. 2018; Hussain-Alkhateeb et al. 2021; UNDP 2018). For example, a recent multi-year project by climate and health practitioners to build a dengue EWS for Barbados employed a co-creation approach and highlighted the importance of co-learning to build strong partnerships between the different actors required to translate scientific knowledge into public health action (Stewart-Ibarra et al. 2022). The team described how their co-creation process spanned all stages of EWS development, from identification of the problem to model building and implementation. They found that this engagement process built trust and shared ownership in the EWS, which are crucial to the system’s ultimate implementation and usefulness in informing public health action. Ultimately, only trusted EWS, which provide clear and useful warnings to decision-makers can lead to meaningful action.
1.1.1.1 Who needs to be convened?
Researchers, including environmental epidemiologists, spatial data experts, meteorologists, climate scientists, modellers, entomologists and infectious disease specialists, among others, provide the core analysis needed to develop an early-warning tool able to forecast future health risks, and to co-develop the technical guidance and standardised methodologies to interpret warnings. Early engagement of policymakers at the appropriate level from different ministries and agencies, such as, but not limited to, the Ministry of Health, national meteorological services and emergency services, ensure a greater chance of the EWS being used for a public health response, whereby the warning can be translated into widespread action. Government involvement is critical for the institutionalisation of any EWS. Ultimately only governments have the authority to create the institutional arrangements and legal frameworks, determine the mandates, roles, and responsibilities of interacting partners, disseminate early warning alerts to the public or take national-level public health action (IFRC 2012; UNDP 2018). In particular, the current lack of formal partnerships between health and climate sectors in many countries has been highlighted as a key barrier to the implementation of climate-informed early-warning systems. To date, few health and climate sectors have the mandate to focus on climate services for health and there is an urgent need to build stronger partnerships between these sectors in many countries, to facilitate research and implementation of EWS (Stewart-Ibarra et al. 2022).
Other important partners include the private sector, civil society organisations, non-governmental organisations, and the public themselves. The private sector may provide expertise in technology, infrastructure, data, or telecommunication and media to support the dissemination of warnings (UNDP 2018). Civil society organisations or the Red Cross Red Crescent Movement and their network of community-based volunteers are often highly trusted by communities and play a vital role in many countries before and during emergencies, including health emergencies. Finally, it’s important that the public trust the EWS and understand what action they should take on the warnings that are received, in a similar way to how the public understand and act upon probabilistic weather forecasts. It will often also be crucial to involve the public in the development of an EWS, for instance in helping to define the locally appropriate solutions, participating in risk assessments, identifying particular at-risk groups, informing response initiatives and emergency plans, and providing feedback and evaluation on how to improve the system (IFRC 2012; UNDP 2018).
1.1.2 Identify the problem
EWS usually target epidemic or outbreak-prone infectious diseases, as these can exhibit rapid surges in case numbers that threaten significant morbidity, mortality and pressure on the health systems (World Health Organization 2005). Common vector-borne diseases targeted by EWS include; dengue, chikungunya, Japanese encephalitis, Rift Valley fever, West Nile virus, Ross River virus, malaria, Murray Valley fever, Lyme disease, and Yellow fever.
Public health decision-makers may be interested in a range of policy questions that could be informed by a climate-informed EWS. These could include questions on when and where to target interventions (primarily vector control efforts), when to begin enhanced surveillance of a vector-borne disease, or what the most effective or cost-effective intervention may be in various scenarios. The forecasting targets of the EWS (that is, the epidemiological output of interest from a forecasting model) should align with the policy questions of interest to ensure that the EWS is able to provide relevant and usable warnings to decision-makers. Forecasting targets should also be well-defined at the outset of the co-creation process and well-documented in the reporting of EWS forecasts to allow for clear and rigorous statistical evaluation of the predictive accuracy of forecasts, and to facilitate comparison with alternative EWS models (Pollett et al. 2021). For instance, during the Dengue Forecasting Challenge in 2015, teams generated forecasts for three epidemiological targets representing key features of dengue outbreaks: the overall number of cases in a season (season incidence); the intensity of the epidemic peak (peak incidence) and the timing of peak incidence (Johansson et al. 2019). However, other relevant targets may be of interest to stakeholders, such as the probability of exceeding a user-defined outbreak threshold. Different forecasting targets may inform different response actions. For example, predictions about the timing of peak incidence in a season could inform decisions on when to begin vector control efforts, while predictions about the magnitude of peak and season intensity could inform decisions on whether to employ emergency vector control efforts or increase medical supplies and treatment capacity.
The geographical extent of the EWS should also be defined. EWS can provide warnings of disease risk at the national, regional or local level, depending on the spatial and temporal resolution required for decision-making (Macherera and Chimbari 2016). Moreover, within a country, certain areas may be more prone to outbreaks than others. For example, an EWS for malaria may be more useful in certain highland areas where conditions are rarely suitable for malaria transmission, than neighbouring lowland areas where transmission can occur all year round (World Health Organization 2005). That being said, as climate change shifts transmission zones and erodes natural barriers to transmission, it will become increasingly crucial for an EWS to be able to monitor and give warning of potential outbreaks in unexpected regions (Lee et al. 2021a). The geographical extent of an EWS should be regularly updated as climate change may create more favourable conditions over a wider geographical range.
1.2 Feasibility study
After identifying the target disease, the relevant public health decision-making questions and geographical scope of an EWS, the next stage involves an assessment of the feasibility of an EWS system for the setting and vector-borne disease being considered. Here, a joint identification and assessment is needed of the resources of key partners and institutions involved, the evidence on climate-health linkages relevant to the problem identified, and the data availability of relevant climate and epidemiological data.
1.2.1 Assess resources
Through engagement and collaboration, the co-creation of an EWS should build on existing capacity to maximise the feasibility of integrating the EWS into national programmes. However, a review of EWS found that most did not assess the feasibility of implementing the developed EWS into national programs ahead of development (Hussain-Alkhateeb et al. 2021). Ideally, EWS should be based on data the country has regular access to (such as local or regional meteorological stations or routine epidemiology data). This, to a large extent, is determined by the infrastructure and budget of a country. The type of EWS that can be implemented and maintained depends on the investments in technology (the surveillance system, data pipelines, dissemination means), the investments in human resources (such as the skill level of analysts), and the investments in preparedness plans (including training, drills, and mobilisation of responders) (UNDP 2018). Most EWS identified in a recent a systematic review demanded advanced analytical skills (Hussain-Alkhateeb et al. 2021) which may affect the sustainability of such systems in lower resourced settings and suggests there was insufficient co-design and engagement in the outset of the projects reviewed.
However, while EWS developed in research settings should consider the resources available to implement the tool at the country or local level, in many settings it is also imperative to advocate for increased financial and technical resources as well as capacity building, particularly within the health and climate sectors. This would enable the harmonization and mutual understanding of climate and epidemiological data sets between climate and health practitioners, allowing for better integration of climate data into health decision-making (Lowe et al. 2020b; Stewart-Ibarra et al. 2022). In particular, financial and technical investment is often needed to ensure the accessibility of historical climate and health data, which may be incomplete, use inconsistent formats, or be paper-based, and may require increased training or personnel devoted to data management (Lowe et al. 2020b). During a recent exploration of national stakeholder needs and perceptions in Barbados and Dominica, national climate and health stakeholders reported a lack of technical expertise in geographic information systems (GIS), statistics, modelling and computer programming (Stewart-Ibarra et al. 2019). As well as improving routine data analytics performed at a national or local level, this kind of technical training would aid in the development, operationalisation and interpretation of a climate-driven EWS.
1.2.2 Assess the evidence
There is a growing body of evidence describing and quantifying climate-health linkages; however, not all climatic indicators will be relevant in all settings and for all VBD s, and it is often logistically challenging to collect data on all possible climatic drivers of transmission and include them in potential models. As such, it is important to consider the evidence from the literature and previous EWS when making decisions on relevant data to collect and include in the model selection process.
Climate is an important driver in the transmission of many infectious diseases, influencing both human behaviour as well as the survival and life cycle of the pathogen (Myers et al. 2000; World Health Organization 2005). Climate is a particularly important driver of VBD s due to the additional impact of climate variability on natural history traits of the disease vector (including vector fecundity, development and survival) and transmission of VBD s is typically seasonal (Mordecai et al. 2017; Ryan et al. 2019). Ultimately, the suitability of a VBD for a climate-informed early warning system will depend on its sensitivity to climate variation.
1.2.2.1 Seasonal climate patterns
Temperature is a key climatic predictor of vector-borne disease transmission, with important effects on the lifecycle of the pathogen and the vector, among other factors. For example, the effect of temperature on arboviral transmission occurs through effects on viral replication as well as on Aedes mosquito survival, breeding, host-seeking and blood-feeding patterns (Bellone and Failloux 2020; Reinhold et al. 2018). Consequently, for most VBD s, transmission cannot occur below or above specific temperature extremes. For instance, 16 °C is thought to be the lower limit for the development of the malarial parasite Plasmodium falciparum (Patz and Olson 2006), while temperatures in the range of 18–34 °C are favourable for dengue virus transmission, with optimal transmission between 26–29 °C (Macdonald 1957; Mordecai et al. 2017). Rainfall is another important climatic driver of vector-borne disease transmission; however, the effects of rainfall on transmission are nuanced and dependent on the vector and setting being considered. Increased rainfall results in wet and humid conditions ideal for vector breeding and the creation of larval habitats, as in the case of Anopheles mosquitoes, which require monthly precipitation above 80 mm (Grover-Kopec et al. 2006). However, in some cases, excessive rainfall can also reduce transmission by washing larval habitats away entirely. This has been documented for dengue, as urban Aedes mosquito vectors typically lay eggs in small domestic water containers (Benedum et al. 2018; Koenraadt and Harrington 2008).
1.2.2.2 Extreme climatic events
Compared with seasonal fluctuations in temperature and rainfall, the effect of extreme climatic events (ECE s) on vector-borne disease transmission remains relatively understudied (Alcayna et al. 2022; McMichael 2015). However, ECE s such as flooding, drought or tropical storms can increase the risk of vector-borne disease outbreaks in some settings through the creation of larval habitats. For instance, droughts can, perhaps unintuitively, increase the risk of vector-borne disease transmission if they result in an increase in uncovered water storage containers in a particular setting. Both drought and excessive rainfall were found to increase the risk of a dengue outbreak, with differing delays, in Barbados and Brazil (Lowe et al. 2018, 2021).
The risk of a vector-borne disease outbreak can be further amplified by downstream impacts of ECE s such as population displacement or disruption to public health services, and disease outcomes could be worsened by the effects of concurrent food shortages or famine (Kovats et al. 2003). Understanding the linkages between ECE s and outbreaks of climate-sensitive infectious diseases is increasingly crucial as the climate emergency continues to increase both the frequency and magnitude of ECE s (Romanello et al. 2021). These insights can be incorporated into climate-informed EWS aiming to prevent or mitigate the harms from infectious disease outbreaks.
1.2.2.3 El Niño Southern Oscillation
El Niño Southern Oscillation (ENSO) is an irregular interannual climatic phenomena involving changes in sea-surface temperatures in the Pacific Ocean, which influences weather patterns around the world. ENSO oscillations can result in an El Niño event, with an anomalous temperature increase, or a contrasting anomalous temperature decrease, known as a La Niña event. El Niño and La Niña events are associated with an increased intensity of ECE s across the globe. El Niño events are associated with an average temperature increase globally of 0.5°C and an increased risk of drought in areas including Southern Africa, Southeast Asia, Indonesia and Australia, among others, and increased risk of extreme rainfall along the western coast of South America. Contrastingly, La Niña events have been associated with increased risk of flooding in the western Pacific and an increased frequency of tropical storms in areas such as the Caribbean and the Gulf of Mexico (Anyamba et al. 2019; Kovats et al. 2003). ENSO events can thus lead to increased vector-borne disease transmission through changes in temperature, rainfall or ECE s experienced in a particular setting.
Understanding the associations between interannual climate fluctuations and vector-borne disease transmission is valuable as it could allow for advanced warning of particularly high or low transmission seasons into a forecasting model, through the inclusion of an ENSO indicator (Kovats et al. 2003). Similarly, alternative indices such as the Indian Ocean Dipole (IOD), which measures oscillations in sea surface temperature in the Indian Ocean, are also associated with shifts in climatic patterns and subsequently with vector-borne disease risk, and may be more relevant than ENSO indicators in certain geographical settings (Chuang et al. 2017).
1.2.2.4 Non-climatic drivers
When considering relevant climate-disease linkages to incorporate into a climate- informed forecasting model it is also critical to evaluate other key drivers that may have a substantial impact on disease dynamics. The level of population immunity to a pathogen over time, which is continually shifting as new susceptibles enter the population and, in some cases, protective immunity wanes, is often a key determinant of the magnitude and timing of outbreaks. This is particularly evident when considering emerging diseases that are introduced into entirely susceptible populations. For instance, during the Zika and chikungunya outbreaks in Latin America, research has shown that lack of immunity to the pathogens was a dominant factor driving transmission in the Dominican Republic, with emergence occurring earlier in the year than optimal seasonal climatic conditions (Petrone et al. 2021). For pathogens with multiple serotypes, such as dengue virus, shifts in the dominant serotype in a setting can result in a change in the magnitude and timing of outbreaks (Rajarethinam et al. 2018).
Similarly, environmental or socioeconomic factors play an important role in the interaction of pathogens with vectors and humans. Certain characteristics of the environment can influence the likelihood of an outbreak occurring, including land cover characteristics, the level of urbanisation, the degree of environmental degradation, and the connectivity between regions, among others (Lee et al. 2021a; Semenza 2015; Ryan et al. 2023). The spread of the dengue vector from urban centres to neighbouring rural areas and urbanised forests in Brazil has been strongly associated with changes in the spatial connectivity, as well as forest degradation (Lee et al. 2021a; Lowe et al. 2020a). Where relevant, these variables can also be incorporated into modelling frameworks to improve the predictive power of models (Lowe et al. 2021). Additionally, incorporating climatic and non-climatic drivers within a single modelling framework can help to understand how, say, serological or environmental factors mediate climate-disease relationships and gain a better understanding of how multiple drivers work in combination to shape VBD risk.
1.2.3 Data availability
After considering the evidence on relevant climate-disease linkages and other non-climatic drivers for the particular problem targeted by a proposed EWS, model developers will need to consider the data streams and products available for a given setting. In practice, thinking about the available data streams will often occur in parallel to discussions about the ideal forecasting model structure for the problem at hand, and available data streams will shape the prediction tools. For instance, longer time-series are often required for statistical modelling approaches and will be particularly necessary for climatic drivers which experience significant inter-annual variability and in settings that may experience several unusually heavy rainy seasons in a row (Lee et al. 2021b).
In ideal circumstances, stakeholders should have open and equitable access to data which is updated regularly (Hussain-Alkhateeb et al. 2021; Ryan et al. 2023). In this section, we will describe the available data sources commonly used for developing forecasting models for EWS.
1.2.3.1 Climate data
As the evidence grows on the impact of climate, and of increasingly frequent climatic extremes, on human health, the availability of climate products for use in the health sector has increased (Fletcher et al. 2021). These services are informed by Earth observations, which comprise space or ground based climatic information describing atmospheric, oceanic or terrestrial conditions. Earth observations can be collected via local meteorological stations and using remote-sensing technologies such as satellite imagery (Fletcher et al. 2021).
Although many countries have networks of local meteorological stations, global coverage from local meteorological station data is patchy, can vary in quality, is often not easily accessible, and is often not directly applicable in large scale models (Ceccato et al. 2018). Satellite-derived climate products are valuable complements for local ground observations, as they provide continuous spatiotemporal historical observations (Fletcher et al. 2021). These datasets include a range of variables and vary in their resolution and spatiotemporal coverage (Table 1). When selecting the appropriate data product for analysis, there is often a trade-off to be made between the temporal and spatial resolution of data (sometimes referred to as the data granularity), the length of time-series available, and associated costs of data products that are not open-access. As an example, while the ERA5-Land products range from 1950 until the present at a 9 km resolution (Muñoz-Sabater et al. 2021), Landsat products offer 10-year time series at 30 metre resolution (‘Landsat Missions, U.S. Geological Survey.,’ 2022).
The choice of the gridded climate products should be informed by their ability to represent the local weather conditions and by considering the geographical scale of the problem at hand. For instance, fine scale data (i.e. 10 km2) can allow for detailed studies of how local microclimates may affect disease risk, while coarser data may be more appropriate to characterise climate-disease linkages across regions containing varied microclimates. Fletcher and colleagues recommended comparing global products with local observations where possible, and applying bias correction techniques to correct for data nuances (Fletcher et al. 2021). Collaboration with local meteorologists can help model developers understand discrepancies between satellite and ground-truth data and which bias correction or downscaling techniques might be more appropriate. This is an important step in EWS development as the use of a data product which is not well suited to the study area, or which contains systematic biases, may affect the characterisation of the climate-disease relationship being modelled, and subsequently impact the quality of model forecasts for early warning.
The most commonly used variables for modelling climate-sensitive VBD s are temperature (usually recorded as the mean, maximum, minimum temperature or temperature range in a given time unit) and rainfall (typically recorded as cumulative rainfall in a given time unit), because of their well-established impact on vector survival and breeding (Ryan et al. 2023). However, additional variables such as absolute or relative humidity, wind speed and direction, and diurnal temperature range, and alternative climate indicators, such as the Standardized Precipitation Index (SPI), have also proven useful in assessing risk of vector-borne disease (Colón-González et al. 2021; Lowe et al. 2018; Pley et al. 2021; Romanello et al. 2021).
Additionally, indicators summarising anomalous conditions in the Pacific Ocean, such as the Oceanic Niño Index (ONI) (a 3-month running average of sea surface temperatures in the Nino 3.4 region) have also been used in VBD modelling studies (Lowe et al. 2017).
1.2.3.2 Lags and lead-time
The goal of a climate-informed EWS is to give warnings of increased outbreak risk with enough lead-time to deploy interventions that can mitigate the impact of an outbreak (Runge-Ranzinger et al. 2016). This can be achieved either by incorporating lagged weather variables in the model (such that temperature or rainfall in the previous month, say, is used to predict the number of cases this month), or through access to climate forecast products which can then be used as inputs for infectious disease forecasting models (Morin et al. 2018). Climate forecast products are available at local and global levels, and it is possible to test model performance using different forecast horizons (Colón-González et al. 2021). Uncertainty in climate forecasts can also be incorporated into the modelling framework, and this may be particularly important when considering infectious disease forecasts with several months lead time. One example of this can be found in a study conducted by Lowe and colleagues which used climate forecasts to predict the 2016 dengue season in Machala, Ecuador. Here, the authors used an ensemble of 24 seasonal climate forecasts (each obtained by perturbing initial conditions) as an input into the modelling framework, thereby incorporating the uncertainty in forecasted precipitation and minimum temperature into predictions of dengue cases from 1 to 11 months ahead (Lowe et al. 2017).
1.2.3.3 Epidemiological and demographic data
At their core, most forecasting models for EWS rely on surveillance data from routine surveillance systems capturing weekly or monthly reported cases (as well as deaths or severe outcomes) from a particular disease, often stratified by age or geographical units. Surveillance systems can also provide information about the conditions that enable transmission that may be useful for developing control strategies. In the case of vector borne diseases, vector species and pathogens can also be monitored (Braks et al. 2019). Cases reported through surveillance systems may be suspected cases, confirmed cases via epidemiological link, or laboratory confirmed cases.
Epidemiological data from surveillance systems has several key limitations, including delays in reporting, changes in reporting mechanisms over time and missing values or errors in data entry (Hussain-Alkhateeb et al. 2021). Additionally, for pathogens that result in a large proportion of asymptomatic or mild disease, such as dengue virus, surveillance systems may only detect a small proportion of overall infections. These limitations mean that interpreting patterns in surveillance data can be challenging.
Global climate datasets openly available online (updated from Fletcher et al. 2021)
When available, demographic information is useful to understand the underlying structure of the population at risk. Population sizes change over time, and it is necessary to account for these changes when creating forecasting tools. Census data is the primary source of demographic information, this is typically collected by governmental institutions and made available to the public. However, as these are not performed every year, population projections are necessary to complete the time series. Alternatively, there are gridded products, such as WorldPop, that provide population estimates in raster format (‘WorldPop,’ 2022). Contact patterns, population mobility, age structure and socio-economic conditions, among other variables, can also be incorporated into forecasting models to better capture pathogen spread in a population and the resultant burden of disease.
Epidemiological information from scientific literature can also be incorporated into forecasting models. For instance, estimates on the force of infection, R0, case fatality ratio, or hospitalisation fatality ratio in a particular setting can provide information to parameterise mechanistic models (Lee et al. 2021b; Lessler et al. 2016).
1.2.3.4 Environmental data
Spatial information is also available on environmental or socioeconomic factors, such as land cover, urbanisation or environmental degradation. In these datasets, spatial units or grid cells are categorised into different classes based on satellite imagery of the Earth’s surface. Land cover datasets describe the physical material on the surface of a country or region, for instance showing whether areas are covered by cropland, grasslands, forest or lakes. These may be static or dynamic, depending on whether they consider changes in land cover over time. Land use datasets also contain information on how people use different physical land types, for instance, classifying areas as industrial, agricultural, or recreational. Land use can modulate climate-disease relationships in important ways as discussed in a study by Fletcher and colleagues, which found that mining activity was associated with hotspots in malaria transmission in southern Venezuela, and that the influence of temperature on malaria differed depending on the intensity of mining activity (Fletcher et al. 2022).
A commonly used index is the Normalised Difference Vegetation Index (NDVI), which indicates the ‘amount of greenness’ in a certain area, which can be useful, for instance, in detecting forest areas compared with built-up urban areas. The Copernicus Global Land Service is an example of a data source that offers land cover products that monitor vegetation, water and energy cycle and the terrestrial cryosphere (Buchhorn et al. 2020). Additionally, there are products that track changes in urban development throughout time, which are useful for historic reconstruction of disease drivers and future projections (Liu et al. 2020). Local partners may also have access to local monitoring services of land cover/use data most relevant to the setting and establishing partnerships with local institutions or universities could provide high quality products, ready for model building.
1.2.3.5 Common barriers to accessing data
Climate-informed EWS require collaborative data generation between global and local stakeholders (Ryan et al. 2023), and therefore will rely on legislation or regulations in place surrounding data ownership and sharing (Pley et al. 2021; Ryan et al. 2023). A recent review of tools for modelling climate-sensitive infectious diseases found that the sharing of datasets was a key barrier to tool development and implementation, but noted that climate datasets were generally more easily accessible than epidemiological or health data, which were often found to be difficult to access due to privacy concerns (Ryan et al. 2023). Formalised collaborations between climate and health sectors, for instance through Memoranda of Understanding (MoUs) between different governmental departments or agencies, could help to overcome these barriers and pave the way for the development and implementation of EWS involving multiple governmental bodies (Stewart-Ibarra et al. 2022).
1.3 Developing the tool
Once the feasibility of an EWS has been investigated, considering the resources, evidence and data available to address the problem, the next step in the co-creation process is to develop the forecasting tool. This involves data processing, including the cleaning and harmonising of required data streams, as well as model building, and model validation, where the predictive ability of the forecasting model is assessed.
1.3.1 Data processing
The goal of data processing is to harmonise the spatiotemporal structure of all variables, so that they can be used as inputs for a forecasting tool. Usually, surveillance systems report the number of cases that occurred in an administrative unit per epidemiological week or month (for instance at the district or health centre level). Environmental and climate data products are typically provided in a grid format, which does not align with administrative-level summaries of epidemiological data. As such, it is necessary to process and summarise climate data used so it can be harmonised with the structure of the epidemiological data available. There are multiple ways of summarising spatial information into administrative units, including calculating an arithmetic mean or a population weighted mean. For example, Fletcher and colleagues took the mean value of gridded data per cantón when modelling malaria in Ecuador (Fletcher et al. 2020). The underlying assumption of working with averages is that there is a homogeneous distribution of the variable across the administrative unit, which becomes more of a limitation when working over larger or more diverse areas.
Temporal summaries require careful thought, as not all averages are equally informative. For example, it might be more appropriate to work with accumulated precipitation instead of average precipitation for a given disease outcome. Some studies have benefited from using climate indicators such as the Standardised Precipitation Index (SPI), which is used to characterise drought conditions over a range of timescales (Lowe et al. 2018).
1.3.1.1 Bias correction and downscaling
Global scale gridded climate products contain inherent biases which can result in systematic error when these products are downscaled to smaller administrative units, masking fine-scale ground conditions (Fletcher et al. 2021). In some cases, data from global scale climate products are consistently different from local observations, for instance, containing too many rainy days or temperatures that are consistently too high. There are a wide range of bias correction methods which are useful for overcoming these constraints. Bias correction methods generally use observed values to compute an adjustment factor. For example, the Delta change method, commonly used for climate change impact modelling, incorporates long term systematic deviances into historic observations. Additionally, the resolution of some products is coarser than that required by users, requiring downscaling to a finer resolution. Statistical downscaling is common practice and often occurs alongside bias correction. The monthly WorldClim products, for example, are a downscaled version of the CRU time series, simultaneously using the WorldClim v.2 product as reference for bias correction (Fick and Hijmans 2017).
1.3.2 Building the model
At the core of an EWS is a forecasting tool that is able to use information on the interaction between climate and disease to create predictions based on current or future climatic conditions (Morin et al. 2018). The goal of a forecasting tool is to provide warnings with sufficient lead time to deploy control strategies, to prevent or minimise occurrence of outbreaks. However, increased lead time of warnings often comes at the expense of increased uncertainty in model predictions. This trade-off will need to be evaluated during the co-creation of the tool, and time horizons can vary from short-term predictions, over days or weeks, to predictions for an entire season, to years or decades (Colón-González et al. 2021). Fundamentally, effective tools must include climate and epidemiological information, produce an output that can give warning of future disease risk, be well described and validated in a transparent manner, be clearly communicated to decision-makers to allow for public health action, and, ideally, be named and open source (Pollett et al. 2021; Ryan et al. 2023).
Forecasting tools include mechanistic, statistical, and machine learning approaches (Table 2 and 3). The choice of modelling approach will depend on the aim of the study, the type of data available and what is needed from the model output (Lee et al. 2021). Additionally, modern model building techniques often incorporate or blend together different aspects of statistical, mechanistic or machine learning approaches. In this section, we will give an overview to commonly used statistical and mechanistic modelling approaches. Machine learning approaches are detailed elsewhere (Bzdok et al. 2017).
Historically, the use of complex spatiotemporal Bayesian statistical models, and the fitting of mechanistic models to data has been limited because of the long computation times required for statistical inference of model parameters. However, techniques such as Markov Chain Monte Carlo (MCMC) and alternative computational methods have allowed for the more efficient estimation of a posterior distribution of model parameters. One example commonly used in Bayesian spatiotemporal statistical modelling is the use of integrated nested Laplace approximation (INLA) for posterior estimation, which can be easily implemented using the R-INLA package in the R programming language (Rue et al. 2009).
In recent years, researchers in the infectious disease modelling community have emphasised the importance of accurately incorporating uncertainty surrounding forecasts, in a similar way to the routine reporting of uncertainty associated with climate forecasts. As such, model forecasts are moving from deterministic point predictions (where predictions of incidence or outbreak risk have no associated uncertainty) to probabilistic forecasts, which include an associated probability distribution of the modelled outcome (Colón-González et al. 2021; Held et al. 2017).
Common modelling approaches used to forecast outbreak risk for VBDs
1.3.2.1 Statistical models
Statistical models are mathematical models able to make inferences from a sample regarding the entire population, under a set of assumptions about the probability distribution giving rise to the observed data (Cox 2006). These models are able to explore relationships between one or more variables when sufficient data are available and are useful for making predictions or inferences about an outcome of interest. Statistical regression models are flexible and relatively easy to build and operate, which is reflected in the frequency of their use in forecasting tools (Lee et al. 2021b; Morin et al. 2018; Ryan et al. 2023).
In their simplest form, regression models are able to quantify the associations between covariates and the risk of disease. In practice, it’s often not possible to measure and access all the variables associated with a disease and it is necessary to use additional elements to account for excess variation in the outcome variable, such as random effects. Hierarchical or mixed-effect models can include terms to account for seasonality in the data or interannual variation, for instance due to changes in control interventions over time. Advanced spatially-explicit models can also take into account the underlying spatial structure of the data, using a range of assumptions about the dynamics between locations (Lee et al. 2021b).
Often there are many potential model formulations, particularly when considering multiple climatic variables at different lead times (for instance monthly rainfall with a 1:6-month lag), and model selection criteria can be used to determine the most appropriate candidate model. For example, information criteria such as Widely Applicable Bayesian Information Criterion (WAIC), Deviance Information Criterion (DIC) or Bayesian Information Criterion (BIC), aim to identify the most parsimonious model, balancing model simplicity with the need to minimise distance from the fitted and observed values (Schwarz 1978; Spiegelhalter et al. 2002; Watanabe 2013).
1.3.2.1.1 Statistical models for VBD s
The use of spatial and spatiotemporal models in infectious disease modelling has increased in the last decade as a result of higher computational capacity and the increased availability of gridded environmental and demographic datasets. Hierarchical generalised linear mixed models and Bayesian algorithms are often used when modelling risk of climate-sensitive VBD s because of their flexibility (Hussain-Alkhateeb et al. 2021). For example, temporal random effects can be used to recreate seasonality in the data and account for interannual variations (Figure 2, Lowe et al. 2018), whereas spatial random effects allow models to ‘borrow strength’ from neighbouring regions.
As an example of this kind of approach, Lowe and colleagues used a Bayesian temporal modelling framework to forecast outbreaks in Barbados. Their model used temporal monthly and yearly random effects to account for seasonality and interannual variation in the data (Figure 2) (Lowe et al. 2018). This work followed an earlier space-time modelling approach developed by Lowe and colleagues to forecast dengue risk in Brazil, which was later validated to consider its application in a prototype EWS (Lowe et al. 2016, 2014). Colón-Gonzalez and colleagues further extended this method by incorporating a superensemble approach to forecast dengue outbreaks in Vietnam, which was operationalised into an EWS called D-MOSS, see Case Study #3 below, (Colón-González et al. 2021).
Predicted dengue incidence in Barbados between 2000 and 2016 with 95% credible intervals, compared to the observed incidence in the same period. Predictions were produced with a hierarchical mixed model using a Bayesian framework (Lowe et al. 2018)
1.3.2.1.2 Limitations of statistical models
The power of statistical models depends on the extent and quality of the data used to inform them. Large amounts of data are necessary for obtaining precise estimates, which makes them suitable for endemic diseases, although less useful for (re-) emerging diseases for which high-quality data is often lacking (Lee et al. 2021b; Stewart-Ibarra et al. 2022). Additionally, data quality is often a major constraint when developing statistical prediction models (Hussain-Alkhateeb et al. 2021). Furthermore, statistical models are only able to reliably make predictions within the scope of the used data, which narrows their ability to make out of sample predictions.
1.3.2.2 Mechanistic models
1.3.2.2.1 Mechanistic modelling approaches
Mechanistic modelling approaches, also referred to as dynamic modelling approaches, explicitly model disease states, and track changes in these disease states through time at either an individual or population level (Heesterbeek et al. 2015). One of the most common types of mechanistic model is the compartmental, or SIR model, which, in its simplest form, divides the population into compartments of ‘Susceptible’, ‘Infected’ and ‘Recovered’ individuals and tracks how individuals in the population move between compartments over time during an epidemic. At the other end of the spectrum are individual-based models, which track the disease state of individuals in a population, each of whom has particular associated demographic information and potentially an associated contact network. For VBD s, individual-based models can track humans in a particular setting, or model disease vectors. In addition to this, mechanistic models can be deterministic, where rerunning the model with the same initial parameters will always result in the same model outcome, or stochastic, where the role of chance in the processes driving transmission is considered and model outcomes over subsequent runs will vary (Heesterbeek et al. 2015). Overall, the appropriate level of model complexity, and the extent to which models incorporate stochasticity, will depend on the research question being considered.
Mechanistic models aim to capture the underlying processes driving infectious disease transmission and can incorporate information on population immunity, contact patterns, population mobility and control measures. As a result of this, mechanistic models offer the potential to improve forecasting of vector-borne disease incidence, particularly when considering diseases or settings where there are multiple important drivers of transmission (Lessler et al. 2016). The ability to consider multiple drivers of disease transmission within a single modelling framework also allows us to avoid incorrectly attributing variation in disease risk to a single variable, for instance a climatic variable, and instead gain understanding in how transmission is shaped by multiple important drivers working in combination.
Similarly, by explicitly modelling the underlying biological mechanisms driving transmission, mechanistic approaches can be used to provide policy makers with information on what may happen if certain public health actions are or aren’t taken during the course of an outbreak. For instance, to inform how the imposition or timing of control measures or the introduction of a vaccination program might affect disease burden and transmission in a particular setting. Indeed, a recent report landscape mapping available tools for climate-sensitive infectious disease modelling found that the majority of identified tools used mechanistic or dynamical population modelling approaches, with varying structures, highlighting the wide applicability of mechanistic approaches in the modelling of climate-sensitive infectious diseases (Wellcome Trust 2022).
1.3.2.2.2 Mechanistic models for vector-borne diseases
Mechanistic models for vector-borne diseases can explicitly model human disease states, mosquito disease states, or both, with varying levels of complexity. For instance, several mechanistic models of dengue transmission use a simplified model structure based on direct host-to-host transmission (Andraud et al. 2012). Models describing human and vector disease states are often based on the Ross-Macdonald theory, originally developed to describe mosquito-borne pathogen transmission and model vector control strategies (Reiner et al. 2013). Additionally, for instances where the research question is focused on the spatial structure of transmission, metapopulation models can be used. These model infectious disease transmission between different ‘patches’ or populations based on assumptions of human or vector movement between populations.
1.3.2.2.3 Limitations of mechanistic models
Mechanistic models rely on assumptions about the transmission process of the VBD in question, and as such are less suitable for instances where the causal pathways underlying transmission are poorly characterised. Furthermore, complex mechanistic models, with many individuals or compartments, can be computationally costly to fit to data, posing challenges for model verification and validation and appropriate uncertainty quantification. Mechanistic models are not typically as widely used to forecast VBD s and in the 2015 Dengue Forecasting Challenge, mechanistic models were found to perform less well than statistical models which didn’t incorporate information on the biological mechanisms underlying transmission (Johansson et al. 2019). However, mechanistic models have shown great utility in other epidemic scenarios and continuing research into the causal pathways underlying transmission will lead to the development of improved mechanistic models, which can also incorporate multiple data streams to improve their predictive ability.
1.3.2.3 Semi-mechanistic and ensemble modelling
1.3.2.3.1 Semi-mechanistic
While we have presented statistical and mechanistic methods as two separate modelling approaches, in reality many models will incorporate statistical and mechanistic elements. For instance, mechanistic approaches are reliant on statistical inference of model parameters. Moreover, semi-mechanistic approaches, which further blend mechanistic and statistical model frameworks, have been applied when forecasting infectious disease dynamics. For instance, Funk and colleagues used a semi-mechanistic framework to produce forecasts during the 2013–2016 West African Ebola epidemic where a compartmental mechanistic framework was combined with a time-varying stochastic rate of transmission between individuals (Funk et al. 2019). In this instance, a semi-mechanistic approach was able to provide more timely and accurate forecasts, as researchers were unable to parameterise a fully mechanistic model in real time during an evolving outbreak (Funk et al. 2019).
1.3.2.3.2 Ensembles
It has been argued that the dynamics of disease transmission are too complex to be entirely captured by one model. As an alternative, data-assimilation methods have been proposed as a tool for combining multiple model forecasts to produce more accurate predictions (Colón-González et al. 2021). Such techniques have been tested for forecasting outbreaks of dengue in Vietnam, which showed a meaningful improvement compared to the individual models (Colón-González et al. 2021). The Dengue Forecasting challenge, where different teams of researchers submitted dengue forecasts for San Juan, Puerto Rico and Iquitos, Peru during multiple dengue seasons, found that groups that used ensemble approaches had improved forecasting ability and calibration than other groups. Additionally, an average of all the forecasts submitted was found to routinely outperform other models for all the forecasting targets considered at each time point (Johansson et al. 2019). This makes intuitive sense, as different modelling approaches are associated with different assumptions and limitations, so an ensemble could in theory leverage the different strengths of each. Ensemble models are also able to better quantify uncertainty by considering where component models have high levels of disagreement compared with where models give very similar predictions. Particularly, ensembles including both mechanistic and statistical approaches may leverage the advantages of each, and allow for improved predictive ability.
However, it should be noted that model averages (for instance by weighting predictions from each model equally or with weighting based on previous performance) may not be the best option when the forecast target is concerned with the likelihood of an extreme event occurring. For instance, an outbreak of sufficient size that a health system is completely overwhelmed. Here, understanding the extremes of individual model predictions included in the ensemble may be more useful to inform public health planning, in a similar way to how ensemble modelling of extreme weather events is conducted by meteorological teams.
1.3.3 Model validation
1.3.3.1 Out-of-sample testing
Once a forecasting model has been developed, the most important test of the model’s forecasting ability is its performance in predicting out-of-sample data, usually from the end of the time-series being considered. Out-of-sample testing data can be prospective data that had not yet been collected at the time of model development or data that has already been collected but is deliberately withheld from the model building phase in order to test the model. This train-test approach to model building, where one set of data is used to train the model and another is used to test its performance, is a common approach used to evaluate machine learning algorithms. This approach helps to assess if a model is underfitting – where the model does not learn enough from past data to predict future disease incidence – or overfitting – where the model fits too closely to noise in past data and is unable to generalise to unseen data and predict future disease incidence.
1.3.3.2 Cross-validation
In cases where, due to data constraints, it is not possible to leave a substantial testing data set out of the model building process, cross-validation techniques can be implemented. Cross-validation allows for a full use of the data, as each data point is used for both testing and training (Bergmeir and Benítez 2012). For instance, in a leave-one-out cross-validation approach, blocks of the dataset are left out at a time (for instance, a year of data or spatial unit) and the model’s ability to predict each block is assessed (Lowe et al. 2016). One limitation of this approach is that it can cause difficulties with time-ordered data, as data from later years or months may be used to predict outcomes in earlier time points. Other methods such as Time-Series Cross Validation (TSCV) can also be used, where training sets only contain observations occurring prior to data in the training sets (Hyndman and Athanasopoulos 2021).
1.3.3.3 Metrics for model evaluation
Traditional metrics used to evaluate the epidemic model forecasts have focused on measuring goodness-of-fit, or how close model point predictions are to the true outcomes that occur. For instance, metrics such as the Mean Absolute Error (MAE) or Root Mean Squared Error (RMSE) which quantify the error of the model fit to data (Chowell 2017). However, a successful probabilistic forecast should also be able to correctly quantify uncertainty or confidence in predictions. In recent years, efforts have been made by researchers in the infectious disease modelling community to incorporate techniques already commonly used in weather and economic forecasting, to evaluate the success of model forecasts based on the entire probability distribution of forecasts, rather than specific point estimates. This allows forecasts to be evaluated based on both their ability to select between different possible outcomes and also to accurately quantify the uncertainty associated with their predictions (Held et al. 2017).
With this in mind, model evaluation should aim to assess the predictive performance of a model according to the paradigm of ‘maximising the sharpness of the predictive distribution, subject to calibration’ (Gneiting et al. 2007). Here calibration refers to the ability of a model to correctly quantify its own uncertainty in making predictions, while sharpness refers to how concentrated those predictions are within a narrow range (Funk et al. 2019).
Strictly proper scoring rules jointly assess calibration and sharpness and can be used as a metric of overall forecast skill (Gneiting and Raftery 2007). Commonly used metrics include the logarithm score, the continuous ranked probability score (CRPS) and the Brier score.
1.3.3.4 Choice of reference model
During model validation, the predictive ability of the candidate forecasting model should be compared to the current best-practice model available. In practice, this is often a seasonal model (i.e. where the cases in a given week or month are compared with cases at the same time in previous seasons), or a simple model to detect trends in surveillance data (i.e. whether cases are increasing or decreasing compared to previous weeks). In particular, comparisons should be made with any current early-warning models or signals used by the Ministry of Health and other partners to trigger response activities. The implementation of an EWS is a time-consuming and potentially costly endeavour and it is therefore essential that the relative benefit of a climate-informed EWS is well-established during the co-creation process.
1.4 Implementation and monitoring
Implementation of an EWS at the regional, national or global level relies on institutional arrangements for data sharing between relevant partners, as well as upon standard operating procedures to consistently track forecasting model outputs in time and space to identify warning signals of increased outbreak risk (Hussain-Alkhateeb et al. 2021). Currently, many promising EWS models are developed purely as academic research and are not operationalised within local climate and health sectors. As part of the co-creation process, consideration should be given from the outset as to how the EWS will be handed over from model developers to local public health authorities, and data scientists/managers and software engineers responsible for longer term maintenance, development and interpretation of the forecasting model.
EWS warning signals are meaningless unless they catalyse action. Risk communication creates meaning from information, prompting an appropriate public health response. Historically, most failures in EWS occurred due to miscommunication rather than any other failure in the system (UNDP 2018).
During the co-creation process, researchers should work in collaboration with key partners and stakeholders to understand how best to communicate model outputs to partners, relevant institutions and the general public. The construction of a user-friendly interface or application, using clear and accessible language or graphics, has been highlighted as a strategy to improve engagement with, and the impact of, a climate-informed EWS (Wellcome Trust 2022). Ultimately, the communication strategy (flow and frequency of warnings) and the technology used to disseminate warnings (SMS, social media, television news, radio broadcasts, local loudspeaker) will determine the reach and speed of the system (UNDP 2018). Commonly, a three staged progressive warning, which may be colour coded (green, amber, red), is used to communicate increasing risk. Typically, this might be ‘be aware/watch/be alert’; followed by ‘be prepared’; followed by ‘take action’. Ideally, alert messages will combine the level certainty in the predictions with the potential impact of the outbreak using a decision-matrix, to allow for clear and actionable communication to public health decision-makers (Stewart-Ibarra et al. 2022).
Depending on the stage of the warning, different actions may be required by a variety of actors, ranging from government advisors, health managers, to the general public. Advanced planning and anticipation are key to putting in place effective public health responses (Semenza 2015; UNDP 2018). In cases where a climate-informed EWS gives a warning of increased VBD risk, public health actions that could be taken include; increased entomological or epidemiological surveillance, vector control activities, public awareness campaigns or preparations to increase hospital or treatment capacity (Semenza 2015). Importantly, EWS warnings of increased outbreak risk should have associated plans which recommend changes from the activities that would normally have taken place for a ‘typical’ VBD season (Lowe et al. 2016).
2 Case study 1. Dengue early warning systems in Singapore.
One example of a successfully operationalised climate-driven EWS can be found in the dengue EWS developed by the Singaporean National Environment Agency and collaborating universities (Shi et al. 2016). The EWS is based on a statistical forecasting model where multiple sub-models predict dengue cases 1:12 weeks ahead, with input variables chosen using a LASSO regression approach. Forecasting sub-models incorporate case data, population data, meteorological data including temperature variables (weekly mean temperature, maximum hourly temperature, number of hours of high temperature each week) and humidity variables (absolute and relative humidity), and vector surveillance data monitoring the prevalence of Aedes aegypti in Singapore. The model was assessed using out-of-sample testing and outperformed simpler model approaches based on the mean average percentage error. Crucially, although forecast accuracy was found to decrease with longer lead-times, Shi and colleagues found they were able to predict dengue outbreaks in 2013 and 2014 more than 10 weeks in advance, allowing the government to prepare hospital and diagnostic capacity, intensify vector control efforts and conduct a public communication campaign to mitigate the impact of the outbreak (Seltenrich 2016; Shi et al. 2016).
3 Case study 2. InfoDengue in Brazil.
InfoDengue in Brazil is a successful nowcasting system developed jointly by the Brazilian National Health Ministry, city-level health authorities and various research institutions (Codeco et al. 2018). The system integrates epidemiological, climate and social media data with a pipeline to clean, filter and integrate relevant datasets. The underlying modelling framework uses Bayesian methods to account for reporting delays and to estimate current dengue incidence and the effective reproductive number for dengue. Warnings are communicated using a colour-coded risk classification; green indicates poor conditions for transmission, yellow indicates favourable transmission conditions, orange indicates sustained transmission and red indicates high incidence. The strength of the InfoDengue system lies in streamlining the process of collecting and harmonizing local and national-level data and integrating relevant data streams to provide city-level warnings and situational awareness which can be acted upon by local health agencies responsible for disease prevention and mitigation efforts. Efforts are underway to expand the nowcasting system into an early-warning system able to give advance notice of dengue outbreaks by incorporating tailored seasonal and subseasonal forecasts. This will build on previous work by Lowe and colleagues developing a prototype dengue EWS to produce dengue forecasts ahead of the football World Cup in 2014, expanding this to provide seasonal and subseasonal forecasts to participating Brazilian cities (Lowe et al. 2014, 2016).
4 Case study 3. D-MOSS in Vietnam.
D-MOSS is a dengue EWS funded by the UK Space Agency and co-created by a collaboration of teams in the UK, Vietnam, and other countries. The underlying Bayesian spatiotemporal forecasting model uses Earth observations, seasonal climate forecasts provided by the UK Met Office, and lagged dengue cases in a superensemble of probabilistic dengue models which aim to predict dengue cases up to 6 months ahead (Colón-González et al. 2021; HR Wallingford 2022). Colón-González and colleagues demonstrated that the predictive ability of the superensemble outperformed a baseline model both when considering predictions of dengue incidence (evaluated using the continuous rank probability score, CRPS) and the outbreak detection capability of the model (evaluated using the Brier score) for Vietnamese provinces 1:6 months in advance (Colón-González et al. 2021). Model predictions are communicated to stakeholders through a user-friendly interface with graphs of anticipated dengue cases by month and province as well as maps of the distribution of dengue in neighbouring provinces. D-MOSS has been operational in Vietnam from 2019, Malaysia and Sri Lanka since 2020 and is in the process of being implemented in Cambodia, Laos, Thailand, and The Philippines. Stakeholders in Vietnam currently using the D-MOSS EWS have reported how D-MOSS is ‘supporting to bridge the gap between early warning and early action’ (HR Wallingford 2022). For instance, the National Institute of Hygiene and Epidemiology (NIHE) were able to use model predictions to anticipate a dengue outbreak in Nghe An province and implement enhanced disease response measures including environmental cleaning and spraying, as well as organising training on entomological surveillance and spraying methods for local public health officials.
4.1 Evaluate the early warning systems
4.1.1 Iteration and model tuning
Climate-informed EWS should be continually updated with appropriate corrections to reflect changes in VBD risk over space and time, as well as changes in technology and in user-experience. Evaluation of an EWS and iteration based on the lessons learnt is crucial for the continual improvement of the EWS. Evaluations of EWS which have been implemented in real-world settings, should include: (1) evaluation of the overall effectiveness of the system; (2) evaluation of the system components; and (3) an economic assessment of the cost-effectiveness and affordability of the system (Ebi and Schmier 2005). User experience has rarely been evaluated in EWS but, as highlighted at the beginning of this chapter, convening partners, and understanding how they will use the system is important for the institutionalisation of forecasting tools and the overall effectiveness of the system in preventing or reducing outbreaks (Hussain-Alkhateeb et al. 2021).
4.1.1.1 Overall effectiveness of the system
The first challenge is deciding which benchmark to evaluate the EWS against. Typically, the counterfactual (i.e. what has not happened) does not exist as an effective early warning with early action should have precluded the outbreak from occurring, or been able to reduce the epidemic peak. To overcome this, the reference standard is usually previous outbreaks with high reporting quality (Zhang et al. 2014).
There are two types of error that should be evaluated in an EWS – false alarms (predicting an outbreak that does not materialise) and missed events (failing to predict an outbreak). The consequences for which are very different. Trust in the system can be undermined if warning systems repeatedly call for early action when it is not needed. However, trust can equally be lost if the system fails to trigger a warning while substantial morbidity and mortality could occur in situations that could have been prevented. EWS should include checks and routines to minimise the risk of false alarms and improve the overall specificity of the system, without sacrificing the ability to detect events, or specificity (UNDP 2018). Finally, evaluations should consider the user-friendliness of the EWS for all stakeholders involved, the appropriateness of the EWS in terms of implementation, its coordination and coverage, efficiency (timeliness), impact, and sustainability (Hussain-Alkhateeb et al. 2021).
4.1.1.2 System components
Improvements may only be needed to certain components of the EWS, based on an identification of where a failure or inefficiency arose. EWS can and will fail for different reasons at different points (IFRC 2012). For example, delays in data availability, human error, misclassification, decision-makers not understanding and acting on the information (Hussain-Alkhateeb et al. 2021; IFRC 2012; Zhang et al. 2014). As previously mentioned, miscommunication has tended to contribute to the main failure in an EWS. This can be mitigated through the early engagement and co-learning between key stakeholders during the co-creation of an EWS. However, it may also be advisable to ensure redundancy exists in certain parts of the system to reduce the risk of EWS failure. For example, IFRC guidance suggests ensuring redundancy in two key areas by monitoring multiple indicators or outputs from a forecasting tool, and utilising multiple communication channels used to send warning messages (IFRC 2012).
4.1.1.3 Cost-benefit
Limited research exists on cost-benefit assessments of EWS for VBD s. Cost-benefit assessments for EWS of natural hazards, show that for every $1 spent on flood EWS and disaster risk reduction initiatives, $5 is saved in future losses (Mechler and Bouwer 2018). The ratio of benefits to costs for hurricane warnings is around 3:1, and 2,500:1 for heat waves in densely populated urban areas (e.g. Philadelphia) (Teisberg and Weiher 2009; UNDP 2018). Less is known about the cost-benefit of EWS for infectious diseases. The economic costs of false alarms (i.e. triggering a response for an outbreak which does not materialise) can be high, but difficult to quantify as it is not possible to be certain that actions may have helped prevent the outbreak. Cost-benefit analyses which assess the economic costs of false alarms vs missed events should be carried out to define the optimum alert trigger threshold (Colón-González et al. 2021; Lowe et al. 2016). These should be performed by the public health services based on health economic modelling so that EWS evaluations can be comparable to other health interventions (Ebi and Schmier 2005). Suggested cost-benefit assessments include: the cost per event (e.g. death, hospitalization) avoided (Ebi and Schmier 2005); the costs per reported event; the operating costs per surveillance unit per year (Ding et al. 2015). Furthermore, when establishing an EWS it is important to consider the total costs included in set-up (system development and training) and operating costs (data collection, quality control and signal verification) based on the available budget of the country (Ding et al. 2015).
5 Conclusions
Climate-informed EWS offer great potential for public-health decision-makers to prevent or mitigate the harms of outbreaks of VBD s and adapt to the harmful health impacts of climate change. With advances in the computational capacity needed to run complex forecasting models, as well as in the scientific knowledge of climate-health linkages, the time has come for a shift from academic EWS research to the implementation of climate-informed EWS at global, national, and local scales. The effects of climate change on the magnitude and frequency of VBD outbreaks are already clear in certain settings, and, as climate change continues to shift patterns of transmission, climate-informed EWS with good predictive ability and well characterised uncertainty surrounding their predictions will become increasingly crucial as a climate change adaptation measure. In this chapter we have given an overview of the stages involved in the co-creation of an EWS for vector-borne diseases, bringing together insight from epidemic forecasting and climate services for health communities of practice.
6 Recommendations
Acknowledgements
EF was funded by the Medical Research Council (grant number MR/N013638/1). RL was supported by a Royal Society Dorothy Hodgin Fellowship. SAL was supported by a Royal Society Research Grant for Research Fellows PhD studentship. MLB and RL were supported by the European Union’s Horizon Europe programme (Grant Agreement: 101057554 IDAlert and 101086640 E4Warning). We would like to thank the LSHTM Planetary Health and Infectious Diseases lab and the BSC Global Health Resilience group for their input and support, as well as Erin Coughlan de Perez and Adam Kucharski for providing valuable feedback on the initial conceptualizations and drafts of this book chapter.
References
Alcayna T, Fletcher I, Gibb R, Tremblay L, Funk S, Rao B and Lowe R (2022) Climate-sensitive disease outbreaks in the aftermath of extreme climatic events: A scoping review. One Earth 5: 336–350.
Andraud M, Hens N, Marais C and Beutels P (2012) Dynamic epidemiological models for dengue transmission: A systematic review of structural approaches. PLOS ONE 7: e49085.
Anyamba A, Chretien J-P, Britch SC, Soebiyanto RP, Small JL, Jepsen R, Forshey BM, Sanchez JL, Smith RD, Harris R, Tucker CJ, Karesh WB and Linthicum KJ (2019) Global disease outbreaks associated with the 2015–2016 El Niño event. Scientific Reports 9: 1930.
Bellone R and Failloux A-B (2020) The role of temperature in shaping mosquito-borne viruses transmission. Frontiers in Microbiology 11: 2388.
Benedum CM, Seidahmed OME, Eltahir EAB and Markuzon N (2018) Statistical modeling of the effect of rainfall flushing on dengue transmission in Singapore. PLOS Neglected Tropical Diseases 12: e0006935.
Bergmeir C and Benítez JM (2012) On the use of cross-validation for time series predictor evaluation. Information Sciences 191: 192–213.
Braks M, Giglio G, Tomassone L, Sprong H and Leslie TE (2019) Making vector-borne disease surveillance work: New opportunities from the SDG perspectives. Frontiers in Veterinary Science 6: 232.
Buchhorn M, Smets B, Bertels L, Roo BD, Lesiv M, Tsendbazar N-E, Herold M and Fritz S (2020) Copernicus global land service: Land cover 100m: Collection 3: Epoch 2019: Globe.
Bzdok D, Krzywinski M and Altman N (2017) Machine learning: A primer. Nature Methods 14: 1119–1120.
Ceccato P, Ramirez B, Manyangadze T, Gwakisa P and Thomson MC (2018) Data and tools to integrate climate and environmental information into public health. Infectious Diseases of Poverty 7: 126.
Chen Y, Ong JHY, Rajarethinam J, Yap G, Ng LC and Cook AR (2018) Neighbourhood level real-time forecasting of dengue cases in tropical urban Singapore. BMC Medicine 16: 129.
Chowell G (2017) Fitting dynamic models to epidemic outbreaks with quantified uncertainty: a primer for parameter uncertainty, identifiability, and forecasts. Infectious Disease Modelling 2: 379–398.
Chuang T-W, Chaves LF and Chen P-J (2017) Effects of local and regional climatic fluctuations on dengue outbreaks in southern Taiwan. PLOS ONE 12: e0178698.
Codeco C, Coelho F, Cruz O, Oliveira S, Castro T and Bastos L (2018) Infodengue: a nowcasting system for the surveillance of arboviruses in Brazil. Revue d’Épidémiologie et de Santé Publique 66: S386.
Colón-González FJ, Bastos LS, Hofmann B, Hopkin A, Harpham Q, Crocker T, Amato R, Ferrario I, Moschini F, James S, Malde S, Ainscoe E, Nam VS, Tan DQ, Khoa ND, Harrison M, Tsarouchi G, Lumbroso D, Brady OJ and Lowe R (2021) Probabilistic seasonal dengue forecasting in Vietnam: A modelling study using superensembles. PLOS Medicine 18: e1003542.
Cox DR (2006) Principles of statistical inference, Cambridge University Press, 7 pp.
Ding Y, Sauerborn R, Xu B, Shaofa N, Yan W, Diwan VK and Dong H (2015) A cost-effectiveness analysis of three components of a syndromic surveillance system for the early warning of epidemics in rural china. BMC Public Health 15: 1127.
Eastin MD, Delmelle E, Casas I, Wexler J and Self C (2014 ) Intra- and interseasonal autoregressive prediction of dengue outbreaks using local weather and regional climate for a tropical environment in Colombia. The American Journal of Tropical Medicine and Hygiene 91: 598–610.
Ebi KL and Schmier JK (2005) A stitch in time: improving public health early warning systems for extreme weather events. Epidemiologic Reviews 27: 115–121.
Fick SE and Hijmans RJ (2017) Worldclim 2: new 1-km spatial resolution climate surfaces for global land areas. International Journal of Climatology 37: 4302–4315.
Fletcher IK, Stewart-Ibarra AM, García-Díez M, Shumake-Guillemot J and Lowe R (2021) Climate services for health: From global observations to local interventions. Med 2: 355–361.
Fletcher IK, Stewart-Ibarra AM, Sippy R, Carrasco-Escobar G, Silva M, Beltran-Ayala E, Ordoñez T, Adrian J, Sáenz FE, Drakeley C, Jones KE and Lowe R (2020) The relative role of climate variation and control interventions on malaria elimination efforts in El Oro, Ecuador: A modeling study. Frontiers in Environmental Science 8.
Funk S, Camacho A, Kucharski AJ, Lowe R, Eggo RM and Edmunds WJ (2019) Assessing the performance of real-time epidemic forecasts: a case study of Ebola in the Western Area region of Sierra Leone, 2014–15. PLOS Computational Biology 15: e1006785.
Githeko A, Ototo E, Muange P, Zhou G, Yan G and Sang J (2018) The launch and operation of the malaria epidemic early warning system in Kenya. Science Publishing Group Journal 1 (2).
Gneiting T, Balabdaoui F and Raftery AE (2007) Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 69: 243–268.
Gneiting T and Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 102: 359–378.
Grover-Kopec EK, Blumenthal MB, Ceccato P, Dinku T, Omumbo JA and Connor SJ (2006) Web-based climate information resources for malaria control in Africa. Malaria Journal 5: 38.
Heesterbeek H, Anderson RM, Andreasen V, Bansal S, Angelis DD, Dye C, Eames KTD, Edmunds WJ, Frost SDW, Funk S, Hollingsworth TD, House T, Isham V, Klepac P, Lessler J, Lloyd-Smith JO, Metcalf CJE, Mollison D, Pellis L, Pulliam JRC, Roberts MG, Viboud C and Collaboration INII (2015) Modeling infectious disease dynamics in the complex landscape of global health. Science 347(6227): aaa4339. https://doi.org/10.1126/science.aaa4339.
Held L, Meyer S and Bracher J (2017) Probabilistic forecasting in infectious disease epidemiology: The 13th armitage lecture. Statistics in Medicine 36: 3443–3460.
Hussain-Alkhateeb L, Ramírez TR, Kroeger A, Gozzer E and Runge-Ranzinger S (2021) Early warning systems (ewss) for chikungunya, dengue, malaria, yellow fever, and Zika outbreaks: what is the evidence? A scoping review. PLOS Neglected Tropical Diseases 15: e0009686.
Hyndman RJ and Athanasopoulos G (2021) Forecasting: Principles and practice (3rd edition), OTexts, 380 pp.
IFRC (2012) Community early warning systems: Guiding principles, IFRC.
Johansson MA, Apfeldorf KM, Dobson S, Devita J, Buczak AL, Baugher B, Moniz LJ, Bagley T, Babin SM, Guven E, Yamana TK, Shaman J, Moschou T, Lothian N, Lane A, Osborne G, Jiang G, Brooks LC, Farrow DC, Hyun S, Tibshirani RJ, Rosenfeld R, Lessler J, Reich NG, Cummings DAT, Lauer SA, Moore SM, Clapham HE, Lowe R, Bailey TC, García-Díez M, Carvalho MS, Rodó X, Sardar T, Paul R, Ray EL, Sakrejda K, Brown AC, Meng X, Osoba O, Vardavas R, Manheim D, Moore M, Rao DM, Porco TC, Ackley S, Liu F, Worden L, Convertino M, Liu Y, Reddy A, Ortiz E, Rivero J, Brito H, Juarrero A, Johnson LR, Gramacy RB, Cohen JM, Mordecai EA, Murdock CC, Rohr JR, Ryan SJ, Stewart-Ibarra AM, Weikel DP, Jutla A, Khan R, Poultney M, Colwell RR, Rivera-García B, Barker CM, Bell JE, Biggerstaff M, Swerdlow D, Mier-y-Teran-Romero L, Forshey BM, Trtanj J, Asher J, Clay M, Margolis HS, Hebbeler AM, George D and Chretien J-P (2019) An open challenge to advance probabilistic forecasting for dengue epidemics. Proceedings of the National Academy of Sciences 116: 24268–24274.
Koenraadt CJM and Harrington LC (2008) Flushing effect of rain on container-inhabiting mosquitoes Aedes aegypti and Culex pipiens (Diptera: Culicidae). Journal of Medical Entomology 45: 28–35.
Kovats RS, Bouma MJ, Hajat S, Worrall E and Haines A (2003) El Niño and health. The Lancet 362: 1481–1489.
Lee SA, Economou T, Barcellos C, Catão R, Carvalho MS and Lowe R (2021a) Effect of climate change, connectivity, and socioeconomic factors on the expansion of the dengue virus transmission zone in 21st century Brazil: An ecological modelling study. The Lancet Planetary Health 5: S14.
Lee SA, Jarvis CI, Edmunds WJ, Economou T and Lowe R (2021b) Spatial connectivity in mosquito-borne disease models: A systematic review of methods and assumptions. Journal of The Royal Society Interface 18: 20210096.
Lessler J, Azman AS, Grabowski MK, Salje H and Rodriguez-Barraquer I (2016 ) Trends in the mechanistic and dynamic modeling of infectious diseases. Current Epidemiology Reports 3: 212–222.
Liu X, Huang Y, Xu X, Li X, Li X, Ciais P, Lin P, Gong K, Ziegler AD, Chen A, Gong P, Chen J, Hu G, Chen Y, Wang S, Wu Q, Huang K, Estes L and Zeng Z (2020) High-spatiotemporal-resolution mapping of global urban change from 1985 to 2015. Nature Sustainability 3: 564–570.
Lowe R, Barcellos C, Coelho CAS, Bailey TC, Coelho GE, Graham R, Jupp T, Ramalho WM, Carvalho MS, Stephenson DB and Rodó X (2014) Dengue outlook for the world cup in Brazil: an early warning model framework driven by real-time seasonal climate forecasts. The Lancet Infectious Diseases 14: 619–626.
Lowe R, Coelho CA, Barcellos C, Carvalho MS, Catão RDC, Coelho GE, Ramalho WM, Bailey TC, Stephenson DB and Rodó X (2016) Evaluating probabilistic dengue risk forecasts from a prototype early warning system for Brazil. as 5: e11285.
Lowe R, Gasparrini A, Meerbeeck CJV, Lippi CA, Mahon R, Trotman AR, Rollock L, Hinds AQJ, Ryan SJ and Stewart-Ibarra AM (2018) Nonlinear and delayed impacts of climate on dengue risk in Barbados: A modelling study. PLOS Medicine 15: e1002613.
Lowe R, Lee S, Lana RM, Codeço CT, Castro MC and Pascual M (2020a) Emerging arboviruses in the urbanized Amazon rainforest. BMJ 371: m4385.
Lowe R, Lee SA, O’Reilly KM, Brady OJ, Bastos L, Carrasco-Escobar G, Catão RdC, Colón-González FJ, Barcellos C, Carvalho MS, Blangiardo M, Rue H and Gasparrini A (2021) Combined effects of hydrometeorological hazards and urbanisation on dengue risk in Brazil: a spatiotemporal modelling study. The Lancet Planetary Health 5: e209–e219.
Lowe R, Ryan SJ, Mahon R, Van Meerbeeck CJ, Trotman AR, Boodram L-LG, Borbor-Cordova MJ and Stewart-Ibarra AM (2020b) Building resilience to mosquito-borne diseases in the Caribbean. PLOS Biology 18: e3000791.
Lowe R, Stewart-Ibarra AM, Petrova D, García-Díez M, Borbor-Cordova MJ, Mejía R, Regato M and Rodó X (2017) Climate services for health: predicting the evolution of the 2016 dengue season in Machala, Ecuador. The Lancet Planetary Health 1: e142–e151.
Macdonald G (1957) The epidemiology and control of malaria, Oxford University Press, 280 pp.
Macherera M and Chimbari MJ (2016) A review of studies on community based early warning systems. Jàmbá: Journal of Disaster Risk Studies 8: 206.
McMichael AJ (2015) Extreme weather events and infectious disease outbreaks. Virulence 6: 543–547.
Mechler R and Bouwer L (2018) Climate loss and damage? Perspectives on the science and policy debate. 20th EGU General Assembly, EGU2018, Proceedings from the conference held 4–13 April, 2018 in Vienna, Austria: 9794.
Mordecai EA, Cohen JM, Evans MV, Gudapati P, Johnson LR, Lippi CA, Miazgowicz K, Murdock CC, Rohr JR, Ryan SJ, Savage V, Shocket MS, Ibarra AS, Thomas MB and Weikel DP (2017) Detecting the impact of temperature on transmission of Zika, dengue, and chikungunya using mechanistic models. PLOS Neglected Tropical Diseases 11: e0005568.
Morin CW, Semenza JC, Trtanj JM, Glass GE, Boyer C and Ebi KL (2018) Unexplored opportunities: Use of climate- and weather-driven early warning systems to reduce the burden of infectious diseases. Current environmental health reports 5: 430–438.
Muñoz ÁG, Chourio X, Rivière-Cinnamond A, Diuk-Wasser MA, Kache PA, Mordecai EA, Harrington L and Thomson MC (2020) AeDES: A next-generation monitoring and forecasting system for environmental suitability of Aedes-borne disease transmission. Scientific Reports 10: 12640.
Muñoz-Sabater J, Dutra E, Agustí-Panareda A, Albergel C, Arduini G, Balsamo G, Boussetta S, Choulga M, Harrigan S, Hersbach H, Martens B, Miralles DG, Piles M, Rodríguez-Fernández NJ, Zsoter E, Buontempo C and Thépaut J-N (2021) Era5-land: A state-of-the-art global reanalysis dataset for land applications. Earth System Science Data 13: 4349–4383.
Myers MF, Rogers DJ, Cox J, Flahault A and Hay SI (2000) Forecasting disease risk for increased epidemic preparedness in public health. Advances in Parasitology 47: 309–330.
Obolski U, Perez PN, Villabona-Arenas CJ, Thézé J, Faria NR and Lourenço J (2019) MVSE: An R-package that estimates a climate-driven mosquito-borne viral suitability index. Methods in Ecology and Evolution 10: 1357–1370.
Oluwagbemi OO, Fornadel CM, Adebiyi EF, Norris DE and Rasgon JL (2013) ANOSPEX: A stochastic, spatially explicit model for studying anopheles metapopulation dynamics. PLOS ONE 8: e68040.
Patz JA and Olson SH (2006) Malaria risk and temperature: Influences from global climate change and local land use practices. Proceedings of the National Academy of Sciences 103: 5635–5636.
Petrone ME, Earnest R, Lourenço J, Kraemer MUG, Paulino-Ramirez R, Grubaugh ND and Tapia L (2021) Asynchronicity of endemic and emerging mosquito-borne disease outbreaks in the Dominican Republic. Nature Communications 12: 151.
Pley C, Evans M, Lowe R, Montgomery H and Yacoub S (2021) Digital and technological innovation in vector-borne disease surveillance to predict, detect, and control climate-driven outbreaks. The Lancet Planetary Health 5: e739–e745.
Pollett S, Johansson MA, Reich NG, Brett-Major D, Valle SYD, Venkatramanan S, Lowe R, Porco T, Berry IM, Deshpande A, Kraemer MUG, Blazes DL, Pan-ngum W, Vespigiani A, Mate SE, Silal SP, Kandula S, Sippy R, Quandelacy TM, Morgan JJ, Ball J, Morton LC, Althouse BM, Pavlin J, Panhuis Wv, Riley S, Biggerstaff M, Viboud C, Brady O and Rivers C (2021) Recommended reporting items for epidemic forecasting and prediction research: The EPIFORGE 2020 guidelines. PLOS Medicine 18: e1003793.
Puntasecca CJ, King CH and LaBeaud AD (2021) Measuring the global burden of chikungunya and Zika viruses: A systematic review. PLOS Neglected Tropical Diseases 15: e0009055.
Rajarethinam J, Ang L-W, Ong J, Ycasas J, Hapuarachchi HC, Yap G, Chong C-S, Lai Y-L, Cutter J, Ho D, Lee V and Ng L-C (2018) Dengue in Singapore from 2004 to 2016: Cyclical epidemic patterns dominated by serotypes 1 and 2. The American Journal of Tropical Medicine and Hygiene 99: 204–210.
Reiner RC, Perkins TA, Barker CM, Niu T, Chaves LF, Ellis AM, George DB, Le Menach A, Pulliam JRC, Bisanzio D, Buckee C, Chiyaka C, Cummings DAT, Garcia AJ, Gatton ML, Gething PW, Hartley DM, Johnston G, Klein EY, Michael E, Lindsay SW, Lloyd AL, Pigott DM, Reisen WK, Ruktanonchai N, Singh BK, Tatem AJ, Kitron U, Hay SI, Scott TW and Smith DL (2013) A systematic review of mathematical models of mosquito-borne pathogen transmission: 1970–2010. Journal of the Royal Society Interface 10: 20120921.
Reinhold JM, Lazzari CR and Lahondère C (2018) Effects of the environmental temperature on Aedes aegypti and Aedes albopictus mosquitoes: A review. Insects 9: 158.
Romanello M, McGushin A, Napoli CD, Drummond P, Hughes N, Jamart L, Kennard H, Lampard P, Rodriguez BS, Arnell N, Ayeb-Karlsson S, Belesova K, Cai W, Campbell-Lendrum D, Capstick S, Chambers J, Chu L, Ciampi L, Dalin C, Dasandi N, Dasgupta S, Davies M, Dominguez-Salas P, Dubrow R, Ebi KL, Eckelman M, Ekins P, Escobar LE, Georgeson L, Grace D, Graham H, Gunther SH, Hartinger S, He K, Heaviside C, Hess J, Hsu S-C, Jankin S, Jimenez MP, Kelman I, Kiesewetter G, Kinney PL, Kjellstrom T, Kniveton D, Lee JKW, Lemke B, Liu Y, Liu Z, Lott M, Lowe R, Martinez-Urtaza J, Maslin M, McAllister L, McMichael C, Mi Z, Milner J, Minor K, Mohajeri N, Moradi-Lakeh M, Morrissey K, Munzert S, Murray KA, Neville T, Nilsson M, Obradovich N, Sewe MO, Oreszczyn T, Otto M, Owfi F, Pearman O, Pencheon D, Rabbaniha M, Robinson E, Rocklöv J, Salas RN, Semenza JC, Sherman J, Shi L, Springmann M, Tabatabaei M, Taylor J, Trinanes J, Shumake-Guillemot J, Vu B, Wagner F, Wilkinson P, Winning M, Yglesias M, Zhang S, Gong P, Montgomery H, Costello A and Hamilton I (2021) The 2021 report of the lancet countdown on health and climate change: Code red for a healthy future. The Lancet 398: 1619–1662.
Rue H, Martino S and Chopin N (2009) Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71: 319–392.
Runge-Ranzinger S, Kroeger A, Olliaro P, McCall PJ, Sánchez Tejeda G, Lloyd LS, Hakim L, Bowman LR, Horstick O and Coelho G (2016) Dengue contingency planning: From research to policy and practice. PLOS Neglected Tropical Diseases 10: e0004916.
Ryan SJ, Carlson CJ, Mordecai EA and Johnson LR (2019) Global expansion and redistribution of Aedes-borne virus transmission risk with climate change. PLOS Neglected Tropical Diseases 13: e0007213.
Ryan SJ, Lippi CA, Caplan T, Diaz A, Dunbar W, Grover S, Johnson S, Knowles R, Lowe R, Mateen B, Thomson M, Stewart-Ibarra AM. (2022). The current landscape of software tools for the climate-sensitive infectious disease modelling community. The Lancet Planetary Health 7: e527–536 (https://doi.org/10.1016/S2542-5196(23)00056-6.
Sarma N, Patouillard E, Cibulskis RE and Arcand J-L (2019) The economic burden of malaria: Revisiting the evidence. The American Journal of Tropical Medicine and Hygiene 101: 1405–1415.
Schwarz G (1978) Estimating the dimension of a model. The Annals of Statistics 6: 461–464.
Scott JA, Gandy A, Mishra S, Unwin J, Flaxman S and Bhatt S (2020) Epidemia: Modeling of epidemics using hierarchical Bayesian models. R package version 1.0.0. 2020.
Seltenrich N (2016) Singapore success: New model helps forecast dengue outbreaks. Environmental Health Perspectives 124: A167.
Semenza JC (2015) Prototype early warning systems for vector-borne diseases in europe. International Journal of Environmental Research and Public Health 12: 6333–6351.
Shepard DS, Undurraga EA, Halasa YA and Stanaway JD (2016) The global economic burden of dengue: A systematic analysis. The Lancet Infectious Diseases 16: 935–941.
Shi Y, Liu X, Kok S-Y, Rajarethinam J, Liang S, Yap G, Chong C-S, Lee K-S, Tan SSY, Chin CKY, Lo A, Kong W, Ng LC and Cook AR (2016) Three-month real-time dengue forecast models: An early warning system for outbreak alerts and policy decision support in Singapore. Environmental Health Perspectives 124: 1369–1375.
Smith T, Killeen GF, Maire N, Ross A, Molineaux L, Tediosi F, Hutton G, Utzinger J, Dietz K and Tanner M (2006) Mathematical modeling of the impact of malaria vaccines on the clinical epidemiology and natural history of Plasmodium falciparum malaria: overview. The American Journal of Tropical Medicine and Hygiene 75: 1–10.
Spiegelhalter DJ, Best NG, Carlin BP and Van Der Linde A (2002) Bayesian measures of model complexity and fit . Journal of the Royal Statistical Society: Series B (Statistical Methodology) 64: 583–639.
Stewart-Ibarra AM, Rollock L, Best S, Brown T, Diaz AR, Dunbar W, Lippi CA, Mahon R, Ryan SJ, Trotman A, Van Meerbeeck CJ and Lowe R (2022) Co-learning during the co-creation of a dengue early warning system for the health sector in Barbados. BMJ global health 7: e007842.
Stewart-Ibarra AM, Romero M, Hinds AQJ, Lowe R, Mahon R, Van Meerbeeck CJ, Rollock L, Gittens-St Hilaire M, St. Ville S, Ryan SJ, Trotman AR and Borbor-Cordova MJ (2019) Co-developing climate services for public health: Stakeholder needs and perceptions for the prevention and control of Aedes-transmitted diseases in the Caribbean. PLOS Neglected Tropical Diseases 13: e0007772.
Stolerman LM, Maia PD and Kutz JN (2019) Forecasting dengue fever in Brazil: An assessment of climate conditions. PLOS ONE 14: e0220106.
Teisberg T, J and Weiher R, F (2009) Background paper on the benefits and costs of early warning systems for major natural hazards. Submitted to the world bank group, global facility for disaster reduction and recovery (gfdrr).
U.S. Geological Survey (2022) Landsat missions. Available at.
UNDP (2018) Five approaches to build functional early warning systems.
Wallingford H (2022) D-MOSS: Dengue forecasting model satellite-based system. Available at.
Watanabe S (2013) A widely applicable bayesian information criterion. Journal of Machine Learning Research 14: 867–897.
World Health Organization (2005) Using climate to predict infectious disease epidemics, Geneva.
World Health Organization (2021a) Quality criteria for the evaluation of climate-informed early warning systems for infectious diseases.
World Health Organization (2021b) World malaria report.
World Meteorological Organization (2015) What do we mean by climate services? Available at.
WorldPop (2022) Open spatial demographic data and research. Available at: https://www.worldpop.org/.
Yamana TK, Kandula S and Shaman J (2016) Superensemble forecasts of dengue outbreaks. Journal of The Royal Society Interface 13: 20160410.
Zhang H, Li Z, Lai S, Clements ACA, Wang L, Yin W, Zhou H, Yu H, Hu W and Yang W (2014) Evaluation of the performance of a dengue outbreak detection tool for China. PLOS ONE 9: e106144.