White Paper One Contributor: Campbell-Flatter Goffer and Rei Goffer

Contribution

Q1. How good are the forecasts today and what are the unresolved challenges in the weather/climate forecasting? (e.g., before looking 10 years ahead, look 10 years back – what has changed? 

How good are forecasts today?

• Significant global progress in forecasts has been made in recent years and continues to improve year by year. This is clear when looking at long-term statistics of the leading NWP systems, as well as forecast metrics around high-impact events, e.g. tropical cyclones. The increased adoption of forecast-driven weather tools, especially by the commercial sector for decision-making, also suggested major progress.
• This "perspective" article by Alley/Emanuel/Zhang from last year is particularly relevant
• The frontiers of weather forecasting are curious opposites: extremely near-term, “warn-on-forecast” nowcasting for convective phenomena, and sub-seasonal forecasting are the two biggest buckets
• Lorenz’s comments on [weather] predictability as a problem “partly solved” (2006 ) are still highly relevant
• See example of advancements in forecasting that have lead to wider adoption via ClimaCell use case

What are the unresolved challenges in the weather/climate forecasting?

It is clear that the advances of forecasting are yet to reach their full potential in emerging markets.

• Observation gaps:
  • Gaps exist in observations, especially in remote areas. There exists many underutilized on the ground weather stations, often used for academic purposes but yet potential to also contribute to prediction; There is also the challenge of how ground stations are serviced, maintained and resourced. In the near future, we will hopefully see application of more non traditional observations such as virtual sensors, IOT devices, smartphone sensors and military-grade weather stations.
• Forecasting gaps:
  • There are many “niche” forecasts which are still extremely challenging: rapid intensification of tropical cyclones, severe convective weather, etc.
  • Data assimilation has advanced to initialize forecast models from a more accurate "current state" of the atmosphere.
  • Accurately simulating convection and capturing large-scale dynamical teleconnections are core science challenges.
• Dissemination and usage:
  • There remains major gaps in access and the practical use of weather data.
  • In Africa, for example, weather data is not used for daily decision making, despite increased demand and recognition of its importance from a variety of sectors and applications (ranging from subsistence farming communities, emergency responders, social enterprises and the commercial sector). In the USA, on the other hand, industries ranging from supply chain management, agriculture and aviation are highly dependent on high resolution forecast-driven decision making tools.
  • There is delayed transmission, relay, use and feedback of information and customised products within the value chain because of limitations in the integration and timely transmissions of data and information between the mandated institutions and the end-users.

Managing Expectations

Q3. Major technological breakthroughs and new applications (e.g. Artificial Intelligence, machine learning, new data, cloud computing) – where are they expected and what will be their impact?

• This is an area where the private sector is actively innovating and has developed tools that are mature
• The atmospheric sciences as a field has always been at the forefront applying new technological breakthroughs for weather forecasting - from numerical simulation of PDEs to big data to statistical modeling.
• Innovations will come from transitioning from deterministic to probabilistic forecasts informed by very large ensembles of high-resolution NWP simulations. Bayesian statistical approaches like BMA or model dressing promise ways to leverage this data to create fine-tuned (“sharp” and “calibrated”) probabilistic forecasts which will have a great deal of value for many types of decision-making
• See here for an example of CimaCell’s decision-making tools
• ^^ is especially exciting for sub-seasonal forecasts -> identifying, with higher specificity, when large scale weather patterns will drive heat, drought, flood, etc. There are huge implications for the global agricultural supply chain as these techniques continue to develop.

Q4. Technical improvement – accuracy & reliability and translation of data for decision making vis -à-vis value & impact

• Continued development of diverse epistemic communities which bring stakeholders from decision-maker and authorities will help lead to more useful forecasts.
• Connecting the weather forecast to “decisions on the ground” will help boost reliability (by forecasting the right things) and utility (by providing the right information and context)
• There is much opportunity for applying design thinking and user-centric design methods for serving vulnerable communities with decision-making tools.

Systems Operation

Q7. How to integrate seamless weather/climate forecasts into user sector specific forecasts, -where do we address specific sector users weather/climate forecasts needs?

• This depends entirely on the user's needs. Making data easier to access and process will help more users gain insights for decision making.
• “Address[ing] specific users weather/climate forecast needs” is likely a job best suited for the private sector in partnership with public and community partners. The endeavors required to build better/more sophisticated weather forecasts are likely to be very capital intensive, and require technology investments on timescales unattractive to private enterprise. But delivering and constructing value from weather forecasts and climate information to meet specific consumer and user needs requires high investment in a specific set of skills and services and is beyond the scope and mandate of most of the public agencies which will make those investments. Thus, this is a great example of where public-private partnerships can “complete” the value chain and ensure that taxpayers and corporations can derive as much value as possible from the weather enterprise.
• Technology best serves a user when it is designed with rapid and iterative feedback from the user. When considering how to address the needs of vulnerable communities through forecasting, vehicles that enable the user to be a part of the design process from the beginning (e.g. leveraging human centered design methods can be beneficial). Partners with existing and trusted dissemination channels must also be equipped with access and knowledge to best utilize forecasts.

Resources

Q9. What will be needed in terms of investment for infrastructure, to realize the vision of future weather and climate forecast?

• Improved observation systems - we continue to lack critical weather observations in sparsely populated parts of the world and over oceans. Improved remote observing systems - such as autonomous drones or other systems which can probe and measure the 3D volume of the atmosphere - will have a significant impact on global forecast quality. The private sector will play a critical role in the execution of this.
• Democratization of weather and climate data - Weather and climate data are massive , and it’s increasingly problematic to get this data into the hands of users. The old model of storing data in centralized archives - ostensibly part of high-performance computing systems, which likely produce the data - is not capable of scaling to support the diverse user group expected to want this data over the next decade, for two reasons: (1) Remote users likely cannot get access to them, and (2) security concerns make it unwieldy to support large user bases on these systems. A solution is needed to make data open and accessible for diverse stakeholders from across the globe. Cloud-powered data archives and support infrastructure are likely a key enabler.
• High performing financial sustainable NMS’s

Q10. What would be optimal mechanisms to support developing countries through the advanced future forecasting systems?

• It would be helpful to target funding toward projects that maximize scalability. For example, cloud-based software solutions deployed in one country can be seamlessly ported to assist other countries with similar needs.
• To build on the above point, it’s not about porting - we’d go so far as to propose “systems as a service”: cloud-based infrastructure which can be customized and re-deployed with a single click of a button.
• It would also be helpful to target funding toward projects that will demonstrate the value of a public-private interaction, inclusive reach, and lead to eventual self-sustainability.

Q12. Based on your views on the impact of these developments on the way NMHSs will be organized in the future, your advice to NMHSs – where to put effort and how to avoid unjustified investments?

• NMHSs should engage in community-based modeling and data initiatives / consortiums. Confederating activities and targeting inter-country collaborations will help scale core NHMS technology and capability development
• Resources and experimentation of public-private engagement with the goal of eventual financial sustainability.

PPE: Present and Future

Q14. How do you foresee the evolving roles of public, private and academic stakeholders in the future forecasting enterprise?

• We should expect larger investment and resources to enter through the private sector as the role of weather and climate issues are raised in activities across many global industries. However, to really create value, the agents which bring these resources will need guidance from and collaboration with the traditional sectors of the Enterprise, else it will end up being a very inefficient way to realize positive impacts from the forecasting enterprise.
• We will likely see the expansion of stakeholders included in the Weather Enterprise, including an active contribution from NGOs, social enterprises and international development organizations.
• Expect a shift in the public and academic sectors with regards to investment on climate - given our current challenges with mitigation of climate change, there will be an imperative to aggressively study weather and climate adaptation. This will likely manifest as a shift in public funding from climate towards weather applications.