Social Sciences in Ecological Forecasting

Date: December 13, 2019

Post by: Kira Sullivan-Wiley1 and Jaime Ashander2

Series contributors: Mike Gerst3, Kathy Gerst4,5, Kailin Kroetz6, Yusuke Kuwayama2, and Melissa Kenney7

1Boston University, 2Resources for the Future, 3University of Maryland, 4USA National Phenology Network, 5University of Arizona,  6Arizona State University,  7University of Minnesota

Ecological forecasting involves models and ecology but is also a fundamentally people-centered endeavor. In their 2018 paper, Mike Dietze and colleagues outlined the ecological forecasting cycle (Figure 1 is a simplified version of that cycle) where forecasts are designed, implemented, disseminated, iteratively reassessed and improved through design.  This cycle is about a process, and in each part of this process there are people. Wherever there are people, there are social scientists asking questions about them, their actions, and how to improve decisions made by these insights.

So we ask the questions: How might ideas from the social sciences improve ecological forecasting? What new opportunities and questions does the emerging interdisciplinary field of ecological forecasting raise for the social sciences?

This post introduces a series of posts that address these questions, discussing opportunities for the social sciences in Ecological Forecasting Initiative (EFI) and gains from considering humans in forecasting research. Thus, our aim is to better describe the role of social scientists in the ecological forecasting cycle and the opportunities for them to contribute to EFI.

Figure 1  The process of producing a forecast, which may begin at any step depending on which stakeholder group initiates.

So where are the people?

At EFI, we’re interested in reducing uncertainty of forecasts, but also improving the processes by which we make forecasts that are useful, useable, and used. This means improving forecasts, their design, use, and impact, but it also results in a range of opportunities to advance basic social science.

To do this we need to know: Where are the people in this process? Where among the boxes and arrows of Figure 1 are people involved, where do their beliefs, perceptions, decisions, and behavior make a difference? Are there people outside this figure who matter to the processes within it? Figure 2 highlights critical groups of people involved, and some of the actions they take, that are integral to the ecological forecasting process.

Figure 2 Critical actors in the ecological forecasting process, linked with important actions

Figure 2 moves beyond the three-phase process described in Figure 1 because the process of ecological forecasting is predicated on funding sources and data, often provided by people and processes outside the forecasting process. So when we think about where social science belongs in ecological forecasting, we have to look beyond the forecasting process alone.

Making forecasts better

If EFI wants to make this process work and produce better forecasts, which of these actors matters and how? How might different stakeholders even define a “better” forecast? One might think of “better” as synonymous with a lower degree of uncertainty, while another might measure quality by a forecast’s impact on societal welfare. Ease of use, ease of access, and spatial or temporal coverage are other metrics by which to measure quality, and the relative importance of each is likely to vary among stakeholders. Social science can help us to answer questions like: Which attributes of a forecast matter most to which stakeholders, under what conditions, and why?

While a natural scientist might assess the quality of a forecast based on its level of uncertainty, a social scientist might assess the value of a forecast by asking:

  • What individuals are likely to use the forecast?
  • How might the actions of these individuals change based on the forecast?
  • How will this change in actions affect the well-being of these or other individuals?

Building “better forecasts” will require a better understanding of the variety of ways that stakeholders engage with forecasts. The posts in this blog series will shine a spotlight on some of these stakeholder groups and the social sciences that can provide insights for making forecasts better. Posts in the series will discuss issues ranging from how stakeholders interact with forecast visualizations to the use of expert judgements in models to when forecasts should jointly model human behavior and ecological conditions. Considering these questions will help forecasters design forecasts that are more likely to increase our understanding of these socio-environmental systems and enhance societal well-being.

These examples hint at the range of potential interactions between the social sciences and ecological forecasting. There is a wealth of opportunity for social scientists to use the nascent field of ecological forecasting to ask new and interesting questions in their fields. In turn, theories developed in the social sciences have much to contribute to emerging interdisciplinary practice of ecological forecasting in socio-environmental systems. As we can see, better ecological forecasting may require us to think beyond ecological systems.

Making Ecological Forecasts Operational: The Process Used by NOAA’s Satellite & Information Service

Date: November 18, 2019

Post by Christopher Brown; Oceanographer – NOAA

My last blog briefly described the general process whereby new technologies and products are identified from the multitude available, culled, and eventually transitioned to operations to meet NOAA’s and its user’s needs, as well as offered some lessons learned when transitioning ecological forecasting products to operations, applications, and commercialization (R2X). In this blog, I introduce and briefly describe the steps in the R2X process used by NOAA’s Satellite & Information Service (NESDIS). NESDIS develops, generates, and distributes environmental satellite data and products for all NOAA line offices as well as for a wide range of Federal Government agencies, international users, state and local governments, and the general public. A considerable amount of planning and resources are required to develop and operationalize a product or service, and an orderly and well-defined review and approval process is required to manage the transition. The R2X process at NESDIS, managed by the Satellite Products and Services Review Board (SPSRB), is formal and implemented to identify funds and effectively manage the life cycle of a satellite product and service from development to its termination.  It is a real-life example of how a science-based, operational agency transitions research to operations. A ‘broad brush’ approach of the process is given here, yet will hopefully be useful in providing insight into the major components involved in an R2X process that can be applied generally to the ecological forecasting (and other) communities. Details can be found in this SPSRB Process Paper.

The first step in the R2X process is acquiring a request for a new or improved product or service from an operational NOAA “user”. NESDIS considers requests from three sources: individual users, program or project managers, and scientific agencies. Individual users must be NOAA employees, so a relationship between a federal employee and other users, such as from the public and private sectors, including academia and local, state and tribal governments, must first be established. The request, submitted via a User Request Form similar to this one, must identify the need and benefits of the new or improved product(s) and includes requirements, specifications and other information to adequately describe the product and service. As an example, satellite-derived sea-surface temperature (SST), an operational product generated from several NOAA sensors, such as the heritage Advanced Very High Resolution Radiometer (AVHRR) and the current Visible Infrared Imaging Radiometer Suite (VIIRS), was requested by representatives from several NOAA Offices.

If the SPSRB deems the request and its requirements valid and complete, the following six key steps are sequentially taken:

  1. Perform Technical Assessment
  2. Conduct Analysis of Alternatives
  3. Develop Project Plan
  4. Execute Product Lifecycle
  5. Approve for Operations, and
  6. Retire or Divest

These steps are depicted in Figure 1.

Figure 1. Key SPSRB process steps.  Credit: Process Paper, Satellite Products and Services Review Board, 2018, SPSRB Improvement Working Group, Ver. 17, Department of Commerce. NOAA/NESDIS, 23 July 2018, 29pp.

1. Perform Technical Assessment and Requirements Validation

A technical assessment is performed to determine if the request is technically feasible, aligns with NOAA’s mission and provides management the opportunity to decide the best ways to process the user request. For instance, a user requests estimates of satellite-derived SST with a horizontal resolution of 1 meter every hour throughout the day for waters off the U.S. East Coast to monitor the position of the Gulf Stream.  Though the request does match a NOAA mandate, i.e. to provide information critical to transportation, the specifications of the request are currently not feasible from space-borne sensors and the request would be rejected.  On the other hand, a request for 1 km twice a day for the same geographic coverage would be accepted and the next step in the R2X process – Analysis of Alternative – would be initiated.

2. Conduct Analysis of Alternatives

An analysis of alternatives is performed to identify viable technical solutions and to select the most cost-effective approach to develop and implement the requested product or service that satisfies the operational need. An Integrated Product Team (IPT) consisting of applied researchers, operational personnel and users, is formed to complete this step. In the case of SST, this may be consideration of the use of data from one or more sensors to meet the user requests for the required frequency of estimates.

3. Develop Project Plan

The Project Plan describes specifically how the product will transition from research to operations to meet the user requirements following an existing template. Project plans are updated annually. The plan consists of several important “interface processes” that include: 

  • Identifying resources to determine how the project will be funded.  Various components of the product or services life cycle, from beginning to end, are defined and priced, e.g. support product development, long-term maintenance and archive. Though the SPSRB has no funding authority, it typically recommends the appropriate internal NOAA source for funding, e.g. the Joint Polar Satellite System Program;
  • Inserting the requirements of the product and service into an observational requirements list database for monitoring and record keeping;
  • Adding the product and service into an observing systems architecture database to assess whether observations are available to validate products or services, as all operational products and services must be validated to ensure that required thresholds of error and uncertainty are met; and,
  • Establishing an archiving capability to robustly store (including data stewardship) and to enable data discovery and retrieval of the requested products and services.

4. Execute Product Lifecycle

Product development implements the approved technical solution in accordance with the defined product or service capability, requirements, cost, schedule and performance parameters.  Product development consists of three phases:  development, pre-operational and operational.  In the development stage, the IPT uses the Project Plan as the basis for directing and tracking development through several project phases or stages.  In the pre-operations stage, the IPT begins routine processing to test and validate the product, including limited beta testing of the product by selected users. Importantly, user feedback is included in the process to help refine the product and ensure sufficient documentation and compatibility with requirements.

5. Approve for Operations

The NESDIS office responsible for operational generation of the product or service decides whether to transition the product or service to operations.  After approval by the office, the IPT prepares and presents a decision brief to the SPSRB for it to assess whether the project has met the user’s needs, the user is prepared to use the product, and the product can be supported operationally, e.g. the infrastructure and sufficient funding for monitoring, maintenance, and operational product or service generation and distribution exists. The project enters the operations stage once the SPSRB approves the product or service. If the user identifies a significant new requirement or desired enhancement to an existing product, the user must submit a new user request.

6. Retire or Divest

If a product or service is no longer needed and can be terminated, or the responsibility for production can be divested or transferred to another organization, it enters the divestiture or retirement phase.

Each of NOAA’s five Line Offices, e.g. Ocean Service, Weather Service, and Fisheries Service, has their own R2X process, that differs in one way or another to that of NESDIS. Even within NESDIS, if a project has external funding, it may not engage the SPSRB.  Furthermore, the process may be updated if conditions justify, such as additional criteria are introduced from the administration.  The process will, however, generally follow the major steps involved in the R2X process: user request, project plan, product/service development, implementation, product testing and evaluation, operationalization, and finally termination.

Acknowledgment: I thank John Sapper, David Donahue, and Michael Dietze for offering valuable suggestions that substantially improved an earlier version of this blog.

Making Ecological Forecasts Operational: Some Lessons Learned By NOAA

Date: July 22, 2019

Post by Christopher Brown; Oceanographer – NOAA

Ideally, newly developed ecological forecasts deemed useful should be transitioned to operations, applications or commercialization to benefit society. More succinctly, if there is no transition, there is no outcome. NOAA develops and transitions ecological forecasts to fulfill its mandates and role in the protection of life, property, human health and well-being, and in stewardship of coastal, marine, and Great Lakes environments. A depiction of this general process (Figure 1), affectionately called “the R&D funnel”, illustrates how new technologies and products are identified from the multitude available from numerous sources and are culled and eventually transitioned to meet NOAA’s needs.  Unfortunately, while several ecological forecasting projects at NOAA have been successfully transitioned to operations, many projects remain primarily in a research mode. To better understand the reasons behind this, NOAA’s Ecological Forecasting Roadmap program conducted an unpublished study in 2014 that compared ecological forecasting projects that had been successfully transitioned to operations with those that languished or failed in order to identify common characteristics related to the success or failure of the transition. (NOAA defines operations as “sustained, systematic, reliable, and robust mission activities with an institutional commitment to deliver specified products and services”.) Based on the comparative analysis of nine projects, a list of “lessons learned” for transitioning ecological forecasts to operations, applications or commercialization (R2X) were compiled. The salient points are listed below:

  • Identify the “owner” or group responsible for operationally producing the product or service as early as possible in the process.  In baseball vernacular, find the catcher’s mitt;
  • Engage the users, stakeholders and decision makers, from researchers to management, as early as possible to establish user needs and obtain routine feedback for setting and updating product requirements;
  • Find and secure funding of the product or service to ensure its transition, verification, sustainment and improvement;
  • Plan and document, to the best of your ability, as many steps in the life of the product or service, from research to operations to termination, including the entities responsible, the major milestones, and the funding required. This activity will focus attention on the steps that need to be taken and provide information required in the formal R2X process; and
  • Include plans for the sustained collection, analysis and archive of relevant data necessary for product verification.
Figure 1. R&D Funnel