By Steve Jackson
“Just the facts” is a phrase that has come to mean objectivity is primary. This is the Academy’s mantra as well. In providing the facts about our efforts to provide objective and unbiased information about climate issues, I think it’s also useful to note “All models are wrong, but some are useful.”
The statistician George E.P. Box’s famous aphorism would have been a fine subtitle for the panel “Actuaries Climate Risk Index” at the fall meeting of the Middle Atlantic Actuarial Club on Nov. 20, 2019. Box elaborated: “Since all models are wrong the scientist must be alert to what is importantly wrong. It is inappropriate to be concerned about mice when there are tigers abroad.”[1]
With this metaphor in hand, we might say that the primary theme of the session was distinguishing mice from tigers when modeling changing weather conditions and the impact of those changing conditions on losses. A second recurrent theme was the humility that must follow from recognition that “all models are wrong.” Finally, the presentations at this panel emphasized that weather conditions are becoming more severe, and that while increasing losses in recent years associated with extreme weather may follow from changing climatic conditions, those losses most certainly also follow from changing patterns of development—some of which increase risk exposure and some of which decrease that exposure.
The panel consisted of presentations by Margaret Conroy, MAAA, FCAS, Ph.D., and me, Steve Jackson, Ph.D. Conroy is principal at Analytic Solutions and chief actuary, P&C for Windsor Strategy Partners; she discussed the Actuaries Climate Index (ACI). I am the assistant director for research (public policy) at the Academy; I discussed the Actuaries Climate Risk Index (ACRI). We concluded by jointly discussing ongoing efforts to develop version 2.0 of both the ACI and the ACRI.
Why Do Actuaries Care About Climate Risk?
Before describing the ACI, Conroy presented evidence for the increasing severity of losses from extreme weather events in North America. Relying in large part on data drawn from the National Oceanographic and Atmospheric Administration’s Billion Dollar Losses database, she illustrated the range of climatic events that increasingly produce very large losses, and the geographic scope of those very large losses—encompassing much of the continental United States. By the end of September 2019, “billion-dollar weather and climate disasters” in 2019 alone numbered 10, ranging from severe weather in the Southeast Ohio Valley in late February to Tropical Storm Imelda in late September, from Missouri River flooding in the second half of March to tornadoes across the Southeastern states in April. In 2018, 14 billion-dollar disasters accounted for $91 billion in losses; other weather-related losses accounted for at least an additional $10 billion.[2]
Actuaries Climate Index
With losses this large, it makes sense for actuaries to have for their own use, and to provide to policymakers and the public, an educational tool describing changes in the conditions that produce the extreme weather events. The Actuaries Climate Index (ACI) is an objective indicator of changes of six climate-related conditions since 1990 in the continental United States and Canada. The six monitored conditions are high and low temperature, precipitation and the lack of precipitation, wind power, and sea level. The index relies on publicly available data to provide a monitoring tool for past climate trends, built in a manner designed to be statistically robust yet easy to understand. Developed jointly by four actuarial associations (the Academy, the Canadian Institute of Actuaries, the Casualty Actuarial Society, and the Society of Actuaries), in the fall of 2016 the ACI went live with graphs depicting the trends in each of the six components and in the index as a whole. The graphs are available for the U.S. and Canada combined, for each country separately, and for any of 12 large regions in North America.
The ACI is distinctive in several respects from other indices of changing climatic conditions. First, many other indices focus on averages; the ACI focuses on extremes. For example, rather than looking at the average temperature, the ACI relies on the frequency of daily high temperatures above the level of the 90th percentile during a reference period, 1961–1990. Second, the ACI incorporates six indicators of changing climate, while most indices include only one or two. The inclusion of more than one indicator creates an intrinsic challenge: how to combine the apples and oranges of, for example, temperature and rainfall, into a single index. The ACI meets this challenge by normalizing each indicator for each region-month combination, subtracting from the observed value the mean of that indicator during the reference period, and dividing the difference by the standard deviation also from the reference period.
The resulting metric for each indicator captures the variation from the mean of the reference period in standard deviations. For each indicator (high temperatures, for example), a standardized value of zero indicates that the value equals the reference period mean; positive values indicate larger values than the reference period. Without knowing precisely the nature of the distributions of these values, we can still recognize an increase of 2 standard deviations as an anomaly—an indication that the likelihood is very high that the true value is larger than the mean of the reference period.
These standardized metrics can then be examined for individual components, combined into a single index for a single location, or aggregated to create metrics for regions and super-regions (i.e., countries and continents). It is useful to note that while the standard deviation for each indicator during the reference period is, by definition, 1, the standard deviation of the combined ACI, reflecting both the sum and the correlations of the variances of the separate indicators, is only 0.45. As a result, a standardized individual indicator of 2.0 reflects a clear anomaly; at the same time, an ACI value of 0.90 reflects a similarly anomalous value for the index.
What these metrics are telling us is that for the United States and Canada combined (and for the United States by itself), the climatic indicators are significantly more extreme than they were during the reference period. The five-year moving average of the ACI has been greater than 1 since fall 2018, and it continues to increase. The steady increase of the ACI since the early 1990s reflects varying degrees of change in the individual indicators. As of winter 2019 (and again looking at five-year moving averages), sea level changes are more than 2.5 standard deviations above the reference period, followed by high temperatures, precipitation, and low temperatures, with values ranging from 1.9 to 1.4 standard deviations. Wind and dry days do not appear materially different from the reference period.
The importance of the different indicators varies by region as well. For the Midwest region of the U.S., precipitation drives the ACI higher, with high temperatures also contributing. For the Central East Atlantic of the U.S., sea level drives the ACI higher, with secondary contributions from both high temperatures and precipitation. In the Southeast Atlantic region, sea level also drives the index primarily, with high temperatures also contributing. Perhaps the most conspicuous difference in the changes observed in the indicators across regions contrasts the increase in sea level in the Southeast Atlantic region—almost 4 standard deviations higher than the reference period—with the decrease in sea level in Alaska (more than 4 standard deviations lower than the reference period) offsetting this result somewhat.
The ACI clearly reveals a pattern of increasingly frequent extreme weather and sea level change. Taken as a whole, these measures indicate a warmer climate in the last 28 years compared to the 30 prior years. Not all indicators move together, and different regions of the U.S. and Canada have experienced distinct patterns. But these results need be tempered with humility; there are mice, and perhaps tigers, roaming around in the assumptions and simplifications required to construct this index.
At the conclusion of the session, Conroy and I turned our attention to some of the sources of uncertainty in the ACI and where we might head in crafting a version 2.0. But, first, we turned to a presentation of the Actuaries Climate Risk Index (ACRI).
Actuaries Climate Risk Index
The ACRI builds on the foundation of the ACI to answer the question: If, as the ACI indicates, the climate is warming, is the extreme weather and rising sea level that results producing increased losses of property and human lives not attributable to increased populations and development in areas subject to extreme weather? The unequivocal answer from the soon-to-be-released version 1.0 of the ACRI is … maybe. While this result may surprise many who consider the billion-dollar losses described above as clear indication that climate-related losses are increasing, there is a clear difference between asking whether those losses are increasing and asking whether they are increasing due to changes in weather, rather than changes in the value of housing and other development in areas most at risk of extreme weather.
Answering this question depends on combining the data on which the ACI is built with data on property damage, crop losses, and human lives lost and injured due to weather-related events. Publicly available data from the Sheldus database makes this possible. Sheldus—Spatial Hazard Events and Losses Database for the United States—is a publicly available database (for a fee) with county-level hazard dataset that records losses of property, crops, lives, and injuries. The database draws primarily on records from NOAA’s Monthly Storm Reports, which are now reported in NOAA’s own Storm Events Database, and is maintained by a research center at Arizona State University.
Reinforcing the message from the billion-dollar disasters data, the Sheldus data also shows significant increases in losses since the end of the reference period. In 15 of the 26 years since 1990 analyzed (the ACRI analysis ended with 2016 data), the inflation-adjusted losses were greater than the losses in the single year with the largest losses during the reference period, 1965. In the 11 of the 26 years post-reference period, when losses were smaller than the largest loss during the reference period, the average loss was 40% larger than the average loss across all years, peak and non-peak, during the reference period.
The ACI clearly indicates that the weather has become more extreme since the end of the reference period. The data on losses indicates that losses have also increased during that time. The ACRI gives us reason to believe that a portion—a relatively small portion—of those increased losses are probably due to the changes in climatic conditions.
In a prior effort to construct the ACRI, some years ago the four associations that had constructed the ACI hired a consulting firm, Solterra Solutions, to assist. Their efforts identified the most promising datasets for losses in the U.S. and Canada; the current ACRI relies on the same U.S. source but found insufficient data in the Canadian source, leading version 1.0 of the ACRI to cover only the U.S. Solterra also provided a functional form based on work from the United Nations Development Programme, with which to relate climate indicators and economic losses. The current ACRI relies on that same functional form, adapting it to include all regions in one estimating equation, where Solterra estimated relations separately for each region.
While several models were explored (and results from several reported in the research update, which will be published shortly), a relatively simple regression of a linear transformation of an exponential model was found to be the most robust. That model controls for inflation, region, seasonality, and exposure risk. This model still has many vulnerabilities, also reported in the research update, and puts this effort squarely with other efforts reviewed by the U.S. Government Accountability Office in 2017 when it concluded: “Methods used to estimate the potential economic effects of climate change in the United States … and the studies that use them produce imprecise results because of modeling and other limitations but can convey insight into potential climate damages across sectors in the United States” (emphasis added).[3]
The principal insight that arises from version 1.0 of the ACRI comes from documenting a best estimate of property losses in the U.S. since 1990 that might be attributed to changes in climatic conditions, controlling for exposure risk. The ACRI estimates that $24 billion in losses occurred in the US between 1991 and 2016 as a result of changing climate risk (with a 90% confidence internal ranging from $2 billion to $45 billion). This amounts to a little less than $1 billion per year, roughly 5% of all weather-related property losses during that time period. While some might regard this as a small estimate, it is important to note that this is consistent with the non-systematic estimates produced by the Intergovernmental Panel on Climate Change, which, in its last full report in 2014, noted: “Economic costs of extreme weather events have increased over the period 1960–2000. … However, the greatest contributor to increased cost is rising exposure associated with population growth and growing value of assets.”[4]
While there are many studies illustrating the substantial losses (of property and lives) that will occur by the end of the century if climatic change continues as most scientists expect, absent urgent and significant changes in policy and behavior), the ACRI belongs to the relatively small group of studies that have looked objectively and retrospectively at losses already occurred and places a specific number (with significant uncertainty) on those losses. If climate changes as the IPCC expects, the application of the ACRI methodology would be expected to show increasing losses in the future.
Toward Version 2.0
At the conclusion of the presentation, Conroy and I presented some of the ideas currently being discussed as ways to improve both the ACI and the ACRI. Much of that discussion focused on improvements to the ACI, which would then allow further improvements to the ACRI.
The most important issue being examined is that of granularity: The ACI currently relies primarily on data from a gridded dataset on a 2.5-by-2.5-degree grid, roughly 175 miles squared. These grid points are then averaged at the regional level to obtain the basic building blocks of the ACI. There are three levels of challenge presented here. First, the grid points themselves are based on reporting stations that occur unevenly throughout North America, but which represent the most granular level of reporting. Second, the distance between grid points (and the fact that only land-based grid points are included) means that many coastal locations—where major storms often hit—are measured based on grid points 100 miles or more away (this is true for New Orleans, for example, and for the entirety of the South Carolina coast). Third, in averaging the grid points across environmentally heterogeneous regions, a good deal of extremity of weather events is lost. When the winds of Hurricane Katrina in New Orleans are averaged with the relatively calm weather of northern Georgia as the storm struck, the result would be moderated, even for the days when Katrina struck. Because our measures are calculated over a month’s time, even for New Orleans itself, the monthly average is not nearly as extreme as Katrina. Finding ways to make the data more granular, and the aggregations of data more sensitive to the extremes, is one of the highest priorities.
While this issue and others being examined reflect possible areas for refinement of the ACI (and the ACRI), they should not obscure the objective value of version 1.0 of either ACI or ACRI. Both are tools designed to educate actuaries, policymakers, and the public on broad trends and patterns—and both do so in an objective and “just the facts” manner. But both can be improved. And a few more general lessons might follow from the experience with these climate indices:
- Models that are as useful as possible at one point may become less useful as technology and data availability make more information available, requiring refinement;
- Models may have a fair number of mice scurrying around with the tigers; and
- Those creating and using these models need to practice humility, especially when crafting novel tools for assessing emergent risks.
STEVE JACKSON, PhD, is assistant director for research (public policy) at the Academy.
Endnotes
[1] George E. P. Box, “Science and Statistics,” Journal of the American Statistical Association, Vol. 71, No. 356 (Dec., 1976), page 792.
[2] Billion-dollar losses from “2018’s Billion Dollar Disasters in Context”; Climate.gov; other weather-related losses from NOAA Storm Events database, accessed on December 4, 2019, with results tabulated by the Academy.
[3] GAO 17-720, “Climate Change,” September 2017.
[4] IPCC, 2014: Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects.
