Up To Code

Communicating Uncertainty: The Importance of Acknowledging Limitations

Communicating Uncertainty: The Importance of Acknowledging Limitations

By David Driscoll

“I know one thing; that I know nothing.”

If you recognize the quotation above as one commonly attributed to Socrates, congratulations. You are probably someone who does well in trivia contests.

If you recognize the quotation as one that is commonly incorrectly attributed to Socrates, greater congratulations. You may well be someone who has had to dispel plausible but false notions on more than one occasion.

Refuting falsehoods that are widely held to be true is not a task for the faint of heart, as the example of Socrates shows. For leading his pupils in critical examinations of their society’s accepted verities, he was charged with corrupting the youth of Athens and promoting “false gods,” adjudged guilty, and sentenced to death. Socrates’ defense at his trial is commemorated in Plato’s Apology, which quotes him as stating not that he knows nothing, but rather that he is wiser than those who imagine they know things that they do not, for he is fully aware that there are certain things he does not know. The apocryphal quotation presented above is suspected by some of being an oversimplification of Socrates’ actual remarks as presented by Plato.

The experiences of Socrates, Galileo, Darwin and others whose life work has consisted of challenging the prevailing beliefs of their epochs all point to a major difficulty faced by those who must tell others that what they have long thought to be right may in fact be wrong: Humans prefer certainty to ambiguity in nearly all matters. This trait has been well documented in psychological research.

In many ways, actuaries make their living catering to this preference. The longtime motto of one of the actuarial organizations to which I belong suggests as much: “The work of science is to substitute facts for appearances and demonstrations for impressions.” We employ our examination-validated statistical abilities and knowledge of financial matters to place monetary values on obligations that are conditioned upon events whose timing and/or cost cannot be known with certainty, and we are the accepted providers of valuations needed for transactions related to such obligations. How much money must a retirement plan accumulate to provide a now-62-year-old Connecticut state employee with a monthly payment of $5,000 for the remainder of his life (perhaps with continuance to his now-61-year-old-spouse after his death, should his spouse survive him, or maybe not; perhaps with annual adjustments for inflation, or maybe not)? How much money must be set aside to cover the likely cost to repair a home and its contents on the north side of Minneapolis should they be damaged by fire (perhaps with exclusions of damage resulting from certain sources of combustion, perhaps with caps on overall reimbursements or coverage of personal property in the dwelling, perhaps with reflection of changes in the cost of household items since they were purchased)? How much must be charged to provide a 59-year-old woman residing in Los Angeles with health care coverage for one year (taking into account myriad policy provisions related to access to specialized services, deductibles, mandatory coverage provisions stipulated in applicable law, required co-payments, lifetime maximums, exclusions for pre-existing conditions, and claims for which others are deemed liable)?

As a profession, we are quick to remind the world at large that actuaries are ready, willing, and able to provide answers to such questions.

Our work serves purposes above and beyond the mere fulfillment of an emotional need for concrete answers in the presence of chaos. Most importantly, perhaps, it promotes a proper allocation of economic resources to the provision of financial security programs in both the public and private sectors. However, it is important to for us to bear in mind that what actuaries provide are nearly always estimates, not final answers, and that these estimates—even if based on good data and methods—are quite likely wrong, primarily because the assumptions on which they are based are highly unlikely to be realized exactly. This does not diminish the utility of our work, but it does mean that special care must be taken in communicating its results to users who do not necessarily understand that what actuaries are tasked with quantifying is often impossible to measure precisely.

The clear conclusion this leads to is that actuaries ought to think about providing information about risk, in the form of the variability of potential outcomes, as part of, or in addition to, their work products. Furnishing such information not only enhances completeness and transparency in the advice provided to a principal but may also be encouraged on the grounds of what might be termed the defensive practice of actuarial science. The omission from traditional actuarial work products of good explanations of what the numbers mean and caveats that actual outcomes will not necessarily match projections may result in a disgruntled principal who does not correctly understand the information provided and is thus inclined to attribute the consequences of adverse experience to “mistakes” on the part of the actuary. The human tendency to be angered when confronted with evidence that settled expectations are “untrue” is never stronger than when the party producing such evidence is the same one who established those expectations originally.

The evolution of actuarial practice over the years points to an increasing focus on ranges of possible outcomes in addition to point estimates. This has undoubtedly been driven in part by the growing availability of computational tools that can perform the necessary calculations. I believe it has also been impelled by changes in the Code of Professional Conduct and actuarial standards of practice (ASOPs). Arguably, fulfillment of precepts 4 (“… shall take appropriate steps to ensure that the Actuarial Communication is clear and appropriate to the circumstances and its intended audience”) and 8 (“The Actuary should recognize the risks of misquotation, misinterpretation, or other misuse of the Actuarial Communication…”) of the Code is enhanced by inclusion of information about the risk associated with actuarial measurements presented in a statement of actuarial opinion. More important, ASOP No. 41, Actuarial Communications (which, the Academy’s Council on Professionalism’s Applicability Guidelines remind us, is applicable to all assignments in all practice areas) has long indicated that the actuary “should consider what cautions regarding possible uncertainty or risk in any results should be included in the actuarial report.” In practice-specific standards of practice, we have seen a strengthening of risk-related communications requirements. Pension actuaries, for example, have been required for some years by ASOP No. 4, Measuring Pension Obligations and Determining Pension Plan Costs or Contributions, to include the following language, or something substantially equivalent, in reports of the results of measurements of pension obligations:

Future actuarial measurements may differ significantly from the current measurements presented in this report due to such factors as the following: plan experience differing from that anticipated by the economic or demographic assumptions; changes in economic or demographic assumptions; increases or decreases expected as part of the natural operation of the methodology used for these measurements (such as the end of an amortization period or additional cost or contribution requirements based on the plan’s funded status); and changes in plan provisions or applicable law.

This language must be accompanied by a statement of whether an analysis of the potential range of future results was conducted (and if so what the results were) or was deemed to be outside the scope of the assignment. Recently adopted ASOP No. 51, Assessment and Disclosure of Risk Associated with Measuring Pension Obligations and Determining Pension Plan Contributions, which applies to pricing or funding valuations of pension systems made on or after Nov. 1, 2018, will require the inclusion of risk assessments in reports on the results of such valuations.

A statement quite similar to the one that opened this piece, both in terms of the sentiment it conveys and its frequent incorrect attribution, is “the beginning of wisdom is the admission of one’s ignorance.” Acknowledging one’s own ignorance may or may not put one on the path to wisdom, but straightforward statements of the limitations of our work, and their implications for its users, is an increasingly important part of good actuarial practice.

 

DAVID DRISCOLL, MAAA, EA, FSA, FCA, is a member of the Actuarial Board for Counseling and Discipline.

print

Next article Professionalism: Our Ongoing Conversation With the Public
Previous article Buffet’s Greatest Hits

Related posts