By David Driscoll

I write this shortly after Christmas of 2016. As always, the season was marked by the challenge of finding presents for people who already have every material thing they really need or want. For such recipients, I often purchase things that everyone needs to replace sooner or later, such as gloves or umbrellas. Less commonly, I buy them a book that I think most anyone would find a worthwhile read.

One such book is The Checklist Manifesto: How to Get Things Right, by Atul Gawande. First published in 2009, it became a New York Times bestseller and continues to sell well today. In the book, as in his 2007 article in The New Yorker on the same subject, Gawande argues that the probability of good outcomes in the performance of professional work of great technical complexity can often be significantly improved by the use of checklists to ensure that necessary steps have been taken. A surgeon by trade, Gawande cites multiple examples from the practice of medicine to illustrate how the incidence of bad outcomes in his profession could be, and had been, reduced markedly through the use of checklists.

Gawande’s examples include descriptions of the resistance that has often confronted the assertion that so elementary a tool could be of help in improving the work of highly trained physicians. There is probably universal agreement that checklists are appropriate for making sure that one has packed appropriate gear for a camping trip, or that the functioning of the safety equipment in an automobile has been verified, but many have difficulty believing that professionals with high levels of training and experience would find checklists beneficial in their work. Experts, after all, are generally regarded as such because of their ability to recognize the unique and important characteristics of a given situation and to adapt their actions to its demands. A checklist, by contrast, is effectively a hard-and-fast set of rules. Its use does not require a high level of professional education or experience.

But as Gawande points out, the complexity of modern medical practice, and of modern technology generally, has created a void that in many cases can appropriately be filled by a checklist. In an intensive care unit, for example, there are simply too many aspects of patient care that must be executed appropriately to rely on anything less than systematic checking to establish that they have been. In the field of aeronautics, Gawande recounts how advancing aircraft design required the introduction of checklists to be used by pilots in takeoff, flight, landing, and taxiing. The more complex our work becomes, it seems, the more we can benefit from the use of checklists to ensure that everything necessary for a good outcome has been completed.

Others (notably Jay Jaffe, in the April/May 2014 edition of The Actuary) have taken note of Gawande’s writings and the potential for actuaries to use checklists in their work to guard against errors. The development of the kinds of checklists Jaffe describes would generally have to reflect the tasks required to prepare specific actuarial work products. As a member of the Actuarial Board for Counseling and Discipline, I particularly want to encourage actuaries to think also about the use of checklists to ensure that their work complies with the standards of professionalism embodied in the Code of Professional Conduct and actuarial standards of practice (ASOPs).

The impetus for this suggestion is (as one might expect) recognition of the growing complexity of doing all one must do to remain “professional.” While many aspects of the Code seem not to require a checklist to ensure compliance (e.g., you probably do not need a checklist to ensure that you have not unintentionally violated Precepts 9 or 12), other requirements have grown in their complexity and specificity in recent years in ways that would make it difficult for even a seasoned practitioner with a highly developed sense of professionalism to fulfill them instinctively.

A good example is Actuarial Standard of Practice No. 41, Actuarial Communications. Almost all actuarial work products are subject to at least some aspects of ASOP No. 41, and yet I think it is safe to say that it receives far less attention from most actuaries than do the ASOPs that apply specifically to their individual areas of practice. For this reason, I think many actuaries would find it beneficial to take a look at the most recent reports they have issued and ask themselves the following (which, in fact, is adapted directly from Section 4.1.3 of ASOP No. 41):

Does my report identify or state:

the intended users of the report?
the scope and intended purpose of the engagement or assignment?
the acknowledgement of qualification, as specified in the Qualification Standards?
any cautions about risk and uncertainty (per section 3.4.1 of ASOP No. 41)?
any limitations or constraints on the use or applicability of the actuarial findings contained within the actuarial communication (including, if appropriate, a statement that the communication should not be relied upon for any other purpose)?
any conflict of interest, as described in section 3.4.2 of ASOP No. 41?
any information on which the actuary relied that has a material impact on the actuarial findings and for which the actuary does not assume responsibility (see section 3.4.3 of ASOP No. 41)?

Another area of professionalism that might merit such a review is data quality. In December, the Actuarial Standards Board adopted a revised edition of Actuarial Standard of Practice No. 23, Data Quality. While the vast majority of actuaries have a strong appreciation of the importance of data quality in their work, the specific actions and disclosures required to do it justice have been changed in the new edition of ASOP No. 23. It might therefore be helpful to review reports issued in the near future using a checklist constructed from the disclosure section of the revised ASOP (which is effective for work products issued on or after April 30, 2017):

Does my report identify or state:

the source(s) of the data used?
any limitations on the use of the actuarial work product due to uncertainty about the quality of the data or other information relevant to the use of the data (as discussed in section 3.1 of ASOP No. 23)?
whether the actuary performed a review of the data and, if not, the reason for not reviewing the data and any resulting limitations on the use of the actuarial work product (as discussed in section 3.3 of ASOP No. 23)?
unresolved concerns the actuary may have about questionable data values that are relevant to the use of the data and could have a significant effect on the actuarial work product (as discussed in section 3.3(b) of ASOP No. 23)?
discussions of any significant steps the actuary has taken to improve the data due to identifying questionable data values or relationships (as discussed in section 3.3(b) of ASOP No. 23)?
significant judgmental adjustments or assumptions that the actuary applied to the data or to the results, or are known by the actuary to have been applied to the data, to allow the actuary to perform the analysis (as discussed in section 3.4(c) of ASOP No. 23)?
the existence of results that are highly uncertain or have a potentially significant bias of which the actuary is aware due to the quality of the data or other information relevant to the use of the data, and the nature and potential magnitude of such uncertainty or bias, if they can be reasonably determined (as discussed in section 3.4(c) of ASOP No. 23)?
the extent of the actuary’s reliance on data and other information relevant to the use of the data supplied by others (as discussed in sections 3.5 and 3.6 of ASOP No. 23)?
the disclosure in ASOP No. 41, section 4.2, if any material assumption or method was prescribed by applicable law (statutes, regulations, and other legally binding authority)?
the disclosure in ASOP No. 41, section 4.3, if the actuary states reliance on other sources and thereby disclaims responsibility for any material assumption or method selected by a party other than the actuary?
the disclosure in ASOP No. 41, section 4.4, if, in the actuary’s professional judgment, the actuary has otherwise deviated materially from the guidance of ASOP No. 23?

These examples are certainly not comprehensive, and it is undoubtedly possible to create checklists that are better suited to the review of particular work products. However, I believe these examples illustrate that it is not necessarily difficult to create checklists that facilitate compliance with aspects of professionalism that can easily be overlooked in the preparation of actuarial work products. I encourage all who read this to try this in their work, and to see whether it does not result in finished products that are more fully “up to code.”

DAVID DRISCOLL is a member of the Actuarial Board for Counseling and Discipline.

Up To Code

A ‘Checklist Manifesto’ for Professionalism?