Feature

Model Behavior—How a modern modeler can add value and drive effective business decisions

By Olyvia Leahy

While actuarial models are by their nature quite scientific, the process of developing and adapting a model requires a bit of art as well. This is where a skilled actuarial modeler really adds value, and where a model goes from being a formality of business processes to an extremely valuable decision-making tool.

With the recent introduction of several new standards and regulations, companies now find themselves scrambling not only to understand the intricacies of International Financial Reporting Standard (IFRS) 17, principle-based reserving (PBR), Life Insurance Capital Adequacy Test (LICAT), and U.S. tax reform, but also to decipher exactly how to integrate these changes into their already complex models.

Aside from these most recent regulations, companies deal with ever-changing rules to the game and a competitive landscape where the advantage goes to those that are most agile. There is a benefit to making current models more accurate. Another benefit is modeling things a company has previously never modeled. Ultimately, more reliable models can help drive business decisions and better assure that companies are taking the appropriate steps to plan for the future.

This article explores five ways to take a current modeling process and make it more effective, including: utilizing resources, embracing the infamous “black box” actuarial software, , designing robust solutions, identifying limitations, and most important—being tenacious.

Exhaust Available Resources

Actuarial modelers come from many different backgrounds with many types and amounts of experience. But even for modelers with decades of time under their belts, it’s foolish to go it alone. There are several resources that should be utilized while tackling a modeling project. Three main categories are people, documentation, and technology.

First, for a modeling solution to qualify as a solution, it must solve a modeling problem (and hopefully not create three more). This means that before all else, the problem needs to be clearly defined. It is always a good idea to reach out to other people who may have a different perspective on an issue to ensure a modeling solution is actually a solution. Often other professionals are able to see things that may be in the modeler’s blind spot. This saves a lot of time that would otherwise be wasted creating ineffective solutions as well as potential debugging later.

In addition, reaching out to other modelers is a great way to leverage past solutions. Modeling issues have a way of developing patterns that can almost be anticipated. Therefore, leveraging past solutions lets a modeler draw similarities between issues in and across models. This is most easily achieved by leveraging the knowledge and expertise of others.

Especially in large companies, actuaries and modelers may frequently move around roles and departments, making it difficult to get someone’s firsthand experience from a previous modeling project. For this reason, it is extremely important to maintain thorough documentation of past modeling issues and solutions. Accurate documentation allows a modeler to utilize the knowledge of people who may no longer be easily accessible.

Once the problem is clearly defined, a solution should be developed with all available technologies in mind. A good way to do this is to start by identifying what would be considered an ideal solution. Sometimes this can be thought of as what happens in “real life” (as opposed to what happens in a model). Because the disconnect is often that a model cannot perform a function exactly how it occurs in “real life,” recognizing this case gives the modeler a point to start from and a basis to judge against. From here, a modeler can weigh what technology and tool options allow the fewest concessions possible from the ideal solution, while still fitting into company standards.

Embrace the Black Box

When confronted with difficult modeling challenges, it is all too easy to blame the “black box.” Over the past couple decades, so-called black-box actuarial software has grown in popularity among insurance companies, primarily for its ability to control calculations. This kind of control is particularly useful for adhering to internal corporate modeling standards, National Association of Insurance Commissioners model regulations, and audit requirements that developed under Sarbanes-Oxley. While black-box software has become more flexible and transparent, most calculations still remain somewhat closed to the end-user. Many modelers use this as an excuse not to be innovative, when in actuality, black-box software is extremely powerful. The key is to be creative in leveraging its built-in functionalities.

Software is becoming ever more customizable at the request of its users, so users have to understand all the features available to them.

First and foremost, it is important to understand the software inputs and intended calculations. Developer documentation can often provide this information. When it is difficult to understand what the hidden calculation is doing, recreations of the calculation outside the black-box software—in Excel, for example—are always a good place to start. When the calculation is too complex to easily recreate, controlled testing can often give some insight. Such testing would involve changing inputs in isolation and then monitoring the change in output. How output varies compared to expectation will usually give a clue as to why the calculation doesn’t seem to be performing as expected. For example, if changing a key input results in no change in output, the first inclination may be that the input is not actually being used. This result could mean something as simple as an incorrect switch or input placement in the model. When in doubt, contacting the developer can be a quick way to get answers to straightforward questions about software functions.

For more ambitious modeling endeavors, the first step is to have a good understanding of the overall software. Developers are constantly improving modeling software, so users should attend training sessions when available, stay informed of new feature releases, and keep the software up to date with the newest versions. Software is becoming ever more customizable at the request of its users, so it is also important for users to understand all the features available to them. Once modelers have a good grasp of the overall software, the key to innovative modeling solutions is to be creative. The creativity allowed in modeling is limited only to the creativity of the user. However creative the solution, though, it is important to have it fully vetted for validity and to make sure it complies with all applicable modeling standards—especially the actuarial standards of practice (ASOPs) promulgated by the Actuarial Standards Board (ASB). Note that the ASB is drafting a fourth exposure draft of a proposed ASOP on modeling for all practice areas; while not binding guidance, actuaries will wish to review the exposure draft when it is released and will need to stay abreast of developments with this potential new ASOP.

Some common creative solutions in black-box software include process automation and the off-label use of predefined features. Process automation involves combining several small processes in the software and/or outside the software. For example, information from the model can be exported to Excel. From here, an open-source programming language could be used to achieve some manipulation of the output, and then the results can be fed back into the modeling software in a usable form. The trade-off to be cognizant of here is flexibility versus control. As for off-label use of predefined features, the key is to thoroughly understand the feature that will be leveraged. Often features are added to black-box software for very specific purposes, but that does not necessarily mean they are not useful for other purposes. If a feature can be broken out into what basic problem its solution addresses, that solution can usually be adapted to fit other issues with the same root problem or framework.

As a rule, if two solutions achieve the same end, the simpler solution is better than the complex solution.

Design Efficient, Robust, and Ambitious Solutions

Efficient solutions are usually the most desirable solutions. When designing modeling solutions, the largest error a modeler can make (after not clearly defining the problem) is being needlessly complex. As a rule, if two solutions achieve the same end, the simpler solution is better than the complex solution. In the case where more value is derived from a more complex solution, the decision will need to be made if the value added outweighs the additional complexity. When judging the complexity of a solution, a good check is on its level of redundancy. Needlessly complex solutions can often be simplified by removing redundancies—either by redesigning the solution more carefully, or by utilizing other solutions already present in the model.

Robust solutions are

  1. adaptable, meaning they are designed with the anticipation that model design and structure often change for many different reasons;
  2. transparent to the extent that future users of the model will be able to follow the logic and build off the solution; and
  3. verifiable so that proper vetting can be performed on the model.

While ideal solutions are admittedly difficult to achieve, it is important to keep these qualities in mind when negotiating compromises. Some of these characteristics may be considered more valuable than others depending on the purpose of the model.

Ambitious solutions are clever, unique … and may even seem a little crazy. They step outside the box and often develop from pure necessity. They address the most stubborn issues, often when a modeler is running thin on options. These kinds of solutions are usually difficult to create, but there are some key skills a modeler can lean on if the goal is to create a great solution. These include an extremely thorough understanding of the problem, perseverance to work through trials, and patience to spend more time thinking before doing. While ambitious solutions are more likely the exception and not the rule, they are the kinds of solutions that transform a model from sufficient to exceptional.

Identify—and Mitigate—Limitations

The most effective modeling processes are realistic modeling processes. This simply means having a realistic understanding of the limitations that a specific model must be designed around. There are two main types of modeling limitations: those that can potentially be fixed with creative modeling solutions and those that cannot. Those that cannot are limitations like company modeling guidelines, model and assumption regulations, and standards of practice. As mentioned previously, these kinds of factors are important in maintaining the control as well as reliability of the model. With few exceptions—time, for example—almost every other limitation is only so because that is how it gets categorized.

Time as a limitation is worth mentioning individually. Because modeling projects often work through deadlines and deliverables, it is important to consider how long it is anticipated that solutions will take to design, test, implement, and check. In addition, it is important to consider the trade-off between time spent now and time saved in the future. For example, a solution may take a week longer than desired to develop but could save many hours of effort each quarter moving forward.

An effective modeler narrows down limitations so that the final modeling solution is as valuable as possible. Often it is difficult to eliminate limitations entirely, but mitigating them is very doable. For example, if software and time limitations make performing a certain calculation impossible, acceptable approximations can usually be developed. Important factors to consider when filtering out limitations are the materiality and necessity of the solutions they may be preventing. If a solution is necessary, a workaround for the limitation will need to be designed. If the solution addresses a problem that is not very material, the limitation can probably be ignored.

Developing a model that is more than just useful often involves trying something a little different than what has been done in the past—and being prepared to work through a few failed solutions before landing on a success.

While limitations are often viewed as a negative, they should be considered a welcome challenge to any modeling process. Operating around limitations inspires creativity and fosters engagement in a project and encourages collaboration. If limitations are well-identified and -understood, they don’t have to be debilitating.

Be Tenacious

When working through difficult modeling exercises, one of the most valuable attributes a modeler can develop is tenacity. Rarely do truly innovative solutions present themselves immediately or without rigorous testing. In addition, if a solution is innovative, it can take a good amount of proof and convincing to get others comfortable with the solution. For these reasons, it is important to use a methodical approach when designing and testing solutions, and even more important, to adequately document new strategies and procedures.

Good solutions address clearly defined problems, consider all details, and follow a logical order of thinking. Part of being a tenacious modeler is having the ambition to think big and the discipline to test small.

Modeling in an industry as highly regulated as financial services, it is easy to default to doing things the way they have always been done. Tenacious modelers challenge the norm and base solutions on comprehensive problem-solving. They aim to never take results for granted and understand every facet and impact of a proposed change. They draw logical connections between theory and practice and can persuasively defend those relationships.

Finally, being a tenacious modeler is not necessarily about creating the perfect model or solution. All models make concessions and compromises. A useful model is created by deciding its goal or purpose, as well as how accurate and reliable it needs to be. Developing a model that is more than just useful often involves trying something a little different than what has been done in the past—and being prepared to work through a few failed solutions before landing on a success.

Innovative solutions are developed by not being afraid to just try something. Tenacity in actuarial modeling is having the persistence to approach a challenge with the mindset that a solution exists … it is simply a modeler’s job to find it.

 

OLYVIA LEAHY, MAAA, ASA, ACIA, is an actuarial modeler in U.S. life insurance pricing at John Hancock in Boston, Mass.

Print Article

Next article A Long Stroll Through Avoca Cemetery—An actuarial study … and a few meditations and meanderings
Previous article Data Disasters: Learning From History—Case Studies in Analytic Decision-Making

Related posts