3. Incorporating expert judgement
Given machine learning algorithms are parameterized on historical data, how can shocks or changes in the claims environment, or additional external information, including expert judgement, be incorporated?
Overview
This is a common concern, though you could also argue that this is already a problem with any model, including the chain ladder, that uses historical data to predict the future.
It can be helpful to think of ML as one of the tools available to an actuary, a skill set to develop, rather than a solution in itself. Different approaches and different ML techniques and framing will be useful for different problems, or for saving time on different parts of the reserving process. ML can also be used to provide additional insights into the data that are not possible with current techniques.
ML can in fact allow additional data to be incorporated as described below. It may also allow you to separate out and understand specific features, eg to quantify and identify an historic shock say, enabling you to apply better informed assumptions going forward.
The use of ML in reserving is still in its early days and there is a lot of work to do to bring the process to maturity. However, we would argue that this is not a reason for not starting to apply ML now. Members of the working party give examples of how you can build confidence in the system and start to incorporate ML in this article from the March 2022 edition of The Actuary.
What is judgement anyway?
It’s worth taking a step back and contemplating what is judgement and how has it been acquired.
We might think of judgement as something built up over years of experience. It’s typically an instinctive response rather than a quantitative calculation. It’s something we acquire through experience, through seeing what works and what doesn’t. But, just because we can’t simply or quantitatively explain our judgements, does that mean a machine learning algorithm can’t outperform an experienced human in predictive accuracy?
Experience from other domains such as radiography and cancer detection evidences that machines can now outperform skilled experts in complex predictive tasks.
Perhaps there is a neat explanation as to what human judgement is and why machines can approach and exceed human accuracy in predictive tasks. We might think of the judgment we have built up as equivalent to slowly training a neural network, the one in our heads. Years of observing what works and what doesn’t has built and reinforced that neural network that we call judgement. Just like the machine learnt neural networks our judgement is a bit of a black box that we don’t really know why it’s right but we know it often is. However if pushed we could give a rationale for my judgement informed decisions, they are not random guesses!
So, given there’s an underlying rationale to evidence our judgements, machines should be able to learn this too … provided they have access to enough of the relevant experience and use a suitable learning algorithm.
Explicit use of external information
If there is external information that is relevant then it can be explicitly included into the model building process either
- as additional explanatory features in the training data; or
- as an offset feature that the model starts from when building towards the target outcome.
Offsets can also be a good way of incorporating judgements (that can be quantified) into the model build. Another way to incorporate judgement would be as the prior in a Bayesian approach, there are some Bayesian machine learning approaches but it is accepted that they are less prevalent.
The time dimension
Since reserving is an extrapolation exercise, accurately reflecting the time dimension often requires a lot of judgement. Using a ML algorithm that can better handle the time dimension than traditional techniques may lead to greater insights and reduce the need for judgemental interventions.
Neural network based sequence models, LSTMS, GRU’s and most recently Transformers are very effective at modelling the sort of sequential data we often see in reserving problems and we are exploring this further. For example Kevin Kuo utilised LSTM in his 2020 paper Individual claims forecasting with Bayesian Mixed Density networks.We are also interested in investigating different structures of models, outside of the traditional triangles that actuaries habitually use.
Real world considerations
You will always need a real world sense check on any model. Communication of model issues and uncertainties is an important part of an actuary’s work over and above the modelling itself. Scenarios can be particularly helpful here. They are a useful communication tool and can aid discussion and decisions on assumptions which are often better made with input from others who have more oversight or particular expertise than the actuary.