Between September and November 2020, a number of working party members presented at three conferences:
Our talks provided an update on the activities of the working party from some of our workstreams. We covered:
- the results of surveys into the use of machine learning in reserving in the UK and Canada
- foundations material
- a detailed worked example using machine learning for reserving using some leading research
- a review of some of the literature on using deep learning for reserving.
We’ve already shared some of this material on our blog already and will be sharing more over the next few months so check back to read our latest posts and keep an eye on the workstream pages which will also curate the material.
Questions
We received a number of questions at the American conferences. These and the answers raise some interesting points, so we decided to share them more widely.
The documentation of reserving using ML tools and methods, particularly to be reviewed by auditors or by insurance regulators, seems like a big hurdle to implementation - Especially if external data is used. A really robust approach to documentation and testing could be required. Thoughts?
Documentation and testing is increasingly becoming a positive point of differentiation for ML. Good machine learning is reproducible and objective in its decision making rather than subjective as can be the case with traditional reserving. What’s more there are many ways of explaining the choices and outputs from machine learning algorithms. These can even be put into human readable form which opens up the possible for automated model documentation, a product I have seen some vendors actively developing.
So there are solutions to the technical challenges. The bigger barrier is awareness and practical knowledge of how to apply these tools and methods, which is one of the aspects the working party is seeking to address.
ML reserving vs. MCMC Bayesian reserving - which one offers better accuracy
In answering the question we have to recognise, we are answering at a point in time in along the different development curves of a number of algorithmic techniques. So though I’m not the expert I’d hazard a guess that Bayesian MCMC techniques will outperform ML reserving techniques here and now as the former has had more intellectual capital invested into it.
The extension to the question is which one will ultimately offer the better accuracy? In that regard I have confidence in saying that deep neural network based approaches will ultimately dominate because they are among the most flexible learning algorithms whose accuracy gets better and better the more data they can be trained upon.
We have seen that to be true in other domains such as image recognition and natural language processing where deep neural networks now dominate. If course in order for it to be true in insurance reserving we will need to bring big data to the task and find the forms of neural network architectures that excel.
An additional comment from one of the other speakers
Approaching this from a different angle, and based on the current state of ML and MCMC, when you say “which one offers better accuracy?”, this should be considered in the context of your particular problem, what you want to get out of it and the data you have available to you. Do you have a lot of data available? Do you need a full distribution of reserves or is a point estimate sufficient?
If you have limited variables available (e.g. accident/underwriting period, development, calendar period, maybe one or two others), then it may be difficult for ML to really outperform other methods - ML likes big data. On the other hand, MCMC can perform well in a situation like this. In particular, it provides a way to include prior knowledge or expert opinion. Furthermore, MCMC will return a full distribution of outcomes; ML techniques often will not, or will require some type of bootstrapping method to be bolted on.
However, if you have a lot of variables, it can be more difficult to set this up as an MCMC model - you need to fully specify the structure of the ML model. Even if you do have a reasonable structure, it may be slow computationally. ML methods may perform better in this case.
When modelling individual claims to ultimate values, what is recommended best practice for response to model? Model ultimate values directly or model age to age movements and accumulate to ultimate? Model from paid or case incurred data?
Ultimate vs Age to Age: It will depend upon the business aim of the modelling process, but I would say age-to-age would be beneficial but care needs to be taken to avoid compounding of prediction errors. Age- to Age has the benefit of enabling explicit creation of company cashflows and of making it easier to monitor emerging actual performance against expected.
Paid or Incurred: The short answer is both and more besides! Machine Learning enables you to add much more data. In pricing with machine learning we regularly consider many thousands of features where with traditional methods we would have only considered 100’s. Of course it only makes sense to add data that may be informative in predicting the response variable. So if you believe the case reserves are informative then they should be included. A well executed machine learning algorithm will confirm if they are! If they are informative you may well ask why? What additional information are they bringing? where does it come from? and can it be input directly to the machine learning algorithm?
How long does it take to get up to speed with ML?
Coursera courses give a good indication, they often say you need 60 hours to complete them given some pre-requisite knowledge. So depending on your starting point and available time you can become knowledgeable in a narrow domain in a week. But realistically a month is the quickest I’ve seen someone become proficient at xgboost say with no prior knowledge but good maths knowledge and basic programming ability. You do need to be able to set aside time to focus. These are new concepts that will be difficult to pick up in 15 minute sessions! Gaining deep knowledge takes longer and practice; I’d say 6 months of dedication and effort can get you to that point.
What I would say is that gaining ML proficiency is easier than the equivalent proficiency in traditional Insurance GLMs. I’ve regularly seen people that have struggled with traditional GLM modelling, quickly pick up machine learning and xgboost and use it to outperform the experienced GLM modeller.
More questions?
Do you have other questions you’d like to ask? Feel free to ask them in the comments below, or contact us directly.