COVID-19 has every bank thinking about how to modify its reserves. Some of the largest institutions have taken double-digit hits on that basis. Community banks, understandably, are uncertain about what exactly they should be doing to prepare their balance sheets for the upcoming environment.
Naturally, many bank CEOs and CFOs will turn to their incumbent ALLL models to help understand the magnitude of credit risk that they may be facing. Others will look to their stress testing process for guidance. While the instinct to utilize these resources to navigate the choppy seas of COVID-19 is a good one, we fear that most community bankers will emerge from that process with little more than a queasy stomach.
The problem is that far too many ALLL, CECL and stress testing models rely on statistical modeling techniques that require extensive domain expertise to work properly. Don’t get me wrong – sophisticated statistical models can work in this environment, but without an in-depth, air-tight grip on how they work, they simply won’t.
Regression models are an excellent example of this problem. If you run a regression model for your ALLL (or ACL) or stress test, I’d be willing to bet that you’ve already put the unemployment rate in the high teens or low twenties. I’d also be willing to bet that when you got the result, your jaw dropped. Don’t worry – your model is probably wrong.
I once had a professor explain that regressions were like hammers – a tool for a specific kind of job. But when you first discover what a hammer can do, everything starts to look suspiciously like a nail. COVID-19 looks like the biggest nail since 2008, and we’re all trying to bash it with our regression hammer. The results, naturally, don’t make sense. Why?
Because regression models were trained on the last downturn, and the next downturn is always different enough to throw those models out of whack. Without extensive exploration and modification (which community banks don’t have the time for at the moment) existing models are simply unable to properly adapt to the fact that the COVID-19 recession started in a different corner of the economy, will have a far deeper trough, may differ in duration (V-shape vs U-shape vs L-shape), and has a far different regulatory, monetary, and fiscal policy framework associated with it.
Statisticians and data scientists will argue that regression models can, perhaps, be calibrated to capture the error that we’re currently seeing. The reality is that most community banks don’t retain a statistician or data scientist on staff to attempt to adjust these models – and that leads to “Regression Depression.”
What’s the solution? Unfortunately, there is no magic bullet here. It really comes down to three choices (from worst to best – if you ask us):
- Continue to operate a regression-based model and utilize the results, despite the issues listed above. This is the worst option. A model that you don’t trust isn’t worth the hard-drive space it’s saved on. Forget being strategic – you can’t even check the box.
- Recruit external support. While not the worst option, it can certainly get expensive. Many practitioners and vendors of regression-based models will be more than happy to take your money in exchange for explanations only they will understand.
- Revisit your model selection. While this may take a bit of time, and the current crisis is immediate, it is important that management teams have faith in the model they’ve selected – even in times of extreme uncertainty.
It is important to stick to common-sense analytical techniques that don’t require a PhD to understand. Black box models that are inflexible are unacceptable. Yes, models should be data-driven with a strong analytical foundation. However, these models should also be flexible—and bankers should be able to easily understand and unpack them. Remember, the primary goal is not precision, but instead to provide direction, a trend line, and quickly drill down to the critical questions that need to be asked.
I have a lot of empathy for banks in this situation. They have spent a lot of money and time preparing these models to get ready for CECL. However, recognition that these models are failing in the current environment should not be viewed as a negative. It is part of the learning process.
Learning from failure is always the best way to learn, as long as you apply those lessons by making changes.
And for many banks, that means replacing your regression model with a new data-driven model that doesn’t require mastery of statistics and can be understood and easily used for the current environment.
Guy LeBlanc is the Director of Client Analytics at Invictus, which provides fully customized stress testing, ALLL, and CECL solutions to community banks.
