Back to the Future: Understanding the Benefits of ALM Backtesting

Backtesting Your Interest Rate Risk Model

Throughout 2023, regulatory bodies have advised community financial institutions to undertake more extensive interest rate risk (IRR) management procedures. One important aspect which is often misunderstood in the process is backtesting. While the term can be defined as evaluating a strategy, theory, or model by applying it to historical data, it is interpreted on a broader basis regarding asset/liability management. Below are some basic procedures to implement when reviewing your institution’s ALM model to ensure it is producing meaningful results.

Backtesting Procedures

Model Validation: Today, all model vendors provide a third-party validation letter. The letter is usually found within the reports themselves or is available upon request. The model validation not only increases the reliability and confidence in the model but can also illustrate the significant strengths and drawbacks for users. Steps should be taken periodically to confirm that the letter is on file and a new report is obtained whenever new versions are released. Although this certification eliminates concerns for the calculations and structure of the model itself, the inputs and assumptions used are much more important factors in the model outputs and backtesting results.

Input Review: When working with any type of model, the adage goes “garbage in, garbage out.” This pertains to interest rate risk models, as they are only as good as the data which is fed into them. Input review involves the review and inspection of all imported or entered data. If the model your institution utilizes performs a file import from the processor for loans, non-maturing, and time deposits, it is necessary to verify these inputs are accurate on a consistent basis. Some of the questions to ask when reviewing the input data include:

• Do the imported balances match what is presented on the core system?
• Are time deposits separated by term or jumbo status?
• Are loan floors, caps, and margins being captured properly?
• Are the yields or rates being imported the same as what the processor reports?
• Is the model producing dynamic cash flows for the loans and bonds such as mortgage-backed securities?

Assumption Review: All models rely on a set of starting points or default assumptions. Many times, financial institutions use these defaults not knowing if they are applicable to their institution or not. The assumption review process involves checking all assumptions used by the model and confirming they are appropriate and reasonable. Shift sensitivities (betas), time lags, asset pre-payment speeds, and non-maturing demand decay rates are all items that should be reviewed at minimum on an annual basis. The assumptions within the model play an important role when assessing interest rate risk exposure and should be monitored closely.

• An example of assumption review would be comparing historic treasury and federal funds rates against the financial institution’s core deposit rates (NOW, money market, savings) to test rate sensitivities.
• On the asset side, comparing the loan prepayments speeds (CPRs) being used in the model against what has occurred historically can be helpful when adjusting those speeds for various rate scenarios.

Comparing Projected Interest Income & Expense: Once the model methodology, inputs, and assumptions have been verified, the next step is comparing the model projections to what happened given the specified review period (most models use 12 to 24 months). This process is what is typically thought of when discussing backtesting. Because unknown future events will inevitably affect the financial institution’s balance sheet, efforts to normalize these unplanned events in the model are necessary. For instance, if the institution’s asset size rose significantly during the review period, it would be inappropriate to compare the interest income unless you were able subtract that portion of interest income coming from the additional earning assets. Because many externalities can occur at the financial institution over a 12-month horizon, this process can be considered difficult to perform when there are drastic changes at the bank.

• Is the interest rate simulation being reviewed similar vs actual rate changes?
• Was there growth/shrinkage to the balance sheet?
• Did the balance sheet mix change (ex. reduction of loans, increase in securities)?
• Did the institution undertake any deposit strategies such as CD specials?

Backtesting is a critical component of asset/liability management as it often results in adjustments or changes to the inputs and assumptions to improve the process going forward. Keep in mind that models are attempts to recreate real-life situations with many variables to account for, especially ALM models. Because of this, your IRR reporting will inevitably have some degree of variance from what occurs. Prudent risk managers know this, and that being able to identify those variances and adjusting the model for better performance is crucial for successful asset/liability management.