How Well Does the Cleveland Fed’s Systemic Risk Indicator Predict Stress?

An important tool for monitoring financial stability is a stress index, a constructed measure that indicates whether the banking or financial sectors are “under stress.” A number of such indexes were developed after the financial crisis of 2007–2009, and their primary purpose is to alert the attention of policymakers to conditions that may warrant taking actions. How well these indexes perform can be evaluated along several dimensions. An analogy is a sophisticated burglar alarm system: A good system is reliable in that it will give a clear signal when attention is needed. It is timely in that it will give enough advance notice so that corrective action can be taken. It is straightforward in that the alarm buzzes to get the observers’ attention and then activates other systems to assist observers in making decisions about what actions to take. A good system is also ongoing in that it continues to function during a break-in and will turn off once the threat has disappeared (while retaining a record of when the attack occurred). Finally, the alarm gives valid signals in that it rarely gives a warning of an attack if an attack has not happened.

This Commentary provides a retrospective, narrative analysis of one such index, the Cleveland Fed’s systemic risk indicator (SRI), as a tool for monitoring stress in the financial sector. My approach is to examine the SRI’s daily index values from 2001 to 2020, a period that predates and includes the financial crisis, to see how well it signaled times of known stress along the dimensions above. I find that the index is reliable, timely, and valid. It signaled each of the stressful periods that occurred over the period while signaling only five “false positives,” times when the index predicted stress that did not materialize. Furthermore, those false signals lasted only one day, with one exception that was two days, after which the index went back to normal. Finally, it provided good ongoing information during the financial crisis.

Definition of the Indicator

The SRI is centered on the distance to default (DD), a measure developed by Merton (1974) for firms such as banks that are highly leveraged.¹ This measure is based on a subtle difference between what we measure as value in terms of the price of a bank’s stock (the “value of its equity”) and the value of the bank’s total assets (the “value of its assets”). These differ because equity holders get a return on their assets only after the loans are paid off. If the bank cannot manage to pay off its loans, then it is in default and the equity holders get nothing. Because of this, the value of equity is lower than the value of the assets of the bank.

In Merton’s stylized model, a “default barrier” (D) is defined for each firm as the particular value of the firm’s assets in a future time, T, below which the firm will no longer be able to service its debt in full, the firm will be able to pay its equity holders nothing, and the firm will have to use its full assets to pay off its lenders. D is defined by Merton (as well as much of the subsequent literature that uses distance to default) as short-term debt plus one-half of long-term debt, a number that can be computed from bank balance sheets. Distance to default (DD) defines the number of standard deviations that the firm’s value is from D in the context of a stylized model. The distance to default of a single bank (DDi) is computed from the value of the bank’s assets, the value of its equity, volatility values for the bank’s assets and equities (indicators of uncertainty about the future value of those assets and equities), and yields on US Treasuries.

It is important to note that the DD should not be taken too literally as a measure of the probability of default. It is an indicator rather than an exact measure of the default probability for reasons that include the fact that the default barrier is somewhat arbitrary and does not include regulatory capital measures that might shut a bank down. In spite of this, the DD measure that we define is commonly used in risk analysis, portfolio analysis, and formal regulatory discussions of the rating of institutions. It is a concept that is quite familiar within the financial profession.

The SRI builds upon Merton’s DD by calculating the DD for a portfolio of banks, and for each of the individual banks that make up that portfolio. The SRI first calculates the DD for the portfolio of banks using the KBE index, which is an index traded on the Nasdaq exchange. It currently includes 84 banks, each of which is weighted so that the KBE index represents an average price for the banking sector. Because the index is heavily traded, options are sold based upon it, which allows one to calculate the volatility of the portfolio from the options, and from this a “portfolio distance to default” (PDD). The individual DDi’s for all included banks are then averaged with the same set of weights that is used for the KBE index to form an average distance to default (ADD). The SRI is formed by comparing these two numbers.

While individually the quantities PDD and ADD might seem to be good stress indicators, they do not perform exceptionally well as stress indicators on their own. They both embody leverage, balance-sheet strength, and volatility, but these are not sufficient to define triggers that show when regulators should pay attention to stress in the financial sector as a whole. PDD and ADD do marginally better than other indices, such as the average value of banking stocks, in indicating systemic financial stress.

The PDD’s and ADD’s strengths and weaknesses as systemic indicators on their own are due in part to the factors that drive each bank’s DD. A major driver of the levels of each bank’s DD on a day-to-day basis is the value of the bank as measured through the joint equation that calculates the bank’s asset value from its equity price and the bank’s asset volatility. These levels change with the equity prices, and to some extent, the volatility of the bank equity market. As a measure of financial stress, the levels of the averages of these DDi’s share, to some extent, the same disadvantage that a composite indicator of a financial equity index and a volatility index of financial equities would have. Namely, their indication of financial stress would often be driven by factors affecting the economy at large, especially in their false positives.

However, the relationship between the PDD and the ADD does better at capturing common shocks that affect the entire industry. The SRI calculates this relationship as the difference between the two (ADD − PDD). This measure shares some of the good points of the ADD and PDD in predicting stress. If both the ADD and PDD are reduced by a common stress factor, then their difference (ADD − PDD) will be closer to zero. However, in many of the periods of financial stress examined here, including events surrounding the Bear–Stearns failure, the difference was more timely in predicting the seriousness of the event. Precisely why this is so is the subject of ongoing research. As defined, the SRI is usually a positive number and decreases with stress.

How Well Has the SRI Performed in the Past?

A look at the historical values of the SRI in figure 1 indicates that during periods of intense stress, the level of the index falls below 0.1 and then rises to 0.5 and stays at this level for some period of time. So I choose a trigger value of 0.1 to examine how well the index performs in the almost 20 years for which we have data. The way that I assess this record is by examining how well the SRI has anticipated known periods of stress since February 2001 considering two questions: Were there any false positives during this period, and how soon, as a stressful period was developing, did the SRI alarm go off? I argue that the SRI with a trigger point of 0.1 has an excellent track record as an indicator of financial-sector stress for the entire period.

This exploration is made somewhat difficult by the fact that defining periods of financial stress is subjective.² For the purposes of this Commentary, I define them as periods that have been named in the financial literature by pundits, academics, and financial columnists. In some cases, the name simply is a label of the assumed cause of the financial stress.

I have identified 15 subperiods of stress, listed in table 1. These are also shown in a timeline in figure 1 along with the SRI. Because the SRI is a daily measure and possible crisis events unfold slowly, several of the stress periods are listed multiple times in figure 1, with the dates of various subcrisis events during the crisis denoted. In all, there are nine separate events.

Table 2 shows the behavior of the indicator for the same time frame. Its track record is remarkably good. Not only does it catch the major episodes of financial stress, it also anticipates them with a remarkable foresight. The Sarbanes–Oxley Act is anticipated on the day of the first real discussion leading up to it, six months before its passage. Although it was widely anticipated to give the financial sector trouble with its complication of capital requirements during a time of slow growth, in the end the difficulty for the financial sector did not pan out. However, the SRI detected this stress on the day that all the high-ranking officials of the White House and Federal Reserve Chair Alan Greenspan agreed that the Enron scandal necessitated a thorough accounting reform bill.

The Bear–Stearns problem is seen by the index as a major systemic problem well in advance of the time at which the financial press thought that it would be a problem. Also of note is the Lehman bankruptcy, which not only was associated with an index value indicating a highly elevated level of stress but was also presciently ahead of the curve in that the index shrank to negative levels on the Friday before the rest of the markets were roiled by the Fed’s announcement that there would be no bailout. Further, the index also seems to learn from past experience. Increasing globalization caused Brexit and the Chinese selloff to send the index lower. However, these two experiences seemed to “teach” the index that financial-sector stability was less exposed to global events, so that the Italian debt crisis did not affect it nearly as much.

I counted five false-positive events, which the SRI cleared up quickly: Four of them lasted one day on the index’s radar, and one lasted two days. In each case, I could find some reason for the false positive. On three of the days, a large positive outcome in the equity markets gave a common shock that was picked up by the stress index. In two of the cases, all that I could find on that date was a regulatory announcement that might have caused the stress, along with higher-than-average volatility of bank equity prices. In each of these cases, neither the common shock nor the news was enough to sustain the index’s indication of high stress for more than two days. At five, the number of false positives over a 20-year time span is small.

The overall rule from the historical experience of the index is this: When the difference between ADD and PDD is less than 0.1 for more than a couple of days, it has always been verified to represent an underlying stress that represented a risk to the stability of the banking sector. The periods of large stress, such as happened with the financial crisis of 2007–2009, are characterized by long periods during which the indicator seldom goes above 0.5. In this sense, the indicator stays “on” while the financial sector is still stressed.

The lack of false positives in this indicator can be compared with the performance of simpler indices that might be used to show financial stress. For example, a component of the SRI, bank equity volatility, could be an indicator of stress, showing as it does, uncertainty about the future value of banks. However, whatever reasonable threshold value of the volatility that one chooses, banks go through many sustained periods of high volatility that represent very little stress to the financial system, but rather uncertainty about future earnings. This can occur when the structure of the system is sound, and indeed, can be the result of banks feeling confident enough in the system that they underwrite more profitable yet riskier projects. The SRI distinguishes between these periods and actual stress periods quite well, so that these common shocks to the system that increase profitability will register in the SRI after only a day, but then the index returns to normal levels. However, when the SRI has been sustained below a level of 0.1, it has always indicated stress, and it has signaled that stress early.

Conclusion

A good financial-stress indicator is reliable, timely, straightforward, valid, and ongoing. The SRI has demonstrated most of these qualities. For reliability and timeliness, we see it has provided an early signal of every period of financial stress throughout the past 20 years, and in the largest crisis during the period it indicated a major problem even before the Bear–Stearns bailout and continued in high alarm mode throughout the financial crisis. The SRI is straightforward to interpret in that any time the indicator reaches 0.1 and stays lower than 0.5 for more than a few days, it indicates that the financial sector is seriously stressed. The index provides an ongoing signal of a stress period, remaining at a depressed level throughout such periods. It is valid in that it has a small number of false alarms in its history, and in all of these cases, the false alarm went away after a day or two.

The reason that the SRI works well is that it combines measures of balance-sheet strength, volatility, and correlation of the asset values of the major banks with the forward-looking characteristics of option prices. A possible disadvantage of the indicator is that it is terribly complex, so that it is sometimes difficult to disentangle why it works in the way it does during a period of financial stress. Further, in its current form, it does not single out which banks are responsible for a crisis (although this is in principle possible). The fact that the indicator raises the alarm when balance sheets are weak and banks are highly leveraged, along with the fact that information hitting the financial sector is both highly volatile and highly correlated, means that false positives are minimized. This minimization is balanced by the fact that the information is acquired from option values that are forward-looking and thus the indicator incorporates the latest information, so that when a financial crisis is real, it trips the SRI warning early.

The SRI is one of many possible indicators that could be used. Future research will study how well the index performs compared to these other methods and where it outperforms or underperforms them. For the time being, however, there is much that recommends the SRI as an early indicator of financial stress.