This article was originally post on StrexGroup.com. Click here for the original article or keep reading below:

With the bank stress-testing season just around the corner, it is worth taking a look at what stress testing is and, perhaps more importantly, what it is not.

Stress testing is a way to evaluate the stability of a system/structure under extreme abnormal circumstances. Simulated stress testing requires applying adverse assumptions to a model of the system/structure and observing the impact on variables critical for its stability.

In the banking industry, this means evaluating the simulated impact of adverse market and economic conditions on individual financial institution’s liquidity and capital positions. This exercise is useful not only for contingency planning, but also for thinking through key operating and regulatory issues.

No bank can afford poor grades on regulatory stress tests, also known as Comprehensive Capital Analysis and Review (CCAR) and Dodd-Frank Act Stress Testing (DFAST), because of the negative perception as well as the regulatory restrictions that may accompany low grades.

Even though the objectives of CCAR and DFAST are to assess the ability of large banks and financial companies to continue operations through adverse conditions, stress testing can also create a false sense of security because of what it is not.

Many internal and external factors impact a bank’s liquidity and capital positions at any moment. Despite advances in computer simulations, human limitations restrict testing to only a small fraction of thousands, if not millions, of possible combinations of these factors. Therefore, no simulation can envision all possible stressful scenarios, and thus prepare for “unexpected and highly consequential” events, also known as black-swan events.

And therein lies the problem. Nobel laureate physicist Richard Feynman, who played a key role in identifying the catastrophic failure of O-rings as the primary cause of the shuttle Challenger disaster, once said: “It is not what we know, but what we do not know which we must address, to avoid major failures, catastrophes and panics.”

No one can predict a black-swan event. But that does not keep people from trying and having their predictions guide preparations for extreme adversity. Nassim Nicholas Taleb, former derivatives trader, a respected voice on black swans and distinguished professor of risk engineering at New York University says that “through some mental bias, people think in hindsight that they ‘sort of’ considered the possibility of such event; this gives them confidence in continuing to formulate predictions. But our tools for forecasting and risk measurement cannot begin to capture black swans. Indeed, our faith in these tools makes it more likely that we will continue to take dangerous, uninformed risks”

Therefore, passing stress tests, involving what during normal times may seem like some extreme assumptions, can lull organizations into believing it as a sign of strength to survive extreme crises or black-swan events. This is not too different from the pre-2008 confidence that a reasonable level of VaR in relation to average daily revenues meant everything was hunky dory. Everyone knows how that story turned out.

Stress testing is an important component of uncertainty management, but it leaves out the part most responsible for causing devastation from unexpected events. Here’s why.

Managing uncertainty deals with two items: possible outcomes and probabilities of outcomes. So the spectrum of uncertainty can be divided into 4 situations. First, known outcomes with known certainty (100% probability), or “Known-Knowns,” have no uncertainty and thus need no addressing. Second, situations where specific outcomes are unknown, but their probabilities are known, or “Unknown-Knowns,” can be managed effectively through traditional risk management because expected values can be defined and addressed. Third, situations where outcomes can be defined and are thus known, but their probabilities are unknown, or “Known-Unknowns,” can be addressed through capital management or stress testing of what is known. Fourth, situations where outcomes can’t be envisioned and are thus unknown, and obviously their probabilities are also unknown, or “Unknown-Unknowns,” pose a difficult problem. These Unknown-Unknowns are what black swan events are all about. There is no way to know in advance how such situations may unfold.

This is not just a theoretical concept. The crash of 2008 was a black-swan event where even on the morning of September 15, 2008 – the day Lehman’s bankruptcy triggered events leading to the crash – it was hard to envision what laid ahead in the next few hours, days or months.

To manage uncertainty effectively, all 4 situations must be addressed simultaneously. So what should be done to deal with “Unknown-Unknowns” that can be so devastating?

The problem is that all current initiatives, including stress testing, to manage uncertainty and risk require knowing either outcomes or probabilities, and therefore are “event-centric.” Solutions can be planned if one can define a problem event. This means that the “knowns” can be addressed either through controls to reduce probabilities, priced into transactions to cover their adverse impact or cushioned through a robust capital structure. This enables the first of the 3 situations to be addressed effectively.

However, black-swan events can’t be envisioned. Therefore, an event-centric approach can’t work and a completely different approach is needed. According to Prof Taleb, instead of trying harder to predict black swans, “We should try to create institutions that won’t fall apart when we encounter black swans.”

The way to deal with black swans is by taking an approach that, instead of being event-centric, is “damage-centric,” regardless of the event that causes the damage. Focusing on maximum damage from the risk arising from uncertainty, or extreme-tail risk, can ensure that institutions won’t fall apart in extreme crises. Unfortunately, despite all the sophistication of risk management, today there is no metric for extreme-tail risk and is urgently needed. Just like precise measures, such as earnings, profitability and volatility, extreme-tail risk must have a measure to gauge and manage it.

The insurance industry’s commonly used concept of a “probable maximum loss” model can be an effective measure for extreme-risk management. It can provide a simple, but powerful, objective and continuous measure of an institution’s strength or fragility in relation to extreme risk. Bear Stearns and Lehman had not just too much extreme-tail risk. They also didn’t know how close to the edge of the precipice they were operating.

So stress testing is a useful exercise, but its focus has become about capital adequacy, which because of its subjective and only-known assumptions can offer no assurance of survival in the next extreme crisis. According to William Coen, Secretary General of the Basel Committee on Banking Supervision, “The answer isn’t simply to say you need another X basis points in capital. Maybe the better response is to require the bank to present a detailed plan on how it’s going to better manage these risks and to make sure the bank sticks to that plan.”

A significant part of that plan must include a way to deal with the full spectrum of uncertainty and risk, including extreme risk and black swans to ensure that the institution can continue operations throughout times of economic and financial stress.

About the Author

Karamjeet Paul, Managing Principal of Strategic Exposure Group, has developed expertise in extreme tail risk. His commentaries have been widely published. He is the author of Managing Extreme Financial Risk: Strategies and Tactics for Going available for purchase on the Elsevier Store. Use discount code “STC3014” to save 30%!

he has over 30 years of operating, finance, treasury, and exposure managment experience, giving him unique expertise in identifying and addressing critical risk-exposure-reward and other strategic issues at the highest levels in large organizations. Mr. Paul’s perspective has been gleaned from hands-on experience in large global organizations, startups in entrepreneurial settings, and consulting assignments.