1.1 FROM ARTIFICIAL INTELLIGENCE TO RISK MODELLING
We are engineers specialized in artificial intelligence. We have been working together for about 25 years, and early in our careers we spent a lot of time on applications of neural networks, Bayesian networks, and what was called data mining at this time. It was almost a generation before the current popularity of these techniques.
Back in the 1990s, few industries had the means to invest in artificial intelligence and data mining: mostly banking, finance, and the defense sector, as they had identified applications with important stakes. The defense usually conducts its own research, and so, quite naturally, we spent a lot of time working in research and development with banks and insurance companies, for applications such as credit rating, forecasting financial markets, and portfolio allocation. We were fortunate enough that the French Central Bank was our first client for this service, for several years. Thanks to a visionary managing director, the French Central Bank created in the early 1990s an AI team of more than 20 people working on applications ranging from natural language processing to credit scoring.
Our conclusion was mixed. Machine-learning techniques were generally not better than conventional linear techniques. This mediocre performance was not related to the techniques themselves, but to the data. When you try to predict the default of a company from its financial ratios, you will always have several companies with exactly the same profile, but that will not share the same destiny. This is because the observed data do not include all of the variables that could help predict the future. The talent and the pugnacity of the leader, the competitive environment, and so on, are not directly represented in the accounting or financial data. However, these nonfinancial indicators are the ones that will make the difference, all things being equal otherwise. Finally, in rating or classification applications, and whatever the technique used, the rates of false positives or false negatives were usually very close.
This is even more applicable when you are trying to predict the markets. We were most of the time trying to forecast the return of one particular market at various horizons, using either macroeconomic variables, or technical variables. We would have been largely satisfied with a performance just slightly better than flipping a coin. Again, the performance of nonlinear models was comparable to other techniques. In a slightly more subtle way here, the limitation was expressed through the dilemma between the complexity of the model and the stability of its performances: to get a model with stable performance, this model must be simple. The best compromise is often the linear model.
About 10 years ago, a Head of Operational Risk for a large bank asked us to think about the use of the Bayesian networks to model the operational risk, and to seek to evaluate possible extreme events. Not surprisingly, she was advised to do so by the former managing director of the French Central Bank, which we mentioned previously.
We were immediately intrigued and interested in the subject. We liked the challenge of leaving aside for a while the “big data” analysis to work on models based on “scarce data”! We thought, and continue to think, that the work of a modeler is not to look for mathematical laws to represent data, but to understand the underlying mechanisms and to gain knowledge about them. It is not surprising that one of us wrote his PhD thesis on the translation of a trained neural network into an intelligible set of rules.
Going back to operational risks, or more precisely to one of the requirements of AMA (advanced measurement approach), the problem was formulated mathematically quite simply, but seemed to require an enormous work.
The mathematical problem was to estimate an amount M such that it could only be exceeded with a probability of 0.1%, regardless of the combination of operational risk events that could be observed in the forthcoming year.
In practical terms, this meant answering several questions, all of them more difficult than the other:
- Identification. What are the major events that my institution could be exposed to next year? How to identify them? How to structure them? How to keep only those that are extreme but realistic (that is, how not to quantify a Jurassic Park scenario!).
- Evaluation. How to evaluate the probability that one of them will occur? If it occurs, how to evaluate the variability of its consequences?
- Interdependencies. All adverse events will not happen at the same time. However, certain events can weaken a business and make other extreme events more likely. For example, a significant natural event can weaken control capabilities and increase the risk of fraud. How to evaluate the correlations between these events?
Once we became acquainted to the problem, we did two things:
- As consultants, we studied closely the risk management system of the bank.
- As researchers, we studied the state of the art on the question of quantification.
We must admit that if we were impressed by the work done by our client, this was not the case on the state of the art.
This customer, which is one of the largest French banks, serves today nearly 30 million customers with more than 70,000 employees, and covers most of the banking business lines, even if it does not compare in th...
