People respond to incentives, and so if we want to take on much bigger challenges, we need to collaborate across thousands and in some cases hundreds of thousands of people. How do you get 100,000 people to work together? It’s not that easy. In the old days, it was religion and before that it was simple fiat rules, tyranny. The Egyptians built some beautiful pyramids, but they did that with hundreds of thousands of slaves over decades. If we rule out slavery as a possible means of societal advances, there really isn’t any other choice. If we need 100,000 people to cure cancer, to deal with Alzheimer’s, to figure out fusion energy and climate change…I don’t know of any other way to do that other than financial markets: equity, debt, proper financing and proper payout of returns. I think that in many cases [finance] probably is the gating factor. That, to me, is the short answer to the question about why finance is so important.
As the technology advances, we might soon cross some threshold beyond which using AI requires a leap of faith. Sure, we humans can’t always truly explain our thought processes either—but we find ways to intuitively trust and gauge people. Will that also be possible with machines that think and make decisions differently from the way a human would? We’ve never before built machines that operate in ways their creators don’t understand. How well can we expect to communicate—and get along with—intelligent machines that could be unpredictable and inscrutable?
Illustration : Adam Ferriss
The result is that modern machine learning offers a choice among oracles: Would we like to know what will happen with high accuracy, or why something will happen, at the expense of accuracy? The “why” helps us strategize, adapt, and know when our model is about to break. The “what” helps us act appropriately in the immediate future.
It can be a difficult choice to make. But some researchers hope to eliminate the need to choose—to allow us to have our many-layered cake, and understand it, too. Surprisingly, some of the most promising avenues of research treat neural networks as experimental objects—after the fashion of the biological science that inspired them to begin with—rather than analytical, purely mathematical objects.
There is a caveat to all of my modelling work, a small detail that I haven’t yet revealed. It is this. What I haven’t mentioned is that I had a fifth model. It was called “ask my wife”. Lovisa Sumpter is a very talented individual. She is an associate professor of mathematics education in Sweden, where we live, and a qualified yoga instructor. She also has a much better record than her husband in football betting. When she was still a student, Lovisa correctly predicted the outcome of every one of the 13 matches in the Swedish Stryktipset. The chance of getting these right by picking randomly is 1 in 3 to the power of 13 (or 1/1,594,323). Although the pay-out for her winning week was relatively small, she remains proud of being one of the few people in Sweden to “get 13 right”.
A simplified version of the problem goes like this: Imagine that you are imprisoned in a tunnel that opens out onto a precipice two paces to your left, and a pit of vipers two paces to your right. To torment you, your evil captor forces you to take a series of steps to the left and right. You need to devise a series that will allow you to avoid the hazards — if you take a step to the right, for example, you’ll want your second step to be to the left, to avoid falling off the cliff. You might try alternating right and left steps, but here’s the catch: You have to list your planned steps ahead of time, and your captor might have you take every second step on your list (starting at the second step), or every third step (starting at the third), or some other skip-counting sequence. Is there a list of steps that will keep you alive, no matter what sequence your captor chooses?
At a dinner I attended some years ago, the distinguished differential geometer Eugenio Calabi volunteered to me his tongue-in-cheek distinction between pure and applied mathematicians. A pure mathematician, when stuck on the problem under study, often decides to narrow the problem further and so avoid the obstruction. An applied mathematician interprets being stuck as an indication that it is time to learn more mathematics and find better tools.
I have always loved this point of view; it explains how applied mathematicians will always need to make use of the new concepts and structures that are constantly being developed in more foundational mathematics. This is particularly evident today in the ongoing effort to understand “big data” — data sets that are too large or complex to be understood using traditional data-processing techniques.
Our current mathematical understanding of many techniques that are central to the ongoing big-data revolution is inadequate, at best. Consider the simplest case, that of supervised learning, which has been used by companies such as Google, Facebook and Apple to create voice- or image-recognition technologies with a near-human level of accuracy. These systems start with a massive corpus of training samples — millions or billions of images or voice recordings — which are used to train a deep neural network to spot statistical regularities. As in other areas of machine learning, the hope is that computers can churn through enough data to “learn” the task: Instead of being programmed with the detailed steps necessary for the decision process, the computers follow algorithms that gradually lead them to focus on the relevant patterns.
The Gaussian copula is not an economic model, but it has been similarly misused and is similarly demonised. In broad terms, the Gaussian copula is a formula to map the approximate correlation between two variables. In the financial world it was used to express the relationship between two assets in a simple form. This was foolish. Even the relationship between debt and equity changes with the market conditions. Often it has a negative correlation, but other times it can be positive.
That does not mean it was useless. The Gaussian copula provided a convienent way to describe a relationship that held under particular conditions. But it was fed data that reflected a period when housing prices were not correlated to the extent that they turned out to be when the housing bubble popped. You can have the most complicated and complete model in the world to explain asset correlation, but if you calibrate it assuming housing prices won’t fall on a national level, the model cannot hedge you against that happening.