Probability has been the mathematical language of uncertainty for over 300 years and has shown its usefulness in wide-ranging contexts from gambling to meteorology, economics, and political science. Used properly, it can help us learn from observation and construct better forecasts of what’s to come. But such forecasts are always subject to possibly flawed model assumptions. There will always be room for improvement. As the Danish aphorism goes: “It is difficult to make predictions, especially about the future.” What recent experience has taught us is that the alternative—listening to paid spokespeople and pundits, often biased toward conflict, prone to spin tales about momentum and upsets—is worse.
People respond to incentives, and so if we want to take on much bigger challenges, we need to collaborate across thousands and in some cases hundreds of thousands of people. How do you get 100,000 people to work together? It’s not that easy. In the old days, it was religion and before that it was simple fiat rules, tyranny. The Egyptians built some beautiful pyramids, but they did that with hundreds of thousands of slaves over decades. If we rule out slavery as a possible means of societal advances, there really isn’t any other choice. If we need 100,000 people to cure cancer, to deal with Alzheimer’s, to figure out fusion energy and climate change…I don’t know of any other way to do that other than financial markets: equity, debt, proper financing and proper payout of returns. I think that in many cases [finance] probably is the gating factor. That, to me, is the short answer to the question about why finance is so important.
As the technology advances, we might soon cross some threshold beyond which using AI requires a leap of faith. Sure, we humans can’t always truly explain our thought processes either—but we find ways to intuitively trust and gauge people. Will that also be possible with machines that think and make decisions differently from the way a human would? We’ve never before built machines that operate in ways their creators don’t understand. How well can we expect to communicate—and get along with—intelligent machines that could be unpredictable and inscrutable?
Illustration : Adam Ferriss
The result is that modern machine learning offers a choice among oracles: Would we like to know what will happen with high accuracy, or why something will happen, at the expense of accuracy? The “why” helps us strategize, adapt, and know when our model is about to break. The “what” helps us act appropriately in the immediate future.
It can be a difficult choice to make. But some researchers hope to eliminate the need to choose—to allow us to have our many-layered cake, and understand it, too. Surprisingly, some of the most promising avenues of research treat neural networks as experimental objects—after the fashion of the biological science that inspired them to begin with—rather than analytical, purely mathematical objects.
There is a caveat to all of my modelling work, a small detail that I haven’t yet revealed. It is this. What I haven’t mentioned is that I had a fifth model. It was called “ask my wife”. Lovisa Sumpter is a very talented individual. She is an associate professor of mathematics education in Sweden, where we live, and a qualified yoga instructor. She also has a much better record than her husband in football betting. When she was still a student, Lovisa correctly predicted the outcome of every one of the 13 matches in the Swedish Stryktipset. The chance of getting these right by picking randomly is 1 in 3 to the power of 13 (or 1/1,594,323). Although the pay-out for her winning week was relatively small, she remains proud of being one of the few people in Sweden to “get 13 right”.