Reading Benjamin Recht's The Irrational Decision

I learned a lot from this book. When I picked it up, I thought it would focus on Artificial Intelligence and how it is going to impact the way we live - that’s the topic of the day, after all. But it doesn’t do that. Instead, it gives a history for the layperson of the development of computing since the mid-twentieth century, through different threads, with the overarching theme of the sorts of questions people thought computers would be able to solve better than humans could. AI comes into it, but only as far as it represents the current state of what has been a long path. He doesn’t dwell on metaphysical questions - what is intelligence? what is consciousness? - but instead looks at the practical question of when high-powered computer tools will be useful, and when they will not be able to compete with the power of the human brain. It is neither pro- nor anti-technology; it is about when certain technologies are useful, and when they could lead us astray.

Computing is about prediction. One chapter is devoted to games, and how many developments in computing were tied to people trying to figure out how to make computers good at chess, checkers, go, poker, or even rock-paper-scissors. Games involve prediction because the player is trying to look ahead: “if I do this, how likely is it to affect my chances of winning?” Because games can become so complex - the number of possible paths of a game of chess is massive - the problem has to be broken down into smaller problems, looking ahead just a couple of moves, or it quickly becomes unwieldy. An interesting side effect of this line of research is that over time humans became better players, learning from computer strategies.

Another chapter is devoted to research using randomized clinical trials. These are the “gold standard” in medical, natural science, and, where possible, social science research, and they too are about prediction: based on an analysis of a RCT, what do we think will be the likely future effect of a prescribed policy (e.g. take this drug for this condition, or use this tax credit to get people to change their behavior). When reading this chapter, interesting as it was, I was left unsure about the fit: where is Recht going with all this? But in the end it comes together in his central question: when are the predictive abilities of computing helpful, and when not?

Computing is helpful when the parameters of the analysis are clearly delineated. Consider his rule for when machine learning will be good at pattern recognition:

Does conventional wisdom say there should be a classification rule that is stable over time?

Can we not write a computer program that does it?

Can we collect a lot of data?

Are the stakes low if you make an error?

Do you have access to a ton of computing power?

So, for example, the use of machine learning to predict the structure of a protein from its amino acid sequence was a really good idea - it answers “yes” to all these questions. But notice how contained that problem is. AI that generates text, or responds to your prompts, takes us into something very uncontained. And so the text it generates is flattened, can often be wrong, and is tied to trying to satisfy the prompt.

When is human judgment superior? When, say, should a data-driven (to use the buzz word in policy) recommendation be over-ridden by a human? Recht uses the example of driving a car. Since the 1950s, computer scientists have predicted that a world of self-driving cars is just around the corner. But we are still not at that corner, and what self-driving cars exist are constrained to a few specific spaces, and remain very expensive. It’s because humans have the capacity to be good drivers, good in the sense of having a super rapid understanding of what is going on in the road, and what other drivers are doing and are likely to do, and of surprising hazards (in my neighborhood, if I spot a deer in someone’s front yard, I know that the animal, which is very stupid, could easily just decide to jump on to the road in front of my car - I have to slow down and assess this likelihood). We are great at processing a lot of different sorts of information in a split second. When I taught my children to drive, they were naturally panicked at first: there are so many things to be thinking about at once! But they get the hang of it. That we can drive around cities in these weighty and powerful machines in a coordinated way without constant crashes is a marvel of civilization, really. But our brains are good at this, and it is really hard to get a computer to be this good at it.

I can think of other examples. One is management. A lot of AI-pushers talk about how firms can become much more efficient, and yes, of course there are computer tools that have made business more efficient - spread sheets, word processing, etc. But the applications have to be for specific and limited tasks. I have a former colleague who is trying to use AI to figure out how to schedule class sizes and rooms for future semesters, and, sure, it’s probably going to be helpful at that linear programming problem. But when I worked in management, a huge part of my job was dealing with the uniqueness of individuals and how to encourage them, get them to work as a good team, resolve disputes, sense when something is going wrong, when somebody just doesn’t seem to be themselves these days. When I taught management I would tell my students that whenever they started a new job, they should quickly try to assess the culture of the place, and get to know the people as people and not just as job classifications. All of this is not something any AI program is going to help with, since AI works as a predictive model (what words are most likely to be the correct ones given this prompt?) that is based on averages of what is out there in text, not on your actual colleagues.

Another is the arts. There have been to date over 600,000 articles¹ on how AI can write novels and make music that fool people into thinking it is the real thing. But fakes are fakes, genuine art requires a human with something they want to express, and criticism and evaluation of art cannot be done through the predictive capabilities of machine learning. And yet, in my field of cultural policy, there are researchers, all with good motives, who look for ways to quantify the unquantifiable, so that there can be an analysis of what cities are most arts “vibrant”, or what arts presenters are most “efficient”.

I recently was part of an online thread (in which I followed Thumper’s rule) on what are the best Key Performance Indicators for works of public art. And the point of this would be to guide future decisions on public art. And I don’t think KPI’s would lead to better decisions - they could very well lead to worse decisions - as to the selection of works of art and their setting.

And this gets to the subtitle of Recht’s book: “How we gave computers the power to choose for us.” It was the mistaken idea that with enough data analysis we could optimize all of our personal decisions and our public policies. But we can’t. Developments in computer science have led to some very good things - it’s great that they can quickly analyze proteins. But the human brain, with its capacity to understand the other humans it encounters, their art, or even just whether they want you to go first at a four-way stop sign, is a wondrous thing, that we should not be hell-bent on replacing.

1. A personal estimate.