AI at 70: Born in the USA

Mostly, it was. Although Alan Turing and other WWII-era scientists would play a role, there’s wide agreement that AI didn’t really get off the ground until a trio of Americans organized an academic conference in Dartmouth, New Hampshire, in the summer of 1956. That’s when the new field got its name: artificial intelligence, a term coined by the conference organizer, John McCarthy.

Whatever McCarthy would later do as a scientist, he’d proven himself to be one of the best advertising slogan writers of all time: Artificial Intelligence as a concept was easy to grasp, and the term sparked the public imagination. It caught on instantly.

British historian Jerry Kaplan has an example of why the comparison to human intelligence and consciousness has been a thorny one, sometimes limiting the imagination. Suppose that from the beginning, airplanes were called and thought of as “artificial birds”? Would we say they don’t really fly because they don’t flap their wings?

Dartmouth ’56, present at the creation: John McCarthy, Marvin Minsky, and Claude Shannon—for AI, they were like the 1927 Yankees, MGM in 1939, or The Traveling Wilburys; a rare collection of diverse talents. Jerry Kaplan got to meet John McCarthy decades later, who he describes as giving off the distracted, mad genius air of Dr. Emmet Brown in Back to the Future.

Why was Claude Shannon such a big deal? His 1937 MIT doctoral thesis on the use of electrical relays showed how simple on-off switching could form logic gates capable of performing useful results. When MIT published the paper in 1940, Bell System and IBM took notice. Shannon’s paper was based on the then-already century-old Boolean algebra. Its critical contribution was binary arithmetic, soon to be nearly the universal computer standard. Shannon’s main contribution to science was pure logic, not electrical switching, important as it was to his evolving grasp of computer science.

Cybernetics, which we vaguely use as describing the whole field of computing, once had a narrower meaning in the late 1940s and early 50s: the study of how nerve signals and involuntary reflexes built into biological systems formed an important, if poorly understood, role in the evolution of human consciousness, and that at least some of this instinctive learning could be modeled with artificial means. Clearly, they were on to something.

Maybe surprisingly, the cybernetics movement, led by Norbert Wiener, wasn’t wedded to the wave of the immediate future, von Neumann-style digital computers that processed data in sequential order. Cyberneticists started with analog computers and stayed with them too long. Eventually, conventional off-the-shelf digital computers could handle the kinds of programs beloved by the original postwar generation of bio-based cybernetics.

But some of their ideas, and more of their contrarian spirit, were absorbed by the connectionists, the neural net guys, the next generation of rebels. They grew up, so to speak, in the shadow of their wildly profitable sibling, IBM’s mainframe data processing.

Creating artificial intelligence was like reaching a state of ideal communism: it was assumed to be presently unattainable, but always out there somewhere in the distance if you wrote enough lines of code to define it. In the Fifties and maybe for half the Sixties, two main camps of AI were peaceful neighbors: the “Neats” who believed in structured rules and the “Scruffies” who pursued neural networks.

In AI’s early years, rules-based “toy” worlds and projects often looked impressive to the general public. By 1959, off-the-shelf computers could be programmed to play checkers. It got plenty of coverage.

Margaret Boden has published on the subject since 1977. She often referred to GOFAI, Good Old Fashioned AI, to summarize the basics of rules-oriented work, and one key early concept is heuristics. Aside from being the H in HAL 9000, heuristics narrow a search by ignoring what is likely (but not guaranteed) to be less relevant. Heuristics can be dismissed as programming tricks, but it amounts to focusing attention.

GOFAI uses frames, which Boden explains simply as data hierarchies. So, for example, “a person in a room” is subdefined. “Person” includes sets like “head,” subdivided into “eyes” and “ears” and so on. “Room” has a floor and a ceiling and four walls and windows. Of course, it’s possible to subdivide every object in the room to such detail that it takes a long time to process any change, any movement in that basic framework. So, especially because they were dealing with the slower, less detailed computers of sixty years ago, they had to simplify.

It’s tough to program in common sense, because there’s so much of it and we seldom need to describe it explicitly. Humans know that objects have weight and will fall to the floor. They know that some materials break when dropped. Humans instinctively know that particularly heavy objects can be dangerous. And if you “tell” a computer that, it asks what about, say, helium balloons?

Or take another inconsistency. When one object is in front of another, your view of it is partly blocked. Video game programs “know” this, and don’t bother calculating the look of parts of the image you won’t see. It saves time and system resources. But what if the object in the foreground is made of glass? The more exceptions, the more chances that some choices will be wrong, sometimes erratically, not showing up at first.

Neural nets were a more radical experimental approach. In essence, teach very little and let the machine figure it out. It wasn’t based on how normal digital computers processed data. Instead, it was modeled on idealized and simplified human brain neurons and how they behave. Their artificial equivalents were called perceptrons.

The first perceptron-based computer had an array of 400 photocells, 20 x 20. That extremely crude “eye” trained an equal number of neural units, adjusting them until the human operator agreed that the system had read the pattern correctly. At first, they were physical circuit boards. Later, they were virtual. The idea worked, although the extremely limited number of training inputs and the slow computing speeds of the era tended to mute its marginal successes.

Gradually, as all of AI began to suffer from a regrettably well-deserved history of overpromising and underdelivering, the Scruffy brothers and sisters of neural nets absorbed (and probably deserved) blame from the others for being especially premature, overhyped and promotional.

The connectionists, as the neural net people sometimes called themselves, were all but left for dead (rhetorically) after 1969, when Seymour Papert and Marvin Minsky published their debunking book, Perceptrons. For the first time, I looked into why: they conclusively proved that a single-layer perceptron cannot perform the XOR logical function. It’s more significant than it sounds.

Then and later, well through the end of the century, the whole field was prone to periodic “AI winters” when funding, publication, and careers dried up in response to a lack of real progress. The “Neats”—the OG rule-based logical programmers—made slow progress. Margaret Boden describes some of the stirrings in the Eighties as (my metaphor, not hers) like the Sorcerer’s Apprentice scene in Fantasia, the shards of the shattered brooms of the connectionists began to twitch back to life, unite, and revive.

The next time I return to this subject, British researchers take the spotlight. Once Geoffrey Hinton’s 2006 paper explained how multi-level, deep learning was possible, the Scruffy faction seized the steering wheel. They still need the well-defined, predictable Neats for all sorts of things. But deep learning is a neural technology. Hinton won the Nobel Prize. So, later, would the founder of the UK’s DeepMind, Demis Hassabis.

A real-life simple challenge for visual AI over the decades was a consistent, taunting failure to recognize cat faces, or the differences between cats and dogs. Examples of partly successful efforts were chronicled in Daniel Crevier’s 1994 AI: The History of the Search for Artificial Intelligence, but success rates rose very slowly until the “deep learning era” began to show public results in around 2010-12. Then the new techniques ran the scores up dramatically and almost instantly. They’ve only gotten better and faster.