AI and Human Revelation

Growing up, we were forbidden from taking IQ tests. The reason is historical. My father was given an IQ test when he was in elementary school. It was decided that he was ‘mentally retarded’, to use the parlance of the day. He scored 68, one less than Forrest Gump. Years later, after facing substantial discriminatory headwinds, he received his PhD from Columbia University in Operations Research. Not many doctoral students in Columbia’s 269-year history would have done quite as badly on the IQ as he did. The test was thus seen, within the family, as deeply flawed.

My mother, whose own Columbia PhD was in the History of the Philosophy of Science, had a more conventional intelligence than my father. Where my father had an intuitive resentment of the test, my mother had a rational objection: the IQ test only measures what can be measured – and the most important forms of intelligence are immeasurable.

While IQ tests may be statistically valid, they fail when it comes to the individual.

All of this history is relevant today because of the onslaught of AI. ChatGPT scores a 155 on the verbal IQ test – putting it in the top 0.1% of humanity. However, while many have been wowed by ChatGPT and its brethren, I have been far from impressed. These systems are wonderful recombinative, pattern-based systems. Their ability to form sentences, pictures etc… is stunning as is their ability to quickly analyze massive data sets. However, as anybody who has used them for a while can attest, they are also pretty stupid. In their way, AI systems are the inverse of my father. They score high on the tests, but their intuitive abilities are actually quite limited.

As a result, amid the rush to AI adoption, I’ve been something of a skeptic. I’ve used AI image generation, but I’ve also learned to limit my expectations. When it comes to ChatGPT, I mainly use it just to see what it will say – not to produce any sort of product. For most of my writing, I find I’m far faster than ChatGPT – at least when it comes to saying something that is actually interesting.

Precisely because of its limitations, AI has done a fantastic job of revealing the limits of the quantitative analysis of intelligence. This sort of quantitative analysis has been all the rage lately, with EQ (emotional quotient), SQ (social quotient) and AQ (adversity quotient) joining IQ in the pantheon of human quantification. Psychometric tests abound to demonstrate each of these quotients and try to define how people will perform. In fact, a significant majority of major companies require psychometric testing for new hires.

Statistically speaking, the benefits of psychometrics are clear. For example, the retention of new hires can be dramatically improved. If you are staffing an army or a major corporation, these tests can make sense. The work of these organizations is on a statistical scale.

But two kinds of human work are fundamentally not statistical. Innovation and adaptation. Innovation is delivered by those who break the existing, statistically predictable, reality. Adaptation, on the other hand, is an adjustment of existing systems in order to deal with unpredictable changes in the environment (business or otherwise). Put another way, innovation is self-driven change while adaptation is externally imposed.

Large companies, built almost like giant human clockworks, often struggle with both innovation and adaptation. When the world around them inevitably shifts, cracks appear in the foundations of their operations. Openings are created for newcomers to rise up and redefine the reality they operate in. When it comes to adaptation, much of it can be handled by standard approaches – and there are change managers, project managers and business analysts who deliver fantastic results. Sometimes, though, more fundamentally outside-the-framework thinking is required. The people who do this work are often hard to find and have a hard time being found. Hiring systems, whether AI-based or more conventional, will inherently have a hard time identifying this sort of individual. Even once they are identified, there is a fairly good chance they won't rock their psychometric evaluations.

If you see somebody who just doesn't fit - but still somehow has a track record of success - take a flier on them.

It might just work out.