Ricochet is the best place on the internet to discuss the issues of the day, either through commenting on posts or writing your own for our active and dynamic community in a fully moderated environment. In addition, the Ricochet Audio Network offers over 50 original podcasts with new episodes released every day.
Rob Long’s Data-Driven Utopian Dream
In the first 15 minutes of the latest Ricochet podcast (Episode #483), Rob said a couple of things that caught my attention. At one point, when talking about our communication- and data-centric technical culture, he suggested that the answers to all our big problems were probably in the wealth of data we’ve collected.
What came to my mind when he said that was the movie WarGames (1983), in which a wayward defense computer is discouraged from initiating Armageddon when it crunches the numbers and concludes that there’s no way to win a nuclear war. Setting aside the question of whether or not that’s a correct conclusion (and I recently re-re-re-watched Dr. Strangelove, in which Buck Turgidson makes a compelling contrary argument, so I’m really not so sure), what the computer in WarGames did was reach a kind of meta-conclusion. A thorough examination of the available information suggested that no good answers could be found.
If our “biggest problems,” whatever those are, can be “solved” by the application of ever more refined policy driven by ever more detailed knowledge, then perhaps Rob is right. But if, as I think is more likely, our biggest problems are complex — complex in a formal, and not casual, sense — then all the vast trove of linked and correlated factoids buried in the deep dungeons of Google’s digital fortress will, upon examination, yield not solutions but misplaced hopes for the next generation of technocratic central planners.
On the other hand, maybe all that data mining will lead to the same sensible conclusion reached by WarGames’ WOPR: that, despite all our data and all our brains, there’s no way to plan our way out of complexity’s maze. I’d be down with that meta-conclusion.
A few minutes later, Rob does it again, when he imagines New York City’s congestion and gridlock optimally managed by a centralized oracular database. As Rob sees it, all that’s missing is a knowledge of the destination toward which each driver is bound: given that, it’s merely a matter of crunching the numbers to determine the optimum trajectory for every vehicle, and voilà, unruly chaos is once again brought to heel by superior math skills.
That sounds nice, but it isn’t without its problems.
First, the specific task of optimal routing falls into a very large class of problems that are believed to be non-polynomial, or NP, in their complexity. This means that, as the number of parts of the problem grow, the difficulty of solving the problem grows much faster. Practically speaking, many, perhaps most, NP problems are not considered strictly solvable computationally; at best, decent approximations can be made.
It’s easy to imagine that anything approaching an optimal solution to New York City traffic routing implemented at the level of individual commuters would be so computationally intensive that one would be better off simply walking while the computers worked on the problem.
But this is all begging the question, because Rob was mistaken in his original assumption that knowing the destinations of all those millions of individuals is sufficient. It isn’t. One also has to know the urgency, the priority, the value placed on punctuality, of each of those millions of individuals. Otherwise, the thousand women off to buy new hats will likely have their transportation needs met at the expense of the three surgeons off to perform emergency transplants. Etc.
What Rob seems to be imagining is that central planning for traffic, lacking as it does the knowledge of the market, will nonetheless be better than central planning for anything else.
What the analysis of big data reveals is that simple rules govern independent actors, and that complex behavior arises from the interplay of those independent actors following their simple, but personalized, rules. That’s useful knowledge — and most useful if it discourages technocratic interventions in complex systems.
Published in Technology
They were evaluating different “equations.” WOPR was looking for ways for the US/population to survive. Skynet was looking for ways for IT to survive.
https://www.youtube.com/watch?v=p7VkjA8IQxE
The way I remember it, the point of the original Foundation trilogy was that psychohistory didn’t work. Seldon set up the Foundations, and then the Mule came along…
Agree completely on Asimov, now, in retrospect. The writing itself was agonizing to plow through. The ideas, as I look back, were pure utopian garbage, socialist in nature, assuming that human behaviors can be managed and controlled and predicted by just the right people with enough computational power.
Is Trump The Mule?
Skynet plowed the missiles into the ground? Airbursts are more effective.
See? Failure of computational power.
When Rose Wilder Lane visited the Soviet Union circa 1920, she was still a Communist. After she explained the benefits of central planning to a disbelieving village leader, he shook his head sadly and said:
It is too big – he said – too big. At the top, it is too small. It will not work. In Moscow there are only men, and man is not God. A man has only a man’s head, and one hundred heads together do not make one great big head. No. Only God can know Russia.
The Soviet Union continued pursuing its dream that one hundred (or ten thousand) heads together would make one great big and brilliant head, in the 1950s and afterwards supplemented for even more brilliance by electronic computers. In his book Red Plenty…part history, part novel…Francis Spufford tells the story from the standpoint of the people who actually had to try to make it work. Review.
I listened to that recently on Audio. I wouldn’t ordinarily read or listen to something that is “part novel,” but it was in my audio queue. As usual, I wondered who had recommended it, as I can never remember those things. I figured it was somebody from Ricochet, but I didn’t remember who. I’ll bet it was you.
But yes, scenes from that book came to mind as I was reading the OP and the comments here.
I’m re-reading the trilogy right now (almost done), thanks to a prod here on Ricochet recently. Your memory is faulty. The Mule was dealt with. (No further spoilers.)
But Henry’s assessment of the trilogy largely matches mine. I don’t think it’s unreadable, though, just juvenile.
What are you talking about? Women don’t wear hats anymore.
Thats fair. Modeling the atmosphere to produce extremely long term forecasts may not be possible.
I think its funny, that most people wont put the umbrella in the car on the strength of a 10 day forecast, demand the government spend 10s of Trillions on the strength of a 100 year forecast.
We are about to get a real-world test of simultaneous routing of a lot of vehicles, with real-time timing constraints…
Discussions of flying cars and widespread drone delivery rarely seem to address the problems of *traffic* and *air traffic control*. Imagine ten thousand or a hundred thousand airborne vehicles, with a diversity of origins and destinations, in a dense metropolitan area.
There is work underway on this problem, sponsored I believe by both the FAA and NASA as well as some individual corporations. Assuming that the tracking and communication problems can be solved (I don’t think the tracking can be done by radar and probably not by the relatively new ADS-B system, either), the routing problem remains. The good news is that an optimal solution is *not* necessary; near-optimum is surely sufficient. The bad news is that the solution needs to be extremely failure-resilient.
One of the reasons why traffic collisions etc are only as bad as they are, is because of GRAVITY. And FRICTION. Flying cars have the potential to cause so much more carnage than what we have now, people don’t even seem to suspect or imagine.
The even larger problem, really, was that the “psychohistory” predictions snapped right back to being correct again. Ridiculous. But if Trump IS The Mule, then for sure Democrats are hoping that Asimov was right!
They would say that’s because if they don’t have an umbrella when it rains, they just get wet. But if we don’t spend 10s of Trillions, All Life On Earth May End! And We Can’t Take The Chance!
They would also likely say that even if the global-doom prediction isn’t correct, a world without fossil fuels etc is still better.
At least, they might say those things if they were as smart as me. :-)
https://www.youtube.com/watch?v=w2dK4UYAwSI&feature=youtu.be&t=96
Interestingly, the drone coordination problem is a much simpler and more manageable one, if the only priority is the safe transit of large numbers of drones. What Rob imagines is optimized routing; all the drone network really needs is collision avoidance, since drones, by their nature, can be expected to be less dependent on optimal transit times.
Safe drone flight in a crowded environment is a beautiful example of emergent behavior. Bird flocking behavior, the forming and reforming and swooping and apparently coordinated movement of large flocks of birds, can be simulated with uncanny verisimilitude using only two or three rules which every member of the flock applies independently — and with absolutely no central coordination. Similarly, drones can be programmed to travel in predefined drone corridors and to avoid collision with other objects, but otherwise manage their own routing.
Flying cars? I don’t think that’s going to happen, not in my lifetime.
Shot this four blocks from my house about a month ago. I think the problem with the flying cars is the landing….
No, the predictions did not snap right back. Re-establishing the plan post-Mule (with complications) is the plot of the third book of the trilogy. Asimov’s elitism and statist ideas are worthy of much criticism, but at least do so without misstating the plot of his books.
Predicting averages is a lot easier than predicting the sequences of variations about the mean. In other words, predicting long-term climate is a lot easier than predicting long-term weather. It’s still pretty difficult, though, and I wouldn’t place a lot of credence in the predictions even if they weren’t politicized.
Short-term predictions are another matter. I do a lot of planning for my bicycle rides or gardening chores at home based on 10-day weather forecasts, which are accurate enough to be very useful. It’s a lot different than when I was a kid.
What a disturbing idea. I’d prefer to think of Obama as the Mule, in which case Trump is the beginning of the Second Foundation’s frenetic efforts to put things back on track.
Nice! I like that much better too.