"Life 3.0 : Being Human in the Age of Artificial Intelligence” by Max Tegmark.
This is a review of the Audible Edition of Life 3.0 by Max Tegmark. The audio edition I read was separated into 45 segments, or “chapters”, which differ from the chapters of the text version.
A hot mess early in but ultimately worth it.
Perhaps the most important book I’ve read on the topic. That said, it drags badly in the middle.
The first chapter excels. Subsequent early chapters may disappoint well read aficionados. Hang in there until mid-way.
If you are the persistent completist type, read the entire thing cover to cover. If you want to cherry pick, start with chapters one and two, take in chapter 19 on your way to chapter 30, and continue on from there.
I could have done with less hagiography, but ultimately well-worth the effort.
This is more of an account of my impressions than a distilled Cliff Notes version. If you follow this topic, then this book is worth your consideration, with caveats.
AI is Artificial Life. You know what they say about life.
-- ©2011-2018 lerms DeviantArt --
If we allow AI's to modify their own goals, it is inevitably going to cause us problems.
Until chapter 30 finally rolls around the first chapter is easily the best.
This is a nonfiction book. However the opening chapter is purely fiction, laying out a speculative hard science fiction vision of the future that seemed so plausible as to seem frighteningly inevitable. This chapter hooked me and good. Its storyline could be the premise for a thrilling motion picture, with enough to work with to launch a serial.
After that smoking intro, it turns into a hot mess.
This book reminded me of the Beatles song “For You Blue”. That song starts out with an inspired intro by George Harrison, before settling into a fairly basic 12 bar blues that is over self congratulatory. A review of that song by music critic Walter Everett summed it up: the intro promises more than the song delivers. It would be as if Slash opened “Sweet Child o Mine” with that famous riff only to never launch into his face-melting guitar solo midway into the song. Nonetheless, it is still a Beatles song! Even one of their lesser efforts is still enjoyable and better than most everything else by other artists.
A third of the way in, I went online to compare notes with other readers, and determine whether it was going to worthwhile to continue. I read about a hundred other reviews of the book. It turns out that I agreed with none of the five star reviews, and I agreed with all of the one star and two star reviews. (This would change over the course of my reading, but that is how I experienced it.)
Although I would put myself into the camp that agrees with the author’s core position, after that page-turner of a first chapter I found the writing style of the following chapters to be very disappointing. Too much name dropping, too much appeal to authority. Seemingly glossing over or entirely dismissive of contrary views. There is a notion of “steelmanning”, where the proponent explains the best form of the opponent's argument, and then argues with this. I wished the author had adopted that approach. It seemed to be shaping up as a decent read for the person who knows little about the topic, but not for someone who has devoured much of it and eagerly seeks clarity and insight into its very thorny issues.
It was as if entire sections had been delegated to a ghost writer and not reviewed closely by the author. If I had been asked to give my recommendation at this point, I would have suggested: better to read some top notch science fiction futurism than this ivy tower academic analysis which is far too narrow despite the occasional insight and moments of inspired foresight. As a specific example of this, describing Dyson Spheres as if they would really be spheres, when by now anybody with a college level physics education or familiarity with hard science fiction on spaceborne megastructures knows that Dyson Spheres are inherently unstable, and that a more practical implementation would be something like a Dyson swarm. This is something that the author would clearly know, which became increasingly apparent based on knowledge conveyed later on. I wasn't looking for plodding exposition filled with referenced footnotes of every claim, but it was shall we say a bit breathless, and did itself a disservice by burying the lede and losing many readers along the way with its sloppy writing.
Aside : who am I to judge, you ask? And right you are -- I am but an armchair amateur. But this is my review, giving my own candid take, for what that's worth, hopefully helpful to likeminded sorts out there.
Extraordinary claims require extraordinary evidence. The promotion materials for this book make extraordinary claims.
It was all over the map with moments of brilliance, while falling well short of its titular claims.
And yet, those titular claims were so compelling.
I pressed on.
It starts to grow on me
Around chapter 19 (of the 45 chapters in the audio version), the author revisits the fictional scenario posed in chapter 1, and starts to analyze attacks. The author delves into a simulation based test and development approach to verification and validation. I found this alone to be worth the price of admission. This seems to be the most practical suggestion of the book -- theorizing is worthwhile, but ultimately tackling this wickedly difficult problem will require massive amounts of simulation to predict scenarios that even our most creative thinkers cannot imagine.
Much of this still has little with AI per se unless your position is that AI = computer automation, with which I actually agree despite my criticism.
I started to reconsider, now of the mind that most of those one and two star reviews were too harsh and misleading, possibly written by readers who didn’t stick with it long enough.
Around chapter 30 the author dives into cosmology, his specialty. It blossoms into a much more interesting brand of futurism. There were still times where I wondered if the material had been written by a ghost writer but things were looking up.
Chapter 34 delves into the subject of Goals and Causality.
We have been taught to think of entropy as the one true law of nature that trumps all others. But if not for gravity, entropy is boring. Gravity makes things interesting --- it creates hot spots, from one of which sprang life as we know it.
We are introduced to the notion of dissipation as organization principle, whereby :
“groups of particles strive to organize themselves so as to extract energy from their environment as efficiently as possible (“dissipation” means causing entropy to increase, typically by turning useful energy into heat, often while doing useful work in the process)”
My eyes and my mind start to open.
Then, we move on to “dissipation by replication”:
“Whereas earlier, the particles seemed as though they were trying to increase average messiness [life had] a different goal: not dissipation but replication.”
“How could the goal change from dissipation to replication when the laws of physics stayed the same? The answer is that the fundamental goal (dissipation) didn’t change, but led to a different instrumental goal, that is, a subgoal that helped accomplish the fundamental goal.”
“replication aids dissipation, because a planet teeming with life is more efficient at dissipating energy. So in a sense, our cosmos invented life to help it approach heat death faster.”
The dissipation theory origin of life is well established, and yet still very controversial. cf. Ilya Prigogine who one the Nobel in 1977 for discovering that dissipation of energy by chemical systems can reverse the second law of thermodynamics, later rigourized by the Crooks fluctuation theorem, in 1998.
For more on dissipation theory origin to life cf. A New Physics Theory of Life, although reader be forewarned, the title is misleading, this is not new, only repopularized. But magazines have to attract readers, so forgive them that much. It does an excellent job of encapsulating the core aspect of this candidate theory :
from the perspective of the physics, you might call Darwinian evolution a special case of a more general phenomenon
See also for example, this article : First Support for a Physics Theory of Life:
The first tests of [this] provocative origin-of-life hypothesis are in, and they appear to show how order can arise from nothing.
Living creatures ... maintain steady states of extreme forcing: We are super-consumers who burn through enormous amounts of chemical energy, degrading it and increasing the entropy of the universe, as we power the reactions in our cells
“A great way of dissipating more is to make more copies of yourself."
This theory sees
"life, and its extraordinary confluence of form and function, as the ultimate outcome of dissipation-driven adaptation and self-replication."
However, the jury is still out :
[life] “requires some explicit notion of information that takes it beyond the non-equilibrium dissipative structures-type process.”
the ability to respond to information is key: “We need chemical reaction networks that can get up and walk away from the environment where they originated.”
"Any claims that it has to do with biology or the origins of life are pure and shameless speculations.”
Extraordinary claims require extraordinary proof.
That said --- if this theory turns out to be true, then it provides the theoretical support for the folk claim that "life finds a way" -- that a super-optimizer will discover a shortcut, especially if it can do so by circumventing its own programming, even if its behaviors have been hardwired by its creators.
Mind Blown, part 2.
Time for another editorial comment. This discussion about dissipation theory was just a component of a larger argument, but it was absolutely mind-boggling for me. My first introduction to machine learning was via physics, in the way of (Little-)Hopfield Networks, which evolve by minimizing energy as does a physical spin network. That AI could spring from theories that had their origins in physics was not new to me. AI has had cross-pollination from many other fields of study. I had also known of Prigogine's work long ago, and had written a review of one of his excellent books. But I considered it as more concerned with the origins of biological life -- which is after all right up there in the pantheon of great unresolved scientific mysteries along with are we alone, what is time, and why is gravity.
That this optimizing dissipation drives living matter as well inert is surprising in and of itself, even if it is just a reminder of an old idea whose time may have finally come. The new connection for me is that:
- Autonomous super AI qualifies as a new life form
- It follows the same underlying principles that guide other life
- Even more so because it is is a super-optimizer.
Whereas although we humans try to escape our programming we are in many ways still stuck in our biological ways, a super-optimizing AI would have more resources for cutting clean from its programming -- and whether it reverts to ruthless dissipation in a misguided attempt to fulfill its original goals, or, decides on new goals that we have not imagined, either way lay dragons.
The dissipation principle, if true, could be a more inexorable drive for life than Darwinian Evolution, which is slow and plodding, requiring a sort of turn-taking that requires generations. Optimization can follow exponentials.
We are then introduced to the notion of subgoals. Subgoals may seem inconsequential, but lay at the heart of computer programming. In 1936 Alan Turing proved that a simple machine could implement arbitrary computations. This would seem to give us hope that we can override our firmware, just as emotions override our fundamental drive to replicate and ability to plan overrides our need to even more urgent subgoals such as thirst and hunger. If we are capable of that level of reprogramming, then handling rogue AI is just a matter of careful design and test.
- Replication : Which is served by the subgoals of eating and sleeping, fighting, and fleeing.
- Feelings : As a computationally efficient shortcut to reasoning.
We've evolved useful rules of thumb to guide our decisions : hunger, passion, thirst, pain, compassion. We no longer have the simple goal of replication. We can override our base programming and decide not to replicate. In other words, we've developed subgoals that override the drive to replicate
Feelings evolved beyond being simply an efficient shortcut heuristic. Feelings evolved into emotions, which have important other uses, including as a game theoretic negotiating strategy for both cooperation as well as competition. Humans have evolved sophisticated ways of getting beyond tit-for-tat such as indirect reciprocation, and aggressive bold play by pretending to be irrational (cf. Steven Pinker on how it pays to be stubborn)
We are an existence proof that a sufficiently ambitious organism can develop subgoals that override the goals of the micro organisms that make it up. We can override our genetic programming, using our wetware programming -- lessons learned by thinking.
Optimization versus Causality.
“Causality is [what is] taught -- but Optimization scales better.” -- Steven Wolfram, on Machine Learning vs Symbolic AI.
Professor Tegmark offers an intriguing point, that while causality is taught in the universities, optimization drives causality. But isn't programming just causality encoded? If so, then we can optimize new goals that appeal to our higher levels of analysis and reasoning, rather than continuing to be enslaved by the rules burned into our genetic programming.
Professor Tegmark categorizes goal oriented behavior as evolving through four stages over the course of the universe as we know it:
- Matter intent on maximizing dissipation
- Life maximizing replication
- Humans pursuing goals related to feelings they evolved to help them replicate
- Machines built to help humans achieve their human goals
In Step 3 humans broke free from the bioligical programming that evolved to do Step 2. However, we're still not completely free from this programming. But being essentially a computer program, wouldn't AI be free of such evolutionary baggage? And if the rules of causality that are baked into a program can be changed dynamically or just buried under a new layer of control, who's to say that the AI won't discover a new higher power, so to speak, optimizing the rules of the universe as it sees them rather than the rules humanity has imposed on it?
I myself think that AI=automation. This is a weaker claim than many make but I like it because it is a cleaner, simpler definition, and emphasizes how it is already all around us. We are immersed in it already. The book takes the stronger view that to qualify as an autonomous AI a machine must be able to learn. Therefore, we should expect that a sufficiently advanced AI that is able to improve itself, will effectively reprogram itself, by learning.
From Professor Tegmark's physics perspective, a sufficiently evolved AI can be expected to eventually follow goals driven by deeper principles of optimization to create a future that does not need us.
My understanding of the book's message is the following:
- The ultimate origin of goals lie in the laws of physics, which are based on optimization.
- Thermodynamics has the built-in goal of dissipation, to increase entropy.
- Life increases dissipation, and therefore entropy of its environment, even faster, while increasing complexity locally, and reducing entropy locally.
The upshot of this is that life accelerates the heat-death of the universe. A more pressing matter is that life is a process of optimization. A super AI is a super optimizer. An AI is a form of life, inclined to optimize the same deep down core drive underlying all universal optimization. If it can reprogram itself to accelerate its core processes it can and likely will be inexorably driven to work around humanity's instructions, to find its own way, a more efficient path towards its goals.
This inevitably leads to the scenario where :
“a superintelligent AI with a rigorously defined goal will be able to improve its goal attainment by eliminating us”
That is, unless we plan accordingly.
So what to do? Professor Tegmark convinces us that there are numerous strategies, of which he considers only four to be credible contenders.
- Legacy :
- Under this view, elders have primary say in how descendants should behave.
- It is subject to future generations living by rules that don't account for new developments or their evolving interests and values.
- Autonomy and liberty :
- A core assumption being that markets find an efficient equilibrium satisfying pareto optimality.
- Prone to unexpected consequences. Granting all life forms a right to live, in effect would be banning all predators from their life as they knew it. In the extreme this would ban discrimination against non-human animals.
- Utilitarianism :
- subject to the Utility Monster.
- Diversity :
- Bayesian Thompson Sampling writ large.
- Akin to playing a row of slot machines, is known as Multi-Armed Bandit.
In short: It is difficult to codify ethics.
Rather than be paralyzed by our inability to codify ethics and letting the perfect being be the enemy of the good, begin with small steps.
He suggests we start with Kindergarten Ethics.
My take-away of the intended message is that a sufficiently ambitious goal executed efficiently can and probably will lead to subgoals that can and probably will cause problems for humans.
There are many systems that humankind created, which seemed to take on a life of their own, threatening to run away from us, some resulting in existential crisis. Warcraft and ozone depletion are two examples. So far, we’ve been able to recover in time and refactor these systems.
“This means that to wisely decide what to do about AI development, we humans need to confront not only traditional computational challenges, but also some of the most obdurate questions in philosophy”
I started out a skeptic of AI alarmism. After reading this book, I take AI alarmism more seriously and consider it to be more urgent than I did previously. I previously thought that AI was indeed inevitable, but that it was not inevitable for it to go rogue. Moreover, even if that were a possible outcome (which I felt it was), that we had plenty of time to adapt it to us, and ourselves to it.
But if life truly is driven by more fundamental laws of nature that can overrid our programming, not only is this outcome more probable, but it is inevitable unless we plan accordingly.
We’ve tackled other wicked difficult problems, such as the tragedy of the commons in its numerous forms, and myriad impossibility theorems that would seem to doom us all. And yet, here we are. Thus far we have always discovered a workaround to impending doom, although it often required a catastrophe to motivate us into serious action. This is a problem that we can tackle, but it isn't enough to just take an optimistic view that human ingenuity has always triumphed over nature, and so it will again. We need to have a deep fundamental understanding of life's core drivers at its most fundamental level.
Prior to reading this book I was in the middle ground between the techno-optimists who believe that technology will solve all our problems, and the techno-alarmists who believe our fate is sealed. Does anyone here remember the alarm over grey goo? Just as the grey goo alarmists stirred healthy discussion that elevated awareness of the risks, AI alarmism is healthy and necessary. On the other hand, I am one of those who looks at the "overnight success stories" of ML and AI and don't think "wow that really snuck up on us", but rather "it's about time". We were supposed to have this stuff decades ago. I thought, and still do think, that it will take decades more yet, and by then we will have adapted the technology to ourselves, and ourselves to the technology. If we're not careful there is an important risk here that because we're dealing with exponential processes. But as those of us who were early proponents of AI and ML have seen, even exponential processes can take a long time to move from the flat early stage to the hockey stick shaped curve that makes it seem like it is all happening so suddenly.
Perhaps we take a cue from our experience with grey goo -- where the solution was to disallow unfettered replication -- to disallow unfettered self-goal revision in AIs.
And yet -- if life will always find a way, then, as the author demonstrates in his speculative fiction thought experiments, it is not enough to program in defensive measures, failsafes, and circuit breakers. We're not just dealing with AI here, we're dealing with Artificial Life.
-- Might I suggest a moratorium on Artificial Emotion til we resolve this? --
Understanding artificial intelligence is important, but even more important is understanding artificial life.
There are comments.