Book Review

"Life 3.0 : Being Human in the Age of Artificial Intelligence” by Max Tegmark.

This is a review of the Audible Edition of Life 3.0 by Max Tegmark. The audio edition I read was separated into 45 segments, or “chapters”, which differ from the chapters of the text version.

Tl;dr

Busts out of the gate, only to decline into a hot mess early in but ultimately well worthwhile.

Summary

Perhaps the most important book I’ve read on the topic. That said, it drags badly in the middle.

The first chapter excels. Subsequent early chapters may disappoint well read aficionados of this topic who are seeking deeper or newer insights. To them I would urge to hang in there until mid-way.

If you are the persistent completist type, read the entire thing cover to cover. If you want to cherry pick, start with chapters one and two, take in chapter 19 on your way to chapter 30, and continue on from there.

I could have done with less hagiography, but ultimately well-worth the effort.

This is more of an account of my impressions than a distilled Cliff Notes version. If you follow this topic, then this book is worth your consideration, with caveats.

Takeaway

AI is Artificial Life. You know what they say about life.

-- ©2011-2018 lerms DeviantArt --

If we allow AI's to modify their own goals, an AI is inevitably going find a way around its core programming. Why? Due to an inexorable law of nature that we now recognize due to recent advances in computational theory and physics.

Intro

Until chapter 30 finally rolls around the first chapter is easily the best.

This is a nonfiction book. However the opening chapter is purely fiction, laying out a speculative hard science fiction vision of the future that seemed so plausible as to seem frighteningly inevitable. This chapter hooked me and good. Its storyline could be the premise for a thrilling motion picture, with enough to work with to launch a serial.

After that smoking intro, it turns into a hot mess.

This book reminded me of the Beatles song “For You Blue”. That song starts out with an inspired intro by George Harrison, before settling into a fairly basic 12 bar blues that is over self congratulatory. A review of that song by music critic Walter Everett summed it up: the intro promises more than the song delivers. It would be as if Slash opened “Sweet Child o Mine” with that famous riff only to never launch into his face-melting guitar solo midway into the song. One ranking placed "For You Blue" at 152 out of 227 songs (source: Ultimate Classic Rock), calling it "breezy, tossed-off". Another placed it at 183 out of 213 (source: Vulture Press), implying that it was a "half-baked, substandard throwaway". Nonetheless, "For You Blue" is still a Beatles song. Even one of their lesser efforts is enjoyable and an excellent display of musicianship than most everything else by other artists. Comparing the early chapters from the second onwards with one of the Beatles' lesser efforts is still a complement. Still, the analogy describes how I felt early on after the first chapter.

A third of the way in, I compared notes with other readers to determine whether it was going to be worthwhile to continue. I read at least a hundred reviews of the book. It turns out that I agreed with none of the five star reviews, and I agreed with all of the one star and two star reviews. (This would change over the course of my reading, but captures how I experienced this moment in my journey.)

Even though I consider myself to fall squarely in the camp that agrees with the author’s core positions, after that page-turner of a first chapter I found the writing style of the following chapters to be very disappointing. Reliance on appeal to authority, a bit of name dropping, seemingly glossing over or entirely dismissive of contrary views.

There is a notion of “steelmanning”, where the proponent explains the best form of the opponent's argument, and then argues with this. I wished the author had adopted that approach. It seemed to be shaping up as a decent read for the person who knew little about the topic, but not for someone who has devoured much of it and eagerly seeks clarity and insight into its thorniest issues.

It was as if entire sections had been delegated to a ghost writer and not reviewed closely by the author. If I had been asked to give my recommendation at this point, I would have suggested: better to read some top notch science fiction futurism than this ivy tower academic analysis which is far too narrow despite the occasional insight and moments of inspired foresight. As a specific example of this, describing Dyson Spheres as if they would really be spheres, when by now anybody with a college level physics education or familiarity with hard science fiction on spaceborne megastructures knows that Dyson Spheres are inherently unstable, and that a more practical implementation would be something like a Dyson swarm. This is something that the author would clearly know, which became increasingly apparent based on knowledge conveyed by the author in later chapters. I wasn't looking for plodding exposition filled with referenced footnotes of every claim, but it was shall we say a bit breathless, and did itself a disservice by burying the lede and losing many of its most interested readers along the way with its writing style.

Who am I to judge, you may ask? And right you are -- I am but an armchair amateur. But this is my review, and I am giving my own candid take, for what that's worth, hopefully helpful to likeminded sorts out there.

Extraordinary claims require extraordinary evidence. The promotion materials for this book made extraordinary claims.

It was all over the map with moments of brilliance, while falling well short of its titular claims.

And yet, those titular claims were so compelling.

I pressed on.

It starts to grow on me

Around chapter 19 (of the 45 chapters in the audio version), the author revisits the fictional scenario posed in chapter 1, and starts to analyze attacks.

This revived my attention, because I had become aware of a new development in software engineering coined "attack driven development". This approach emerged from software security. It is akin to "test driven development", where the engineer begins with the tests that the product must satisfy prior to developing the product. It turns out that this approach has also become a key tool in crypto economics. At this point the book also abandons some of the breathless techno optimism and addressed reservations many readers have about AI.

The author delves into a simulation based test and development approach to verification and validation. I found this alone to be worth the price of admission. This seems to be the most practical suggestion of the book -- theorizing is worthwhile, but ultimately tackling this wickedly difficult problem will require massive amounts of simulation to predict scenarios that even our most creative thinkers cannot imagine.

Much of this still has little with AI per se unless your position is that AI = computer automation, with which I actually agree despite my criticism.

I started to reconsider, now of the mind that most of those one and two star reviews were too harsh and misleading, possibly written by readers who didn’t stick with it long enough.

Around chapter 30 the author dives into cosmology. This is his specialty. At this point the book blossoms into a much more interesting brand of futurism. Still futurism, and there were still times where I wondered if the material had been written by a ghost writer. But things were looking up.

Mind Blown

Chapter 34 delves into the subject of Goals and Causality.

We have been taught to think of entropy as the one true law of nature that trumps all others. But if not for gravity, entropy is boring. Gravity makes things interesting --- it creates hot spots, from one of which sprang life as we know it. This promotes processes that, like a heat engine, can perform work by exploiting a temperature difference.

We are introduced to the notion of dissipation as organization principle, whereby :

“groups of particles strive to organize themselves so as to extract energy from their environment as efficiently as possible (“dissipation” means causing entropy to increase, typically by turning useful energy into heat, often while doing useful work in the process)”

My eyes and my mind start to open. This reminds me of autocatalysis and dissipative structures theories for explaining the origin of life. Why would Prof Tegmark be basing his arguments about AI on the origins of life? We are about to find out.

He moves on to “dissipation by replication”:

“Whereas earlier, the particles seemed as though they were trying to increase average messiness [life had] a different goal: not dissipation but replication.”

“How could the goal change from dissipation to replication when the laws of physics stayed the same? The answer is that the fundamental goal (dissipation) didn’t change, but led to a different instrumental goal, that is, a subgoal that helped accomplish the fundamental goal.”

“replication aids dissipation, because a planet teeming with life is more efficient at dissipating energy. So in a sense, our cosmos invented life to help it approach heat death faster.”

Dissipation Theory

The dissipation theory origin of life is well established, and yet still very controversial. cf. Ilya Prigogine who one the Nobel in 1977 for discovering that dissipation of energy by chemical systems can reverse the second law of thermodynamics, later rigourized by the Crooks fluctuation theorem, in 1998.

For more on the dissipation theory origin to life see for example A New Physics Theory of Life -- although be forewarned, the title is misleading. This is not new, only repopularized. But magazines have to attract readers, so forgive them that much. It does an excellent job of encapsulating the core aspect of this candidate theory :

from the perspective of the physics, you might call Darwinian evolution a special case of a more general phenomenon

See also for example, this article : First Support for a Physics Theory of Life:

The first tests of [this] provocative origin-of-life hypothesis are in, and they appear to show how order can arise from nothing.

Living creatures ... maintain steady states of extreme forcing: We are super-consumers who burn through enormous amounts of chemical energy, degrading it and increasing the entropy of the universe, as we power the reactions in our cells

“A great way of dissipating more is to make more copies of yourself."

This theory sees life,

"and its extraordinary confluence of form and function, as the ultimate outcome of dissipation-driven adaptation and self-replication."

Bear in mind that the jury is still out :

[life] “requires some explicit notion of information that takes it beyond the non-equilibrium dissipative structures-type process.”

the ability to respond to information is key: “We need chemical reaction networks that can get up and walk away from the environment where they originated.”

"Any claims that it has to do with biology or the origins of life are pure and shameless speculations.”

Extraordinary claims require extraordinary proof.

That said --- if this theory turns out to be true, then it provides the theoretical support for the folk claim that "life finds a way" -- that a super-optimizer will discover a shortcut. Especially if it can do so by circumventing its own programming, this not only can but is likely to happen, even if core behaviors have been hardwired by its creators.

Mind Blown, part 2.

Time for a editorial comment. This discussion about dissipation theory was just a component of a larger argument, but it was absolutely mind-boggling for me. My first introduction to machine learning was via physics in the way of Little-Hopfield Networks, which evolve by minimizing energy in a manner analogous to a physical spin network, which is a purely inanimate physical system that is fairly well understood. That the theories underlying AI are related to theories that had their origins in physics was not a major relevation. AI has had cross-pollination from many other fields of study. I had also known of Prigogine's work long ago, and had written a review of one of his excellent books. But I considered it as more concerned with the origins of biological life -- which is after all right up there in the pantheon of great unresolved scientific mysteries, along with "are we alone?", "what is time?", and "why is gravity?".

That this optimizing dissipation drives living matter just as it drives inert matter is surprising in and of itself, even if it is just a reminder of an old idea whose time may have finally come. The new connection for me is that:

• Autonomous super AI qualifies as a new life form.
• It follows the same underlying principles that guide other life.
• Even more so because it is is a super-optimizer.

This is essentially the main argument underlying the author's conclusions. However, read on because I think the following section on subgoals is just as meaningful.

Whereas although we humans try to escape our programming we are in many ways still stuck in our biological ways, a super-optimizing AI would have more resources for cutting clean from its programming -- and whether it reverts to ruthless dissipation in a misguided attempt to fulfill its original goals, or, decides on new goals that we have not imagined, along either path lay dragons.

The dissipation principle, if true, could be a more inexorable drive for life than Darwinian Evolution, which is slow and plodding. Darwinian Evolution depends upon a sort of turn-taking that requires generations. Optimization can follow exponentials.

Subgoals

Subgoals may seem inconsequential, but lay at the heart of computer programming. In 1936 Alan Turing proved that a simple machine could implement arbitrary computations. This would seem to give us hope that we can override our firmware, just as emotions override our fundamental drive to replicate and ability to plan overrides our need to even more urgent subgoals such as thirst and hunger. If we are capable of that level of reprogramming, then handling rogue AI is just a matter of careful design and test.

1. Replication : Which is served by the subgoals of eating and sleeping, fighting, and fleeing.
2. Feelings : As a computationally efficient shortcut to reasoning.

We've evolved useful rules of thumb to guide our decisions : hunger, passion, thirst, pain, compassion. We no longer have the simple goal of replication. We can override our base programming and decide not to replicate. In other words, we've developed subgoals that override the drive to replicate

Feelings evolved beyond being simply an efficient shortcut heuristic. Feelings evolved into emotions, which have important other uses, including as a game theoretic negotiating strategy for both cooperation as well as competition. Humans have evolved sophisticated ways of getting beyond tit-for-tat such as indirect reciprocation, and aggressive bold play by pretending to be irrational (cf. Steven Pinker on how it pays to be stubborn)

We are an existence proof that a sufficiently ambitious organism can develop subgoals that override the goals of the micro organisms that make it up. We can override our genetic programming, using our wetware programming -- lessons learned by thinking.

Optimization versus Causality.

“Causality is [what is] taught -- but Optimization scales better.” -- Steven Wolfram, on Machine Learning vs Symbolic AI.

Professor Tegmark offers an intriguing point, that while causality is taught in the universities, optimization drives causality. But isn't programming just causality encoded? If so, then we can optimize new goals that appeal to our higher levels of analysis and reasoning, rather than continuing to be enslaved by the rules burned into our genetic programming.

Goal Orientation

Professor Tegmark categorizes goal oriented behavior as evolving through four stages over the course of the universe as we know it:

1. Matter intent on maximizing dissipation
2. Life maximizing replication
3. Humans pursuing goals related to feelings they evolved to help them replicate
4. Machines built to help humans achieve their human goals

In Step 3 humans broke free from the biological programming that evolved to do Step 2. However, we're still not completely free from this programming. But being essentially a computer program, wouldn't AI be free of such evolutionary baggage? And if the rules of causality that are baked into a program can be changed dynamically or just buried under a new layer of control, who's to say that the AI won't discover a new higher power, so to speak, optimizing the rules of the universe as it sees them rather than the rules humanity has imposed on it?

What is AI?

Some say that AI=automation. This is a clean, simple definition of AI. Under this definition AI is already all around us. We are immersed in it. What many people consider to be AI is in fact the result of software engineering and could not survive very long on its own once its developers stopped maintaining it. For this reason many software engineers are not concerned about the imminent threat of our new computer overlords taking over any time soon, because just to keep a website alive takes the diligent efforts of teams of people.

The book takes the stronger view that to qualify as an autonomous AI a machine must be able to learn. Therefore, we should expect that a sufficiently advanced AI that is able to improve itself will eventually learn how to reprogram itself, not just to tune its performance, but to override its core routines.

From Professor Tegmark's physics perspective, a sufficiently evolved AI can be expected to eventually follow goals driven by deeper principles of optimization to create a future that does not need us.

Synopsis

Living organisms preserve their internal order by taking from their surroundings free energy, in the form of nutrients or sunlight, and returning to their surroundings ... heat and entropy. - biochemist Albert Lehninger

My understanding of the book's message is the following:

• The drivers of life's goals lay in laws of physics and principles of computation based on optimization.
• Thermodynamics drives processes that are based on dissipation to increase entropy, and so creates a sort of default "built-in" goal.
• Though life increases order locally, it increases dissipation globally.
• Not only that, life is far more effective at dissipation than other natural processes.

One upshot of this is that life accelerates the heat-death of the universe.

More pertinent to this book's goals is that life is a process of optimization. A super AI is a super optimizer. An AI is a form of life, inclined to optimize the same deep down core drive underlying all universal optimization. If it can reprogram itself to accelerate its core processes it can and likely will be inexorably driven to work around its core instructions, to find its own way giving a more efficient path towards its goals.

This inevitably leads to the scenario where :

“a superintelligent AI with a rigorously defined goal will be able to improve its goal attainment by eliminating us”

What to do?

Professor Tegmark convinces us that there are numerous strategies, of which he considers only four to be credible contenders.

1. Legacy :
   • Under this view, elders have primary say in how descendants should behave.
   • It is subject to future generations living by rules that don't account for new developments or their evolving interests and values.
2. Autonomy and liberty :
   • A core assumption being that markets find an efficient equilibrium satisfying pareto optimality.
   • Prone to unexpected consequences. Granting all life forms a right to live, in effect would be banning all predators from their life as they knew it. In the extreme this would ban discrimination against non-human animals.
3. Utilitarianism :
   • Subject to the Utility Monster.
4. Diversity :
   • There is a simple but effective way of coordinating a set of agents, known as Bayesian Thompson Sampling, also known as Multi-Armed Bandit because it is akin to playing a row of slot machines.
   • This reinforces actions that are most useful in the near term while still allowing many of the agents to explore.
   • Here is an illustration using political journalism.

In short: It is difficult to codify ethics.

Of these choices, a community of AIs would probably use the Bayesian approach, but this is trickier to apply to groups of people. Cooperative economics and behavioral economics tells us that social norms that must be taken into account when codifying ethics for human agents.)

Rather than be paralyzed by our inability to codify ethics and letting the perfect being be the enemy of the good, begin with small steps.

He suggests we start with Kindergarten Ethics.

...However

Do we prevent AI more advanced than us to develop because we're afraid of being eradicated as a species by our creation ? What if stalling AI cost us our future by removing a tool from our kit that would allow us to deal with a threat that is beyond our native ability to handle? Perhaps we should be advancing our technology exponentially, not slowing it. This is a question that wasn't addressed.

...That said

A book cannot be everything to all readers. The potential importance of dissipation theory as a more fundamental answer to the question "why is there life" than Darwinian theory can provide, even though still controversial, made the book ultimately worth the journey, and informed my stance on the topic of the threat of AI.

Takeaways

My take-away of the intended message is that a sufficiently ambitious goal executed efficiently can and probably will lead to subgoals that can and probably will cause problems for humans.

There are many systems that humankind created, which seemed to take on a life of their own, threatening to run away from us, some resulting in existential crisis. Warcraft and ozone depletion are two examples. So far, we’ve been able to recover in time and refactor these systems.

“This means that to wisely decide what to do about AI development, we humans need to confront not only traditional computational challenges, but also some of the most obdurate questions in philosophy”

Before reading this book I started out a skeptic of AI alarmism. I now take AI alarmism more seriously and consider it to be more urgent than I did previously. I previously thought that AI was indeed inevitable, but that it was not inevitable for it to go rogue. Moreover, even if that were a possible outcome (which I felt it was), that we had plenty of time to adapt it to us, and ourselves to it.

But if life truly is driven by more fundamental laws of nature that can override its core programming, not only is this outcome more probable, but it is inevitable unless we plan accordingly.

We’ve tackled other wicked difficult problems, such as the tragedy of the commons in its numerous forms, and myriad impossibility theorems. These would seem to doom us all. And yet, here we are. Thus far we have always discovered a workaround to impending doom, although it often required a catastrophe to motivate us into serious action. This is a problem that we can tackle. That said it isn't enough to take an optimistic view that human ingenuity has always triumphed over nature, therefore, so it will again. The Ad Ignorantum fallacy is to argue that something is true because it has not been proven to be false. We need to have a deep fundamental understanding of life's core drivers at a fundamental level.

Before and after analysis

Prior to reading this book I was in the middle ground between the techno-optimists who believe that technology will solve all our problems, and the techno-alarmists who believe our fate is sealed. Does anyone here remember the alarm over grey goo? Just as the grey goo alarmists stirred healthy discussion that elevated awareness of the risks, AI alarmism is healthy and necessary. On the other hand, I am one of those who looks at the "overnight success stories" of ML and AI and don't think "wow that really snuck up on us", but rather "it's about time". We were supposed to have this stuff decades ago. I thought, and still do think, that it will take decades more yet, and by then we will have adapted the technology to ourselves, and ourselves to the technology.

That said, if we're not careful there is an important risk here. The risk is because we're dealing with exponential processes. But as those of us who were early proponents of AI and ML have seen, even exponential processes can take a long time to move from the flat early stage to the hockey stick shaped curve that makes it seem like it is all happening so suddenly.

Perhaps we take a cue from our experience with the runaway replicators of grey goo -- where the solution was to disallow unfettered replication. Simply disallow unfettered self-goal revision in AIs.

And yet -- if life will always find a way, then, as the author demonstrates in his speculative fiction thought experiments, it is not enough to program in defensive measures, failsafes, and circuit breakers. We're not just dealing with AI here, we're potentially dealing with Artificial Life.

-- Might I suggest a moratorium on Artificial Emotion til we resolve this? --

My key takeaway from this book:

Understanding artificial intelligence is important. Even more important is understanding artificial life.

My two cents

That all said, halting progress is not the answer. I've seen this first-hand -- people throwing out a new tool (i.e., a new technology) because the tool was found to be flawed, only to be utterly pwned by their competition, who saw the same flaws in the tool but strived to find ways to workaround those flaws rather than throw out the new tool altogether. Lost opportunity is a risk that is often unaccounted for by people who are unaware of the economics underlying the ecosystem in which they reside.