Consciousness may be the greatest illusion of all. Or at least the greatest mystery. “It is in the brain that the poppy is red, that the apple is odorous, that the skylark sings,” wrote Oscar Wilde. But how does the brain construct conscious experience? For the past 25 years, Christof Koch has been trying to provide an answer. At the Allen Institute for the Brain, where he is chief scientific officer, Koch is steering an ambitious project to map the complex networks of the brain. The goal is nothing short of understanding how we function in the world—how we see, fall in love, feel pain, marvel at a night sky full of stars.
In his most recent book, Consciousness: Confessions of a Romantic Reductionist, Koch writes that “consciousness is a fundamental, an elemental, property of living matter.” He puts forth an “integrated information theory” that he believes provides the first rigorous scientific theory to explain consciousness. On a recent afternoon at the Allen Institute in Seattle, Koch took time to explain the captivating, if not entirely clear, theory. Koch, formerly a professor of cognitive and behavioral biology at the California Institute of Technology, has a capacious mind and a piercing intensity. In our discussion, his explanations ranged across brain biology and artificial intelligence, infamous patient Terri Schiavo and the film Her.
Why do you say consciousness is not an emergent property of the brain?
How can a computer or the Internet be conscious?
So was “Watson” on Jeopardy conscious?
What is “Integrated Information Theory?”
Do you believe Terri Schiavo was conscious?
How does Integrated Information Theory explain consciousness where other theories don’t?
How can a subjective sense of blue be reduced to immaterial information?
You write that Integrated Information Theory is “an elaborate form of panpsychism.” What is panpsychism?
When you write things like, “the entire cosmos is suffused with sentience,” you sound pretty New Age-y.
It sounds like a contradiction to say consciousness is both panpsychic and quantifiable.
The self is often called an illusion. How do you define the self?
You write that, “some deep and elemental organizing principle created the universe.” What do you mean?
What would you be if you weren’t a scientist?
Interview Transcript
Why do you say consciousness is not an emergent property of the brain?
Well, I mean, “emergent” fundamentally means that at some level it’s not there, but then at a different level something is there. So the classical example of an emergent property is the wetness of water. If you take one or two molecules of HO, they’re not wet. But then you take trillions of them, 1023, and then suddenly you get this property that it clings to surfaces that we know as wetness. With consciousness, for the longest [time] I also believed, well, it’s an emergent property—small brains don’t have it and then once you get to some bigger brains they do have it. But the problem with that assertion is that consciousness is so fundamentally different from anything else. Neurons and neuromechanisms are so different from the perception they can generate. My perception of having pain, or my perception of seeing red, is so radically different from the neurons that support the perception of seeing red or the neurons that support the perception of having pain that it’s totally unclear how we can go from one to the other.
So finally then, I at least came to the conclusion that it is a fundamental property of certain pieces of highly organized matter, that highly organized matter such as brain are capable of, and do have, experiences. We just live in a world with space and time and mass and energy, and also where complex organized matter has conscious experience; and that’s just the character of the world we are in. Now we can study this property; and the property that organized matter has to have in order to have consciousness; and how much consciousness [it has]. And if every piece of organized matter has consciousness, what about a fetus? What about an early born? What about a patient with severely compromised … [with] most of his brain dysfunctional? What about a dog? What about a cat? What about a worm? And then of course, I can ask the same question: If this is true, what about other pieces of organized matter that didn’t evolve but that we built, like computers or like the Internet? To what extent do they have conscious experience? To what extent does it feel like something to be there?
How can a computer or the Internet be conscious?
Well I mean, unless … In some sense it’s fairly obvious. Once you don’t have the notion any more that of course, we have in our culture for thousands of years that there is this magic thing—the soul, right? So if you believe in the thing called a “soul,” and only we have it and nobody else [has] it, right, then of course, you know you can ask what’s the nature of the soul. And of course it’s been impossible to understand what a soul is and what’s it made out of and where it was before I was born and where it’s going to go after I die. But once you get rid of this notion of a soul as just, as not being compatible with anything else I know and not consistent; then if you believe that consciousness is a property of certain types of natural systems, then in principle you should be able to replicate in a different natural system—once you think about it in that way. And yes, so in principle, once a computer is complex enough to begin to rival the human brain, then in principle, why should it not also have conscious experience?
So was Watson on Jeopardy conscious?
Well, okay, so there the difference is: So Watson wasn’t conscious, Watson is just very good, was programmed. It doesn’t feel like anything to be Watson, right? Watson was very cleverly programmed by a team of, you know, up to 100 engineers and scientists to answer a particular question. Now of course, you know, if you ask something related like you know, what’s the weather like today, it probably couldn’t answer that. But it was programmed exactly to answer Jeopardy.
But let’s take this process fast, forward. Let’s wait 20 or 30 years, right. In fact, let’s take the recent movie, Her, right. Now you remember Samantha, right? So Samantha was the name of this entity. Think of it like Siri but in 20 years from now, right. So now I have this highly conversant voice that talks like a human and has the same memory and can, you know, remember things and feel things and you have to ask your question: Does it feel like anything to be Samantha? In fact, at some point, she asks herself that in the movie, Her. Does it feel like anything? Does she actually have feelings or is she just a set of very complicated input-output routines? And this now really depends on your metaphysical stance and it depends on your theory of consciousness. Interestingly, the dominant theory today, the dominant sort of, metaphysical assumption about consciousness that’s known as “functionalism”—and most philosophers these days are functionalists—would say yes, if she replicates all the causal relationships, if she simulates all the causal relationships of a human brain, then there would be no question. She would be conscious.
Now Integrated Information Theory of consciousness takes a somewhat different tack. It says, well, if you build a neuromorphic brain that has, you know, let’s say copper instead of axons, and transistors instead of neurons, then if you have the same cause-effect repertoires as [the] human brain, then this entity would actually be conscious. But if you simulate the brain like Samantha did, you would not be conscious. So the IIT claims that it doesn’t feel like anything to be Samantha. Essentially, for the same reason that a computer simulation of the weather, of a storm, of a tornado is never wet, right? You can simulate these days on modern computers; you can simulate the details of a tornado and of a hurricane. But it’s never wet inside the computer, right. It simulates the effect of water, but it’s never wet. For the same reason, IIT says that you can simulate consciousness, but this system itself will not have consciousness. In order for it to have consciousness, you have to replicate the detail—not simulate, but you have to replicate the actual cause-effect repertoire of the brain.
What is “Integrated Information Theory?”
Integrated Information Theory is the attempt by one particular, very gifted individual—a psychiatrist and a neuroscientist, Giulio Tononi—he’s at the University of Wisconsin, in Madison—to make sense of all this knowledge, know all, that we have on the one hand about the philosophy of consciousness, but more importantly, about your conscious experiences and about the neurology of consciousness, and to put that into a rigorous scientific axiomatic theory that tells you quite precisely what consciousness is, how it originates, which systems have it, and which ones do not have it. And which also gives rise to a whole set of predictions that you can test in the lab, in the clinic, to again measure and ascertain which systems are conscious and which ones are not. Because in clinical practice, it is very often problematic, for certain classes of patients, to really be sure that the patient is conscious.
You may remember Terri Schiavo? You know, many years ago in Florida. So, she was a patient whose heart had stopped for 15 minutes, and she was in a so-called persistent vegetative state, or permanent vegetative state, where she clearly was … she had wake-sleep periods but when she was “awake” it was impossible to communicate with her. And so in patients like this, they may look normal. If you just see a picture of them, she may look like she’s smiling, but she doesn’t respond in any meaningful way to you. So here, it is not easy in this patient—and there are a few thousands of these patients in the U.S. alone—to ascertain to what extent do they feel anything. Does it feel like something to be Terri Schiavo? Or is it just sort of a brain mechanism? There still are some brain stem reflexes left so she’s clearly alive but without any conscious experience. Just like you are in a deep sleep. In a deep sleep, you’re clearly alive but you don’t have any conscious experiences.
Do you believe Terri Schiavo was conscious?
No. I mean, together … I looked at, you know, I looked at all the evidence. There were four separate clinical teams of medical experts that went over this question. All the evidence suggests that she, like thousands of sad cases like her, was sort of operating on brain stem mechanism. Yeah, occasionally you know, if you shine a bright light, occasionally she would track. Occasionally, she would, you know, look like she [was trying] to smile. But you know, if I film somebody for hours or days, yes, occasionally I can interpret you know, I can interpret if I really, if I’m hoping for a signal—like if I’m the loved ones, or the parents of this unfortunate patient—I can interpret there is something there but if you do this over a long time and if you’re a little bit more detached from it, you clearly see there doesn’t … there didn’t seem to … All the evidence indicated that there wasn’t anybody home anymore That she literally, Terri Schiavo, her conscious self, had died 14 years earlier. The shell of her self, particularly her brain and the brain stem, was still operating. And at autopsy, it was … the brain showed all the cerebral cortex had shrunk by more than a factor of two. So clearly she couldn’t see anything. All of her visual brain at the back was sort of degenerated. So all of that supports the contention: No, that she was not there. It didn’t feel like anything to be her. She, the her, she wasn’t present anymore.
How does Integrated Information Theory explain consciousness where other theories don’t?
Well I mean, a) there really aren’t that many consistent theories around. There are lots of hypotheses, but if by theory you mean a well formulated axiomatic body of short statements from which you can derive a mathematical calculus that you can then turn into predictions, there really isn’t any good alternative around. So … But that’s exactly what I like about the theory. It seems to be very plausible. It shares many elements of a conceptual framework that people have grappled with over the years. So for example, David Chalmers, a philosopher, he in his work 20 years ago already surmised that there was something key about particular types of information—that’s essentially what the theory says. It says particular types of systems generate a particular type of information called, “integrated information,” and it is integrated information that is experienced as experience. That’s what … it’s an identity relation. The theory essentially says that a particular type of information that makes a difference with the system in and of itself is what experience is. So it’s a fundamental assertion that tells you have a complicated, you have an organized system, it makes a difference to itself, and this difference that it makes to itself you can describe in mathematical terms—that’s what conscious experience is.
In a sense it takes us back to Pythagoras—you know in Sicily, the Greek philosopher and mathematician in Sicily who said that everything is ultimately numbers. It ultimately makes this assertion that consciousness is a particular informational structure in a high dimensional space. We don’t experience it as such. We experience it as sounds and sights and the tastes and the pains and the pleasures of life. But it says conscious is this high dimensional structure in these abstract spaces, and from there I can make predictions and I can turn toward neuroscience and toward the clinic to test to what extent these predictions are valid. And it also makes extrapolation. It also says which types of artificial systems, like which types of computer may or may not be conscious.
How can a subjective sense of blue be reduced to immaterial information?
You do it the other way around. You start with your sense of blue. You ask: What is it that makes a sense of blue, blue? Well for one, there is a sense. I do have experience of blue; it exists. It is the way it is because it’s different from any other experience I could have. So it’s different from black; it’s different from seeing you; it’s different from seeing a frame of the movie Blade Runner; it’s different from tasting, you know, wine. So it’s very different and … But it’s integrated. I cannot … So for example, you know, I see this video on a blue background but while I see the blue background, I see a lot of other things at the same time. I cannot separate that out. It’s holistic, it’s integrated; something philosophers have much remarked upon. It’s also structure. When I see a blue sky, it’s blue above, it’s not blue to the below. It’s blue on the left or blue on the right, so it has a sub-structure. And there’s only one blue. It’s not that I experience blue while simultaneously, there’s another Christof that experiences something very slowly and there’s another Christof that experiences something at an incredibly fast scale. There’s only one conscious experience.
And so, then you ask, given all these different properties about blue or any other experience—a kind of blue—you ask what is the mathematical calculus? How does a system have to be structured in order to support that? And that gives rise to a mathematical calculus. If you now, if you take those sort of, ways of defining your phenomenal experience, if you take them seriously and ask what mechanism can support them, then you turn into this … So you translate into mechanisms and then you give rise, you get out this calculus, that’s this calculus that says well in order to have an experience that has these different properties, you need to have a particular system that makes a difference to itself that is structured in a particular way; there’s only one of these systems and it has to make a difference to itself. So there has to be a mechanism like a neuron or like a transistor, it can be on and off—the type of information it has has to be integrated, has to be highly differentiated and then you get to IIT. So it’s a systematic process of going from the different aspects of conscious experience to a particular mechanism that supports. But you start out with experience. You don’t start out with a mechanism like a brain and ask how does a brain mechanism predict “y” conscious experience. You start with your conscious experiences and you ask what is the underlying mechanism have to be? What property does it have to obey in order to have conscious experiences?
You write that Integrated Information Theory is “an elaborate form of panpsychism.” What is panpsychism?
So panpsychism is a very ancient philosophical belief. It goes back to, in our culture, to Plato and it’s very widespread. If you talk to Buddhists … I spent last year, a week with His Holiness, the Dalai Lama, so he and his Buddhists have reasons—are certainly all panpsychists—that assumes that consciousness is very widespread. Now it depends exactly what type of panpsychist you are—of course, there are many different forms—but essentially, it says that it’s very, very widespread and certainly may extend throughout the entire animal kingdom or may even extend wider.
Now as a philosophical position it’s appealing, but it’s very difficult to get anything more out of it. Like you want to ask, well are there all-system conscious? What makes them conscious? Are there any systems that are not conscious? And so panpsychism doesn’t answer these questions. IIT, Integrated Information Theory, does. It makes certain very predictive—it makes some very specific predictions. It says, for instance, probably all complex neurobiological systems—all creatures that have brains—may well have consciousness, including bees and worms and octopi, etc. It may also be possible that if you build a brain out of wires and transistors, that you find consciousness there too.
But it also says that other systems like, for example, digital simulations will not be conscious. And it also says that there isn’t super-consciousness, that if I take a bunch of … For example, clearly answers the question that you and I, although we’re interacting, there’s no super-consciousness, there’s no right now Christof-Kevin or Kevin-Christof super-consciousness. There’s you, your conscious, you have conscious experience; I have conscious experience. We interact, but right now there is no über-consciousness. So [IIT] makes a number of very precise predictions that panpsychism was never able to make so it’s a much richer, more quantitative, more scientific form of panpsychism. But it does share the basic intuition that consciousness is graded and it’s much more widespread than we think in nature.
When you write things like, “the entire cosmos is suffused with sentience, ” you sound pretty New Age-y.
Yes, so here I have to confess that the nice turn of phrase I probably shouldn’t … It’s a beautiful turn of phrase, right? And so as a writer, you want to use it but … So you always have to make a judgment. As a scientist, you know, I ought to be very careful what I say. Yes because IIT again says that you’re conscious, I’m conscious, but it’s not … there isn’t this sort of woven consciousness everywhere. What I meant to say [was] that it’s much more widespread than we think. Many animals may have it; in fact, most animals for all I know will have it and lots of other systems may have it, so that’s what I meant by it. I may be taking too much of a license here as a writer compared to being a very accurate and precise scientist!
It sounds like a contradiction to say consciousness is both panpsychic and quantifiable.
Yeah, but I … So that’s the advantage of IIT. So IIT—finally, we have something that philosophers have not been able to do over the last 2,600 years with a precise mathematical definition of what it is, how to measure it. In fact, that’s … It has this number called ϕ that measures sort of the quantity. It has an informational structure that measures the quality of it so you can now make some very precise statements about consciousness. And while I don’t think Integrated Information Theory is the final word on consciousness, it’s certainly, it’s a big step in the right direction, and those sorts of theories will take us in a direction where we can turn, just like we’ve turned many other subject from metaphysics into physics, and ultimately into scientific subjects, to be able to ultimately explain this most mysterious of all phenomena—how does subjectivity, how do feelings, how does experience get into the physical world.
The self is often called an illusion. How do you define the self?
Well, the self is this sense of ego, this sense of continuity in space and time—that I wake up you know, this morning, I’m the same that I was yesterday, and I’m the same that I was last year, and then when my mom points at a picture of me, you know, where she says, “Well see? That’s you at 2 years old”—I have no idea it’s me, but I believe her. I say, well that’s my self, right. Although I know in the intervening time everything—a molecule in my body—has changed, I still believe that sort of is me.
So partly, it’s an illusion, because of course, we know we’ve changed radically, we have different … we may have different make up—we certainly have different memory—but still, I have this sense, and I think I need this sense in order to act in the world and to continue to act and to take advantage of experiences I’ve stored up over my lifetime. So partly, the Buddhists are right in the sense that it is partly illusory. But ultimately what remains is, and I’d like to end on this, with this phrase, what remains ultimately is my experience of having a self, my experience of having experiences as such. That is the most fundamental observation I can make about the universe and ultimately science has to explain how this feeling of having experiences comes into the physical world.
You write that, “some deep and elemental organizing principle created the universe.” What do you mean?
I find myself in a universe that’s conducive to life, right. This is the discovery over the last 30 years known as the “anthropic principle,” that everything in this particular universe seems to be extraordinarily balanced, so that you can have emergence of stable elements, you can have stable stars around which you can have stable planets and then you have natural evolution that gives rise to complex features. We also live in a universe where complexity begets conscious experience—that’s the heart of something like Integrated Information Theory, that complex piece of organized matter gives rise to consciousness. So isn’t it amazing that we are in this universe. Is that a chance event? Is there an infinite number of universes and we just happened to find ourselves in this one particular universe? So now for a reason I’m utterly unable to justify, I find this all deeply mysterious and deeply beautiful—that I do live in this universe, that is conducive to life, that is conducive to conscious experience. And I see looking back over the last 13 billion years of cosmic evolution, and then 4.3 billion years of terrestrial evolution, and then maybe 4 billion years of biological evolution, that here, we are very complicated creatures and the evolution is ongoing. So it’s all very mysterious, but it’s all very beautiful to me. In that sense, I guess I was born with this sense of mystery. I find that very, very surprising.
What would you be if you weren’t a scientist?
A climber. A professional climber. Climbing’s very addictive because in climbing, you can certainly experience … It’s all about conscious experience. You’re out there on the edge, you’re out there on the sharp end of a rope, and you’re hyper-conscious of the world. Your ego sort of recedes. It’s a form of meditation because you are engaged to such an extent with the world, with the environment, you have to pay such attention to every little unevenness in the rock, that your inner voice—this constant critic that’s always in your head—is completely silent and you just experience the raw feels of the world. It’s a wonderful, it’s an ego-loss, it’s a very addicting experience so I wish in a different life, I could just be a climber all the time.
