Wired Science

Jeff Hawkins got famous for developing the software behind the Palm Pilot and the Treo SmartPhone. They are two of the most important pieces of consumer electronics since -- well, ever.

Jeff has since returned to his first love, neuroscience, specifically on the question of how the brain creates the mind. Now he has a new company, Numenta, that's taking its cues from the structure of the brain to build the first genuinely intelligent machine.

Adam Rogers: Jeff Hawkins, thanks for coming to WIRED Science.

Jeff Hawkins: Thanks for having me, Adam.

Adam: I'm just a huge fan of the Palm and the Treo, and I was inseparable from a Palm for a couple years. But that said, I've got to know -- how does the person who invents those get interested in neuroscience?

Jeff: I was just starting my first job at Intel, and I just fell in love with this problem of how the brain works, and I started reading about it, and I decided that it was a problem we could solve in our lifetimes. Basically, how does the brain work from a theoretical point of view. And I also --

Adam: That's computer science hubris, you know.

Jeff: Well, we could talk about that, but I was more interested purely from the neuroscience point of view. Like okay, this is -- if we're going to understand, if we are our brains, everything we've done in our lives, all our culture, our politics, our art, it's all a product of -- and who we are is all a product of our brains. And we had absolutely no idea how the thing worked.

And yet we had tremendous amounts of data about how it's connected and wired, and the physiology and the chemistry, but there was no -- Francis Crick, one of the co-discoverers of DNA, wrote this piece in "Scientific American" in September, 1979, and it was sort of like the "Emperor Has No Clothes."

He said there was an issue all dedicated to the brain, and in the very last piece, Francis Crick wrote one and he said, "Well, we know all these facts but we haven't a clue how this thing works, and don't pretend we understand our work."

Adam: That's sort of an astounding thing for scientists to acknowledge.

Jeff: Yes, but of course, he already had made his fame and fortune, and he was -- only a person who's really well-respected can say that. But to me it was like yeah, he said "We have all this data, but we really don't have a theory." And I said, "Well, we should be able to figure that out." So I basically at that point in time said "I'm going to spend my life working on this," and believe it or not, the whole Palm thing? That was actually sort of my day job.

Adam: Like I said, that's just astounding, to be able to say that this, as a consumer device that so many people carry and rely on, and so many people carry and rely on Treos, the idea that well, yeah, but that's a hobby.

Jeff: Well, I quit my job in the mid-eighties, and full-time I went to become a neuroscientist. And I got into a graduate program in neuroscience at Berkeley, UC Berkeley, so I left that field. And what I discovered, it was very, very difficult to work on theoretical -- theories of brain function at that time, in the mid-eighties.

You just couldn’t do it. You couldn’t get funding, you couldn't work in labs, there weren't professors doing it. And so everyone thought it was a great idea, they said, you really can't do this now, and I was very bummed out. And so what I decided to do is go back into industry and work a number of years and mature and figure out how to make institutional change, not just scientific and engineering changes, but institutional changes.

I figured neuroscience would mature. I needed to be able to afford to be a student again. So I consciously went back into the computer industry, saying I'm going to work for a number of years to build up a little kitty so I can afford to do this, and for all those things to happen. And it turned out that well, Palm came out of that, and it turned out to be very successful, and it--

Adam: So it was bigger than a little kitty.

Jeff: It was big, and it was also something you can't walk away from after four years. So I had to balance my time between these two, but really, the brain has been the driving force behind almost everything I've done.

Adam: Researchers know a lot of structural information about the brain. What doesn’t that tell them about how the brain works? Shouldn’t that be enough, kind of?

Jeff: Well, no, in fact, how the brain actually operates and even what it does is largely a mystery.

Adam: I suppose if you open the hood of a car, just knowing what all the pieces are doesn’t tell you how a car works, right?

Jeff: Exactly. I use that analogy sometimes. It'd be like I gave you a car engine, and you didn't even know what it does, right? And I said here, tell me how's this thing function. You'd first have to figure out what's its main purpose, and you might say well, it's made of metal, so I'll spend all this time studying what the properties of metal are. But that might not -- that's not going to tell you what the engine does. In fact, that's what happened in neuroscience. We focused a tremendous amount about neurons themselves, and so we know a lot about neurons.

Adam: Individual structural unit.

Jeff: Yes, and how they work, and how the individual synapses are formed, and so on. But the larger-scale structure of the brain is -- it hasn't really been focused on in terms of a theoretical point of view.

Adam: Well, all right, I'll buy that. So tell me how the brain works, then. What do you got?

Jeff: Well, what do we got? So, I can show you on the model here if it's all right with you. Here's a model of a--

Adam: This is your brain on Wired.

Jeff: It is a brain -- a model of a brain. But the outer wrapping of a mammalian brain or a human brain is this wrinkly thing on the top, and that's about 60 percent of the volume of the brain. That is your neocortex. All mammals have one, and non-mammals don't. And the humans have a big one.

Adam: So this is one of the things that divides humans from other mammals--

Jeff: This is -- yeah, and we have a big one. This is why we are -- oops.

Adam: This is the -- oh, that's going to hurt somebody, if you lose that.

Jeff: Well, there we go. That's going to hurt someone.

Adam: This is the thing that keeps you breathing, right?

Jeff: No, that's the cerebellum, it's half the cerebellum. We'll leave that there. If I were to take the neocortex off the top of your brain, which wouldn't hurt, because there's no nerve endings in your brain--

Adam: But it wouldn’t do you any good, either.

Jeff: It wouldn’t do you any good, but it wouldn't hurt. And you were to iron it flat, you would get a structure that's about the size of a dinner napkin. It's about 1,000 square centimeters, and it's about this thick. It's about two to three millimeters thick. So when this gets in your head, it gets all crumpled up to fit in your skull.

Adam: So it fits.

Jeff: Yes.

Adam: Sure.

Jeff: But it's a thin sheet of cells, and this sheet of cells is basically you. All the thoughts you have, all the things you can tell me in life, all the things you’ve learned that you could relate to me, your language, your thinking, your mode of behavior, all high-level, vision, is in this sheet of cells called the neocortex.

Adam: Like this.

Jeff: Yeah. So my neocortex is talking, and yours is listening right now. And my neocortex is moving my hand right now. The rest of the brain is kind of like breathing and, you know, sex, and things like that, which is important.

Adam: I was going to say, boring stuff.

Jeff: Boring stuff. But this is really where you build a model of the world, and when we go to school, we are basically programming this system. So this, it turns out there's two amazing facts about this sheet of cells. One is that different areas do different things. So there's vision areas and hearing areas, and nowadays they do these imaging studies and they can find when you're playing backgammon, what part of the brain is active? Which neocortex we're looking at.

The other thing that's amazing about it is that those different areas look nearly identical, and they're extremely flexible. It looks like they can almost learn anything. And what makes the vision area vision and the hearing area hearing and the language area language is because of the way they're connected together.

But it's the same basic algorithm underlying it. So what we've discovered is what that algorithm is and how the sheet works. So the trick to understanding how this whole thing works is how does that system learn? How does it store information? And it stores it in a hierarchy. So you have this raw sensory data coming in the bottom, and then the first level of hierarchy learns very simple structure, and then the next level learns more complex structure on that, and the next level a more complex structure on that, and it builds up that way. That's how we train our children.

For an example, just think about language. When you learn a speech, for example, it's based on words you already know. You don't have to relearn those words, and you don't have to store them again. They're sort of being reused. And when you learn a new word, it's based on the same syllables you had. And if you learn a new syllable, it's based on the same letters. And literally, there's a hierarchical structure to language. It turns out there's a hierarchical structure to everything in the world.

In language, vision, social interactions -- it's a fascinating area, but that's really why the brain can work, because the world is structured hierarchically, and we model that in a hierarchical brain.

Adam: And so the neocortex can pick all that up and respond?

Jeff: If the two match up. If the world is hierarchical and the neocortex is hierarchical, then it can learn the structure. If it's random, it can't learn it.

Adam: But you’ve actually taken that theoretical model and applied it to a company, to Numenta. What does that model let you build?

Jeff: What I decided at the time is that the best way of accelerating the science -- accelerating the science -- was to give an alternate venue for it in the for-profit world. Again, I live in Silicon Valley and I'm familiar with the processes of commercialization.

Adam: Sure, sort of starting up technology companies.

Jeff: Listening to other people at work and things. So we decided to take the technology -- take the science and the theory and turn it into a technology, and allow people to apply it to all kinds of problems. We write software that configures computers anywhere from a single-CPU laptop to a very large cluster computer that implements the hierarchical memory that's in the cortex.

So here's how you build this stuff. You start off -- it's software, the designer, you can figure this memory system. It's like configuring your cortex. How big is it, how much memory are you going to allocate to it, and so on. Then you feed in sensory data. If I was doing a vision system, I would have to show it movies or pictures, moving -- time has to be involved.

You have to show moving objects, like a child holding things, moving it around, and then it learns. And it takes a while to learn, just like humans take a while to learn, and the system has to learn on its own. You can't program it. And then it builds up this model of the world, and then you show it new patterns it's never seen before, and it understands them.

It says oh, I know what that is. Or even though it's a brand-new pattern, never has been input before, and it says oh, yeah, I know what that is, and I have expectations about what it's going to do next, and so on. But there's no computer today that can do even reasonably good computer vision.

Adam: Yeah, they can't tell the difference between you and your luggage.

Jeff: Exactly, exactly. Or I have a friend who proposed the grand, million-dollar AI prize, which is to have a computer that can recognize pictures of dogs versus cats. Now, turns out that's a really hard thing to do. There's nothing like that today. Now that may not seem very dramatic, cats and dogs, but it gets at the fundamental problem.

Those things which seem so easy, and that humans find -- you know, a three-year-old can do that, right?

Adam: Right.

Jeff: Dog, cat, you know? But the biggest computers in the world, the most powerful computers in the world, can't do that today. But we, through this technique of hierarchical temporal memories, which is what we call them -- hierarchical temporal memories, because they use time -- we're making good progress on those problems.

Adam: Wow. How's the neuroscience community responded to you sort of walking in and saying not only do I have it figure out, but I can also build a computer that can do it?

Jeff: Well, you be careful. When you're in the scientific academic world, you don't walk in and say oh, I figured it out. You say here's what people know and here's what we're trying to understand. Here's the progress I've made, and so on.

Adam: Academics moves a little slower than TV.

Jeff: Yeah. The academic world has responded very well to this. I was very careful and I went through and did all -- approached it very properly. When I created my institute, I was worried that it wouldn’t be well accepted, but it was. I got great scientists to work there. It got to the point where many neuroscientists were asking, "Could I come to Redwood Neuroscience? How come you haven't invited me yet?"

And I published a book that outlined the beginning of this theory three years ago. It's called "On Intelligence." It didn't have all the details in it yet at that time. And that book has been very well received by neuroscientists. As in any academic field, there's some people who will dismiss it, but I've been really thrilled by the reception. This year in November, I'm giving the major talk at the Society for Neuroscience Conference.

Adam: Wow, that's huge, right?

Jeff: Which is over 30,000 people. And so I kind of represent this sort of -- I'll tell you a funny story. I used to think neuroscientists would say oh, there's that technology guy or that engineer, he doesn’t know anything about neuroscience. And they spent some time with me and they say, no, he really knows his neuroscience.

Adam: Nice.

Jeff: But then they also say, numerous scientists come up to me and say, you have the ideal job. And I say, what's that? He says, "Well, you figured out how to go and independently fund your research and have fun and be outside of this crazy business world of academic."

Adam: They all wish they were you.

Jeff: Yeah, well, some do, I wouldn’t say all, but some people say you’ve found the best of both worlds. You get to do the science you want to do.

Adam: That's great.

Jeff: And it's good science, so.

Adam: Jeff, I'm looking for the next Treo having a hierarchical temporal memory.

Jeff: Well, that may be a little while, but we'll work on that.

Adam: Thank you. Thanks for coming.

Jeff: It was a pleasure, Adam. Thank you for having me.