The Marketing AI Show brings you a special episode from MAICON 2021! Learn some of the history behind AI at powerhouses including Google and Facebook.
You can listen now on your favorite podcast app, or keep reading for more for a summary and transcript of this episode.
In this week's episode, show host Paul Roetzer sits down with Cade Metz, technology correspondent at The New York Times, and author of Genius Makers: The Mavericks Who Brought AI to Google, Facebook, and the World.
This special episode took place during MAICON 2021 when Paul and Cade sat down for a fireside chat to discuss the history of AI.
In this episode, Paul and Cade discuss some of the key events that took place over the past few decades that contributed to where AI is today.
Through research and experiences, Cade shares:
[00:05:35] Details on the AlphaGo moment (and even more at [00:31:48])
[00:07:55] The significance of 2012
[00:22:19] Learning about Yann LeCun
[00:25:29] How OpenAI and GPT-3 fit into this
[00:41:09] Cade on “complicated feelings” about AI
Disclaimer: This transcription was written by AI, thanks to Descript.
[00:00:00] Paul Roetzer: I'm joined by Cade Metz of The New York Times and author of an amazing new book, Genius Makers. Cade, thank you so much for joining us at MAICON. So I have to tell you a story. I don't think you and I talked about this, but I had reached out to you in 2019 to keynote our original conference that was going to be in person and we couldn't make schedules work back then.
[00:00:28] But back then you hadn't written Genius Makers yet. But you had been at Wired Magazine and then you were at The New York Times and I'd followed your writing, I think since the AlphaGo, which I know we're going to talk a little bit about AlphaGo today and DeepMind, but I'd followed your writing very closely.
[00:00:44] So then you came out with Genius Makers in March of 2021, right? It came out earlier this year. So I was on spring break in the first week of April. And I was actually listening to the book as I was walking around on vacation. And I've been studying AI since 2011. And I've read pretty much every book you could read on the topic.
[00:01:07] But I'm coming out as a liberal arts writer, I came out of journalism school. But for these 10 years, I've been trying to comprehend why we weren't further along. So when I started following AI in 2011, nobody was talking about AI and marketing and sales. It was in academia and it was just starting to be commercialized a little bit with IBM and some other major players.
[00:01:29] So I started researching and writing about it and all the time, trying to figure out why isn't this a bigger thing? Why isn't the marketing world moving faster? And as I'm walking around, listening to your book, I realized that it was because most of the innovations had just started to happen in the last nine years.
[00:01:49] And I'd read that before in different books, like different stories of Hinton and ImageNet, which we'll get into a little bit. But for some reason, when I heard your stories, these kinds of firsthand insider stories of how this all occurred over the last nine to 10 years, my world just changed. Like it literally just changed my perspective on the significance of what's happening.
[00:02:09] So I just wanted to first thank you for being here to tell this story, because I think your book is the most comprehensive I've ever seen in terms of these inside stories behind the people who actually are building this stuff. So it's just an incredible book. And I just want to start by thanking you for being here to tell this story.
[00:02:28] Cade Metz: I appreciate you saying that. And what interests me the most is that it changed your perspective. And that's what I aim to do with this book. There has been so much hype around AI. The term gets thrown around so often, but few people understand what that means, but, you know, because it gets applied to everything and, you know, the irony is that as all this extreme hype has built up, there has been real change.
[00:02:54] But the trick is to separate the hype from what is actually happening. And that's what I wanted to do with the book, but also, just tell a good yarn, right? This is a story about some amazing people in some amazing situations. And if you can do both, hopefully that's an effective book.
[00:03:15] Paul Roetzer: So let's take a step back to your career journey.
[00:03:18] Because again, I came out of college in 2000. None of this stuff was in the public realm. I mean, I know that AI was being researched in 2000. But we weren't talking about this stuff early in my career. And so you were at PC Magazine from 1994 to 2007 covering computer-related tech, I'm sure covering the big players in this space. In those days were you researching AI or writing about AI back at PC Magazine?
[00:03:48] Cade Metz: Every now and again. You know, that period is what we often call an AI Winter. When people had lost faith in the field. You have this cyclical nature to the field that you see time and again, where the hype will build up.
[00:04:09] People get excited, research intensifies, coverage from the media of the research intensifies, and then it doesn't live up to its promise. And then you sort of descend into this trough of disillusionment. And that was one of those times. And so every now and again I would write a piece, but things really changed when I got to Wired Magazine around 2012.
[00:04:38] When you start a new job like that, often you're pitching your potential editors about what you're going to cover. Like, what are the big themes going to be? Well, AI was not one of the big themes that I pitched, but soon after I got there, it became one of the primary things that me and my team covered.
[00:05:00] Because that change has suddenly happened, it was right around there when things took off. And we can talk about why that happened and what the ramifications are, but it's right there, 2012.
[00:05:11] Paul Roetzer: Yeah, 2012 a lot changed as you kind of talk about in the book. So at what point did you decide to write the book?
[00:05:21] You know, when did you start looking on say, okay, there's a bigger story to be told here about AI beyond just the superficial stuff and the stuff coming out of academia? Like there was a real story and real change was starting to occur. When did you see that coming?
[00:05:35] Cade Metz: Well, it was right after the so-called AlphaGo moment.
[00:05:39] So 2016 when the London-based AI lab, DeepMind, built this system to play the ancient game of Go. And they took it to Seoul, South Korea. And I was lucky enough to be there for this moment when that machine beat one of the world's best Go players, really the best Go player of the past 10 years, a guy named Lee Sedol.
[00:06:01] That was something that most people in the Go field, most people in the AI field, did not think would happen for decades. And it happened in this remarkable way that I wrote about for Wired as I was coming back, having seen what happened with the technology, but also having spent time with people like Demis Hassabis, who led the DeepMind lab, continues to lead the DeepMind lab and others. I resolved to write a book about them.
[00:06:30] And to tell this story through the people. Any good story is about people, whether you're writing about technology or anything else. The story as I started to build it, and even after I pitched it to publishers, it hadn't been accepted, continued to evolve and it became a richer and richer story as the technology improved and as we started to see what it could do and what it couldn't, the problems that it could cause; it became an even better story than I thought, but it was after Korea that I decided to write it.
[00:07:08] Paul Roetzer: So I definitely want to come back to AlphaGo. If you haven't seen the documentary, AlphaGo, Cade's in that documentary, but it's just a remarkable moment in human history. It's a remarkable story that's told. Demis is one of the two. There are a lot of players in this book. There are a lot of stories being told, but Demis is certainly one of the core figures of the book, along with a guy named Geoff Hinton, a University of Toronto professor and AI researcher.
[00:07:35] And you start the prologue off with the story of Hinton and that kind of 2012 moment you had referred to earlier. To really draw in the reader and kind of make you realize that there's way more to this than a bunch of technology and digital transformation – a term thrown a lot around, a lot in marketing.
[00:07:55] So take us back to 2012. What happened with Geoff Hinton and his team, and what was the significance of that moment in, not only AI, but potentially in human history?
[00:08:08] Cade Metz: Well, the significance is multilayered. In one sense, you had a technological advance. Geoff Hinton, who was a professor at the University of Toronto, and two of his students built a system to recognize objects and images.
[00:08:24] That's something that computer scientists have struggled and companies have struggled to do for years. Just being able to recognize your face as we look at this image of you, recognize cats or dogs or whatever else in an image…that's a very, very hard thing to do. And as the years passed, as the decades passed, machines got better and better at that, but they weren't even close to as good as a human who can recognize things in an instant.
[00:08:57] Well, Geoff and his two students built a system that could do that with an accuracy that no one thought was possible. It's similar to that AlphaGo moment. It's something that happened and people didn't think would happen for years. What's even more remarkable though, than them building this system that works so well, is that some of the biggest companies immediately recognized the significance of this, and Geoff realized how interested they were. And what you have at the beginning of the book is this moment where he auctions his services off to the highest bidder. And you have Google, Microsoft, Baidu in China, one of the largest internet companies in China, and DeepMind, by the way, Demis's Hassabis’s lab, who would end up building that Go machine, all bidding for Geoff's services.
[00:09:54] Paul Roetzer: Through a real wild email bid process, it’s like reading a novel - those pages where you're telling the story of how that all unfolded.
[00:10:02] Cade Metz: It’s a remarkable and crazy story. And you know, I often tell people that if you're writing an article for The Times, or if you're writing a book, the struggle is often to decide how you're going to begin.
[00:10:14] Sometimes you spend weeks to start, how are you going? Well, once I had this story, the book just had to begin there. There was no other option because it lays out everything that's going to happen over the next 10 years. All the big players are already there from the scientists, Demis, Geoff, and Geoff’s two students, to the companies. Google is there, Microsoft is there, China, and this may surprise some people is a player from the very beginning, in the form of Baidu, who’s also part of that auction. It’s remarkable how that amazingly dramatic story is a microcosm of everything we've seen in the 10 years since.
[00:11:02] Paul Roetzer: So we hit an inflection point where you talk a lot about this idea of neural networks and that's a term, again, a lot of our audiences: marketing managers, directors, VPs, CMOs…you don't hear neural networks too often in the marketing realm, but it's a term that I think has become known enough now within the public realm that you're probably at least familiar with the concept.
[00:11:23] And that's really the story he tells: this idea of these neural nets, going back to the 1950s and 60s and really that there was this belief by a very small group of scientists and researchers who thought they could, in essence, emulate the human brain. Find a way to think the way the human does to make possible all kinds of new, amazing things.
[00:11:44] And then it just doesn't happen for what? 50 years. And so that moment with ImageNet with Geoff and his team, all of a sudden proves these theories from 50, 60 years earlier, that this actually might be possible. So there's an inflection point from technology, but as you alluded to, you talk a lot about how it also then triggered the arms race for talent and technology within the major players in the space.
[00:12:13] Cade Metz: That's right. That's why those companies are bidding so much for Geoff and his two students in that moment in 2012. It's not just that they built something that can recognize objects and images. That alone is a powerful thing, right? That ended up driving self-driving cars. That's how self-driving cars see the world around them.
[00:12:32] They can recognize pedestrians or street signs or whatever else through a neural network. But the idea is more powerful than that. It's the same idea that now recognizes spoken words, when you speak commands into your iPhone, it's what is helping to drive drug discovery today. There's a big result recently from DeepMind, the same lab that built that Go machine, that could help us develop new medicines and treat disease.
[00:13:04] The list goes on. You now have systems that can better understand natural language, so the natural way you and I talk, and may eventually lead to chatbots, which can carry on a conversation. All of this and more is driven by this one idea, a neural network. And as you said, it's an idea that dates back to the fifties, and never quite worked. That's what happens to the book.
[00:13:28] You have this moment in 2012, where the idea starts to work and then you flashback to the fifties and you see this idea struggle over decades. And you see the entire tech community time and again, lose faith in this idea to the point where most of the community thought people like Geoff Hinton were nuts for continuing to work on it.
[00:13:52] And then it finally works. It’s often a great story: people who might seem to be crazy, but in the end, ended up having the idea that can make a difference.
[00:14:05] Paul Roetzer: And so from a timing perspective, to bring this back to how we started this, and my own kind of realization in April of 2021 after 10 years of studying this space, this is late 2012, that this is happening.
[00:14:19] So we're talking about less than nine years ago that the arms race just starts. So when you look across marketing, thousands of vendors many now claiming some forms of AI and machine learning and natural language generation, all these terms that as marketers, we're starting to hear. It really wasn't until that moment in 2012, where people started to believe you could commercialize this technology and the race was now on to acquire that talent.
[00:14:46] So Geoff Hinton, rather than just taking a job at Google for a couple million a year, forms a company with his two researchers and sells it for $44 million. But he had been working on this prior at Microsoft in 2009.
[00:15:08] Cade Metz: That's right. So he actually had it working for speech recognition before this moment, 2012. And he was inside Microsoft building a system that could recognize spoken words with an accuracy that had previously been impossible. So you could see it work with speech, but still people said, all right, fine.
[00:15:30] It works with speed, it’s just not going to work with anything else. Then in 2012, it works with images and that's when people really woke up. And I think it's worth pointing out very briefly why this idea is, in the end, so powerful and has proven to be. And that’s that it learns these tasks on its own, right? For decades, the way we build artificial intelligence or any other technology, for the most part, is if you wanted to build, say a speech recognition system, you put thousands of engineers into a room and you had them line of code by line of code, rule by rule, try to define all the words that we speak. Each little piece they had to define or try to define, to try to tell the computer what to do in each microsecond.
[00:16:23] A neural network is powerful because it could learn that task on its own. You give it thousands of hours of spoken words, say technical support calls, and it analyzes those. And it identifies the patterns in those words that you and I speak, and it learns to recognize those sounds on its own.
[00:16:48] That is always going to be faster when it comes to developing the technology than having the engineers do it by hand. And that's why we've had all these gains over the past 10 years in all those areas I talk about.
[00:17:00] Paul Roetzer: You told a great story about Microsoft, because again, Hinton's first real play within all the major tech companies was to go there and work on their technology.
[00:17:10] But they'd been working on that for 20 years. It wasn't like Microsoft just realized AI was a thing, they had entire teams dedicated to doing this. And I think you told the story of the one guy who basically sat in the room and watched the demonstration and realized that 20 years of his life had just gone to waste because this thing in two weeks was able to do what his team had done in 20 years.
[00:17:31] It was a remarkable
[00:17:32] Cade Metz: It was a remarkable moment. It's got him. Chris Brockett’s a great guy. He's originally a linguist, and this is the type of person that was hired on to try to build a system that could understand natural language, not a computer scientist, a linguist who tried to define every aspect of language for the machine, rule by rule.
[00:17:50] He spent years and years and years doing that. And then he's sitting at the back of the room for this demo of these two researchers who built something in a matter of weeks that could outperform the stuff that he had spent years on.
[00:18:08] And he was literally taken to the hospital. He thought he was, he thought he was having a heart attack. It turned out to just be an extreme panic attack. But that shows you how things can switch in this field, when you get something new and different. That was actually an old technology, even before neural networks started to work.
[00:18:33] They have advanced things even more than that very simple idea that he witnessed.
[00:18:39] Paul Roetzer: I think there's just so many stories like that--kind of these characters that show up for a few paragraphs, but it just brings home the point that you're trying to make. And I think throughout the book, there's this constant battle between the connectionist and the symbolist. So the symbolists write all the rules, you have to teach the machine intelligence. The connectionists believe it can learn on its own. So there's that battle. Then there's the battle between the people who want AI to be good, and the people who want to make profits with the AI, and there's just this constant struggle.
[00:19:16] So let's go through a couple of the other characters, the real people within the book that really help kind of tell the story of where you started and kind of where AI started, its origins, and then lead us into where we are today. So you start off very early with Frank Rosenblatt. Who was Frank Rosenblatt?
[00:19:38] Cade Metz: He was a Cornell University professor in the 1950s, who believed in this idea we've been talking about: the idea of a neural network. He actually built a working system, and demonstrated it for my current employer, The New York Times, back in the late fifties. It was a very simple system at the time that could recognize large printed letters.
[00:20:06] So a printed letter A or a printed letter B, and it worked. After it worked, he told The New York Times among others, that it was going to work with everything else, meaning it was going to recognize sounds and recognize images and end up somehow building itself on an assembly line and flying into space and doing who knows what. All this was reprinted in The Times.
[00:20:31] He was a little bit ahead of his skis there, which shows you what can happen in the AI field. As things work in simple ways, it's very easy to extrapolate and see it working in others. But then you had that AI Winter set in eventually, when the world realized that this idea was not going to live up to these sort of grand promises that he and others had laid down. And most of the field switches to other ideas and, and switches back to that symbolist idea that you discussed that rule by rule.
[00:21:12] Cade Metz: You see this happen time and again, over the years where there's a little bit of progress, and then people move on. The situation was that, although the neural network idea was sound, we now know in hindsight, we didn't have the volume of data and the computer processing power we would need to make it work.
[00:21:34] So in order to analyze all those sounds, analyze all those images, you need a lot of that. You need a lot of sounds, a lot of images and you need a lot of computing power. To go through it all and learn those patterns.
[00:21:47] Paul Roetzer: When we come back around to Facebook and Google and Amazon, you start to realize now why what they're doing is possible.
[00:21:56] So back in Rosenblatt's days, you had the antagonist Marvin Minsky who, if I'm not mistaken, writes a paper basically debunking Rosenblatt's perceptron, and that, if I remember correctly in like 1969 to 1971, in essence stops all research, the academic papers stop, the citations of the paper stop.
[00:22:19] And then we really go into the mid-1980s now before some new AI researchers sort of pick up the baton and that's where we get introduced to Yann LeCun. So who is Yann LeCun?
[00:22:33] Cade Metz: Well, he was an engineering student in France, and in the mid-eighties, Geoff Hinton and Yann LeCun at the same time, started working on basically this missing piece for a neural network.
[00:22:51] It worked well, as we said, with printed letters back in Rosenblatt’s days, but it couldn't do much more than that. It was missing this mathematical piece. Geoff Hinton, that main character in the book we talked about, who had ended up really changing things 2012, found this piece at the same time, Yann LeCun, this student in France was doing something similar and they end up meeting.
[00:23:19] Yann LeCun goes to the University of Toronto, does a postdoc with Geoff and they become part of a group of people that revived this idea in the eighties, and you do see a revival and it starts to work in new areas. You even have, at Carnegie Mellon, you have grad students there build an early version of a self-driving car using this idea. It can see the road around it, learn to see the road around it, using this idea of a neural network. So you see some real progress before we enter into another trough in the 1990s and 2000s
[00:24:01] Paul Roetzer: And LeCun, if you're following along and recognize that name, he now heads up and founded the Facebook AI Research Lab, FAIR, with Zuckerberg in 2013.
[00:24:16] So again, you have the ImageNet moments where Hinton's team proves the viability of what he had now coined as deep learning, kind of rebranded neural networks as deep learning, and Facebook jumps in and recruits LeCun to come and build a research lab. And now all the big companies are realizing is you have to have research labs that are working on interesting things to attract the top minds to advance what's going on.
So Zuckerberg starts making his bets on AI. Around the same time you've got Google Brain. So the Google Brain emerges from Andrew Ng, the founding lead of Google Brain. You may know that name as the founder of Coursera as well, and also his time at Baidu.
[00:25:06] So you start to see that these names are kind of all emerging and yet they also sort of come from the same tree, not only from the academic tree, but you start to see the influence of people like Elon Musk and Peter Thiel from an investing standpoint. And so Musk goes on to help found OpenAI.
[00:25:29] Now let's talk a little bit about OpenAI for a moment and Sam Altman and his team, because we start to really move into the realm of things that are having an immediate effect on marketers, because one of the major innovations from OpenAI is this concept of GPT-3, which if you've attended other sessions during MAICON, you've heard about a machine's ability to generate language.
[00:25:53] So tell us a little bit about Sam Altman and OpenAI and kind of what they're trying to do there.
[00:26:01] Cade Metz: As you described it in the wake of Google and Microsoft and Baidu bidding for Geoff Hinton’s services, and him eventually going to Google for $44 million, that set a high price for the talent.
[00:26:15] The reality was that there were few people on earth who were working on neural networks at the top. And then it starts to work. So it’s just basic supply and demand, right? You’ve got a small community of people who know what's happening and you have enormous demand for their talent. And so one by one, you see these people go to the big companies for enormous dollar figures. And as that happens, Elon Musk and Sam Altman, two big names in Silicon Valley, they want to get in on this. And there's this key meeting they have at the Rosewood Hotel in Silicon Valley, which is sort of a famous hangout for venture capitalists, where legendarily so many deals go down, right on Sand Hill Road, the main stretch for VCs and Silicon Valley. They meet to discuss whether or not there's still an opportunity for them to create their own lab, whereas all the talent with the big company. In a lot of ways it was, but they pried some key talent away from other players and built this new lab to go after the same sorts of goals as DeepMind, that London lab, which by the way, was acquired by Google the year prior, 2014.
[00:27:38] Cade Metz: So as part of that rush for talent, you have this new player led by Elon Musk and Sam Altman come in and you're right.
[00:27:49] They’ve been at the poor front of this effort to build systems that can really understand natural language, GPT-3, as you call it, being the primary example. It's a remarkable system that can generate its own tweets, write its own poetry, you know, imitate Donald Trump or Joe Biden.
[00:28:11] You know, it's not perfect. I do want to make clear that it does all those things, but not every time, but it's a remarkable system that can feed so many other technologies, and it is already doing so.
[00:28:27] Paul Roetzer: So let’s go back. I want to spend a few minutes on, you know, really the other main characters.
[00:28:32] We talked about Hinton and we touched on Hassabis, but I have for a couple of years now, actually, after the AlphaGo documentary and reading your stories from it, I had come to believe that he was potentially going to be the most important person of our generation.
[00:28:51] Once you understood what he was setting out to do and once they had the resources of Alphabet/Google behind them, it's hard to not think he's going to have a massive impact on society. So in 2010 Hassabis forms DeepMind with two other guys and their stated goal is the creation of artificial general intelligence. So take a moment and just explain to us what does that mean?
[00:29:18] Like what is artificial general intelligence and how is it different than the kind of artificial intelligence we have today?
[00:29:27] Cade Metz: Basically it's an effort to build a machine that can do anything the human brain can do. You know, that term artificial intelligence has been thrown around for decades, and like we said, applied to everything.
[00:29:39] They wanted to show that their ambitions were bigger than that. They wanted to build a truly intelligent machine. And that's the stated aim of OpenAI, by the way. Altman's lab wants to do the same thing. It's an incredibly ambitious goal. I do want to underline again this point that these folks, Hassabis, Altman, the people working for them, they don't necessarily know how to get there.
[00:30:15] We don't know how to build such a system. That is their ambition. That is truly what they're aiming to build. We'll see if they get there anytime soon. It's likely not going to be soon. It's such a difficult task. Many times that notion is overblown. We're not as close as some people would lead you to believe.
[00:30:39] But it's a big part of what happened over the past decade and why Google, for instance, paid $650 million for DeepMind lab, why some of those folks left some of the big companies and went to OpenAI to do the same thing. It’s something that drives a lot of people, that notion.
[00:31:03] Paul Roetzer: So 2004. So they founded DeepMind in 2010. They sell to Google in 2014, and there are so many facts in this book. Like, I don't know if they were ever in the public before, but I certainly hadn't seen them, such as Facebook offered double what Google paid for DeepMind, but Hassabis didn't trust Zuckerberg, basically, and the ethical use of what they intended to create.
[00:31:24] So they didn't even really entertain the Facebook offer. And also how Page learned about DeepMind on a private jet flight when he heard other billionaires talking, it's just such a wild thing. So, they get bought in 2014. And then I know you were there for the Go match with Lee Sodol in South Korea in 2016.
[00:31:48] Let's take a moment and just explain that experience. And again, the significance of it. It’s almost like we had the ImageNet moment in 2012, then we had the AlphaGo moment in 2016, arguably bigger potential impact. What happened there and what was your experience being, in essence, a participant in this event?
[00:32:11] Cade Metz: One of the reasons it was so powerful is it's something that anyone can relate to. Right? We're all games players on some level, you know, whether it’s board games when we're kids, or for anything else. This was a moment, that like I said, most of the AI world and the Go world didn't think would happen, that you could build a system that could beat the world's best players. Go, for those who don't know, is exponentially more complex than chess. There are more possible moves on a Go board than atoms in the universe. So it's not like chess where you can build a rules-based system and sort of look ahead to the end of the game and beat a human that way. You have to play Go by intuition.
[00:32:59] Top players talk about this phenomenon. Sometimes they just play by feel. So in order to beat the best players, you've got to build a machine that at least mimics that, and people didn't think that was possible. Well, DeepMind built such a system and they took it to Korea to play Lee Sedol, one of the world's best players.
[00:33:19] They often refer to him as the best player of the past decade. What you also have to understand is that Go is a national game in Korea as it is in Japan and in China. So you had an entire country and large parts of the continent focused on this match.
[00:33:43] Paul Roetzer: 200 million people I think you cited as bigger than the Super Bowl.
[00:33:46] Cade Metz: Exactly. And as the match would ebb and flow, you could feel this whole country ebb and flow. As the machine took the first two games and then a third to win a match, you could just feel the sadness envelop the country, and then when Lee Sedol came back and won the fourth game, in a way matching the machine in a way, learning from the machine, that was an equally remarkable moment. You could feel the elation spread throughout the country. It was a way for the layperson to understand what was happening.
[00:34:28] This was a system driven by neural networks that could learn to play the game on its own and learn to get that good on its own. So basically they build a system that learned from human Go moves. And then once that was built, they pit the machine against itself and it played millions of games against itself learning the sort of skills that would allow it to beat one of the world's best humans.
[00:34:57] Paul Roetzer: And in essence, what it's doing is calculating probabilities. It's looking at predicting what the human would do. It's making moves based on what it thinks are the greatest probability of winning. So I know there's the famous or infamous, depending on you look at it, move 37 in game two.
[00:35:16] Which again, if you watch the documentary, I cried, like, I mean, there's just this moment when you see what it does and the impact it has on humans. And I've watched the documentary probably five times and that moment still gets me. So just take one moment to explain what happened with move 37 and what was the significance of that specific move?
[00:35:36] Cade Metz: So like I said, the system initially learns from human moves, okay? So it knows what a human would do in a given situation for the most part. Okay. When it reaches that moment and game two, it knows if it plays that move, which we now call move 37, if it plays that move, that's not a move a human player would make.
[00:35:59] There was a one in 10,000 chance to calculate that a human would make that move, but it made the move anyway because it has trained beyond what a human could do. It's played all those games against itself. And then it decided that was a good move despite the fact that a human would never play it. That was a remarkable moment.
[00:36:20] And that's something that was remarkable even to the people who built this. They were amazed by this. They were amazed, you know, the next day when they looked at the figures and saw that there was a such a slim chance that a human would make that move and it did it anyway. It was something that no one in the Go world expected and certainly the designers of the system didn't expect it.
[00:36:48] These are not Go players, right? These are computer scientists. They built something that plays well beyond what they are capable of. That's a remarkable thing.
[00:36:58] Paul Roetzer: So when you set out the write this book, just reading the book cover alone, you can realize this is not a story necessarily about the technology itself.
[00:37:08] It is a much bigger story about the people, their goals, the impact these goals can have on society, the people who are good-intentioned with their AI research, the people, the companies, the governments who maybe are bad intentioned or who are making decisions based on if we don't do this, someone else will.
[00:37:26] And not always asking the right questions of the research they're doing. But there was a couple of lines in your introduction in the book that said “what does it mean to be smart, to be human? What do we really want from life and intelligence we have, or might create?” And I was just curious, did you find the answers to those questions in your research?
[00:37:43] I mean, you've interviewed hundreds of people for this book and then over the nine years prior, do you feel like you've gotten closer to those answers or that the people doing the research have gotten closer to those answers?
[00:37:57] Cade Metz: Well, yes and no. I'm certainly closer, but it doesn't mean I'm close, if you know what I mean. Right? What's so remarkable about the book is that the opinions vary to such a great extent. These are individual people with their own aims and goals and opinions and views of the world. And you get this incredible spectrum. Some people who think that AGI is around the corner and it's going to destroy us.
[00:38:36] Paul Roetzer: Elon Musk is a very prominent supporter of that idea.
[00:38:40] Cade Metz: Exactly. And he's not alone. But then you get people on the other end of things who see things very, very differently.
[00:38:48] Paul Roetzer: Zuckerberg? So again, names you would recognize like there are very prominent people who don't agree on this stuff, even remotely close.
[00:38:58] Cade Metz: And you see that in the book when Zuckerberg and Musk sit down for dinner at Zuckerberg's house.
[00:39:03] And this thing gets hashed out real time. What I do know is that we have real progress and you can see the progress with each passing month. Just this month OpenAI, the lab we've been talking about, has released a new system specifically designed to write its own computer programs called Codex.
[00:39:33] And this is a remarkable thing that it can do this, that you can in plain English say, “Write me a program that will make it snow on a black background.” And it does it like that. And you can see it appear in an instant. That's a remarkable thing. But the flip side of that is that, you know, you could ask it to do something else, a moment later.
[00:39:58] And the code won't exactly be right, and it won't run. Right. And you sort of have to roll the dice. So maybe five times out of 10, a system like this will get it exactly right and really wow you. The other five, it's still flawed. And what it really lacks is the ability to reason. Which you and I have. Machines don't have that, and it's not clear how we’re going to get there.
[00:40:26] Or even if we should. These questions about whether or not we should build the technology are valid. Even this simpler technology that starting to work…it creates all sorts of problems, as you see in the book. One after another, concerns are raised because of this one idea that it started to work.
[00:40:50] Paul Roetzer: I saw you tweet in response to a forbes.com review of the book: “I get excited when people see the humor in my book, but I am far more pleased when they see the humor and the sadness and the complicated feelings in between, including my own.” What do you mean by that?
[00:41:09] Cade Metz: Well, me, like so many people in the book, sees where this technology has gone wrong and can potentially go wrong in the very near future. Whether it's autonomous weapons, disinformation, these are systems that not only recognize things like spoken words or images, they can create their own. They can create an image of a person that looks like the real thing.
[00:41:42] They can create a version of your voice. It sounds like you. They can put words into your mouth. That is a potentially very, very dangerous thing. We've all seen the effects of disinformation online over the past several years. That's disinformation generated by humans. What happens when you can have a machine generate that on an infinite scale?
[00:42:07] Then it really becomes a problem. How do we deal with all these things? These are things that people who are closer to these ideas struggle with, the people who are closer than I am. You know, they spent their careers building this technology and they question, in the end, how will it be used, how can they prevent it from being used in the ways that they don't, especially when it's in the hands of these very, very large public companies with so much money at their disposal.
[00:42:41] Paul Roetzer: And unfortunately, there's a number of great books on this topic, but the whole basic premise is, if we don't someone else will. So if Demis and his team truly believe that by building AGI, they'll learn to control it. But if someone else gets to AGI first, without those constraints built in, then it might go wrong, then they feel it is truly like their mission in life to achieve this.
[00:43:03] Now, could it lead to massive negative implications? Absolutely. But their belief is that if they don't do it, someone else might get there first; maybe a foreign government with bad actors. And so there are some people that certainly do it for profit.
[00:43:23] But it seems like many of these leading researchers we talked about today, are doing it to prove it's possible. I mean, academic people spend their lives researching theories, like physicists spend their whole lives on theories that they may never prove to be true, but it motivates them to see that. Others do it for commercial reasons.
[00:43:43] But it seems like some of these key people are truly doing this because they think this is going to happen. This artificial general intelligence will occur maybe in their lifetime. And if they can get there first, then they can figure out how to control it.
[00:43:58] Cade Metz: It's true. And it's a real phenomenon. It's a real belief. You know, I talk about it in the book as a belief. This is not necessarily posturing from the scientists. This is something they really believe in, that they think is going to happen and that it could be a danger and that they need to build it in a safe way in order to prevent that from happening.
[00:44:26] Paul Roetzer: I mean, I could literally sit and ask you like 20 questions about every chapter of the book, but I want to pay attention to the time here. And I think a good way to maybe wrap up this conversation is to bring this back down. We started with some very practical conversations around the use of deep learning and voice assistance and language generation, and Microsoft.
[00:44:45] We didn't touch on this, but Microsoft invested a billion dollars in OpenAI to license the language generation technology. And this stuff is very real. AGI it may happen in a year. It may never happen in our lifetimes. We don't know, and neither do they. I think that's the key. You touched on Go. Most AI researchers in 2015 didn't think it was possible. You tell the story of Facebook having this sort of like moonshot to do this. And then here's Demis's team like three weeks later, like “yeah, we did it already.” They don't know when these major breakthroughs are going to happen, but what we do know is deep learning has, for the last nine years, proven it can be applied to vision, to computer vision, image recognition,
[00:45:29] it can be applied to language with speech recognition, natural language, generation, translation, text to speech, speech to text…there are all these practical applications. What would be your takeaway for the marketer, for the business leader who maybe, like Microsoft in 2009, just kept thinking like this isn't real, this stuff’s not going to apply to me…”
[00:45:50] You've done the research. You've written all the articles. You've interviewed all the people. How real is this technology right now? And what impact do you think it's going to have on the real world? The real business world in these coming years without major leaps forward? Again, the real tech we're already seeing?
[00:46:11] Cade Metz: Oh, it's, it's very real. But you know, it's worth describing how it is and will be used. I was just talking to someone today who runs a company that helps libraries and newspapers deal with their vast archive of photos. This is a technology that can go through hundreds of years of photos, identify people, tag the photos, help organize them, and help you sell those photos to websites or other newspapers or whatever else.
[00:46:49] That's a real application. And there are countless ways this technology can be used. We're seeing it, like I said, drive the rise of driverless cars. That rollout is going to be slow. Again, a machine can't reason like a human. So that's why we don't see them everywhere, but the technology needed to identify what's going on on the road has improved by leaps and bounds over the past 10 years.
[00:47:16] And you're going to see that technology continued to improve. And more and more, you're going to see it on the roads, other types of robotics as well. You're seeing this in warehouses where you have systems driven by neural networks that can learn.
[00:47:37] To identify objects and pick them up and place them in a new bin. This is what goes on every day in an Amazon warehouse, where they're struggling to hire people to do that. Machines increasingly are able to do that. There are so many areas where the labor is needed and the machine is starting to provide it.
[00:48:01] You're going to see this on the internet. You're going to see it in the real world. We're already seeing it both places.
[00:48:09] Paul Roetzer: And I know you touched on it in the book, but this whole idea of–and we've talked about throughout the conference–that the personalization and convenience you experience with Netflix and Spotify and Google Maps and GMail and Amazon shopping and all of these things, none of it happens without the advancements of the last nine or 10 years; that these deep learning kind of neural nets, they are embedded within everything you're using anywhere personalization is happening, voice recognition, machines being able to talk, it's all possible. And in business and in marketing, we just haven't got to that inflection point yet where it's seamlessly integrated into everything. And that, to me, when I read the book, it was just like, it's coming.
[00:48:47] I always wondered why in 2014 & 2015 all the major marketing software companies weren't infusing AI into everything. And when I read your book I was like, “Oh, because Google hadn't done it yet.” AWS wasn't the powerhouse it is today. There weren't machine learning algorithms off the shelf we could get from Facebook and Microsoft and Google, but that's changed.
[00:49:11] And you mentioned the data is there. The computational power with Nvidia GPU and Google TPU: we have the ability now to realize the promise of AI from the 1950s, what those researchers back then theorized, we might not be building the human brain or recreating the human brain and giving us, you know, reason and logic and consciousness.
[00:49:33] And like, we're not saying that's gonna exist to machines, but the tech is there now to have real impact on people's businesses, on their careers. And so, you know, just in closing, if you haven't read Cade's book, I've got it here. And I've read it twice now. Read it–I promise it will change your perspective on what's possible.
[00:49:55] You'll loo at the current state of business and society differently, and you'll look at what's possible in your own career differently. So, um, Cade, I just want to, again, thank you so much for your time, for your insights, for your articles and all the research and running you've done in the recent years, that's helped me.
[00:50:13] I'm sure it's helped others like me, but I'm just so grateful for your time and insights and that you were able to join us as a part of this conference this year.
[00:50:22] Cade Metz: Very glad to do it. I enjoyed the conversation. Thank you.
[00:50:24] Paul Roetzer: I look forward to the future stuff I wanted to ask what's next, but I'm going to save that for our next conversation.
[00:50:30] Thanks, Cade. Thank you again. And thanks to everybody for being a part of Marketing AI Conference 2021.