The Marketing AI Show—the podcast that helps businesses grow smarter by making artificial intelligence approachable and actionable—has officially dropped!

You can now listen to the first four episodes of the podcast on your favorite podcast app. Keep reading for more on what to expect in episode two.

Episode 2: Christi Olson, Head of Search Advertising, Microsoft 

Christi Olson is the Head of Search within the Global Media team at Microsoft, leading all paid search efforts for the company. She has over a decade of experience in digital marketing leading both in-house and agency teams across the retail, travel, automotive and consumer electronics industries. 

She has been recognized as one of the top 25 most Influential Paid Search experts annually since 2014 by PPCHero, and additionally, is a recognized global speaker on Digital Marketing Strategy, Search, and Cognitive AI. She is a published author in the Applied Marketing Analytics Journal and regularly contributes to Forbes, AdAge, Marketing Profs, Search Engine Land and Search Engine Journal.

In this episode, the main topic is conversational AI. Christi describes conversational AI as the technology that can speak and listen, allowing anyone to engage with it.

Paul and Christi discuss the prevalence of conversational AI in our everyday lives—from voice search to chatbots and how they affect consumer behavior. She also answers several questions she gets from her marketing peers: 

• When it comes to voice search, where are the answers pulled from? And how does that affect my brand’s ability to show up?
• What is the difference between voice search and digital assistants?
• How can marketers get started with conversational AI? And what role will it play for the future of marketing?

[Video] Watch the Full Interview 

Read the Full Transcript

Disclaimer: This transcription was written by AI 🤖, thanks to Descript. 

[00:00:03] Paul Roetzer: Welcome to the marketing AI show. I'm joined today by Christi Olson, head of search within the global media team at Microsoft, where she leads all paid search efforts for the company, which is a new goal, which we're going to talk about in a minute. Welcome Christie.

[00:00:17] Christi Olson: Thank you for having me. I'm excited to be here.

[00:00:19] Paul Roetzer: Thanks so much for doing it. So we're going to focus on conversational AI today, but you guys have this great new ebook, I guess we'll call it marketing with a purpose. And I definitely want to spend some time on that. I feel like that could be based on how deep that goes. It probably could be a series of episodes just about each topic.

[00:00:35] So we'll try and touch on some of those topics today, but, uh, why don't we start it with head of search, that's new. So as our lifetime of knowing each other, I think you've been head of evangelism or the last few years. And you've moved into this new role. Tell us a little bit about what your focus is now.

[00:00:51] Christi Olson: Yeah. So for the last five years at Microsoft, I was the head of evangelism, which was a lot of an external facing role going out and speaking at conferences and events and with. [00:01:00] Companies, I would say fortune 1000 companies across the globe as to how does search and digital marketing fit into their strategy and fit into what they're trying to accomplish.

[00:01:12] As a business. I have been in paid search for almost 20 years now and are paid and organic search, and they had an opening to actually lead all of Microsoft's internal search efforts. So I'm running the team that does all of search on behalf of the company, um, globally, which is.

[00:01:29] Paul Roetzer: That's awesome. Yeah. I mean, probably not as busy as, you know, traveling to 80 events a year, but it's busy from different different perspective.

[00:01:37] Christi Olson: Yeah. It's, it's fun. It's also interesting. Cause when I first started at Microsoft 2006, I was in a division in a similar type of role where we were running the internal search at Microsoft. And so it's sort of fun to come back and see how has this evolved over the last 15 years? How has it shifted and changed?

[00:01:54] But now I actually have telemetry over every line of business across Microsoft and global.

[00:01:59] Paul Roetzer: That's awesome. [00:02:00] Well, that's actually kind of gives me a jumping off point here 2006. So obviously our topic is artificial intelligence machine learning, like how it plays a role in conversational, but also search and paid.

[00:02:11] Was it was AI being talked about in 2006 when you first joined Microsoft?

[00:02:16] Christi Olson: Automation was but not AI to the level it wasn't really around 2000 and I'm trying to think of the exact year cause they all blend each other at this point. 2009 when we launched thing that we really started having the conversations about AI and the role that AI plays within the search space.

[00:02:34] Because up until like the 2009 timeframe all the algorithms for optimizing search were manual. So it was literally coders in the background are writing the algorithms. They weren't crying. They were using some aspect of neural networking and the algorithms, but it wasn't fully automated when we launched Bing.

[00:02:51] That was the thing is we went from having a core algorithm, engineer, and somebody responsible for components of the algorithm to [00:03:00] using deep neural networking and thousands of employees dedicated to AI, right? I mean, we have an entire division to AI and research and it's, it's fun when we talk organic search.

[00:03:13] People want to like, what are the ranking factors like? Well, for how we have, I think it was like 275 nodes within the neural network that count four ranking factors and each individual who searches has a slight different variation based on their history, their behaviors, how they've engaged with results, because our goal is to provide the most relevant results, right?

[00:03:34] Based on the query and how people engage and interact with the search result results. So it really is fully AI. Yeah. And I've heard, you mentioned like being is the biggest AI application within Microsoft. I mean, that's kind of how it's viewed is it really is an AI engine. Now. It is, it is fully AI engine.

[00:03:53] So, I speak, like I said, a lot on organic search and paid search. Right. And people will ask like the [00:04:00] algorithm it's like, well, there isn't. And, there are multiple algorithms are all part of different nodes. They're looking at the different factors that come in. Um, I think in, uh, the AI Academy, uh, you have myself and Frederick Dabu go actually go through and talk about the algorithms.

[00:04:17] And we talk about like, how has the algorithm shifted and changed over the last? Well, he goes all the way back to, I think it was like 19, the 1950s, but how, how has. AI has been infused into our algorithms over the last. 15 years, 10 years. And where we're at today, it's just a cool takeaway for the average marketer who may be new to AI and think they missed the boat.

[00:04:44] Paul Roetzer: And the reality is like you just said, even at Microsoft, you think about the big leaders in AI. There's like six of them and Microsoft's obviously one of them. And, and yet it was only 11 years ago that they weren't really doing it yet. And so like in [00:05:00] marketing as a whole, like as a profession, there's so much a lack of understanding and adoption of AI in specific use cases for like the daily life and email or content or paid search, but that's okay.

[00:05:10] Like this is still relatively new. AI goes back decades, but its use in marketing is still a relatively new concept. And so you're, if this is like maybe the first time you're even sitting down and listening to something about it, You're not that far behind, most of your peers still haven't figured this out either.

[00:05:28] Christi Olson: And it's interesting cause like I've, I've sat in multiple different business groups at Microsoft and depending on the group, you talk to depends on their level of sophistication with artificial intelligence, where some of them, it is very basic and is not fully automated AI there to the level of they're creating the algorithms to help them get to like lead scoring.

[00:05:48] And that is the level of sophistication they're at versus when you have like. Microsoft advertising and Bing as a search engine, they're full instruments of AI. And so you have different levels of sophistication [00:06:00] where I'm at now, the team, the team I'm on work, considered global media. We run the media on behalf of all Microsoft from display to social, to paid search.

[00:06:09] Um, it all sits under us. Each team has a different, a slight different application of AI that we're using. We aren't necessarily AI experts. We're not going and doing the coding on the backend, but we are leveraging AI tools and technology. And so even there. The level of sophistication, the level of knowledge and experience varies team to team person, to person.

[00:06:33] Paul Roetzer: So it's never too late. It's dive in and learn. And I think that's probably representative of what we often tell people your future is going to be like as a marketer, you don't have to build the machine learning algorithms. You don't have to know exactly how the algorithms work. What you have to know is what's possible and that there may be better ways to do what you do every day.

[00:06:46] And so your point, if you're managing paid or managing organic or whatever your role is to know, there might be smarter tools out there that can help you do that. Better, more efficiently drive costs down, drive performance up. Um, and so you're not doing go back to 2006, like the all human powered rule spaced approach.

[00:07:04] And that's really how we look at it as like traditional marketing. Traditional automation is okay. All human all the time. We're moving into a phase where it's  part of the time it's machine power, that there is this intelligence that can be built into tools where it takes some of that off of the marketer where they can actually do some of the repetitive tasks or some of the predictive modeling, things like that.

[00:07:23] Christi Olson: Yeah. And that's like, um, we're having the conversation within my team today is, uh, within paid search. There's some automation that can happen on the bidding side. Um, we're talking about, we, we use a tool today for a bidding and we want to actually take their tool. Add in the search engine layer of their automated bidding on top of the tool that's used today.

[00:07:43] So it is the AI layer. That's going to be on top of an algorithm that somebody else is managing. So it's the, how do you add that automation on top of the manual human work that is happening in the tool sets at this point in time, we, um, We are going down that path and probably starting that [00:08:00] next I'm hoping next month we are, we are working on the integration of getting the right level of data into the tool, sets on the backend to make sure that it can happen on a regular recurring basis.

[00:08:10] So it goes back to you have the crawl walk run, right? You're not going to run, but you can start with the crawl, right? And you don't flip a switch and everything just becomes intelligent and automated. This is, it is a process. It's a task one task, one use case at a time, basically. Yep. All right. So let's talk about conversational.

[00:08:28] Paul Roetzer: Yeah. Which was the topic we were gonna focus on today. Just the start with just the broad, what is it? I mean, we hear about it as a kind of a category of marketing now is conversational marketing, conversational sales and AI's role within that. How do you think about what conversational AI is?

[00:08:43] Christi Olson: Yeah. So how I think about conversational AI is its technology that can speak and listen, allowing pretty much anyone to engage.

[00:08:52] So conversational AI. It's the idea of listening and speaking and being able to respond, coming back. And when you break that down, [00:09:00] there's different components in different types of conversational AI that I would almost say that. Um, as I started talking about conversational, AI went all the way back to 2016.

[00:09:10] And when I was talking about this in 2016, not a lot of people had really been engaging with conversational AI today. I would say almost everyone has engaged in the last six months with some sort of conversational AI, whether you realized it or potentially didn't what would be applications of conversational AI?

[00:09:30] Paul Roetzer: Like what would be some examples where someone might engage with it?

[00:09:33] Christi Olson: So I had put it in two different areas. So on the speaking can listen, um, and would be a chatbot is a really simple, conversational AI interface. And right now today, a lot of customer service applications across websites around the globe are chatbots.

[00:09:48] Even some of the telephone trees. So IVR technology are using very basic. You, they ask you, do you want to talk to customer service? Do you want to talk to like, here's the phone tree that is also [00:10:00] a conversational AI. Granted it's not as sophisticated. Unfortunately the telephone side just has not kept up with the internet side.

[00:10:07] It makes me very sad. And the more sophisticated side is when you go down to Siri, Alexa Cortana, the Google assistant, where they are much more sophisticated. Um, and they're using a lot of different cognitive services to help them with those levels of sophistication.

[00:10:23] Paul Roetzer: So, Cortana, you mentioned maybe not as household have a name to the average person, but you know, it is, it's a very innovative application of AI within Microsoft.

[00:10:33] And it actually is probably embedded in more areas than people know. So what is, what is Microsoft play there? What is Cortana and how is it used and how we as marketers, maybe we interacting with it and I even realize it's part of the experience.

[00:10:46]Christi Olson: Yeah. So Cortana has actually changed and shifted over the last five years, four years.

[00:10:52] I'm trying to think since really we, I started engaging with our back 2016 in the Cortana team where originally we were at Microsoft doing [00:11:00] a direct compete to Siri, Google assistant, and Alexa. Right. So we did actually make a play to create our own series of smart speaker devices, similar to like the Google home or to the

[00:11:13] Alexa.  And they've sort of pulled back on that, that by the time we got into the market and maybe this is my, this is my opinion, this is not Microsoft, but have we gone to the market? They were. So those other two are so embedded that we just couldn't get the traction that we needed to say, okay, do we continue down the path of manufacturing devices on the day-to-day basis

[00:11:33] um, we had had a conversational language framework and we made it. So that framework does work with Alexa, Google. Wasn't interested in working with us directly for some odd reason, you would have thought, um, to speak, but essentially creating that conversational framework so that you can ask Alexa a question.

[00:11:52] It can go into the Microsoft ecosystem. So, um, I'm trying to think. When we created this, we had a [00:12:00] framework sort of, of productivity versus utility and function. And so Microsoft and Cortana tends to focus toward productivity. It tends to focus more on helping you get things done within the work stream versus Alexa.

[00:12:13] You asked to play music and engages with other IOT devices. Um, same with Google Home, Google Assistant they're into other devices. Ours is on the productivity side and that's where we've, we've moved from trying to be everything to. Every device to now focusing on productivity. So Cortana, if you have outlook, there is a way to use Cortana in the email to set up meetings work.

[00:12:34] We're trying to look at your calendar. If the other person's using outlook, it'll automatically schedule your meetings for you and it'll send notifications. So it's just a assistant on the backend to help you get things done quicker and easier. That's cool. You've talked a little bit about assistance.

[00:12:48] Paul Roetzer : So, in the course you created for us, the, how may AI help you, you talked about the difference between chatbots and digital assistance. Um, w what's the thinking there? What is a digital assistant, when [00:13:00] you think about like where this is all going, how do you think about digital assistance? And part of it might be what you just said with Cortana where it's actually truly there and predicting needs and like helping take actions.

[00:13:10] Christi Olson: Yeah. So I think on the, um, I think when we, in the course, I think I broke down conversational AI into sort of three areas in digital assistants, sort of bridges, all three of them. Okay. Um, where you talk about Alexa is based on skills and actions. So you are teaching that intelligent agent to do something.

[00:13:27] So you're asking. Alexa or the Google assistant to, to go to that device. Like for me in my house, it's go to my nest, adjust the temperature. So it is you're teaching it to do something that engages. It could be IOT or engage with a website. It can do something on that end, a chat bot facilitates and has a conversation.

[00:13:45] And there you can get to that predictive text. You can get to the predictive of helping accomplish a goal. Um, I joke Q and a and stuff. Question, answer in the age of conversational AI, it's questioning. Action because most people want to get to an action, not just the answer, they want the answer and then they [00:14:00] want to do something.

[00:14:01] And then you have voice search, which queries a search engine, and it's just using voice versus text and typing. And so a digital assistant can use all of those components together to help essentially be your virtual assistant and help you get things done, help you schedule, maintain and organize.

[00:14:21] Paul Roetzer: Do you think we're there, I mean, do you use any of them in terms of like actually helping to manage your life yet versus individual just tasks?

[00:14:29] Christi Olson: I do a little bit with Cortana, uh, more on the work-based. So a great example of this was I was hiring for my team back in October and I was using Cortana to help with. Coordinating scheduling the interviews and getting the documents and data together for the interview scheduling. So I literally automated everything through court, on us so that the meetings were scheduled.

[00:14:47] It pulled all the right documents into my inbox. When I'd go to have that interview, I had everything where I needed it. It was right there and it made my interview process, at least on my end. I don't know if it was as good for the applicants, but it made it much easier [00:15:00] for me to do this instead of having like 50 emails back and forth to coordinate so much quicker and easier.

[00:15:05] But I also use Cortana. Um, just for like, Hey Cortana, what's happening with my work day to day. So literally, um, it's a little bit easier now. I'm not driving to my car. My example used to be, I live 30 minutes away from work. I'd be driving in the morning. I'd have Cortana tell me about my schedule for the day.

[00:15:21] So I start to mentally prepare for the meetings. What I need to get done, sending out messages, setting up like, Oh, remind me to do this at this point in time. Um, so I do use Cortana probably more than the average person wide, um, or more like somebody might use Google assistant or Siri or Alexa.

[00:15:39] Paul Roetzer: The one thing I've found, like I, I tried Alexa for a while and I had to turn it off in my house because like you, I have little kids and they would just ask it stupid things all day long.

[00:15:48] And I would just stop talking to Alexa. Um, so I, I finally just unplugged it and now they think that the Apple only does music, so they don't know to ask it other things. Cause I just like pre-train them, like it [00:16:00] only knows this don't bother be with the other stuff. So, um, I've kind of like honed in how voice can be used in our house.

[00:16:07] Um, but what I found with Alexa was it has all of these skills, but like I had no way of knowing what those skills were like, the discovery of them and fi like you had to really look. And the way I think about AI is, is oftentimes instead of going and saying, what, what are a bunch of skills that I could use?

[00:16:27] I like to say, well, what are the things I do every day? And can it help me do that thing and not just with voice, but with AI in general, like I'm constantly just looking, saying, okay. I, I come up with headlines for blog posts, or I figure out what subject lines use an email or I manage paid media, spend AI helped me do those things.

[00:16:46] I don't need this list of like 2000 skills that the thing has that I. May use once or twice. Um, so I don't know. I just like me as a consumer, like, well, it's a combination as a consumer, but then also we're talking to marketers right now. And so that's [00:17:00] where, um, I touched on this briefly and I did not go into this in depth in the course that I put together, but there's this conversational AI framework.

[00:17:08] Christi Olson: And you, you touched on sort of. Two of the pillars of that framework that marketers need to be thinking about. Discoverability is failing across the board for almost everything. Um, and I think when I talked to Mike on make on, I feel like,

[00:17:27] I guess it's right there for your setting. I, I did say like the only, and I still will hold true to this one. Thing that for me is still really pushing their skills NPR. And maybe it's just, I listened to too much radio and I don't have cable television, but every NPR show says and ask Siri, Alexa, Google assistant to play all things considered on NPR.

[00:17:53] Like they are teaching people. Here's the skill. Here's how you use it. 99% of businesses have not taken that step. And [00:18:00] discoverability,

[00:18:01] Paul Roetzer: I love real quick is, um, it Starbucks and this is like a really practical one. So I drop my kids off every morning, like seven 40. I have ordered through the Starbucks app.

[00:18:10] When you order, it'll say, do you want to add this to Siri as like a request or whatever it is now, it'll prompt me at that time in the morning. Do you want to make this order? And I can just grab it. And I was like, Oh, you know, place the order for Starbucks. And it automatically orders from the right location, the right drink.

[00:18:28] And it's like, that's just genius. It's, it's a thing of safety. Cause I don't have to do anything other than say it, you know, Saturday order Starbucks. So that, to your point about the discoverability, like it's in the app, I'm using any way and it just took an extra step out of extra, probably three steps of could pick the order, confirmed the locate.

[00:18:47] And I'm sitting in there for like two minutes and it's only two minutes, but at the same time, I really appreciate that convenience and Dunkin donuts doesn't have it. So it's back to,

[00:18:56] Christi Olson: It goes back to like, um, I joked with my manager and I was talking to him about coming [00:19:00] up with a framework and it was DUI.

[00:19:01] Christi Olson: And he's like, but that's driving under the influence. I'm like, yes, I know. But discover it reveal any utility and then interaction and interface like DUI. He was not a super big fan of the DUI system, but I'm like, Hey. It's memorable for me, but he goes back to, they discover that utility, they discovered what is the thing that you, they can help you do that is the utility behind the skill and yeah, creating a skill in the app for the sake of having a skill doesn't work it's it goes back to apps in, I don't even remember a year that was 2010.

[00:19:33] Like what people have 400 apps or 200 apps on their phone, but they really only use. 15 to 20 of them on a regular basis. Um, and the I is under the interface and interaction and it goes, how is the person engaging with it? Are they engaging with it with a screen, right? Or are they only engaging with it with their voice?

[00:19:53] So like, if I am engaging with something with a screen that I can then go back and look at, it's different than if you're talking to a [00:20:00] speaker and you can't actually see it. And the example I give is, I don't know if you've ever tried to ask Siri. Um, ask Siri or ask your Alexa how to cook something and for a recipe and advice.

[00:20:12] And by the time it has made it through the ingredients list. I have chicken on the counter and I'm like, it's only heat the oven to four 25. And I do that and I'm like, okay, I need to go back to the 27 other ingredients because it's already telling me, like, now you take in the mix, these things, I'm like, Oh, so like spoken doesn't work, but if it can speak to me and I have the screen to look at it, It goes back to like that framework of how is the user engaging?

[00:20:37] What is the utility? What does that discoverability piece? Cause if they can't discover it, they're not going to use it. If they're not going to use it, why are you grading? If utility isn't there, if the need isn't there, if you're not solving a problem or issue. Will it get discovered in use and then that interface of understanding how they're going to use it so important.

[00:20:55] Paul Roetzer: Uh, you touched on one that I think a lot of marketers wonder, and I don't know that many have the [00:21:00] answer to her. So voice search. So the theory behind voice search, if we don't search on screens anymore, is it's going to, what they're trying to present is the absolute one, correct? Answer to you. So for brands, for marketers, Where does that answer come from?

[00:21:19] So if, if, if the future of my brand's ability to show up is no longer one of the 10, you know, first links on Google, but it's actually the one response that's given. Where is that answer coming from? How, how does the machine learn that? How do they figure out what the right answer is? Is it customized individuals based on voice search history?

[00:21:39] Let me just talk a little bit about voice in relation to search in that aspect.

[00:21:43] Christi Olson: Yeah. And voice search is actually how I got into conversational AI to begin with because so many people were asking that question and even I had that question as I started using. Like, I, I got an iPhone when I got my first iPhone.

[00:21:55] I'm like, where is it pulling this information from? How does it pull the information? Where are we [00:22:00] going? So like going down that discovery path on my end is where I really started digging in. So when you ask a question of a digital assistant, I'm going to leave it at the digital assistant level. Cause I think this is a little bit easier because if you think about it, the digital assistants align to a data source.

[00:22:13] So Alexa. Um, and the Amazon ecosystem pulls some of the information from Google and some of it from Microsoft, it is not like a 50 50 split. Um, and then if it's a product purchase or a product specific bit of information, they pull it from their internal systems because they're going to try to help you make a purchase.

[00:22:31] And they have a lot of shopping and product related information. Okay. Um, if you're on the Google assistant, it's going to pull from Google. If you are on, um, Cortana, it is going to pull from bang. So you have sort of. Two to three sources of truth that pulls from so within those, and I'm not going to go in depth into Alexa per se, because it will Alexa and the Amazon ecosystem, it pulls, like I said, product is from there.

[00:22:57] Uh, other queries, it will sometimes pull it from [00:23:00] Google and being, so it sort of holds true on both. But when you ask a question, like how do I get red wine out of a carpet? I've never had to do that as you can see, I have light colored carpets. Uh, and I have small children, which you can sort of, if I go this way, you can, so the wine is needed and sometimes it gets knocked over, especially during COVID.

[00:23:20] When you ask that question, what ends up happening is the digital assistant then goes to the search engine of record, and it looks for a couple of different things, right? It pulled at either polls from an instant answer, which has the knowledge graph within the search engine or some of them the engineers have hard-coded they have a hard coded responses or essentially dab.

[00:23:39] If I asked what is the capital of Washington? It knows like it's a fact, it just pulls from there. It doesn't mean a search group. If you're a brand, it is pulling a lot of times from the knowledge pain and the instant answers. So the question is like, can you optimize for it? Yes. There is opportunity for all the different questions that get or not all, but most of the questions you get asked to become [00:24:00] that instant answer or that one response that's at the very top of the search bar.

[00:24:04] It's that box that pops up. If you're getting into there that about 40% of the time. Um, well, the last, the last survey I saw that we pulled it and I haven't pulled it for two calendar years now. Um, it was about 40% of the time is coming from the knowledge pane, that instant answer box. So organic search optimization is a play that will help with conversational AI.

[00:24:26] And then it gets to understanding and knowing. How do you optimize for voice versus text? Because voice queries are a lot longer in nature. They're more conversational in nature. They do have traditional sentence structure. And as we think about search, like the paid search side, I'm on, like the queries we see are typically three to five words in length.

[00:24:47] Paul Roetzer: When you have voice, you can tell it's a voice query. Oh yeah. It's a very specific question. I always laugh because, uh, I mean, my, my daughter's eight now, so this would have been, she was like four maybe. [00:25:00] And she had an iPad and she would watch shows on Amazon prime or Netflix. And I have a video of this where she actually would say she would hold the button down.

[00:25:10] She would say the, the episode where the princess meets the unicorn. And then like she would watch and it wouldn't come up. So she would hold it down again. The episode where the princess meets the unicorn and they're in the forest and then she would wait and hold down again. And then she would like, look at me and be like, where's, where's my episode.

[00:25:27] And so she was learning on her own to keep. Adjusting her query to try and get the result that she wanted. I was just like so fascinating to watch. Well, like my son is four and I've been watching him do this since he could start to talk that people think I'm joking.

[00:25:43] Christi Olson: But no, literally we have, we have the Google system, we have an Alexa and we have the Cortana in our house.

[00:25:49] Christi Olson: And he discovered that Alexa can order products and he ordered, he was ordering cookies. And I kind of, like, I kind of asked my husband like, Oh, why are we ordering so many, like, why are Oreos coming? It's because [00:26:00] we did not turn off the purchase capability because we weren't using it for purchase, but he would go to Alexa, be like Alexa cookies.

[00:26:07] And so Oreos are added to the cart. One of us are gone to our Amazon count. We'd add something else we needed and we'd check out and all of a sudden cookies would arrive and we're like, this is odd. We have so many Oreos, that's an opportunity there for Oreo. And they don't even know, like you hear examples.

[00:26:23] Like I, my favorite story on the news of like a child. Annoying to the parents ordering like a $600 dollhouse that happened in Texas, I think three years ago. And the parents were a little bit surprised.

[00:26:37] Paul Roetzer: I, I love my, uh, my daughter I think was leather on that would do this. You know, you ask a question to Siri and so he never had the answer.

[00:26:43] She's like, just ask Google, sorry. He doesn't know anything. So they were comparing the digital assistants. Like which ones should we actually ask the question to? Yeah. I mean, like for brands, it goes back to, and I think they do dive into this and the conversational AI class of, yeah. Going back and try and [00:27:00] understand, like what, what is the intent behind what you're the consumer is getting at?

[00:27:04] Christi Olson: Do you have answers to the questions and the intent they're doing? So like today I actually was using voice this morning because, um, I have a surface book, um, surface book pro and. I'm missing a couple of buttons on the keyboard. Cause again, I have a four year old, 18 month old and they love to pull the keys off my keyboard.

[00:27:23] And I had a, a other keyboard that I've been using and it broke. So I'm now on this massive, giant word that is programmer's special. I think I got everything over one. Pull this out of box in the garage and their keyboard died. Um, There is not a scroll lock on the surface book. Like there's no, there's no scroll lock.

[00:27:45] I was trying to go into Excel. There is one on this keyboard and I detached my keyboard and moved it and was just typing up here. I couldn't get the squirrel keyboards. Like I could not get the, the scroll buttons to work. And so I was like, Hey Cortana, where is the scroll [00:28:00] button on this laptop? There is none.

[00:28:02] And turns out I had to reattach the keyboard to undo the scroll lock on the keyboard because it sets something in the backend of my computer, like using voice. To try to discover something as simple as, Oh, plug the keyboard back in, hit the button, call it. Good. Yeah. I feel like I just ask everything now.

[00:28:20] Paul Roetzer: All right. Well, I want, I want to make sure we leave enough time. I want to talk about this Marketing with a Purpose Playbook and that I have a natural transition into that, but before we do that, um, so for a lot of, a lot of brands, we talk to. Conversational AI to them is like they have a chat bot that maybe has some intelligence in it.

[00:28:39] How would you guide marketers to, to think about conversational AI in terms of how do they really get started and what would be some priorities for them? When they think about over the next one to two years of how it, how integral it could be into their marketing and maybe just what a couple steps might be for them to take to really get [00:29:00] moving in the right direction.

[00:29:01] Christi Olson: So part of the reason this came up so often, um, is as you start to look at the digital assistance and is your brand people, like you said, we're asking questions, we're engaging so much more so as brands think the chatbot is like a jumping off window.

[00:29:23] So two years ago, one of the things he said is that, um, digital assistants will start to be able to talk to other digital assistants. So there's going to start to be this AI framework that happens when I ask. And I'm just going to use this because Cortana is more on the Microsoft side. We were productivity me.

[00:29:39] So when you ask Siri Alexa or the Google stuff or something. They're helping you make that purchase. They're helping you discover that path. So the idea is that you would have a digital assistant that's discoverable by those other digital assistants to help with the action that somebody is going to take.

[00:29:55] Like you said, you have that Starbucks one, you could ask Siri to go ahead and place [00:30:00] that order or Alexa to place the order. And it knows to go to the Starbucks app. It's already saved as your settings. And it does it for you. You don't have to go into the Starbucks app to do that. Next thing, like it set up the action.

[00:30:10] Right based on your behavior and history. So that is why it's going to be important is because as these become more integral to our day to day lives, and as they become more used, that's where it's trending and going, is that it makes it easier for the consumer to engage with your brand. So they don't have to think so.

[00:30:28] The chatbot is a good natural first step because you're getting into conversational AI, you're understanding questions, you're providing answers and actions or answers, and then getting to that action phase. So really understanding how are customers engaging? How are you doing well? How are you not doing well using cognitive services, which I don't think we've really defined, but essentially you have a layer of AI that are cognitive services that help you to help the technology understand.

[00:30:56] Images, voice, text, speech intent, getting to there. You can use them all together. So that way, if somebody types in veggie versus vegetable, cognitive AI can understand that veggie and vegetable. When you're talking about pizza, same thing. Right. And instead of saying, I'm sorry, I did not under, I always did this too.

[00:31:16] I'm sorry. I did not understand because I feel like they go like this whenever they, that happens. So it's asking a bad question or for using well language or verbiage in a way, it didn't fully understand the cognitive services helps you understand and get to that level. So that way. As eventually the goal would be as a brand.

[00:31:37] If you have an action, if you have something you want them to purchase to engage, to book a service, if you're a hair salon, go in and get the hair done because I haven't done it in nine months, I've given up, um, So that's the jumping off point, because at some point they will talk to each other and agents can talk to other [00:32:00] agents, assistants can talk to assistance.

[00:32:01] So starting there embedding in those cognitive services, hearing what works well and what doesn't work well. And then essentially I go back to the word, optimize, optimizing it, right, improving over time so that it does provide the better answer, the better service. And that way, as you get to the point of having an agent that can talk to another agent, It is less of a feeling like you're jumping off a cliff and more of a natural transition over time, uh, getting there and getting to that point.

[00:32:32] Paul Roetzer: So at the core of all of this, and, and this is where I kind of lead into this marketing with a purpose idea is data. And to, to, to, to create this convenience and this personalization for consumers, to be able to respond to them and anticipate needs these intelligent agents need. A lot of information. And so as marketers and even AI is powered by this data, um, there aren't a lot of ground rules that we have to follow [00:33:00] as an industry of how this is all going to be done.

[00:33:02] I mean, obviously there are some, some laws put in place, but for the most part, it's still on each individual organization to be ethical in the way they approach their marketing. So your team created this. Marketing with a purpose playbook, and we'll share the link in the show notes, but tell us just the, the idea, the thesis behind this.

[00:33:22] And I know it evolved from an original idea, but like why, why is this important? Cause I did, I did read through most of it myself before this and it, it felt like it was an entire book on its own, but a lot of really critical things around ethics and transparency and privacy. Why should marketers be thinking about this kind of stuff while they're starting to think about the true applications of AI?

[00:33:44] Christi Olson: Yeah. So this will all go back to my previous role. So prior to October, first of this year, um, my roles of Angeles, we did a lot of different research trying to understand, um, everything from like conversational agents, digital assistants, privacy marketing, where's marketing going, [00:34:00] and this, uh, marketing with a purpose came up in all of this research because as we started digging in and looking at, um, we did research into, um, Millennials and millennials and the different age groups of how they're engaging with brands, tools, technology, how do you build brand love and engagement?

[00:34:19] We started seeing themes coming out around data and privacy. So I had done a whole bunch of privacy research this time. Last year, it got launched, um, April of this calendar year. I'm like, gosh, it feels like forever ago. And. I might even, even as sorry, the police are like, I look I'm like, was that last week?

[00:34:38] Or was that a year ago when I did something because they all like, just COVID has made everything just seemed like it's, we've had 10 years in one calendar year, but we started doing the research and we came up with these sort of cartoons of how do you build brand love. And part of it is. When people talk about inclusive inclusion and inclusive marketing, a lot of people think of diversity and inclusion, [00:35:00] right?

[00:35:00] But people aren't thinking about inclusion in marketing, how do you actually think about consumers and how you are including or excluding them based on the words, the images, the data that you're using. And so this, this sort of came out of brainchild of understanding brand love. Brand, uh, data and data transparency and moving toward inclusive marketing as a pillar of marketing.

[00:35:24] How do you think about inclusion of within your data sets? How do you think about. Including individuals. So they see themselves in your advertising or the language that you use because we've seen so much happen in the last one to three years of how people are even like gender, gender fluidity. Do you say they them Z, Z?

[00:35:47] Or how, how do you reference pronouns as we go into conferences? I, I know there's several conferences where I will put a, she, her. Because that's how I reference myself. Um, my partner who helped, I was a contributor to [00:36:00] the bucket's MJ, DePaul, HMAS brain child. And I was one of the people that I, I gave information.

[00:36:06] I helped write sections of it. She, um, she goes as she, but she also goes as day. And it's been a learning process of like, how do you talk about this? Right. Um, going back to like marketing with purpose and data. That's where I contributed was more on the data side because. There are so many examples of times when.

[00:36:25] When you collect data, you are excluding information without potentially realizing it. So how do you know if you're excluding or how do you know if you're building an algorithm that is discriminatory? And the examples that I remember going back and reading and talking with the ethics team within Microsoft is like hiring.

[00:36:46] I'm not, this is not a legal example, but let's use Microsoft as a whole. When you look at the breakdown of Microsoft and you look at male versus female, it is not a 50, 50 split of men and women across the company. So if you [00:37:00] were to build an AI algorithm based on the current employee set.

[00:37:03] You would skew more predominantly toward men. Um, you would skew less toward people of color. You would skew more toward Ivy league educations. Just based on the background we have does that. Did those factors mean that somebody is going to be successful? Microsoft? No, I'm a woman who went to a state school.

[00:37:23] Um, and some of the most successful people I know, including bill Gates did not graduate college. So as you're building these algorithms, if you take a data set and like the data set of Microsoft employees and how successful they have been, it is directional and good, but you could be excluding highly qualified individuals.

[00:37:42] Because you're only using what you've done in the past, and that's why Apple sounds familiar. It did happen at another major tech company. Earlier this year, they had trained a hiring model to evaluate resumes based on what they knew a great employee to be in the may learn that it was basically trained on.

[00:37:59] Paul Roetzer: White [00:38:00] males from successful like Ivy league schools. And so they had to shut the algorithm down because it was bias. And it's the most practical example, man. And they introduced bias without realizing and similar to, um, in the legal system. They tried using AI algorithms to determine, should somebody be released early yesterday?

[00:38:20] Oh, that is a slippery slope. Anytime you get a, I ended illegal, but. I know where to go. Yeah. Yeah. And they start looking at cases and they start using like how judges sentence individuals. And they started noticing that there are more severe and longer sentences in certain regions for people of color versus.

[00:38:38] Christi Olson:  People that are white Caucasians people of color. And so if you train the algorithm based on the sentences and when people are done for early release, now there's bias in the system to begin with. So how do you get past bias? There's bias and data. So it's not that data is inherently evil or it's good.

[00:38:56] It's is there already bias in the system? And that's where I sort of [00:39:00] contributed on the marketing purposes. Understand there is bias. How do you overcome bias? How do you think about making sure that you have all sorts of different people included in the process so that you aren't intentionally adding bias in, or that you understand where biases might be?

[00:39:16] So you can then gather the right additional incremental data and account for that, with what you are doing?

[00:39:22] Paul Roetzer: I think that book again is great. But one of the takeaways for me was there's a number of checklists in there and provided a couple of frameworks and one was like an ethics and, um, and bias checklist of like, here's the things you need to go through before you plan a campaign or before you use data to understand what's in it.

[00:39:39] So that, that would be really valuable. Yeah.

[00:39:41] Christi Olson: And I would say overall, the entire book we cover. Um, I don't remember how many chapters it was so many versions. Yeah. It's like 80 pages. And I, I, I want to say, um, we probably cut 10,000 words, like giving it to something we were published. We probably cut close to 10,000 words because we're all like, okay, this is too media.

[00:39:58] This is too in depth. [00:40:00] People are going to get this. And it literally was a, like at one point it was a 200 plus page book and we're like, okay, no, one's going to read 200 pages. I have a PDF download from our website. So like, let's start cutting. So I'm trying to remember like how many chapters we have, but some of it's even like inclusion in the words you use.

[00:40:18] And like, how do we speak to each other? Where, um, again, MJ, they, their as pronouns as well as she, her, um, she was talking about her experience of traveling where some people think of like LGBTQ friendly hotels. How do you, what are the words you use to show that you are friendly? And that is the example he used.

[00:40:40] I'm like, how do you show that this is a hotel that is accepting of people, backgrounds, because she said when she was traveling internationally, not all places were and when her and her partner would show up, they would not necessarily have the best experience. Right. So how do you then use the words that people are using?

[00:40:59] Right. To [00:41:00] show, if you are having, if you have an experience that does cater to that demographic, what are the words they use? Not just the words you use, but the words they use as they're searching and understanding, then how do you incorporate that into your marketing? How do you incorporate that into how you speak and what you are writing based on behalf of your brand and across the board?

[00:41:19] I mean, there's so much, I mean, the research, um, at one point we have a three or four different research studies we referenced in there. Um, we're talking to like. We try to distill 500 pages of research down in there, two 80 pages and take the highlights from it. I mean, there's so much absolutely amazing research.

[00:41:37] I would highly recommend reading it. And just for your marketing overall, not just the data, not just the AI piece, but how we do marketing as a whole is shifting and changing to think about inclusive marketing as a pillar to create brand love, tree cream, create brand loyalty that will then help us as we do more with AI and automation.

[00:41:56] Paul Roetzer: Yeah, and I think it's just a movement. You're. I mean, obviously it's, it's gotten [00:42:00] a lot of momentum in the last 12 to 24 months, but most of what I've seen is still more at the business level and the HR level and making sure that it's all integrated in there, but you don't see as much of what you've published, which is what does this mean to marketing and how can we carry these same ideals through to ensure that, uh, what we're standing for as a brand is actually represented in an authentic way  to our audiences and you don't unintentionally have bias in, in your, uh, your campaigns. So yeah, just, I mean, such a critical topic and something I'd love to keep exploring downtown.

[00:42:35] Christi Olson: Yeah. I'll introduce you to MJM Jan, I'm willing to bet would love to come on and speak. Uh, her and I have been speaking about this for months as the playbook was getting written and we, her and I like when we start going down this, we'll talk about examples.

[00:42:48] And then three hours later, like, Oh, we miss four meetings. All right. So this has been awesome. We're going to wrap up this episode as we always do with our rapid fire questions for [00:43:00] Christie.

[00:43:00] Paul Roetzer: Okay. All right. So here we go. Rapid fire to end this up. You ready? So this first one is not meant to be a trick question, voice assistant that you use the most. We asked this of everybody. I did not ask the specific Alexa, Google Assistant, Siri Cortana.

[00:43:25] We don't the last one was don't use, but we already know you use them and we're in between,

[00:43:29] Christi Olson: I'd say Siri and Cortana. Cause I use them both. Like if I'm in my car, I don't use on because Cortana really isn't in my car. So then I use Siri, but if I'm at home and on my PC and on my computer, I am using Katana, both from voice and digital assistant cider.

[00:43:41] All day long. So, I mean, it's pretty equal.

[00:43:44] Paul Roetzer: So I'm kind of like, I'm a Siri guy a lot. Cause my phone is always with me, but when I actually need to know the answer to something, I go to Google, like it's a Google assistant and my kid, that's what I said earlier. My kids would be like, just ask Google, like, why are you messing around with sir?

[00:43:55] He ask us stuff like that.

[00:43:56] Christi Olson: We, we will laugh. Like we have a Google assistant home, but because my [00:44:00] previous role, like I told my husband, like it has to be in the office on a shelf. It can not show up in the background of any of my videos. Um, and I just don't use it because of that.

[00:44:09] All

[00:44:10] Paul Roetzer: right. So the next one more valuable in 10 years, a liberal arts degree or a computer science degree, we didn't get into the more human aspect.

[00:44:18] I know you had some stuff about how conversationally I can actually humanize marketing, but Mark Cuban is famously said that he thinks liberal arts in 10 years is more valuable. And so I just think it's an intriguing question to ask question.

[00:44:32]Christi Olson: I have to say I'm like, it also depends on where you want to go and what you want to do.

[00:44:35] Um, I, I would almost probably say, well, if you want to be coder, then the answer is pretty like you're going to grow into computer science and coding. Your answers there. But I would say the liberal arts degree is valuable because we are becoming experts across so many different topics. So it goes back to you want to be a mile deep and an inch wide or an inch wide and a mile deep.

[00:44:59] And I got my [00:45:00] hand gestures opposite on their bed, but it's the idea of like, um, I have a marketing background. I don't have a coding background, but I actually do talk to people about like the coding frameworks, the frameworks, and what you do on that backend. I'm not going to code it. You do not want something.

[00:45:13] I code

[00:45:15] Paul Roetzer: I'm with you. I can't code anything. My son can now code me and he's like, yeah.

[00:45:20] Christi Olson: But I think it's, I think it's helpful because understand that human component, like you said,

[00:45:26] Paul Roetzer: I think that's what he's getting to is it's almost like the AI is going to code itself in some ways, I think is his theory that so much of what computer science majors have done historically will be done by machines.

[00:45:36] And they'll certainly still have essential roles. Um, but it's going to get to the point where so much intelligent automation exists, that the people who know what to tell the machines, what to do and know what to do with those and know how to interact with humans. Like. That may end up becoming a more a valuable commodity, I guess.

[00:45:53] Christi Olson: Well, it's valuable today. The problem is people like you. I cannot underscore with my team, like the ability to [00:46:00] have clear and concise communications to our partners. Very important. Let's talk about how to send a nice email. It's not passive aggressive, uh, okay.

[00:46:10] Paul Roetzer : Net net effect over the next decade. More jobs eliminated by AI.

[00:46:14] More jobs created by AI, or it's not going to have an impact one way or the other?

[00:46:20] Christi Olson: I'm going to go. It's not going to have an impact. Cause I don't think, I mean, there's going to be a distribution of jobs switching, like you said, there's going to be some of these really. Um, like I hate using this word by what to use it in my head, like menial, like there's menial tasks we do on a day-to-day basis and I can come in and do that.

[00:46:37] So that'll go away. But it doesn't mean that there's not other rules that people still need to play. So if you think about like even fast food restaurants, automation, and I don't know if that's AI based on mission, but automation has come in to help. Like it'll squeeze the ketchup onto the burger and it's the perfect amount of catch-up.

[00:46:53] You still have somebody that is taking the order and moving things along and making sure like, Oh, the customization, [00:47:00] Oh, this has onions. And they said, no, it's like the job didn't go away. It's still there. It's just, they're no longer manually. Squeezing the repetitive tasks go yeah. Repetitive go away, which is great.

[00:47:11] I'm trying to get into my team. What can we automate? Like what can we get rid of? Right. I'm the same way. If we can do it, do it because it's going to happen anyways. So we might as well figure it out. Now report every month you have to have this report and this data in this format the fifth of every month, can we automate it or do you, do you really manually need to go in and do it?

[00:47:32] Paul Roetzer: All right. What does an AI agent win first or at least share with a human, a Nobel peace prize and Oscar, uh, politics or, or none of the above?

[00:47:42] Christi Olson: Well, I've seen the movie trailer, the AI created, and I could not tell you what the movie was about. Although the other trailer that the humans grader was equally like that's odd.

[00:47:53] Um, Try to remember what you have to do to win a Nobel Peace Prize. Some of them have been awarded and I'm like, Oh, I wouldn't have guessed that.

[00:48:05] Paul Roetzer: I like that one a lot because I it's, it's often for advancements in science, in science understanding. And I think like the example I've talked about on the show before is, um, Alfa Fold from DeepMind.

[00:48:19] And the ability to understand protein folding, that was like, nobody thought that could happen for at least a decade and the machine did it. And so you could certainly see these like major advancements coming over climate change, or even vaccines where it became possible because of. A deep neural network.

[00:48:37] Christi Olson: And so you can envision a day where that, I mean, even think about now. Um, little known fact, my daughter who's upstairs and she'll be coming down soon. She has a pretty rare genetic condition, like one in a million. Um, she actually has two different rare genetic conditions on a million. Uh, they had, they, they did the blood sample.

[00:48:55] Early January because of AI and the ability to go [00:49:00] through and understand the coding sequence previously, that would have taken a long time. It was, I think they had the results back in three weeks. They didn't tell us for another couple of months because they were trying to figure out how to explain to us what it was and connect all the dots with the right professionals like AI enabled them to take the DNA sequence and what used to be bajillion machines running it.

[00:49:20] It was less than two weeks or so cover these really weird, rare snippets of DNA missing. And like, uh, I'm trying remember CA the CA GT, like which exact snippet of protein is missing for, for conditions. So, like, I think that's, I'm trying to think, like, I think on the creative side, Not quite there yet, but I think you're right.

[00:49:43] The advancements in science they're already starting to happen. It's just, will they win a prize? Might be better, right. When would that happen? Because we've already started to see some of the benefits. All right.

[00:49:56] Paul Roetzer: Well, this has been awesome. Let me again, we could, there's a couple of [00:50:00] topics within all of this.

[00:50:00] I could have just gone a whole episode on, but I'm just so grateful for your time. I know for all of us, especially with little kids at home, it's such a challenge to carve out 40 minutes to talk and not have them running through the background or that's how I just, I finally had to come to the office to do mine.

[00:50:16] My kids, when they come stomping upstairs, my daughter like crawl behind me, pretending like no one could see here I go downstairs. She's like, did you notice? There was like, yeah, I knew you were there. I tell you a little head, like poke. And I was like, Oh, I was being so quiet though. Yeah. I laugh with my team cause, uh, is, uh, sometimes I'll put her upstairs.

[00:50:32] Christi Olson: So the stairwells right here and she'll literally be like pushing her head through the, through the slots. I'm like, well, if I move my camera up just right, I can be on a call and I can watch her. I'm like, look, she's safe. We're good. It's just all part of what we're living through today, but thank you again.

[00:50:50] Paul Roetzer: Do you have any final thoughts for, uh, for our audience in terms of just trying to process all this stuff and get started is really where everybody is at. I mean, [00:51:00] um, how I started, this is reading, like join something like makeup, where you can go through the Academy, you can take the classes, um, I have literally, I can't even tell you how many hundreds of hours I've spent reading websites, blogs, books, talking to individuals, such as yourself, as well as like for voice.

[00:51:21] Christi Olson: Um, you have the Westwaters who are absolutely that they haven't seen it.

[00:51:27] Scott and Susan are so smart and him and I have been on a couple of panels together that like, Oh, I didn't think about it from this angle. Now let's talk about this and just bouncing ideas off of each other. That's how I learn. So even as you're getting started, find the community, ask questions because there's enough of us out there that have been doing this.

[00:51:48] I mean, I mean, I'm somehow considered an expert for years in on conversational AI, which just boggles my mind. I'm like, I feel like there should be a lot more people with a lot more expertise [00:52:00] singularity, but there are not just if someone's me. Yes. 

[00:52:03] Paul Roetzer: THere's a bunch of engineers, but like to the people who can actually have the conversation and make it understandable, that's the challenge is like taking what the engineers know and explaining it to other people in a way that's like, Oh, okay, this is an intimidating, like.

[00:52:16] Christi Olson: Well it taking it so that marketers and brands can understand it, understand that value. I mean, that's where you can just start to dive in, start to learn, ingest and ask questions. I would normally say find me on Twitter, but, um, since COVID hit, I've actually gone off of a lot of social media because I needed to take time for myself.

[00:52:35] So I'm not on tools and technology 18 hours a day. Like I was, um, but it's, it's not as challenging. Well, it's not that it's not as challenging. It is not as scary potentially as you thought it might be. Um, it just takes a lot of. Investment time to learn and an open mind and a growth mindset that you can learn it.

[00:52:54] And I would say that marketing has purpose playbook while it isn't a hundred percent AI base, it does bring up a lot of [00:53:00] really great thoughts toward marketers and brands of how you can start to go down the path of how do you build ethical? How do you build an inclusivity? Into what you are doing from a marketing perspective.

[00:53:11] And really when MJ and I were going down the path of greediness, literally, we hadn't seen anybody else talking about it from this standpoint. And we literally, like I said, 80 conferences, one year, I've been to a lot of events. No, one's talking about it from that perspective. It's the HR person.

[00:53:26]Paul Roetzer: I agree. We just went through our editorial calendar and we, we put it in there as a thought.

[00:53:29] I was like, we gotta, this needs to be a core part of what we're doing on the business of AI side. And so I was, that's why I was so happy when you shared it in advance of this. It's like gross. Fantastic. Like this is something we can work with and yeah. So I'd love to continue those conversations.

[00:53:45] Christi Olson: Wonderful. Well, thank you. It's such a privilege and honor to get the hour to speak with you as always.

[00:53:53]Paul Roetzer: love catching up with you and usually we just get to do it over, you know, um, you know, over the phone, but it's, it's awesome. Be able to have these conversations and share your insights with everybody. So thank you so much. I appreciate it. Thank you and have a great day. All right. Thanks everybody for joining us. This has been The Marketing AI Show. Until next time. Thanks again.