This week on the podcast, Matt talks with John List, the Kenneth C. Griffin Distinguished Service Professor of Economics at the University of Chicago and Chief Economist at Walmart. List walks through his role at Walmart and how his work has the potential to change the world because of Walmart’s footprint. They then discuss his experience in working with Lyft and United Airlines on loyalty programs and they finish discussing John’s book, The Voltage Effect, and the concept of optimal quitting. 

If you develop solutions that can both help Walmart and help the broader society, you have a lot bigger chance to change the world.

Excellence, professionalism, innovation and collegiality. These are the values the Sam M. Walton College of Business explores in education, business and the lives of people we meet every day. I'm Matt Waller, Dean of the Walton College and welcome to the Be Epic podcast. I have with me today, ProfessorJohn List, who is the Kenneth C. Griffin Distinguished Service Professor of Economics at the University of Chicago. He is also currently in another position as well. He is the Chief Economist of Walmart, which is a new position. As a professor, he has had over 200 peer review, reviewed journal articles and several textbooks. And he's got a couple of popular books as well, including The Y Axis and The Voltage Effect. And today, we're gonna be talking primarily about The Voltage Effect, although I want to ask him a few other things. John, thanks so much for joining me today. I really appreciate it.

Hey, Matt, it's my pleasure. Thanks so much for having me today.

John, I'm really intrigued by your background for many different reasons. And your book, I definitely want to get to, but you have, you're wearing lots of different hats right now. You're a professor at the University of Chicago, which has one of the most distinguished and notable economics departments in the world. I noticed also that you're currently an editor of Journal of Political Economy, which was edited for many years by the famous George Stigler. And, of course, University of Chicago had Milton Friedman and many other really famous professors. How do you, and, but at the same time, you're also Chief Economist at Walmart. How do you manage all of these different roles?

Yeah, look, that's a good question. So so first of all, thank you for the nice words about Chicago economics. I'm very proud member of the Econ department here at the University of Chicago. And you are correct that we do have a long and storied history. When I took over as chair, I started being Chair of our department in 2012. And I think we had something like 18 or 20 faculty, and five or six of them had Nobel prizes. So that was, that was kind of a fun, fun foray so I was chair from 2012 to 2018. But, but you're right, I also wear other hats. I about 20 years ago, I served in the White House in the administration for Bush two. I was an economist giving the President advice. More recently, I've, you know, I've been Chief Economist at Uber and then left and now Walmart. You know, for me, when you look at my research agenda, it is essentially, I use the world as my lab. If I had to make a guess my guess would be 99.9% of your listeners have been a subject in one of my experiments. You don't know it. And it's not creepy, in the sense that I know your name, and I know your behavior. I don't know any of that, nor nor do I want any of that information for confidentiality purposes. But what I do know for example, is when I raise the price of an Uber trip by 10% around Arkansas, I do know how people respond to that price change. So when you think of it that way, this is a micro economist or a behavioral economist who is trying to test economic theory in the field. What other way would I do it, then partner with the best firms and organizations in the world? To help me learn about how economics is applied to the world? My question basically is, how have we gone for so long and acted and so we are trying to explain the world and we are trying to discuss how our theory is working in the world without ever being in the world. That would be how I would ask other academics who aren't doing what I'm doing, how in the world can you sit in the ivory tower, and understand whether your theories are right, or whether your conjectures are right, unless you're out there playing around, and figuring out if that theory is right in one setting or another? So the way that I think about it long and short of it, these activities are complements, not substitutes to being a great professor, not only because my research is better, but it's also because I can bring those experiences into the classroom. And I can talk to the students about what I've learned. The students can be part of it. I have, literally an army of people every summer, who are helping me run field experiments, and then they're getting hands-on experience in terms of collecting data, analyzing data, interpreting data and writing it up. So to me, it's a 100% compliment and that's what allows me to be a much better researcher than I otherwise could be.

I completely agree. And in some ways, you could also say, you could take the other side of it, how has it gone so long? That companies, big companies that that could apply and investigate questions that they're dealing with, with sound theories from things like economics and other fields, and also using field experiments to make decisions about what they should do.

Now, that's a great point. So back in 2010, Jeff Bezos reached out to me and offered me to be the Chief Economist at Amazon. And Jeff had said, you know, "We don't have any economists, so you will be our first economist. And then we want you to grow the number of economists across the firm, because we believe that a major interruption can occur through the lens of economics." Back then I decided not to take it. But he was right. Now Amazon.com has 1000s of economists on site who are changing the world, and changing Amazon using economics. Now firms are starting to, you know, hire chief economists, use field experimentation, use behavioral economics, but I really think we're in the first inning of this baseball game, which, of course, is a nine inning game. And I think we're just scratching the surface. We're starting it at Walmart, which I'm very excited about. But but there are many firms who haven't even begun the process of bringing on a small team of economists or what I would call a scale unit to think about how to scale ideas from the small to the large.

Do you think that part of the reason for this taking so long to happen in our economy, is because of the fact that it's taken a long time for data and information to become so widely available in processing power to scale? To the degree that it has? Do you think that's one of the reasons?

I do think that's one of them. And now, you don't have to have as much imagination to think, "Well, how is an economist going to help me?" An economist is going to help me because we have mounds and mounds of data that we can work on, right? But I think another reason is economists have in a way been gated off for a long time. And what I mean by gated off, you do have people who have been expert consultants, for example, on antitrust trials. So you do have economists being hired that way. You do have economists going into government. But for many years, you you did not have economists like me, who would straddle, who would be able to be successful in the academy and publish research, but then also unlock those secrets that are in academic journals and bring them to companies. I think there's just been a wall. And that's been, I think, one of the major impediments of why we haven't used economics as much in the private sector. Because of this, it's kind of a mythical wall, but it's been, you know, academics you stay over there. Business people, you stay over there. And in the end, maybe we'll gonna learn a little bit from each other. But I think there's been this mythical wall that has not allowed that to happen.

And, you know, I think one other thing is that there's a lot of leaders of companies who've never taken taken a course in economics. In other words, they don't know what it could offer. And I think the same thing is true with fields like operations research, and management science and some of these other fields where, you know, the senior leaders may actually have people in their companies that could do these things, and they're unaware of it, or what could be done with it, for example.

Yeah, yeah, I think you're exactly right. I think that is changing now. So when I think about Lyft, Logan Green was an economics major at UC Santa Barbara. So So Logan appreciated economics and what I what I could bring to the table, he was the founder, and CEO of Lyft. When I think about, you know, for example, Travis Kalanick, he did Computer Science at UCLA. But he was well versed in economics. So he sat through a lot of econ courses, too. But I think in general, you're right. In the, in the past, there may have been fewer economists who headed up major organizations then what you have today, or at least fewer who had been exposed to economics, but I really do think that's changing now.

Wll if you look at a company like Walmart, it has so much data on sales. If you just look at the stores, not counting the omnichannel, areas, but if you just look at stores, they have point of sale data by day. And they have price data, they have inventory data, you know, you can also look at the sales and price of complements and substitutes. And, and of course, with Sam's Club, they also have information about this, you know, it's tied to a specific Club member.

Right.

So you can get even additional information about all the different places they shop, and so on and so forth. I mean all the different clubs they shop at. So it's, it's really exciting that Walmart has decided to go down this path. So but I would think it takes a lot of negotiation to make sure that senior leaders give you the freedom you need to be able to set things up properly. Is that right?

Yeah, so I like, I like your summary. You, you pinpointed some of the major reasons why I picked up my toys from Lyft, and moved them to the bigger sandbox of Walmart. Because Walmart provides a very big footprint in our economy. If you develop solutions that can both help Walmart and help the broader society, you have a lot bigger chance to change the world. And you also have more opportunities to change our scientific understanding of how the world works by being part of Walmart. So on the first point, you're exactly right. And this is important to me as an academic who really wants to change the world, that I'm given the best opportunities. Now, you're, the last part of your passage was about, how are you going to work through with the executive team to figure out what you're going to work on, but also give you some liberty to spread your wings a bit and explore issues that, at the very beginning, everyone might say, that's a loser. But they still allow you some freedom to work on those types of topics. So, so far, so good. I've been starting to work on various parts of the company like the membership program, Walmart+, working on different issues around pricing, working around different issues on the social side. Because look, at the end of the day, back in the early 90s, when I wanted to do field experiments, everyone told me I was crazy and it was a dumb idea. And I shouldn't waste my talents, doing field experiments. That's when I was a grad student back in Wyoming in the early 90s. People allowed me in that graduate program to spread my wings and take a chance. And I did, and thankfully, it worked out. Now, so far, Walmart has shown a tremendous amount of support and flexibility to allow both me and my team to stretch our wings out a bit. So if in a few years, we don't have some fundamental and important discoveries, it will be because of me and my team, not because of Walmart.

Well, it's, that's awesome. You know, when you think about, there's so many questions that are not well answered yet. For example, knowing when an item is out of stock in a store is difficult, even for high, high velocity items. But when an item is out of stock, and you know, it's out of stock, it's hard to know, how many units did you lose in terms of sales, right, because sometimes people switch to a substitute. And sometimes they leave the store. And sometimes they never come back. Because of the, it was the nth stock out that they,

Exactly.

They experienced. And when you analyze data, for example, you're trying to analyze, volume and quantity demanded and price with that kind of data. If you don't know where the stockouts occurred, or how the behavior of the shoppers resulted in maybe say, substitute sales increasing, or people leaving the store, it's it's hard to sometimes judge what's what's actually happened. And then, similarly, some of the items may only sell one unit per month in a given store.

You're hinting at a very important issue that any organization faces. And that's when something important happens like a stockout, or like inflation, or like a competitor changes prices, or there's a supply change, or a supply disruption. How can we think about going from a correlation to something that's causal, because in the end of the day, what we want to do as scientists, is we want to establish causality with the least number of assumptions. We want to make sure that this is a true causal relationship. So in your stockout example, you're exactly right, in the types of outcomes we should care about, are not only things like, Do they go to the next jar of peanut butter, and still spend the same amount of money in Walmart? Or do they go to across the street to Kroger, and we lose that person? And then do we not only lose them that first time? Do we lose them forever? And what you need to do in that particular case, which is one of the issues that we're looking at a bit, is you need to look at some kinds of natural experiments. So sometimes stores naturally stock out just because of bad luck or weather, or supply chain disruption. But then you're right, you need to have data. And you need to have appropriate measures at that individual level to pinpoint, what did they do? You know, you have the you have the stimulus, which is there's a stock out. But then in the end, what does that lead to? And that's a problem of data. So the problem of data, even though Walmart has a lot of data, any organization, you will always have the problem of getting clean data that are measured correctly. And that's exactly what my team in part attempts to do at Walmart.

You know, you mentioned you were working on something related to membership. I think you said just a moment ago.

That's right.

And I remember in the book you were talking about how Lyft they, they, um had a hypothesis that some kind of a membership would work well. But the empirical you know, it was a good thing they didn't roll it out too broadly, because the empirical data showed that it didn't work. And there's certain situations where membership works really well. And others where it doesn't tend to work. Would you mind speaking to that?

Yeah, no, of course. Now, if you back up and and ask, you know, why in the world do we even have membership programs? Why do we have points programs like you, at United Airlines, you fly and then you earn points. And then those points lead to benefits. And in other cases you have, you're a member at Sam's Club, so you can go in and purchase an array of goods for a really good price. So these types of programs, of course, are meant to engender loyalty. Because in the end of the day, once you are lucky enough to attract somebody into the store, or somebody to use Uber or Lyft, you really want to retain them. And that's your job is to continue to give them good products, good prices, good services, so you can retain them and make them happy customers. So economists for years have thought about, you know, what is the economic theory, behind memberships, it goes all the way back to an economist named Walter Roy, who wrote a paper that's titled The Disney Land Dilemma. Because Disney Land, what they used to do is they would charge at the gate, and then they would charge you per ride, where now it's, of course, you just pay at the gate, and then you take as many rides as you want. And on the side, you can purchase a fast pass if you're if your value of time is really high. But at Lyft, we designed a program, which is called Lyft Pink. And of course, before rolling it out, we wanted to test it. And we wanted to make sure that the program made sense, both for the customer and for Lyft. Now, what we found was something that was really interesting, what we found is in this program, which the program is basically, you can pay $10 per month, and then you receive something like 15% off of each trip. That's what that membership program was all about with Lyft. What you find is that in that program, three out of every four people who sign up, they simply take the same number of trips that they would have taken had they not signed up, and they just signed up just to get a better deal. So there was zero incrementality. In these trips, these are, in the book, I call these the 'no goods'. They're the no goods, because there are no goods from the perspective of Lyft, in that, they're signing up to get a good deal but they're not riding more, they're just paying a lower price. So for three out of four people, the membership program is really a bad investment for Lyft. Now, for the one out of four people, those are in the book, what I call the 'Joe goods'. These are people who buy the membership, and then they end up taking more trips. The problem is, at least at Lyft, when you had the three out of four, 'no goods' and the one out of four 'Joe goods' that ends up being not a very good value proposition. So that particular membership program was not a good one to scale. And the generic lesson here is we should always understand that these membership programs come at a potential cost to the firm. And we should always be able to measure effectively, what are the benefits of having this membership program. And in the end of the day, the linchpin more or less comes down to the incrementality. And we should always as a good firm, make sure we're measuring incrementality and we're fine tuning our program. So then everyone's better off not just one side of this market.

You mentioned United Airlines and just the airlines using frequent flyer miles and I don't remember what year those were created. But they they create a switching cost because there's a benefit to staying with the same airline. But but I've noticed over the years, they keep tweaking, you know, and airlines have been known for being heavy users of operations research and techniques like that for scheduling flights and pilots and and then optimize, well they used to call it yield management.

Right. Right.

But I noticed they keep tweaking it over time. Do you have any insights into that?

I don't know. 15-16 year ago, I actually was helping United Airlines with mileage plus program. So Steve Levitt and I worked a little bit with United. United's headquarters is here in Chicago, Illinois. So so we went to the headquarters and worked with them. And you're right, what, what those programs are set up to do is essentially, the more you fly, the benefits actually increase in a convex way. So the benefits for each incremental unit that you fly, get greater and greater and greater. By by having that feature, which is an important feature of membership programs, along with United is really good at bringing in outside rewards. So I am a United Frequent Flyer myself. And when I generate enough points, I can actually use them to take my wife on a trip to Bermuda should I want or to stay at a nice hotel, or to buy a set of golf clubs. So what they're doing is they bring in outside benefits, that are really nice features that I can buy using my United points at sometimes a lower than market price. And that's what a lot of times, organizations forget to do. Because when you look at Uber and Lyft, it's very difficult within the marketplace to give a lot of benefits, because the minute you start to use preferential dispatch, that means another customer is worse off. So you always want to be careful not to make other customers in the system worse off, and the easiest way to do that is to try to bring in benefits from the outside, whether it's McDonald's coupons, or shopping at Walmart, or what have you. You in many cases don't want to disrupt your own market, and you want to have outside benefits that are attractive, in a net way there against a trade, I'll trade my loyalty for some of these benefits that United Airlines can give me.

So one of the topics I wanted to make sure we talked about, there's a few of them. It's something that I've witnessed a lot in students and alumni. And that is I think there's a culture, maybe more so in Arkansas than some other places I've lived, I'm not sure about that, but that's what I kind of observed. This, generally people feel like, quitting is for quitters. Right? And your, your, book talks about this. And, and it's really, it really struck me as something that people need to hear. You mentioned something about 'optimal quitting', which I thought that was a cool phrase. When you yourself, were a two time academic All American golfer in college, and you had this dream of being a pro golfer. But you analyze the data and correct me if I get any of this wrong, but and you realize no, I'm probably not going to be a pro. And so you decided to quit. You decided to give up on the idea. And you then discovered economics. And so part of the point is it's, right, we've got all kinds of alternatives out there, where we may have a comparative advantage. And many times we don't know what they are unless we're trying them.

Right.

And so you tried economics and look what happened to you. I mean, you've you've just excelled beyond the imagination in economics. Would you mind speaking to that a little bit?

Wow. So you have a good, a good memory. And, and you've, you've read The Voltage Effect, which I appreciate. So So you're right. The back half of The Voltage Effect is really about execution. The first half talks about what are the signatures of good ideas that can scale. The second half I think about is four little behavioral economics secrets to execution. One of those is quitting. So you might ask, you know, how did I get to this idea of optimal quitting? And you're right it, it occurred on basically on the golf course and through self reflection. So if you just back up a few years, I would have never gone to college, had it not been for golf. So my dad is a trucker, my grandfather was a trucker, my brother is a trucker, my mom's a secretary. Nobody in my family had gone to college. Okay, so my family basically said, there's no reason for you to go to college. But I had this dream, and the dream was to be a professional golfer. And when I was given a partial golf scholarship, it was really an academic scholarship because we were D2, D3 at UW Stevens Point. But the idea was, I can go to college and play golf. Unfortunately, what I learned very early on was that I was never going to be good enough to be a professional golfer. As you mentioned, I was pretty good. I was Academic All American, I could average 73-74. On my home course, I could average 72. But there was always a shot or two on each side, back nine then front nine that I could never get rid of. And I did look at the data. I looked at people like Steve Stricker, who I played against in high school, he became a famous golfer. I looked at people like Jerry Kelly, who played on the tour for many years, I played against him in high school. And they really improved and continued to improve. And I didn't, I plateaued, I didn't quit the team, let's be very clear, I continued with the team, what I quit, was I quit on the dream. Because I didn't want to be a a club professional. Nothing wrong with that. But I just didn't want to be at the local club, teaching lessons talking about the good old days, I would be perfectly happy with that had that been my comparative advantage. But what I learned is that I was pretty good at economics. Now at this point, you have to think about this is a kid who was raised in Wisconsin, he's a blue collar. And at that time, Wisconsin as I was being raised in the 70s, late 60s, 70s, 80s, there was this mountain of a man called Vince Lombardi. So Vince Lombardi was the coach of the Packers, they won the first two Super Bowls. That's why the Super Bowl trophy is called the Lombardi trophy. He famously said, winners never quit, and quitters never win.

Yeah.

That's how I was raised Matt, that that's how a blue collar kid is raised. You don't quit. Look, you can type in Google, 'quitting and inspirational quotes.' And you will find enough posters to fill every museum in the world. That's how ingrained society is in terms of 'you don't quit.' So on the one hand, society tells us, you shouldn't quit. Okay, that's society's fault. Let me talk a little bit about ourselves. And where does a fault lie with us? Here's where the fault lies with us, as humans, we neglect our opportunity cost of time. Okay, that's a lot of economies. So let me unpack that for you a bit. So I did a recent survey of people who have recently quit their jobs. And I said to them, "Why did you quit your job?" Reason number one, 'I had lost the meaning of work.' Reason number two, 'I didn't get the promotion that I thought it would get.' Reason number three, 'I didn't get the pay raise that I thought I would get.' Reason number four, 'I no longer got along with my co workers.' Dot dot, dot, dot, dot. Reason number 10, 'I didn't like my cubicle anymore.' Okay,

so they missed the real reason.

They missed the important reason. Every one of these reasons was because my current lot in life got soiled. So they started to look around. When the issues around them got so bad, they became unbearable. Now, what I'm here to tell you is you should be just as likely to move by saying, "Look, I looked around, I looked at my opportunity set. My opportunities just got better." Like, look at me, I left Lyft, not because I didn't like Lyft. I love Lyft. I left Lyft because my opportunity set got better. I could go to Walmart and be the Chief Economist at Walmart and play in a much bigger sandbox and play in an org where I have a chance to really change the world. My opportunity set got better. So I took it as humans, we ignore our opportunity cost of time. And that's another reason why we don't make enough changes or we don't quit. One, society. Two, it's ourselves.

What a powerful message when I read that part of the book I, I immediately started wanting to send it to people. I mentioned to you that yesterday, I gave it to a young man that was in my office. And I think that it's I think I've definitely fallen into that trap. A lot. My whole life. I wish I wish you would have written this book 40 years earlier.

Well, I'll tell you Matt, I'll tell you want. I'll tell you what. So right now, a listener might be thinking, well, John, that sounds like theory, do you have data? Pick up the chapter, we have real science around. So Steve Levitt and I ran an experiment, that induce people to quit, whether it was move apartments, move cities, quit a job, quit a relationship. And then 6 12 months later, we go back to them, and the people we induced to quit, were happier than the people who stuck with the status quo. So I have real science behind this insight as well.

John, you at a young age really enjoyed sports trading cards. And you it really, you know, probably taught you a lot, but it also got you interested in field experiments. Is that correct?

No, that is very correct.

And, you know, field experiments, you know, for a lot of economics research, is, some of it is just purely deductive using mathematics. And, and some of it is empirical, using experiments with human subjects. And some of it uses archival data. But you've developed this expertise in field experiments. And, and why is that important?

Let's turn back the clock a little bit, because it it is kind of an instructive story about how I stumbled upon field experiments and literally stumbled upon them. Because when I was a kid, I would have a few paper routes, I would shovel snow in the winter, and I would cut grass in the summer. And I was always paid in cash. And I would take that cash immediately to the local PDQ, which was a convenience store. And I would buy baseball cards, or I would buy football cards, roll it, roll it up to about the mid to late 80s. I was an undergraduate student in college. And these baseball cards actually became valuable. So back then they would have conventions, where people would go to the convention hall, and there would be dealers, and there would be consumers. And the big group of people they would buy, sell or trade baseball cards. So I started to go to those conventions to buy, sell and trade. And eventually I became a dealer. And while I was learning economics in the classroom, I would take those economic learnings to the field or these baseball card shows. And I would test them out. I would say, Look, I just learned this concept about bargaining. In my economics class, does it actually work? Or is it just a theory that is useless? So I would try it out. I would try out I would explore discrimination. I would I would explore auctions and in auction theory. And I would bring those learnings back into the classroom. And it was a give and take for me between theory and empiricism or data. And then thinking about how the theory should be amended. And then that essentially evolved around 1990 into I would run experiments. And I would experiment on how to bargain or negotiate with somebody I would experiment on what kind of auction should I run to sell my cards? And I started to think, should we be testing economic theory in this way. This is kind of a neat way. I go to the field. And I run these experiments where I'm generating unique data To learn about whether the economic science is actually working, when it does, that's great. I can write it up as an academic paper saying, Here's where economic theory is getting it right. When it doesn't, I can say, Well, look, here's where the imperfection is. And maybe we need a little bit of behavioral economics in that part. So for me it, it became a love, a love of learning, a constant curiosity about how we can use economics to change the world. And then this sort of monomaniacal nature, to go to the field, and generate data and try to solve important social problems. So in the early 90s, it was baseball card shows. But by the end of the 90s, I was working with a lot of charitable organizations, and helping them raise money, and starting a research agenda on the economics of charity. And then that rolled into working on starting a pre K school. So I started in 2010, a preschool for three, four and five year olds in Chicago Heights and the idea was, let's use a field experiment to learn about the education production function at a young age. So every child has an equal opportunity as every other child, and then it evolved into running experiments in firms. So as you can see, it all paints the same picture. It's using experimentation in the real world to answer important societal questions such as, why do people discriminate? Why do people give to charitable causes? Why do inner city schools fail? Why do women earn less money than men working in the labor market, these are all issues and many more, of course, that I work on. And the best tool that I can imagine is go to the actual, let's say, market or go to the world and use it as your lab is if you're a hard scientist.

That is really exciting. It's also encouraging that you have such a strong drive to want to change the world. You know, what you're talking about looking at why inner city schools fail, and all these kinds of questions are so important. And now you're able to do something at Walmart, which has such a huge footprint that really could. And Walmart has made a big difference on the world, especially in terms of, you know, the efficiency that they've created. But, John, I really appreciate you taking time to visit with me, and I've really enjoyed reading your book, the voltage effect.

You know, Matt, thanks so much for having me and I look forward to my next trip to Fayetteville and visiting all the Razorbacks.