How will generative AI and synthetic audiences shape the landscape of qualitative research?

In this episode, President of Sago, Isaac Rogers joins us to explore the changing face of qualitative research and the need for more engaging and innovative research designs. We discuss how researchers are adopting multi-phase and iterative research designs, including online discussions, video interviews, and digital diaries, to gather more comprehensive insights from participants. We also explore the emergence of generative AI and synthetic audiences and the challenges around data auditing and security measures to combat fraud in this area. Isaac shares his perspective on the challenges and opportunities in the industry, including the need for incentivization, data auditing against fraud, and the potential of computer-generated open-ended responses.

You can reach out to Isaac on LinkedIn.

Many thanks to Isaac for being our guest. Thanks also to our producer, Natalie Pusch; and our editor, James Carlisle.

Hello, everybody. It’s Lenny Murphy with another edition of the GreenBook Podcast. Thank you so much for taking time out of your day to spend it with me and my guest. All my guests are special, you know that by now. I sound like a broken record, but you know, some closer to home and today’s guest is one of those. Isaac Rogers, president at Sāgo. Isaac, welcome.

Pleasure to be here, Lenny.

So, I say this is special because Isaac and I have known each other for way longer than probably either one of us would want to admit.

Oh, more than a decade, I think is safe. I’ve been in the industry 15 years, so I know I’ve known you the majority of that.

Yeah, absolutely. So, back in the 20|20 days.

Well, actually, I think you were at Rockhopper at the time, I think. And so, that’s how I got introduced to you, so that’s—that really puts it back a long way.

You are definitely taking it back the 15-year mark, for sure. So anyway, it’s a pleasure to have you on. And for our audience, I recently had the opportunity to go to an event with P&G, and Isaac presented some internal data that Sāgo has been collecting around changes they’re seeing in project type and volume and quality that I just thought was really, really interesting and we’re not getting anywhere else, even GRIT, you know? We don’t get that type of information. So, that’s the core of why I wanted Isaac to come because I think that there’s just a lot of great information there to share.

Hey, thanks, Lenny. So, first of all, you know again, pleasure to be here. Love talking about the market research space, the industry, what’s going on. I have been fortunate, been in the industry about 15 years now. As you mentioned, I joined a little company called 20|20 Research that grew and became one of the world’s largest digital qual providers.

That is the issue, isn’t it? How do we get in front of it? Well, let’s dive in, right, rather than me pontificating. So, I think that there was an awful lot of really interesting things that some were, kind of a yeah, we assumed this is what was happening and how things were going to go, some that were pretty darn surprising.

Yeah, absolutely. And so, the questions that continue to come up are, so what’s going on with in-person qualitative research? And so, we looked across our entire dataset, every group we’ve done over the past five years, categorized, organized, and came up with several trends. So, the first thing people say is, you know, “Are people really back doing in-person work?” And the answer is actually, yes.

Sure. A lot of test kitchens? Sensory—

Which, you know, are tough because a lot of facilities are in commercial buildings that don’t allow you to have commercial test kitchens. And so, like, it creates all these really interesting logistics challenges. But there’s no doubt that the world has changed in in-person research. It is back. It is back differently, but you know, I get people all the time going, “Oh, people are really doing in-person groups?” And I say, “Yeah, they are just different than they were pre-Covid.”

Yeah. Which absolutely aligns with what we’ve seen via GRIT with asking about methodology [and option 00:08:54], right? We saw that flip. And we’ve seen over the past few years and figured when 2020 hit that that was one of those things that would stick, right, just like working from home. But I’m glad that it is back to the level that it is. So, that’s fantastic. You know, because the next great fear would be as we go move into, you know, the metaverse, what would that do? But thankfully, that hasn’t happened yet. At least not at scale.

Not at scale. Not at scale. You know, I think it will have an impact at a point, Lenny, no doubt. And it’s coming, but it will take some time.

Yeah. Yeah, I agree. All right, so what else that was one that’s—I’m trying to dismiss it—it’s kind of an obvious one when we think about and looking at the data points, but you found some stuff, particularly around incentives.

Yeah, yeah. You know, again, from, from, from the thousands and thousands of people we recruit every year, we’ve got a lot of data to rely on. And so, one of the trends that has been picked up by a lot of folks and that we spend a lot of time with our clients consulting on is we have an almost an existential problem with respondent engagement. And I know that sounds like a little Chicken Little, sky is falling, but as eart—like, earlier today, I was looking at some of the attrition numbers in some of our various qualitative and quantitative panels and they are much, much bigger than they ever were pre-Covid, and a few things are driving that.

Yeah. So folks, this shouldn’t come as a surprise, right? This is why I really wanted you on, Isaac, because you’re seeing it where the rubber hits the road, right? Because of your volume, it’s unmistakable. But this is a problem has been brewing for a hell of a long time, and then we just reached the perfect storm situation.

It—well, that, that’s—we’re competing with it, so, like—

Yeah, and so we better start thinking like Uber, TaskRabbit, you know, et cetera, et cetera, as well as TikTok and [laugh] Twitter and Facebook. You know, how do we engage? How do we deliver value? We have to think like marketers because those businesses have been incredibly successful at building the value proposition for consumers to engage and perform tasks for a reward that—whether it’s social or fun or compensation. And we’ve always had this expectation as an industry of, well, no, you just do it because it’s, you know, the right thing to do, right?

Well, you know, Lenny, it’s the value prop isn’t just about the money that you have to compensate people. I think it’s a part of it, but like, it’s a fun exercise to do to talk to researchers and say, “Hey, like, let me ask you a question. If you took the last quantitative survey that your company fielded or the last qualitative screener, would you make your mother take that?” And everybody laughed. And they’re like, “What do you mean?”

Yep. Yep. Agree wholeheartedly. And brands are paying attention, too, right? And it’s not just that the usual suspects, you know, P&Gs of the world that have always kind of led the charge, but eh, you know, not much really happened.

Absolutely. And you know, I hear some really positive things from brands that are kind of new for our industry, things like, wait, are we saying that these are our buyers, these are our customers, the people who drive our revenue and we’re going to punish a thousand of them [laugh]? Like, that’s a bad brand experience. And so, I’m hearing that more and more that researchers, CMOs, folks that are in charge of the brand experience for the brand as a whole are going, “It’s not acceptable for us to punish our consumers in this way and put them through surveys.” Like, sure it’s a blinded instrument and all that kind of stuff, but still, like, this just is the wrong thing to do.

All right, so let’s add a little more insult to injury then on this conversation because here came generative AI. And, you know, and it’s not just a buzz term, is I think about the conversations that I have, both with suppliers and with brands, and particularly with brands right now, they are all-in on that promise that we had with big data back in the day of, you know, we’ll have the who, what, when, where, and how through all of the synthesis of all this data, but we couldn’t quite pull it off because we didn’t have the right technology to synthesize it. Well, now we do. And the implications that we’re seeing right now on creating virtual audiences, you know, virtual respondents, that is huge, huge.

Yeah, I think you’re right, Lenny. I mean, we obviously at Sāgo have a lot of investments in AI, we’ve got a lot of partnerships, a lot of our own technology, and we are looking across the entire spectrum of our workflows and seeing how could AI or even generative AI help. And I think, you know, the obvious areas are reporting and analysis. And, you know, there’s a lot of good work being done there; we get a lot of tools in our toolkit that help us there and it’s totally, totally assistive. And I think that’s kind of the obvious one.

Yeah, I couldn’t agree more. And so, this idea that we’re talking about, this has been—in a much less sophisticated level—been done within advertising, you know, other areas for a long time, right? The idea of utilizing—political polling. You know, targeting folks off existing data is something that is normal and customary. So, it’s not a leap to think about how we do that now.

That is our belief at Sāgo for sure that, you know, having access to those consumers, business people to help build that training data, to help project the things that nobody’s asked a question about if there is an existing data set, that is going to be a real key fundamental part of our business. And, you know, when we think about building our panel assets and we’ve got, you know, access to millions of consumers and the depth we know about them and how we start building that out, we put a lot of effort behind what data do we need to collect to help train these AI models that might not have been terribly relevant for survey data collection or qualitative collection, but could be relevant in a data training world that’s a very active conversation here at Sāgo.

Yeah, and I think—so here’s my prediction; see whether you agree. Now that I think that the generative AI thing has really put into focus the value of data as a whole. And we’re already seeing companies, I think last week, Reddit said, “Nope. We’re putting a moat around our data. You want access to Reddit content for your training sets? You’re going to pay us for that.”

Oh no, you’re not full—Lenny, that’s already happening. Like, you know, we have a fairly large amount of business these days coming from working with brands on proprietary datasets, visual data, textual data, all the things that are relevant to their brand, and they’re starting to build these datasets. And in some cases, you know, we are helping them collect the data. And they don’t even know what they’re going to use it for yet, right, but they know that they’re going to need it.

That’s great to hear, not just because I like being told that I’m right. But the—[laugh]—although I make my living by… you know, pontificating and then being right, so it’s good to have that validation [laugh]. But I think it’s a positive vision for the industry. I mean, I remember giving talks on this idea ten years ago and thinking, you know, “Look, it’s going to be scary. It’s going to be weird. It’s going to—but we will get there.”

very cookie cutter. And you know, that’s—you know, we’re all trying to run scalable businesses, and so it’s easier to repeat success than it is to go do something new every day. And I get that. I understand that.

I am multilayered [crosstalk 00:31:45]—

Oh, don’t we know? Don’t we-

[laugh]. And back to our previous point, too, the technology is making that easier now, right? I mean, before what you’re describing, there was a whole lot of work involved to make that usable.

Oh, it’s a click of a button now. And, like, like—and, you know, I still find researchers who are blown away and I’m like, “Hey look, you’re doing an online discussion, our platform, click this button and Linda gets invited to a video interview tomorrow night.” And they’re like, “Wait, is it that easy?” And I’m like, “Yeah, and it’s been that easy for a long time,” but now we see the value in it. And it’s not just about being able to get more data out of Lenny. It’s about learning more about you and learning in different ways and, honestly, providing a richer understanding of who you are as a consumer than just in a, you know, in a 60-minute online video interview.

Yeah. Yeah, agreed. And I would also say that I think—not to keep going back to it, but I think it’s relevant—that all these changes we’re talking about usher in a golden age of qual. I think that research as a whole is going to look and feel experientially be far more qualitative in the way we think about it from a discussion standpoint—I’m not talking about sample size; just the, you know, the techniques of engagement—than it will be quantitative.

I agree. And you know, as, you know, the president of Sāgo who does both qual and quant, I can’t, like—I love both of those methodologies. That being said, it is a very common refrain in our industry that, you know, qual may be in a resurgence at this point for a bunch of different reasons. Over-reliance on quantitative data, difficulty in getting the right audiences, you know, pick your poison, there’s lots of reasons why people may be incorporating qualitative more. But you know, for me, the beauty of qual is—well there lots of things I appreciate about qual, but one of the things a friend of mine once told me, “You know, Isaac, we don’t tell our kids bedtime facts, we tell them bedtime stories.”

But not in—

[crosstalk 00:35:14] it was in a photo that was captured in our [unintelligible 00:35:16] platform. And so, the researcher came back to us and said, “Hey, we got to probe what was going on there.” And turns out this person had a belief—true or not true, I’m not sure—that there was some sort of radio emission. She had a brand new child at home and so she said, “You know, better safe than sorry. I don’t know. It’s maybe weird, [maybe 00:35:33] new protected mother, you know. She got this little baby at home. [unintelligible 00:35:36] I’ve—yeah, I’ve started putting microwave around my microwave—or rather aluminum foil around my microwave, just to—just in case.”

Yeah. And what a great story. And as somebody whose wife is made—buy lots of Shungite to put all around all of the—it’s a stone—

There you, there you, there you go, Lenny. You should—

It’s an interesting question, right? So, better safe than sorry. But [laugh] as I sit here with my WiFi router, right? But that’s a great point. I remember doing research for Kimberly Clark back in the Rockhopper days on cleaning cloths and discovered that a huge audience for them for these clean—this—I forget this specific type, but it was very lush microfiber, you know—wasn’t janitorial; it was car collectors.

Makes perfect sense. And you know, those are the kind of things that I think qualitative can really help you uncover. And one of the reasons I love this industry is—because I’m always just totally curious about the human condition and finding people who have these, you know, interesting uses for baby wipes. Like, it makes me want to go do it myself, right? And so, you know, consumers are just these fascinating animals.

Yep. Absolutely, absolutely agree, right, wholeheartedly. So, I want to be conscious of your time as well as the audience because you and I could go on and on about this type of stuff. So, you did some quantitative research as well. Was there anything—or looked at your quant data—anything else that popped out of that, that we want to make sure that we—

just as big, if not bigger, challenge in the quantitative world. Incentive rates haven’t changed nearly as much, and I think it’s a conversation we need to start opening up. And as a matter of fact, our findings, which will come out a little bit later in July, I believe, we’re going to talk about the incentivization problem that we have in quant. In qual, it’s just so much more obvious because the consumers will tell you, “No, I won’t do that for $75.” Whereas in quant they will just click away and so you don’t know. Did they abandon the survey because it wasn’t worth their time or did they abandon it because you got too many stupid questions?

Yeah. Right, there with you. And, yeah, we can go off on another tangent on that, but I do want to put the point on it that we talked about this idea of synthetic sample or virtual respondents. The use case seems to be early exploratory kind of concept, understanding audiences at a basic level. Hypothesis, right, hypothesis-forming, which I think is a logical use case.

It’s an honor to be in that position. And you know, one of the ways that we’re trying to help—I don’t want to say, “Give back,” but, like, make sure the industry has some visibility into these things is these reports that we’re doing to, you know, just show trends and things. Because at the end of the day, like we want, you know, the industry to know what’s going on, so we’re just more than happy to help.

Yeah. Always appreciate that about the DNA of the company as a whole, right? When you were Schlesinger, you did the same thing. At Sāgo, you’re doing it. It’s just thrilling. And by the way, for an audience who doesn’t know the story, right, this is a company that started literally at a dinner table.

That’s right. That’s right. I’ve actually met researchers who debriefed their clients in Steve’s childhood bedroom [laugh].

[laugh]. So, what a success story with that, too, right? I mean, you obviously have been inside, but I’ve been privileged to have some views over the years of what was going on and it’s just always been so cool. Just, like, a great story. So, for entrepreneurs out there, right, all these things we’re talking about, they are opportunities as well, and you guys are an example of… literally [laugh] you know, starting it in your house and creating now one of the largest research companies in the world.

Yeah. And you know, and on that note, innovation, like you know, keeping an eye to the future because, you know, we have been able to grow and thrive and change because the DNA of this organization is that we are looking to what’s going to come next. If we had stuck in, you know, Steve’s mom’s kitchen table, we would have never grown. If we would have stuck doing just in person, we would have never grown. If we had stuck not getting into quantitative research, not getting into digital qual, not getting into global professional services, we would have not become the company we are today, so you’ve always got to keep an eye on the future. And I think that’s why you know, folks like you educating people on what the future is going to look like, helps firms like us place those bets and really move the industry forward.

Oh, great. I love it when we have a mutual admiration society. So, anything else that you want to share with the audience, Isaac?

You know, Lenny, I think this has been a fantastic session. I really thank you for hosting me. I do want to point out, we will be releasing our quantitative analysis report, kind of the state of the qualitative industry, sometime mid-summer, so stay tuned for that. And hopefully, maybe you’ll have me back.

We’d love to. It’s a great conversation. I know that there’ll be many more, definitely, and let us know—GreenBook—when you release those reports.