Transcripts
Hey, Megan, let's do a podcast. Great idea. What
Speaker:should we talk about?
Speaker:One of the really, really fantastic things about
Speaker:my job in the universe is that I get to meet all sorts
Speaker:of really fantastic people and have really interesting
Speaker:conversations. And I've decided to share those
Speaker:conversations. So, Sam Rogers, welcome. It is
Speaker:so exciting to finally get to have this
Speaker:conversation — actually, I've been thinking about this conversation for the last few
Speaker:months — and to dig into your
Speaker:work and where you are and where you see things right now.
Speaker:So can you start off by introducing yourself? Sure.
Speaker:Yes. Hi, Megan. Thanks for having me. I'm Sam Rogers,
Speaker:founder and CEO of PAICE dot Work. I also run Snap Synapse,
Speaker:where I've spent over two decades helping organizations with their
Speaker:learning management systems, creating learning content, more generally
Speaker:making change stick. For years, I was the guy
Speaker:that companies called when their learning ecosystem wasn't working right. And
Speaker:have helped with migrations, integrations, workflow fixes, all that unglamorous
Speaker:stuff that makes learning manageable at work.
Speaker:Well, and I love the different directions at which
Speaker:you come at this work, and a lot of those nuts and bolts,
Speaker:Sam, make it actually happen in the world,
Speaker:right? Like, it's one thing to design a program, but if you can't make it
Speaker:happen in the world, it's like bears in the woods
Speaker:and whatever bears are supposed to be doing in woods. So I,
Speaker:you know, I also appreciate — the boring, nerdy
Speaker:person in me sees the boring, nerdy person in you and actually making
Speaker:these out. You and I met at Learning Dev camp years
Speaker:and years ago, back
Speaker:when we were just children, I'm sure. You were doing
Speaker:a lot of video and learning design for video, and
Speaker:you also share
Speaker:this ecosystem framework, or
Speaker:sense about you. And you were also one of the first LXD
Speaker:people that I knew that went really deep into AI. Lots and lots of
Speaker:people did, but you're one of the
Speaker:ones who was very early in helping
Speaker:people get comfortable with it, helping people
Speaker:do smart things with it, right? And
Speaker:although I will say that I remembered you made a playlist for
Speaker:Learning Dev Camp that was so catchy — an
Speaker:AI-generated playlist — it was so catchy that I caught myself two weeks later,
Speaker:you know, just kind of humming something in the shower, and I'm like, where did
Speaker:this song come from? Where did this song come from? Where did this song...? And
Speaker:all of a sudden I started singing and realized it was the Learning Dev Camp
Speaker:song. It came from AI.
Speaker:Wow, that is creepy. So, but,
Speaker:when you and I chatted at Dev Learn, right, you're
Speaker:getting really strategic about this and looking
Speaker:forward to the kinds of things that we will need
Speaker:as an industry, as a society, as an ecosystem, not just
Speaker:as learning designers, right? And it's around the assessment of an
Speaker:individual's and an organization's actual capabilities
Speaker:with AI — not just usage. So
Speaker:how did you get to this point? How did you realize we needed this?
Speaker:Well, for better or for worse, I never really think in terms
Speaker:of like what L&D needs. I think more generally about,
Speaker:you know, business needs. And every business
Speaker:needs to use AI well, and of course, most
Speaker:don't yet, right? Because it's new to the business workflow.
Speaker:But they don't even really have the best way to
Speaker:talk about, let alone measure, what good looks like
Speaker:in this new context. So these are
Speaker:problems that I'm solving with PAICE. I built
Speaker:a measurement and workflow system to
Speaker:make AI adoption teachable, governable,
Speaker:real. Not just that people tried the tool, but that it's actually
Speaker:producing outputs that people value. I
Speaker:love it. I love it. Right, so it's not just, "did you use it? Did
Speaker:you try it?" But "are you using it well?" I think that's fantastic.
Speaker:Okay, so stop for a minute. Let's look
Speaker:at PAICE dot work, and I want to start with the super basics. What does
Speaker:PAICE dot work stand for? It's an acronym, right?
Speaker:It is. It's P A I C E
Speaker:and it stands for People plus AI Collaboration
Speaker:Effectiveness. It's a framework basically for measuring
Speaker:if people — how well people can work with AI safely,
Speaker:repeatably, in a way that can be taught, managed, and governed. Here
Speaker:we are at the end of 2025, when lots of orgs have started tracking
Speaker:AI usage. And that's a start. But really
Speaker:I'm more interested in capability. That is the shift where
Speaker:L&D becomes truly essential. I
Speaker:actually use the acronym P A I C E in a couple
Speaker:places. It's also the capability
Speaker:measures of Performance, Accountability,
Speaker:Integrity, Collaboration and Evolution that spells out
Speaker:PAICE. I actually — I think I came up with that one first, but
Speaker:there's also the scoring tiers and the products
Speaker:and all that kind of thing. I maybe went a little too hard on the
Speaker:branding, but having a common mnemonic is definitely helpful.
Speaker:Absolutely. And I really like
Speaker:— common to both of those is the word "collaboration," right?
Speaker:And so I think that that's the key with seeing AI tools
Speaker:as being a collaboration. It's a new kind of partner. I
Speaker:hesitate to anthropomorphize it, but it is a
Speaker:different kind of work. And so I'm really excited
Speaker:about the collaboration, but at the same time, in order to implement tools, in order
Speaker:to use tools well, the people need to collaborate and be able to
Speaker:have different kinds of conversations as well. So I'm really digging the
Speaker:collaboration space on this. Yeah, great, me too.
Speaker:So typically, right when, when I talk to organizations and
Speaker:we say, what are you measuring with
Speaker:your AI tools? They'll say, "wait, wait, wait, wait, wait, wait!" It's too early to
Speaker:be measuring results and business and all this stuff and everybody tries to, but there's
Speaker:this conversation around, should we be measuring dollars? Which
Speaker:is one piece of things, but we also look at, right, organizations
Speaker:are asking their people, "are you using tools?" So self reporting tools usage.
Speaker:Depending on your scale
Speaker:in the organization, your tools actually keep track of who
Speaker:uses them and for how long a session and how many minutes and how
Speaker:many sessions a week and whatnot. That is only though
Speaker:— it's notably — that's only on the approved and paid
Speaker:for tools by the company. Not all the things that you might bring along
Speaker:for the ride as well. People are tracking
Speaker:completion of required training, right? "When we pushed out
Speaker:the compliance training about AI, did you, did you take it?" We all know
Speaker:what an effective learning metric that is. And then
Speaker:several of my clients
Speaker:are, you know, they're — how many of their people, how many of their client-facing
Speaker:people have certified, you know, completed an AI
Speaker:certification program or something like that, right? So still a measure
Speaker:of activity, but kind of a
Speaker:measure of "I have completed a
Speaker:thing." What does PAICE dot work measure,
Speaker:actually? Yeah, I'm glad
Speaker:you brought up the training itself is a perfect
Speaker:analogy here. So the old butts-in-seats metric, you know,
Speaker:it doesn't tell us a whole lot. We know that.
Speaker:We don't just want to show the activity-based checkboxes of attended,
Speaker:and completed, and passed, and — all those things are needed, certainly. They're
Speaker:just not enough. They never really were enough.
Speaker:But they're just the first hurdle that we had to clear in any
Speaker:kind of learning intervention. But very soon after we clear that bar,
Speaker:the next one comes into focus, which is what difference does it make? So
Speaker:just like with any learning intervention, AI can
Speaker:make the kind of difference that the business is hungry for or it
Speaker:can do something else that isn't that. What PACE
Speaker:aims to do is quantify the risks of AI,
Speaker:not on a technological basis, but on the human level.
Speaker:So the shorthand I often give is, it's like FICO for
Speaker:AI risk. That's a very US-centric term, I know, but
Speaker:PAICE is global in scope. It's basically
Speaker:helping the organization assess what are the risks
Speaker:of giving these high-powered tools to our workforce.
Speaker:I think that's really, really
Speaker:so important. I remember an early conversation I had with Josh Cavalier,
Speaker:and there's conversation about why are some
Speaker:organizations adopting AI and why are so few doing it.
Speaker:This was several years ago and we had this conversation around
Speaker:organizational risk. These tools do things, these
Speaker:tools have access to a lot of things. And the, the gateway
Speaker:to that, by the way, are humans. But the,
Speaker:because of the — the possibility of risk is so
Speaker:great, both frequency and impact, that...
Speaker:this is, this is different. This is different than rolling
Speaker:out calculators. This is different than rolling out the internet. This is different.
Speaker:So I think this is such important work.
Speaker:You've written a white paper, and I've had a
Speaker:chance to dig in and give a
Speaker:round or two of thoughts on that, which was so much
Speaker:fun. Thank you so much for that opportunity. There's a couple
Speaker:of terms now we've got PAICE dot work covered. There's a couple terms that
Speaker:I'd like for you, in this context, to
Speaker:discuss and define a little bit more. One is, what do you mean by "AI
Speaker:collaboration"? Yeah, well, well, first I just want to say again,
Speaker:thank you, Megan, for your time and generosity on that version of the white
Speaker:paper. I've since made some pretty substantial revisions that incorporate
Speaker:the feedback that you and others have provided. The new version is much
Speaker:shorter and introduces fewer terms that need definition. But, but
Speaker:the "AI collaboration" one that you kind of foreshadowed there,
Speaker:yeah. What do we mean when we talk about AI collaboration?
Speaker:Because it's not people using AI, it's
Speaker:the ability of a person or a team to reliably
Speaker:produce good work with AI in the loop,
Speaker:with its known limitations, with review checkpoints
Speaker:with the required organizational
Speaker:accountability. So in L&D terms, it's
Speaker:a performance system. This isn't like skill badging or certifications.
Speaker:It's a tool-agnostic means of aggregating
Speaker:human intelligence and AI intelligence
Speaker:to output work.
Speaker:Love it. Love it. Okay, so thinking about
Speaker:that then, what do we mean by "governance readiness"?
Speaker:Yeah, we've seen a lot about AI
Speaker:readiness in general and well, "ready or not, here it comes." At
Speaker:PAICE, we refer more to governance readiness, which
Speaker:is the organization's ability to scale AI without all
Speaker:the confetti. I'm not opposed to confetti and celebrating things; I've
Speaker:been known to throw some confetti on occasion. But this
Speaker:version of readiness includes
Speaker:guardrails, it includes review cycles, it includes escalation
Speaker:paths, ownership; it's hard to take readiness
Speaker:as a term seriously with those elements
Speaker:missing. So that's what we mean when we talk about
Speaker:governance readiness. I kind of like that.
Speaker:So a lot of times "readiness" is in a conversation around literacy and
Speaker:individuals' skill sets, or
Speaker:change capacity, or interest and
Speaker:willingness, right? "Readiness," "willingness," right? But governance is
Speaker:a word nobody really, or very few
Speaker:people, really get excited about. But governance is
Speaker:what we need to make sure that we're doing this in a sane and
Speaker:orderly way that is safe and effective for the
Speaker:organization overall. The readiness to have that
Speaker:governance infrastructure, I think is just at a level above both of
Speaker:those. So I kind of love that a lot.
Speaker:Great. Okay, so behavioral
Speaker:observation is the next thing I want you to dig in
Speaker:because PAICE is not a multiple choice test.
Speaker:That's right, yeah. So, we can
Speaker:— we're very familiar in L&D with testing what people know about
Speaker:something. That doesn't mean that they're going to do anything
Speaker:or that their knowledge would be predictive of what they would do
Speaker:or how people feel about something, you know, with a sentiment survey or something like
Speaker:that. The reason I made PAICE is
Speaker:because those things don't "matter harder"
Speaker:when it comes to AI. Like there's the
Speaker:conversation we have in L&D all the time about trying to get closer
Speaker:to the things that actually matter and what is actually predictive
Speaker:of behavior. But the best thing to observe is behavior
Speaker:itself. So that's what PAICE does.
Speaker:That's what it's built to do. And it's not,
Speaker:it's not asking you what you know about AI. Which is great because
Speaker:you don't actually have to know a lot about AI in order
Speaker:to use AI well, just like you don't have to be a great
Speaker:mechanic to be a great driver, right? Like I
Speaker:want to get this thing where I want to go. The ability
Speaker:to do that is really what PAICE is focused on, of
Speaker:watching that. In the same way that you take a driver's
Speaker:test; there's the written part of the test. Yes, you have to know
Speaker:stuff. But then someone sits in a car with you and they take you on
Speaker:a route that is unpredictable, that maybe you've never been on
Speaker:before. And they're watching how you perform
Speaker:in that circumstance and giving you your
Speaker:license based on how you perform against that
Speaker:rubric. That's the best analogy for what PAICE is
Speaker:doing by watching how people use AI in
Speaker:this kind of simulation circumstance and
Speaker:then being able to provide very specific
Speaker:feedback and ratings based on that assessment.
Speaker:I love this. And actually the way in which
Speaker:you're doing this with PAICE dot work has become my new example
Speaker:of when I'm, when I'm describing Will
Speaker:Talheimer's Learning Transfer Evaluation model. And people are like, "well, what's the difference between
Speaker:a tier 6 decision competence and a
Speaker:tier 7 — sorry, tier 5 decision competence, 6 task competence" — like
Speaker:task competetence. We actually watch somebody do it.
Speaker:And this is a great example by the way.
Speaker:So super, super cool. Now, but here's the thing; as I
Speaker:think back to my own driver's test and what a nerve-wracking experience that was,
Speaker:but that gave me the ticket to do something, right? It literally gave me the
Speaker:keys, right? Or license to drive.
Speaker:This kind of measurement is a
Speaker:bold and maybe challenging move from a
Speaker:measurement perspective, right? I can imagine it makes some people
Speaker:uncomfortable to be assessed this way, particularly
Speaker:if they like to think of themselves as having great skills. Everybody is an above
Speaker:average driver, right? By some miracle of metrics,
Speaker:everyone's above average. Yeah, everybody is above — like, "I'm a good driver, of course!"
Speaker:And what are
Speaker:some of the change components that an organization should
Speaker:be thinking about if they are looking
Speaker:at something like PAICE dot work? Well, the
Speaker:first thing to say is that I built
Speaker:PAICE dot work on Privacy by Design principles. So
Speaker:as far as like the person who sat in the car with you and
Speaker:your nerve-wracking experience of your driver's test, PAICE
Speaker:is built to scale
Speaker:that kind of individual assessment and
Speaker:observation, but it's scaling it using AI, so
Speaker:there's not actually a person watching. So when you go through the
Speaker:assessment, no one knows that you did. No one
Speaker:sees your score unless you choose to show it off. Teams
Speaker:and organizations can still see that their people did something in
Speaker:aggregate. They can still see any gaps and dangers that exist
Speaker:at a higher-level view, which is really what they want anyway.
Speaker:Well, most of the time, I should say. Anyone who's been responsible
Speaker:for compliance reporting knows that there's
Speaker:inevitably a persecution motive that develops within the
Speaker:organization of, you know, "let's fire all the low
Speaker:scores" or the, you know, all that kind of thing that happens. I've designed
Speaker:PAICE to be immune to that. So not even I
Speaker:know how specific individuals are scoring. I just
Speaker:see that anonymized individuals are getting their scores.
Speaker:That way I can't be coerced into helping fire people. You know, the
Speaker:platform can't be used that way. But
Speaker:eventually PAICE dot work will probably have some competition.
Speaker:I'm happy to let the competition take that
Speaker:legally-riddled and misaligned work. Our mission is
Speaker:to enable safer and more effective
Speaker:people-plus-AI collaboration by providing
Speaker:independent capability measurement.
Speaker:We're also a public benefit corporation, so we're fully
Speaker:committed to scaling a profitable company, but we're structured so that we
Speaker:never have to choose between profit and principle.
Speaker:So that's important for people to know: You're not being
Speaker:observed in that traditional sense. This is a new capability that AI
Speaker:opens up that has never really existed before. And it's also important for
Speaker:organizations to know that what you're getting is a view
Speaker:of your people, not a view of individuals,
Speaker:like in a learning management system or something. Okay. And
Speaker:that's, that's perhaps different than what people are expecting from
Speaker:L&D or from their organizations. So probably an important message.
Speaker:I will find the link and add it to
Speaker:this podcast when we publish it, but there was an interesting piece
Speaker:of research that was done with one of the large recruiting
Speaker:firms and they conducted a trial in which
Speaker:they allowed applicants to either interview with a
Speaker:human interviewer, an AI interviewer — actually,
Speaker:they didn't — group one had to interview with a human.
Speaker:Group two had to interview with an AI. And group three
Speaker:got the choice. And there was some interesting,
Speaker:interesting findings that came out. One was that people who
Speaker:interviewed with the AI tools actually scored higher.
Speaker:Humans did the scoring. It was just the interview was
Speaker:with the AI humans did the scoring. And so some of
Speaker:their thinking was that it freed people up. They didn't feel quite so, so
Speaker:judged. And so that's interesting.
Speaker:What they also found in that third group where they allowed people to choose
Speaker:was that women and minorities
Speaker:tended to choose the AI more frequently. Yeah. And
Speaker:their thinking behind this and some of their follow up
Speaker:was that they were feeling like they would be more assessed as an
Speaker:individual and not based on their social identity
Speaker:characteristics. I thought, "mind blowing." Yeah. This is a tool for
Speaker:equity and participation.
Speaker:Yes. And they're probably right, is my
Speaker:instinct. The studies have yet to be done,
Speaker:but AI can amplify whatever it is that we
Speaker:want to focus on and use it for. So one of the big worries is
Speaker:certainly security and privacy
Speaker:and all those kinds of things. I've directed PACE to use
Speaker:AI to make it impossible to deconstruct
Speaker:an individual's identity from what it is that's being gathered. Like if we
Speaker:aim it at Privacy by Design, it stays private. If we
Speaker:aim it towards equity, it works great for that. When we aim
Speaker:towards accessibility, it works great for that too.
Speaker:It's a matter of how we drive that AI output to
Speaker:where it is that we choose to go, where we want to go. So
Speaker:I get a little meta about that and I'm doing my best to
Speaker:practice what we're preaching with PAICE.
Speaker:I love it. I absolutely love it. Okay,
Speaker:this has been fascinating, but I want to take a
Speaker:slight shift here. I am always
Speaker:curious about people's work quirks.
Speaker:What are productivity habits — things that make you either
Speaker:super effective or super annoying to the people around? The
Speaker:variety is always interesting and surprising. I always learn something more about people that
Speaker:I didn't know before. Do you have any favorite habits
Speaker:or tips? Your favorite fidget, your favorite distraction?
Speaker:So I'm probably a pretty quirky worker. I get along well in
Speaker:anybody's office. But 25 years ago I was the only person with three
Speaker:computers on my desk. One development box, one kind of
Speaker:handicapped baseline computer for testing, my own personal laptop.
Speaker:Today I'm still doing something like that. Three computers on
Speaker:my desk now, but I'm also working across
Speaker:ChatGPT, Claude, Gemini, Perplexity, usually a local model like
Speaker:Deepseek, all at once. And I find myself
Speaker:serving as the human transport layer between them all very
Speaker:often. I have my own Obsidian vault, which I
Speaker:am kind of obsessive over. That's where text and
Speaker:all the other files land. I'm like Markdown native,
Speaker:so anything that's coming to me, I'm
Speaker:often like porting in and out of Markdown,
Speaker:which is just a basic text format. And I
Speaker:do all of my, all of my jotting still in a paper
Speaker:notebook, I think, which is... which is probably a
Speaker:quirk at this point. I use mine constantly because just of the
Speaker:quality of thinking that comes out is just better for certain things.
Speaker:What I don't use unless I have to
Speaker:is mobile devices. I was one of the first independent
Speaker:iOS developers long ago. I taught my smartphone cinema workshops
Speaker:for years. I still believe in the promise, but I can't ignore the history
Speaker:since then. So, yeah, I would
Speaker:say don't text me stuff so much. I'm
Speaker:not the guy that wants to do all of that on my phone.
Speaker:And I would encourage anybody who wants to create things that matter to stop
Speaker:consuming things that you don't... don't like through your phone.
Speaker:The costs outweigh the benefits, I think. So
Speaker:I'm always the one trying to find a better way to like
Speaker:not go through the phone for everything. Does that count?
Speaker:It totally counts. Maybe not what one would have expected from somebody
Speaker:who's got three computers running all the time. So I love it. I love it,
Speaker:love it, love it. Sam, thank you. Thank you so
Speaker:much for hanging out with me for a few minutes. Thank you so much for
Speaker:the work that you have done and are doing
Speaker:for the industry. And I will, I am
Speaker:sure I will bump into you at some interesting conference or some interesting
Speaker:conversation again in the future. So thank you so much. Yeah, always
Speaker:a pleasure, Megan. Thanks so much.
Speaker:So how'd that go? Megan? That was so much fun.
Speaker:I've known Sam for years and just really
Speaker:appreciate — he's always surprising, right?
Speaker:So he's this kind of easygoing, unassuming guy
Speaker:and then he starts talking and you're thinking, oh my gosh, he
Speaker:is thinking about things at a very deep level. And I appreciate
Speaker:that so much about him. And he's so willing to share and make it
Speaker:practical with other people. And that's something really
Speaker:awesome about him. He came up to me at DevLearn and literally
Speaker:gave me a paper print out of his white paper
Speaker:and asked for feedback. And I thought, well, that's
Speaker:cool. That gives me something to do on the plane. And so old school.
Speaker:Like he was talking about on paper and pen. Old school. I go out and
Speaker:I'm like scribbling in my thoughts and I'm drawing pictures
Speaker:and, "what do you mean by this?" And "you've got two of those." And "I'm
Speaker:kind of confused here," but "this is really cool. You got to amplify this."
Speaker:And then I had these, like, 18 pages
Speaker:of handwritten notes, scribbles all over, like, how do I
Speaker:get... do I put this in the mail? Do I like, what
Speaker:do I do? So no kidding. I filmed it with my
Speaker:phone. I like, took my phone,
Speaker:and of course after Sam said he doesn't like to use phones. But anyways,
Speaker:the the video camera in my phone and I talked
Speaker:through all of my notes, and it was probably the weirdest way to give
Speaker:feedback, but worked,
Speaker:so. And it was a fun paper to dig into. So
Speaker:it was great to have this conversation. It's cool to see the iteration and the
Speaker:evolution of the work that he's doing. I know you've got
Speaker:one more thing to share, Megan. I myself am really
Speaker:eager to get in and try the tool out,
Speaker:both as an individual and maybe as an organization.
Speaker:And I think that this kind of thing
Speaker:is something we'll see, in the coming years, ore and more of. Sam
Speaker:mentioned he'll have competitors. He absolutely will have competitors
Speaker:for this. But what I love is that this first one
Speaker:is like a research project project. It is
Speaker:an amplifier for us. So super cool stuff.
Speaker:I love my job. This is Meg Fairchild and Megan
Speaker:Torrance and this has been a podcast from
Speaker:TorranceLearning. Tangents is the official podcast of
Speaker:Torrance Learning (as though we have an unofficial one). Tangents
Speaker:is hosted by Meg Fairchild and Megan Torrance. It's
Speaker:produced by Dean Castle Castile and Meg Fairchild, engineered and
Speaker:edited by Dean Castile, with original music also
Speaker:by Dean Castile. This episode was fact checked
Speaker:by Meg Fairchild.
