#122 Learning and Teaching in the Age of AI, with Hugo Bowne-Anderson

Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!

Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work!

Visit our Patreon page to unlock exclusive Bayesian swag ;)

Takeaways:

Effective data science education requires feedback and rapid iteration.
Building LLM applications presents unique challenges and opportunities.
The software development lifecycle for AI differs from traditional methods.
Collaboration between data scientists and software engineers is crucial.
Hugo's new course focuses on practical applications of LLMs.
Continuous learning is essential in the fast-evolving tech landscape.
Engaging learners through practical exercises enhances education.
POC purgatory refers to the challenges faced in deploying LLM-powered software.
Focusing on first principles can help overcome integration issues in AI.
Aspiring data scientists should prioritize problem-solving over specific tools.
Engagement with different parts of an organization is crucial for data scientists.
Quick paths to value generation can help gain buy-in for data projects.
Multimodal models are an exciting trend in AI development.
Probabilistic programming has potential for future growth in data science.
Continuous learning and curiosity are vital in the evolving field of data science.

Chapters:

09:13 Hugo's Journey in Data Science and Education

14:57 The Appeal of Bayesian Statistics

19:36 Learning and Teaching in Data Science

24:53 Key Ingredients for Effective Data Science Education

28:44 Podcasting Journey and Insights

36:10 Building LLM Applications: Course Overview

42:08 Navigating the Software Development Lifecycle

48:06 Overcoming Proof of Concept Purgatory

55:35 Guidance for Aspiring Data Scientists

01:03:25 Exciting Trends in Data Science and AI

01:10:51 Balancing Multiple Roles in Data Science

01:15:23 Envisioning Accessible Data Science for All

Thank you to my Patrons for making this episode possible!

Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor,, Chad Scherrer, Zwelithini Tunyiswa, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Ian Moran, Paul Oreto, Colin Caprani, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Michael Hankin, Cameron Smith, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, Robert Yolken, Or Duek, Pavel Dusek, Paul Cox, Andreas Kröpelin, Raphaël R, Nicolas Rode, Gabriel Stechschulte, Arkady, Kurt TeKolste, Gergely Juhasz, Marcus Nölke, Maggi Mackintosh, Grant Pezzolesi, Avram Aelony, Joshua Meehl, Javier Sabio, Kristian Higgins, Alex Jones, Gregorio Aguilar, Matt Rosinski, Bart Trudeau, Luis Fonseca, Dante Gates, Matt Niccolls, Maksim Kuznecov, Michael Thomas, Luke Gorrie, Cory Kiser, Julio, Edvin Saveljev, Frederick Ayala, Jeffrey Powell, Gal Kampel, Adan Romero, Will Geary, Blake Walters, Jonathan Morgan, Francesco Madrisotti, Ivy Huang, Gary Clarke, Robert Flannery, Rasmus Hindström, Stefan, Corey Abshire and Mike Loncaric.

Links from the show:

Alex's last paper, “Unveiling True Talent: The Soccer Factor Model for Skill Evaluation”: https://arxiv.org/abs/2412.05911
Associated code and data: https://github.com/AlexAndorra/football-modeling/tree/main/40_submissions/MIT_Sloan_2025/01_Paper
Hugo on Blue Sky: https://bsky.app/profile/hugobowne.bsky.social
Hugo’s website: https://hugobowne.github.io/
Hugo on Linkedin: https://www.linkedin.com/in/hugo-bowne-anderson-045939a5/
Hugo on GitHub: https://github.com/hugobowne
Vanishing Gradients podcast: https://vanishinggradients.fireside.fm/hosts/hugobowne
High Signal podcast: https://high-signal.delphina.ai/
25% discount on Hugo’s course on Building LLM Applications: https://maven.com/hugo-stefan/building-llm-apps-ds-and-swe-from-first-principles?promoCode=LEARNBAYES25
Lightning lessons if people want to get a sense of Hugo's teaching style and content:
https://maven.com/p/38a781/building-with-gen-ai-from-first-principles?utm_medium=ll_share_link&utm_source=instructor
https://maven.com/p/a6f9bf/mastering-llm-application-testing?utm_medium=ll_share_link&utm_source=instructor

Transcript:

This is an automatic transcript and may therefore contain errors. Please get in touch if you're willing to correct them.

Speaker: 00:00:04

Today, am thrilled to host Hugo Baron Anderson, an independent data and AI consultant with

a remarkable career spanning education, podcasting, and building cutting-edge AI

2

: 00:00:18

applications.

3

: 00:00:19

Hugo is the creator of the Vanishing Regions and High Signal podcasts, where he explores

the latest developments in AI and data science, and he's also known for his impactful

4

: 00:00:30

teaching, having developed over

5

: 00:00:32

30 courses on DataCamp, reaching millions of learners worldwide.

6

: 00:00:38

In this episode, Hugo shares his journey from mathematics to data science.

7

: 00:00:42

We dive into his new course on building large-language model applications designed to help

data scientists and software engineers collaborate effectively.

8

: 00:00:51

Hugo explains the challenges of working with LLMs from Pug Purgatory to the importance of

first principles and how focusing on problem solving can overcome common hurdles.

9

: 00:01:02

in deploying AI-powered software.

10

: 00:01:04

Hugo also provides actionable advice for aspiring data scientists, emphasizing the value

of continuous learning and engaging with real-world problems over chasing specific tools.

11

: 00:01:16

This is Learning Vision Statistics, episode 122, recorded December 10, 2024.

12

: 00:01:29

Welcome to Learning Bayesian Statistics, a podcast about Bayesian inference, the methods,

the projects, and the people who make it possible.

13

: 00:01:49

I'm your host, Alex Andorra.

14

: 00:01:51

You can follow me on Twitter at alex-underscore-andorra.

15

: 00:01:55

like the country.

16

: 00:01:56

For any info about the show, learnbasedats.com is Laplace to be.

17

: 00:02:00

Show notes, becoming a corporate sponsor, unlocking Bayesian Merge, supporting the show on

Patreon, everything is in there.

18

: 00:02:07

That's learnbasedats.com.

19

: 00:02:09

If you're interested in one-on-one mentorship, online courses, or statistical consulting,

feel free to reach out and book a call at topmate.io slash alex underscore and dora.

20

: 00:02:20

See you around, folks.

21

: 00:02:22

and best patient wishes to you all.

22

: 00:02:24

And if today's discussion sparked ideas for your business, well, our team at Pimc Labs can

help bring them to life.

23

: 00:02:31

Check us out at pimc-labs.com.

24

: 00:02:36

Hello, my dear patients!

25

: 00:02:38

A few days ago, Max Goebel and I submitted a paper to the 2025 MIT Sloan Sports Analyst

Conference titled, Unveiling True Talent, The Soccer Factor Model for Skill Evaluation.

26

: 00:02:51

In there, we tried to isolate soccer players' individual contributions from team effects,

which is no small feat in soccer.

27

: 00:02:59

For this study, Max went and scraped publicly available sources, spanning over 33,000

matches

28

: 00:03:06

and more than 140 players.

29

: 00:03:10

And we're using Hilbert Space Gaussian Processes because, well, they are awesome.

30

: 00:03:16

I could talk about it for hours, so I'll stop here.

31

: 00:03:18

But if you're curious, I put the archive link in the show notes as well as the GitHub repo

with all the data and code to reproduce our analysis, done with BIMC, of course.

32

: 00:03:29

Hope you'll enjoy and feel free to reach out because this is still a work in progress.

33

: 00:03:34

And even though we already have

34

: 00:03:36

lot of ideas and we're already working on improvement, we're currently listening to

feedback.

35

: 00:03:41

So please reach out if you want.

36

: 00:03:44

Okay, on to the show now.

37

: 00:03:49

Hugo Baron Anderson, welcome to Learning Bayesian Statistics.

38

: 00:03:54

Alex and Dora, is such a pleasure to be on your show.

39

: 00:03:57

Thank you for having me.

40

: 00:03:58

Yeah, really thank you for taking the time.

41

: 00:04:02

And this is really fun because I've known you for a long time.

42

: 00:04:07

I've actually taken a lot of your classes on DataFrame when I started learning, like

really learning Python programming and then stats.

43

: 00:04:18

And then patient sense.

44

: 00:04:20

that was back in 2017 to 2019.

45

: 00:04:25

So that's really cool to have you here because like, um, I remember I listened a lot to

all the episodes of data framed, uh, which is the first blog podcast I knew about you.

46

: 00:04:36

And then like you, you've done so many things and we'll talk about that today.

47

: 00:04:40

But, uh, yeah, I'm really happy to, uh, get the opportunity to talk with you today.

48

: 00:04:45

And the first thing is that I met you in person that

49

: 00:04:47

point data New York in last November.

50

: 00:04:51

And that was with Chris Fonsbeck, who is also like the first person I watched the

tutorials from when I started learning Bayesian statistics.

51

: 00:05:00

you know, absolutely.

52

: 00:05:01

And Chris has always been a huge inspiration to me in all the Bayesian stuff he does, but

all the education outreach he does as well.

53

: 00:05:09

And I think we talked about this, but I remember he

54

: 00:05:11

He had a tutorial maybe at PyCon or something like that.

55

: 00:05:14

The GitHub notebooks are still up there, but it was called Introduction to Statistics in

Python or something like that.

56

: 00:05:19

And it was Bayesian.

57

: 00:05:20

I said, why didn't you call it Bayesian?

58

: 00:05:22

And he said, because it's statistics, Hugo.

59

: 00:05:24

And I was like, yeah, right on.

60

: 00:05:28

Yeah, that's a good one.

61

: 00:05:30

Yeah.

62

: 00:05:31

Yeah.

63

: 00:05:32

I mean, yeah, Chris is super, a great inspiration.

64

: 00:05:37

I really love how he teaches.

65

: 00:05:40

That was really an honor to be able to teach with him at PyData.

66

: 00:05:45

And of course, and he'll be back on the show after your episode in a few episodes

afterwards, you folks are going to be able to hear from Chris.

67

: 00:05:54

We recorded actually at PyData just before our tutorial.

68

: 00:05:59

So yeah, I can tell you if you want to hear about, you know, almost 10 years of experience

in the Bayesian world and

69

: 00:06:08

sports analytics realm listen to that one also you know which lessons you can you can take

from so long in the industry and and where pimes he is going well definitely tune in for

70

: 00:06:21

that one folks and also you're the two people i know who work in bays and baseball and of

course i made the baseball joke which you've made many times before but i'm excited to

71

: 00:06:30

have you both on my podcast one of my podcasts together at some point because i actually

do think

72

: 00:06:36

What we see in sports analytics is a leading indicator of a lot of what happens in the

field.

73

: 00:06:40

A lot of the techniques that are applied industrial data science now, of course, we all

know the moneyball story, right?

74

: 00:06:46

You know, like a lot of these things happened in sports analytics first.

75

: 00:06:50

So I'm very excited for that.

76

: 00:06:52

Yeah, that's actually a very good point.

77

: 00:06:53

I mean, I think it's it's in part because, there is there is a lot of money in sports.

78

: 00:07:00

That's the reason.

79

: 00:07:01

So that's the reason.

80

: 00:07:03

And engagement, but it's money, Right?

81

: 00:07:06

Yeah.

82

: 00:07:06

We have a lot of money, so that means we can make more.

83

: 00:07:09

We have better data, and that means we can make more experiments.

84

: 00:07:15

And also, since sports is more, know, like there is rules everywhere.

85

: 00:07:19

So it's a very contained environment.

86

: 00:07:22

So it's easier to make experiments, right?

87

: 00:07:25

Because you control more, because it's a completely invented game, right?

88

: 00:07:31

of that's in a way easier to contain that and do the experiments and then you can validate

the methods and then if they are validated then you can use them in in the broader world

89

: 00:07:41

let's say exactly and this may be a place we go to but i am deeply interested in

90

: 00:07:45

how much of the techniques, tools, methodologies, processes we use are driven by commerce

as well.

91

: 00:07:50

And the famous example that I love is William Gossett at the Guinness Brewery, right?

92

: 00:07:56

Creating the students tea test in order to check out crop yields and what type of alcohol

they produced and that type of stuff, which was literally driven by the commercial needs

93

: 00:08:06

and desires of the Guinness Brewery at the time.

94

: 00:08:10

Yeah, yeah, yeah.

95

: 00:08:11

I love that.

96

: 00:08:12

I love that story and

97

: 00:08:14

So I don't know if you, if you visited the Guinness museum in Dublin, but that's like,

that's really cool.

98

: 00:08:20

I recommend going there.

99

: 00:08:21

And like when I went there, of course I was, know, like it is very hard because if you

don't go there with fellow nerds, you know, you want to like get into the details of that,

100

: 00:08:34

of that story, which is very nerdy, but you can't really, because they don't care.

101

: 00:08:38

It's just like, it's very frustrating.

102

: 00:08:39

I recommend going there with some, you know, fellow open source developers.

103

: 00:08:43

I think that's a great idea.

104

: 00:08:44

Otherwise, it could be dangerous as well.

105

: 00:08:47

Yeah, and I mean, for sure, anytime, anytime, I'll be I'll be happy to come on your show,

even more if Chris is the airpiece.

106

: 00:08:57

He's always a lot of fun.

107

: 00:08:59

And but today we're we're going to talk about you and what you're doing, Hugo, because

like you, you do so many things.

108

: 00:09:07

I'm like super excited for today.

109

: 00:09:09

So first, can you maybe tell us what you're doing nowadays?

110

: 00:09:12

And

111

: 00:09:13

how you ended up working on this.

112

: 00:09:16

Absolutely.

113

: 00:09:16

And as you know, and as some of your listeners may know, I do a lot of podcasting and

interview a lot of people, but rarely are the tables turned.

114

: 00:09:25

And so I'm a bit nervous, but I feel I'm in comfortable hands.

115

: 00:09:30

So I'll tell you what I do at the moment in broad brushstrokes and then kind of talk about

how I got there.

116

: 00:09:38

Generally what I do is help people build software that uses data in some way.

117

: 00:09:43

So this can be software that

118

: 00:09:45

incorporate statistical inference, Bayesian methods, data science techniques, machine

learning, what we're now seeing in the AI space, foundation models, generative AI stuff.

119

: 00:09:55

There are kind of three prongs to that.

120

: 00:09:56

There's all the education I do, so teaching.

121

: 00:10:00

There's a lot of developer relations I do, so helping out frameworks, open source

frameworks, and companies reach developers, but get their tools in hands of people who

122

: 00:10:10

need them.

123

: 00:10:11

And then there's the product stuff that comes along with it.

124

: 00:10:14

So when teaching people

125

: 00:10:16

and doing dev rel, developer relations, lot of the time you're in an organization trying

to figure out how to ship code into software that serves user needs and creates business

126

: 00:10:28

value, right?

127

: 00:10:29

So the way I got into this is my background is in mathematics.

128

: 00:10:36

I went to grad school and did pure mathematics actually, but I've done a bunch of applied

stuff.

129

: 00:10:40

I joke that, you know,

130

: 00:10:43

10 people read my thesis and four people understood it.

131

: 00:10:48

So what I wanted to do was stay in science but enter something just with a...

132

: 00:10:54

a bit more tangibility essentially.

133

: 00:10:56

I ended up working in biology, in biophysics, studying the physics and mathematics behind

cell division and what's called the cytoskeleton.

134

: 00:11:06

So it's essentially the railway track within cells that's dynamic.

135

: 00:11:09

You've got polymers growing and shrinking forming the mitotic spindle for cell division

and these types of things.

136

: 00:11:15

Some wonderful mathematics and physics there.

137

: 00:11:18

I was working with some of the best scientists I'd ever met.

138

: 00:11:22

This is both at the Max Planck Institute for Cell Biology and Genetics

139

: 00:11:24

Dresden then when my boss moved to Yale in New Haven I ended up there and that's how I

ended up in the US as well but what happened was I was trying to my job was ostensibly to

140

: 00:11:40

do mathematical modeling of cell biological phenomena and

141

: 00:11:45

I realized that the biologists I was working with didn't even have access to the

computational tools or education around the tools they needed.

142

: 00:11:53

back then it was IPython notebooks.

143

: 00:11:55

didn't have the Jupyter notebooks yet.

144

: 00:11:57

So it was IPython notebooks and I was helping people import data into these notebooks.

145

: 00:12:01

Pandas had just come out a couple of years before.

146

: 00:12:03

Matplotlib was relatively sophisticated at that point with all the wonderful work that

John Hunter and the whole team had done.

147

: 00:12:10

But it was this confluence of, know, pie data stack stuff having come together.

148

: 00:12:15

allowed me to educate research scientists on the tools they needed to do the work that

they needed to do to do the right science.

149

: 00:12:25

At that time I started looking for jobs in academia.

150

: 00:12:30

while kind of exploring industry.

151

: 00:12:31

was working in New Haven and moved to New York City where I am, I'm in Brooklyn currently.

152

: 00:12:37

And at that time I was like going to machine learning and data science meetups in New York

and it was a real buzz.

153

: 00:12:44

This was a decade ago, right?

154

: 00:12:45

And there was a real energy at every meetup people got up saying, hey, we've got job

opportunities here.

155

: 00:12:51

We'd love to chat with you all.

156

: 00:12:53

And I ended up joining what was then an early stage startup called DataCamp that did

online art education.

157

: 00:13:00

and had just started a Python curriculum.

158

: 00:13:02

Their pitch to me was come in, build product with us, build data science, but data science

education platform.

159

: 00:13:09

I think they had, you know, several hundred thousand people on the platform using R at the

time and they needed someone to come in, build the Python curriculum.

160

: 00:13:17

So I came in and built out the first 30 courses in their Python curriculum with a lot of

partners such as Continuum Analytics that then became Anaconda.

161

: 00:13:25

Anyway, I kind of came deeply embedded in the open source education staff.

162

: 00:13:30

And ever since then I've been doing education while building product at companies and also

Then moving around doing dev rel for so developer relations for open source inspired

163

: 00:13:42

companies So companies that started with an open source framework, but then they're trying

to to productize This year so I've been working full-time for many years for a variety of

164

: 00:13:53

companies this year stuff got so exciting in the space that I wanted to go freelance and

work on a variety

165

: 00:13:59

of different projects, teach a variety of different things and see how that panned out.

166

: 00:14:04

So long story short, now I'm freelance working in a variety of organizations doing dev

rel, education and product stuff.

167

: 00:14:13

Yeah, thanks.

168

: 00:14:15

I really love that story.

169

: 00:14:17

And yeah, for sure.

170

: 00:14:18

The education part is like everywhere in your curriculum.

171

: 00:14:24

And I guess that help.

172

: 00:14:28

shaped quite a lot the perspective you have today on on data science.

173

: 00:14:33

I'm actually curious about that.

174

: 00:14:35

so a question I almost always ask my guest is, do you remember when you were first

introduced to patient stance?

175

: 00:14:42

And also, more broadly for you, since you've worked across so many industries and, and,

and companies, how have these diverse experiences shaped your perspective?

176

: 00:14:57

So...

177

: 00:15:00

I can't remember when I first heard the term Bayesian stats.

178

: 00:15:05

When I was doing my PhD, I actually had to teach like probability and statistics and it

kind of made no sense to me.

179

: 00:15:12

I was like, okay, this is what I'm teaching, but this isn't helping anyone do anything

essentially.

180

: 00:15:19

And teaching...

181

: 00:15:20

the central limit theorem through the calculus is great for people who know the calculus

and love the calculus.

182

: 00:15:26

But then later on, of course, realizing you can use resampling techniques in the bootstrap

to show the central limit theorem emerge from, you know, pretty much any data set is one

183

: 00:15:37

of those aha moments to recognize there's kind of more to statistics and probability than

we initially taught and thought.

184

: 00:15:47

But it was when I was working in biology that

185

: 00:15:50

biologists would come in and they'd give me point estimates and Then you know mean

variance and then they'd be like look I did this t-test and I was like, but wait doesn't

186

: 00:16:01

that assume Normality and they're like, yeah, looks pretty normal to me and I was like,

okay I wonder whether it's normal enough.

187

: 00:16:09

How would we test this and you have another statistical test, right?

188

: 00:16:13

And that's not to say that statistical hypothesis testing

189

: 00:16:21

isn't robust when done correctly.

190

: 00:16:24

Rarely is it done correctly, I think is part of my beef.

191

: 00:16:28

But what I realized then, I was chatting with a fellow researcher and he informed me kind

of the details of the Bayesian workflow and I loved that it really made

192

: 00:16:41

It forces you to make your assumptions clear.

193

: 00:16:44

And then you're not only dealing with point estimates, you get out the entire

distribution.

194

: 00:16:47

That's a blessing and a curse in a lot of ways, but you can always return to point

estimates from them.

195

: 00:16:52

And the heuristics are incredibly straightforward, right?

196

: 00:16:57

So you've got a prior and data and the data is summarized in your likelihood somehow.

197

: 00:17:05

You make your...

198

: 00:17:07

Assumptions clear and then you get out some posterior if you have a lot of data the prior

matters less if you have less data the prior matters more and Then if you get more data

199

: 00:17:16

your posterior can become a prior so that workflow is incredibly principled I Think to

delve a bit deeper in into these things and your audience is probably even far more

200

: 00:17:27

sophisticated than me on on this But

201

: 00:17:32

I think frequentism works to a certain extent in some situations, but I actually don't buy

the frequentist definition of probability.

202

: 00:17:40

I don't think it's the right definition of probability outside a small number of use

cases.

203

: 00:17:45

So even if we think about...

204

: 00:17:48

You know, Black Swan events and that type of stuff, you can't flip a coin a trillion times

in order to figure anything out about that.

205

: 00:17:53

But even thinking about it, we're looking at the position of Neptune and trying to figure

that out, the uncertainty around that has nothing to do with frequentism or the number of

206

: 00:18:05

times Neptune would be in that place doesn't even make sense.

207

: 00:18:08

It...

208

: 00:18:09

What the probability and uncertainty actually encapsulates there is my personal degree of

belief that hopefully accords with yours if we have the same techniques and the same data

209

: 00:18:19

set.

210

: 00:18:20

So I know your question was when did I initially discover Bayesian inference, but this is

kind of the story of why it became attractive to me.

211

: 00:18:27

Then I actually went and started reading ET Jane's and then after that I was like this,

know, yeah, there's a lot here.

212

: 00:18:37

Yeah, yeah, for sure.

213

: 00:18:38

Yeah, and T.

214

: 00:18:39

James is a great, so great intrusion, especially for you, I'm guessing, with your with

your mathematical background.

215

: 00:18:46

Yeah.

216

: 00:18:46

And it must have been something.

217

: 00:18:48

Yeah, that that really spoke to you.

218

: 00:18:52

know, the something I'm also wondering and impressed by all your all your career is you've

learned so many things.

219

: 00:19:06

along the way, right?

220

: 00:19:07

I think it's also because you're a you love education, so you're a great educator, but

that forces you to learn new stuff all the time.

221

: 00:19:15

that's also why I love teaching, teaching personally, you know.

222

: 00:19:20

But something I'm really curious about this, how do you go about learning new methods, you

know, piece, you've learned and taught so many different topics in your career already.

223

: 00:19:31

That's a good question.

224

: 00:19:33

I mean, there are several steps to this.

225

: 00:19:36

First is I'm just perennially restless and looking for new things.

226

: 00:19:42

And especially in a space when there's so many exciting things happening.

227

: 00:19:47

So I think that speaks to like the discovery process in some ways.

228

: 00:19:52

But I think there are several steps to this, Like there's finding things that you're

excited about learning.

229

: 00:19:59

Then there's filtering down to the ones that you're actually gonna learn.

230

: 00:20:03

And then there's the actual learning stuff.

231

: 00:20:07

So finding things, I'm actually in a very fortunate position where I, as part of my work,

I get to chat with interesting people all the time.

232

: 00:20:15

And we may get to this, but I have several podcasts of my own where I just invite people

on who I find interesting.

233

: 00:20:22

so I get to learn from them in real time and then figure out, and people on the cutting

edge.

234

: 00:20:27

mean,

235

: 00:20:28

On my personal podcast, I've had Jeremy Howard on several times and he's kind of really at

the forefront of a lot of the things happening.

236

: 00:20:35

So I get to learn from him in real time.

237

: 00:20:39

The next step, which is finding all this stuff, then filtering down is the toughest part.

238

: 00:20:43

What do I focus my time on?

239

: 00:20:47

In all honesty, there are projects I work on that help me figure out what to filter down

on, but also intuition.

240

: 00:20:55

There aren't enough hours in the day, right?

241

: 00:20:57

So following my intuition, what excites me and also what excites people in the space.

242

: 00:21:04

And then the figuring out how to learn.

243

: 00:21:07

There are just so many incredible resources out there, right?

244

: 00:21:11

So for example, I still go back to a lot of

245

: 00:21:15

Alan Downey's books and I learn things constantly from those.

246

: 00:21:20

also, Jake Vanderplaats' Python Data Science Handbook I constantly go back to, which is

fantastic.

247

: 00:21:26

But a lot of the tools and a lot of the frameworks out there have increasingly better

documentation these days.

248

: 00:21:34

mean, documentation can be in a woeful state, give or take.

249

: 00:21:38

But with the nature of online education these days, there's no shortage of

250

: 00:21:46

places to learn from.

251

: 00:21:48

It's about figuring out which ones are the highest signal.

252

: 00:21:50

And I mean, dude, we live in a world where like Andre Carpathi when he was at Tesla and

open AI or whatever, he's just like, yeah, I think people should know more about

253

: 00:21:59

tokenization.

254

: 00:22:01

I'm gonna like, I probably get this, but I'm gonna do like a two hour video on

tokenization and then use it to rebuild chat, build GPT-2 again.

255

: 00:22:10

And here's the code in a hundred lines of code or something like that, right?

256

: 00:22:14

And this is the first time in history, in all honesty, that we've had access to the minds

of people at the forefront of industry, giving us stuff on YouTube and on GitHub.

257

: 00:22:26

So that's kind of, I suppose if anyone wants to think about how to learn.

258

: 00:22:32

chat with people, see what interests you, build projects and learn through building those

projects and then follow the people who deliver exciting content and engage.

259

: 00:22:43

Yeah, I love that.

260

: 00:22:45

I really love that.

261

: 00:22:47

And yeah, so many things in what you just said.

262

: 00:22:51

First one is, yeah, like we're so lucky to have access to so many quality, educative

content, right?

263

: 00:23:03

free on YouTube, for instance, as you said, or free podcasts.

264

: 00:23:07

That's just incredible.

265

: 00:23:09

in a way, that's like, that's a really good, you know, illustration to me of the fact that

technology is merely a tool, you know, and what counts is what you make of it, right?

266

: 00:23:24

Because you could spend your time on YouTube watching, you know, useless videos.

267

: 00:23:30

But I also do that.

268

: 00:23:33

But don't only do that.

269

: 00:23:34

Exactly.

270

: 00:23:35

But also, think about the stuff that we have available to us now.

271

: 00:23:40

Before ChatGPT, when stable diffusion came out, the summer, the North American, the

Northern Hemisphere summer before ChatGPT, they released a Colab notebook immediately

272

: 00:23:49

where you could play around with text to image, right?

273

: 00:23:54

If you told me five years ago that was gonna happen, I'd say that's out there.

274

: 00:23:57

But it is absolutely incredible how quickly, and that isn't to say you don't have to smash

your head against the wall with coup de colonel errors.

275

: 00:24:06

we gotta deal with a whole bunch of nonsense, let's be clear.

276

: 00:24:09

But the barrier to entry is getting lower and lower, and that's something I'm very excited

about.

277

: 00:24:15

Yeah, yeah, totally.

278

: 00:24:18

Completely agree with Dan, didn't I?

279

: 00:24:20

So yeah, the opportunities are here, but...

280

: 00:24:23

it's you need to be able to, to take them and use them.

281

: 00:24:28

But yeah, it's just absolutely incredible.

282

: 00:24:32

And the like, you've released so many courses now, and I think over 30 courses, I looked

at during when I prepared for the questions.

283

: 00:24:47

And so I'm guessing you reached millions of learners with that.

284

: 00:24:52

What do you think

285

: 00:24:53

are the key ingredients for effective data science education.

286

: 00:24:59

Yeah, I am.

287

: 00:25:01

The most important thing is make sure you get feedback.

288

: 00:25:05

from the people you're teaching, which isn't obvious on an online learning platform,

right?

289

: 00:25:11

Like how to do that.

290

: 00:25:13

But even in the early days at Data Camp, we would make sure to beta test our courses

before releasing them.

291

: 00:25:18

We'd email 100 people who are power users, say, do you want early access to this?

292

: 00:25:23

Get as much feedback as possible.

293

: 00:25:25

And then of course, because it was an online learning platform, you get indicators on what

people are finding easy, what people are finding difficult, that type of stuff.

294

: 00:25:33

So like with a lot of these things,

295

: 00:25:34

these days.

296

: 00:25:35

I think getting feedback and performing rapid iteration is incredibly important.

297

: 00:25:41

And hopefully we'll get to talk about that in the context of LLM powered applications as

well, which is very different to traditional software because you do need to iterate very

298

: 00:25:50

quickly.

299

: 00:25:52

But I think getting feedback from learners is...

300

: 00:25:55

one of the most important things.

301

: 00:25:57

And finally, I actually last week taught a two day workshop on advanced AI.

302

: 00:26:02

Essentially what that meant was building neural networks and transformers and CNNs with

PyTorch.

303

: 00:26:10

But I taught it to a room of 35 people from Deloitte and having their feedback on what

they're seeing in the field, what their needs are, is not only incredibly important, but

304

: 00:26:22

so useful for me as an educator.

305

: 00:26:26

So you can frame it as like listening to market demand or something like that But it's

really making sure that you've got your your ears wide open Perhaps like tuning out from

306

: 00:26:36

all the crap you see on LinkedIn constantly And figuring out what people really need to

learn That then allows you to make it practical You also need to keep people engaged.

307

: 00:26:50

So if you're doing online stuff, you don't just talk for an hour like you maybe show

308

: 00:26:55

people what to do in 10, 15 minutes, and then get them coding along and help them as well

and have rapid feedback.

309

: 00:27:03

And that speaks to flip classrooms as well.

310

: 00:27:07

making sure that the learners have the opportunity to be as active and do as much as

possible.

311

: 00:27:14

How that played out at data camp, for example, is you may recall, we made sure that no

videos were over four or maybe five minutes long.

312

: 00:27:21

And we got people coding in their browser as soon as possible afterwards.

313

: 00:27:25

So just to recap that, feedback is essential in rapid iteration, make it practical.

314

: 00:27:32

We're in an age of multimedia where you can do all types of different things to keep

people engaged and flip classrooms as well.

315

: 00:27:39

Yeah, and the flip classroom concept, is that something you talked about with Andrew

Gellman?

316

: 00:27:48

Because I remember he talked also about that on my show, it's funny to hear that.

317

: 00:27:53

know whether I spoke about that with Andrew, but yeah.

318

: 00:27:58

He's a super cool guy, huh?

319

: 00:28:03

Oh yeah, always a great pleasure to have him on the show.

320

: 00:28:07

Such a great teacher.

321

: 00:28:09

It's like the quintessential idea that you have of a university teacher that makes you...

322

: 00:28:17

passionate about the topics.

323

: 00:28:18

Absolutely.

324

: 00:28:19

Absolutely.

325

: 00:28:20

So many stories.

326

: 00:28:21

always has so many stories and he's also such a curious person.

327

: 00:28:28

That's just incredible to have him on the show.

328

: 00:28:31

Actually talking about podcasts, you have not one but two active podcasts, which is like

as a podcaster, can say, well done.

329

: 00:28:41

That's impressive.

330

: 00:28:44

Of course, I people to go listen to Vanishing Gradients and High Signal because as all

your other shows, are super interesting to listen to.

331

: 00:28:57

So folks, definitely click on the links in the show notes.

332

: 00:29:03

But I'm also interested to hear your origin story of that, what motivated you to start

these shows.

333

: 00:29:11

and what are some of the most exciting topics you explored so far?

334

: 00:29:15

Yeah, that's a great question.

335

: 00:29:17

And what happened was, so I used to do DataFrame at Datacamp and then I started another

company, I was working at Coiled and I was like, I'd like to maybe podcast again.

336

: 00:29:29

I was like, now's not the right time.

337

: 00:29:31

But then I was just having so many...

338

: 00:29:34

interesting conversations with people doing cool stuff in the space and speaking of which

I just got off a call with a guy who he's at DoorDash now working on infrastructure but

339

: 00:29:46

before that he was at Meta working on front-end user facing applications before that he

was at Netflix and before that he was at Uber

340

: 00:29:55

Right?

341

: 00:29:55

So like, as by virtue of the people I get to talk, I was like, dude, we should do a

podcast.

342

: 00:30:01

I was like, this is a no brainer.

343

: 00:30:03

Right?

344

: 00:30:04

So the fact that I get to chat with so many interesting people, I was like, I probably

should start podcasting again.

345

: 00:30:12

The conversations I was having weren't necessarily appropriate for the company I was

working for at the time.

346

: 00:30:17

I mean, they weren't not appropriate, but they just weren't necessarily involved with

coiled and distributed compute and the Pi data space.

347

: 00:30:24

They were far more broad than that.

348

: 00:30:25

So was like, I'll just start my own podcast, which I get to own as well.

349

: 00:30:31

And because I put a lot of love into it and it's conversations with friends, it's kind of

nice to have something which like feels like mine.

350

: 00:30:38

you know, it isn't like when I leave a company that then stays there until

351

: 00:30:42

property of the company as well.

352

: 00:30:44

So that was a consideration there and I for Vanishing Gradients when I have an interesting

conversation with someone I'm like hey let's let's do a podcast and I get a lot of inbound

353

: 00:30:54

these days but most of the stuff I do is outbound and my most recent episode actually was

and this is not intended as self promotion but it's intended as promoting Ravin Kumar who

354

: 00:31:06

is a big Pi MC head and has worked with labs and you know well but talking about all the

355

: 00:31:11

wonderful he does.

356

: 00:31:13

all the wonderful he work work he does at Google Labs including the work on notebook LM.

357

: 00:31:20

and in particular his journey and I mean he works across from training training models to

creating

358

: 00:31:30

front end user applications as well that can interact with these models.

359

: 00:31:34

So a lot of great stuff there.

360

: 00:31:36

So that was a highlight on Vanishing Gradients for me.

361

: 00:31:39

Hi Signal is a very different podcast.

362

: 00:31:41

So Hi Signal is the goal is to speak with the best people in the space to give us Hi

Signal as possible for people working in the data, ML and AI space to advance their

363

: 00:31:55

career.

364

: 00:31:56

So

365

: 00:31:57

My first guest was Michael I.

366

: 00:31:59

Jordan, the Michael Jordan of machine learning, both literally and figuratively, talking

about what he thinks the future of AI needs to be, and in particular, and you'll love

367

: 00:32:10

this, I think your audience will love this, it's a wonderful conversation.

368

: 00:32:14

He was so lucid and clear about how horrible large language models are at handling

uncertainty and how what we need for AI to really be successful is have intelligent

369

: 00:32:26

infrastructure.

370

: 00:32:27

far better at handling uncertainty.

371

: 00:32:29

And because of the Bayesian nature of this podcast, I thought that would be super

relevant.

372

: 00:32:34

But then, of course, I had Andrew Gelman on to talk about what's important in statistics

for data scientists to know.

373

: 00:32:42

had Kiera Farinato from Harvard Business School talking about the cultural aspects of data

science and how organizations need to be structured in order to have very effective.

374

: 00:32:54

Data Science.

375

: 00:32:55

My most recent episode actually was with Hilary Mason talking and she's the first person I

ever released a podcast with back in the day so it was nice to come full circle and that

376

: 00:33:04

conversation is about what happens to data science in the age of AI and I don't want to

give any any teasers there.

377

: 00:33:12

Well no maybe I'll say it maybe it stays the same but some things do do change as well.

378

: 00:33:19

So they're two very different podcasts.

379

: 00:33:22

And I'm very excited to High Signal.

380

: 00:33:24

I do with the team at a company called Delphina and they're wonderful at producing it and

helped me get great guests as well.

381

: 00:33:31

So it's a lot of fun to be working with a variety of people on these two different

podcasts.

382

: 00:33:36

Yeah, yeah.

383

: 00:33:37

I I can guess and yeah, so folks check out the links in the show notes for both the

Finnish ingredients in High Signal.

384

: 00:33:48

Definitely recommend it.

385

: 00:33:49

Of course, I love this, the episode with with Ravin.

386

: 00:33:52

Ravin was a friend of the show and he's a friend period.

387

: 00:33:58

Always very inspiring to talk with him.

388

: 00:34:01

Andrew, we've talked about him.

389

: 00:34:03

Michael Jordan.

390

: 00:34:04

Yeah, I really love that conversation.

391

: 00:34:07

And I've been meaning to invite him on the show, actually.

392

: 00:34:10

So I should get back to that.

393

: 00:34:13

Yeah.

394

: 00:34:13

Well, just well done on all this work.

395

: 00:34:17

Thank you Alex.

396

: 00:34:18

It's just amazing and I know how much work that is to produce a podcast.

397

: 00:34:25

So really impressive that you have not one but two active podcasts.

398

: 00:34:31

Thank you.

399

: 00:34:33

So to be clear, it is a lot of work but it's also a labor of love, right?

400

: 00:34:39

Like you don't only podcast because you do it because...

401

: 00:34:43

You love these conversations and in all honesty, being able to put out a conversation with

Raven where we talked for two hours about all the stuff he's up to.

402

: 00:34:50

And I'm like, how wonderful that I get to, you know, put that out in the world as well.

403

: 00:34:55

So, yeah, yeah, no, for sure.

404

: 00:34:57

I mean, I would not not make that podcast, you know, it's like, it's a, it's a lot of

work, but it's like, it's like, going to the gym or something physical, you know, where

405

: 00:35:09

it's like, you know, it's, it's,

406

: 00:35:12

it's effort, but it's something that really makes your life better in the long run and

even in the short run.

407

: 00:35:20

You know, so it's just, I really love it.

408

: 00:35:23

And even when I started the podcast, actually honestly started for me because I thought

there should be a podcast about patient stats and I love listening to podcasts.

409

: 00:35:33

It was like, that's a shame.

410

: 00:35:35

There is none about that.

411

: 00:35:36

And I learned a lot through podcasts.

412

: 00:35:39

So it like, I should just, you know, try and make one.

413

: 00:35:42

Um, even if there is one listener who is me, uh, I'm fine because, know, I get to talk to

amazing people like, uh, like you and all the, all the gets, gets on your head so far.

414

: 00:35:55

you know, it's I'm, I like that.

415

: 00:35:59

Um, and well, that's a perfect transition.

416

: 00:36:02

Actually.

417

: 00:36:03

Uh, I can, I can tell you have a podcast because that's a great segue to actually talk

about your, um, your latest course.

418

: 00:36:10

because you're saying Ravin works at Google and he's working on LLMs.

419

: 00:36:15

Well, your latest course is about LLMs.

420

: 00:36:18

So, yeah, can you tell us more about that?

421

: 00:36:21

It's called Building LLM Applications for Data Scientists and Software Engineers.

422

: 00:36:27

From what I understood and read from the courses page, your goal is to teach the first

principles that you need to ship robust

423

: 00:36:40

reliable LLM applications.

424

: 00:36:42

So what inspired you to create this course and what can the participants expect to learn?

425

: 00:36:50

Yeah, that's a great question and two great questions.

426

: 00:36:53

So first, I just want to say in all of the like these weird conversations that happen in

social media and otherwise around like AGI is coming versus like, wait, do these things

427

: 00:37:04

actually just memorize everything and have a generative?

428

: 00:37:08

a touch that is like siloed within large organizations.

429

: 00:37:12

I think all of these are very important concerns.

430

: 00:37:14

Don't get me wrong, but we've forgotten that we can speak to computers now, right?

431

: 00:37:19

Which is an incredible thing.

432

: 00:37:21

The fact that anyone can chat with computation in natural language, I think is, has been a

holy grail for our discipline for many, many decades, if not longer, right?

433

: 00:37:33

So that's incredible.

434

: 00:37:35

Now.

435

: 00:37:36

What's not clear is what is a product that delivers value and what isn't.

436

: 00:37:41

mean, ChatGVT is a thin product layer around a particular transformer that took the world

by storm because people could chat with it.

437

: 00:37:50

But it's not clear how these types of systems can...

438

: 00:37:55

deliver actual value to businesses and people trying to make decisions at work, right?

439

: 00:38:00

So there's a huge amount of what we call demo-itis where you can get like a flashy rag

system.

440

: 00:38:05

So that's for those who don't know rag is a form of information retrieval where you have

documents that you want to use in LLM to query essentially.

441

: 00:38:14

You're gonna have like flashy systems that seem cool at the start, but then you get

complications, right?

442

: 00:38:22

Then it's like,

443

: 00:38:23

wait, this thing's hallucinating, wait, I've got monitoring issues, wait, how do I

actually test this properly?

444

: 00:38:29

I can't integrate it with my software stack correctly.

445

: 00:38:32

So we have an incredible technology that we're not certain how to use correctly yet.

446

: 00:38:39

And I'll actually compare it going back to electricity, right?

447

: 00:38:43

When we figured out how to harness electricity and make electricity ourselves.

448

: 00:38:47

as opposed to, you know, Benjamin Franklin holding a stick up on the hill to get hit by a

lightning bolt or whatever.

449

: 00:38:53

It isn't like we suddenly had a light bulb and electricity grid.

450

: 00:38:56

Edison built a lab that I think then became General Electric to figure out how to use

electricity.

451

: 00:39:04

And then the light bulb came, then the electricity grid came to connect everything.

452

: 00:39:09

So I think we're at a really interesting point where we're figuring out how to use these.

453

: 00:39:13

And over the past couple of years, we've developed a bunch of best practices.

454

: 00:39:18

around this type of stuff.

455

: 00:39:21

Now, what's amazing is that a lot of these challenges boil down to data science challenges

such as monitoring, such as testing, such as looking at your data and knowing what what

456

: 00:39:32

the data is.

457

: 00:39:36

And so I think if we take a data science approach to building this type of software, we

can actually be very successful.

458

: 00:39:42

The other side of the coin is I think because now we have all these vendor APIs, we don't

necessarily need to be training models all the time ourselves and such things.

459

: 00:39:52

There's actually a unique emerging opportunity for software engineers to learn how to

build AI and machine learning powered powered applications.

460

: 00:40:01

So that's a kind of a long way of saying we're actually

461

: 00:40:06

kind of at a very nascent critical opportunity to start helping people figure out how to

incorporate large language models and foundation models in general into their pre-existing

462

: 00:40:19

software stack to deliver business value for their organizations.

463

: 00:40:26

Now, you did also ask what can people expect to learn?

464

: 00:40:31

So do you want me to go into that now or?

465

: 00:40:34

Yep.

466

: 00:40:35

Yeah, for sure.

467

: 00:40:37

It's quite straightforward.

468

: 00:40:38

I the details aren't as with most, just as Bayesian, the Bayesian workflow is quite

straightforward, right?

469

: 00:40:44

But if we were to like first approximation, zero thought approximation, because we zero

index in Python land, zero order approximation for traditional software development is you

470

: 00:40:56

build something, you test it, deploy it, right?

471

: 00:40:59

Now,

472

: 00:41:00

What does the software development life cycle for generative AI look like?

473

: 00:41:04

It's actually very different to this.

474

: 00:41:06

You build an MVP, right?

475

: 00:41:08

Something small, you deploy it often internally, you monitor it, you evaluate it, and then

you iterate on that loop, right?

476

: 00:41:15

Now, once you've built this MVP and start iterating, you do want to start thinking around

things such as prompt engineering, maybe doing things with embeddings and vector stores,

477

: 00:41:25

perhaps fine tuning, incorporating business logic into your

478

: 00:41:30

application.

479

: 00:41:31

When you're deploying, you want to think about unit tests, continuous integration.

480

: 00:41:35

When you're monitoring, you want to think about

481

: 00:41:38

tracing, looking at your conversations, looking at your data, general observability, then

when you evaluate, you want to think about this at various levels.

482

: 00:41:46

And I want to spend, I want to index on this a bit more in a second, but you want to

evaluate your individual LLM calls, like, is this giving me the right thing, yay or nay,

483

: 00:41:57

but also generally how it relates to your business logic.

484

: 00:42:00

So if people do come to the course, what they'll get out of it is an overview of the

485

: 00:42:08

entire software development lifecycle and the types of tools and techniques they need

within it.

486

: 00:42:12

Working through particular projects and we give them compute and these types of things.

487

: 00:42:16

We've got some people providing credits for us which is really, really kind.

488

: 00:42:21

And so they'll get an overview of this entire software development lifecycle and how to

build these types of apps through all of these techniques from prompt engineering to

489

: 00:42:29

embeddings to fine tuning, through unit tests and also evaluation.

490

: 00:42:34

And I think this is actually, this comes down to the most important thing.

491

: 00:42:38

And when you ask me, like, how do you construct curriculum and how do you think about

effective data science teaching, you need feedback, right?

492

: 00:42:47

So in terms of evaluation, you need a feedback loop of what you're trying to achieve.

493

: 00:42:55

And we actually, me and my collaborator taught

494

: 00:42:58

a three hour workshop in Austin recently, the MLops World Generative AI Summit there on

these types of things.

495

: 00:43:06

And we used an example.

496

: 00:43:07

We built an example app which consumes LinkedIn profiles and generates structured output

from those profiles.

497

: 00:43:14

So it'll take your role, your history, all of these things, and then automate an email to

you if you seem like an interesting candidate, essentially, right?

498

: 00:43:27

So.

499

: 00:43:28

What you can do, thinking about evaluation, there are different levels of evaluation here,

right?

500

: 00:43:33

Firstly, you can ask, did the LLM...

501

: 00:43:36

get the correct structured output information.

502

: 00:43:40

Did it give me the correct JSON?

503

: 00:43:41

Did it even give me JSON actually?

504

: 00:43:43

Because a lot of the time it won't, right?

505

: 00:43:47

And in fact, what you see is if you just look at the structured output of any individual

LLM call, if you feed a LinkedIn profile, one of the biggest problems is the

506

: 00:43:56

non-determinism there.

507

: 00:43:57

I mean, you'll get different results.

508

: 00:43:59

If you do it 100 times, you'll get different results.

509

: 00:44:01

So that brings in the importance of testing into that, right?

510

: 00:44:07

you can analyze, can evaluate at the individual LLM call, then you can evaluate it at, you

know, did it draft the right email?

511

: 00:44:15

And at that point, maybe you want to sit down with a domain expert to whoever wrote the

emails beforehand and see if it looks like the type of emails they wrote and these types

512

: 00:44:23

of things.

513

: 00:44:24

So these are the first two levels of evaluation.

514

: 00:44:26

But none of this was your business goal.

515

: 00:44:28

They're necessary for your business goal.

516

: 00:44:30

Your business goal was to hire great people, right?

517

: 00:44:34

So then you want to look at, you getting responses to your emails?

518

: 00:44:37

And in the end, to close the loop, you want to look at, you actually hiring better people

or similarly good people at a faster rate with less labor from your staff?

519

: 00:44:52

So these are very much the important things of being able to connect

520

: 00:44:56

your LLM product development back to identifiable business metrics for your company.

521

: 00:45:03

And through types of projects like that, we'll give people hands-on experience, we'll lead

a bunch of it, but get them to build their own applications and we're there to talk them

522

: 00:45:12

through it as well.

523

: 00:45:13

And we've got some guest lectures from people such as Raven.

524

: 00:45:17

Raven's gonna come and give a guest lecture.

525

: 00:45:19

So we've got a really interesting lineup of people far more knowledgeable than me about a

lot of these things.

526

: 00:45:26

Yeah, super excited about that.

527

: 00:45:29

Yeah, that's amazing.

528

: 00:45:30

That's really fantastic.

529

: 00:45:33

And so for listeners who are interested, were kind enough to give them a discount code,

right?

530

: 00:45:44

Absolutely.

531

: 00:45:45

Thank you so much, Hugo.

532

: 00:45:48

I'll give that to you and we can include it in the show notes as well.

533

: 00:45:52

And also we have...

534

: 00:45:55

So we're doing it on maven.com and what they do on maven is they have the courses which

are paid but they have free lightning lessons as well.

535

: 00:46:02

And I gave one recently on what we call getting out of proof of concept purgatory and I'm

actually giving one next week on testing LLM applications using PyTest.

536

: 00:46:12

So that example of the fact that it'll generate different things if you iterate 100 times.

537

: 00:46:18

We're giving a short 30 minute lightning lesson on how to use PyTest to deal with those

types of things.

538

: 00:46:24

well.

539

: 00:46:24

So we can include that in the notes also.

540

: 00:46:27

Yeah, yeah, definitely.

541

: 00:46:28

So let's include the these lighting lessons and and also all the 25 % discount code.

542

: 00:46:35

Thank you so much, Hugo.

543

: 00:46:36

That's very generous of you.

544

: 00:46:38

So, you know, like, you see, folks, it pays to it pays to listen to bass.

545

: 00:46:44

You know, it's exactly like that.

546

: 00:46:46

Yeah.

547

: 00:46:49

Yeah.

548

: 00:46:50

And actually, what, you know,

549

: 00:46:53

Which profile did you have in mind?

550

: 00:46:57

Which profile of people and students did you have in mind when you wrote this course?

551

: 00:47:03

Basically, who do you think is this course for?

552

: 00:47:07

It's for two types of people.

553

: 00:47:08

And I'm really excited to see the interaction effects here, actually.

554

: 00:47:12

It's for data scientists who want to learn more about building software, particularly

using LLMs.

555

: 00:47:20

on the other side is for software engineers who want to learn about using LLMs and AI and

machine learning in software.

556

: 00:47:28

I think there's a lot they can learn from each.

557

: 00:47:30

I'm actually going to step back and let them all do my job for me is why I'm doing that.

558

: 00:47:33

No, I'm just just kidding.

559

: 00:47:35

But

560

: 00:47:37

The idea is that people who have data skills can come and learn a lot of the software

development best practices.

561

: 00:47:43

People with software skills can learn a lot about the data-centric world view and how

looking at your data and monitoring and all of these types of things and evaluation are of

562

: 00:47:53

the essence.

563

: 00:47:56

Okay, okay.

564

: 00:47:58

So folks, you've heard, if you're in these categories, definitely give a try to the

course.

565

: 00:48:06

and click in the show notes.

566

: 00:48:09

Something I'm really curious about is this Pocky purgatory you just mentioned at the end

of your previous answer, Hugo.

567

: 00:48:18

What's that about?

568

: 00:48:19

How does your course help practitioners move beyond these and become able to deploy

LLMPowered software?

569

: 00:48:30

So I spoke to this briefly before, the vision I'm about to give, but if you...

570

: 00:48:35

And I can include a figure in the show notes, but if you plot excitement as a function of

time when building software, traditional software starts off like in the bottom left-hand

571

: 00:48:47

corner, right?

572

: 00:48:48

It's you like, you build like a very small version, right?

573

: 00:48:54

Then you start doing some tests.

574

: 00:48:56

Then you build it up more and more and more.

575

: 00:48:58

And after a while, you have something.

576

: 00:49:02

which will be exciting, whereas LLM inspired software and generative AI inspired software

is the opposite.

577

: 00:49:09

You start off with super excitement, right?

578

: 00:49:11

Something flashy, something that works with five lines of code.

579

: 00:49:14

And then as soon as you try to start to try getting it in your existing software systems,

you've got integration issues, and then it's like.

580

: 00:49:24

wait, I've got hallucinations.

581

: 00:49:26

How do I deal with that?

582

: 00:49:27

And then it's like, wait, I've got monitoring issues.

583

: 00:49:31

And also I can't, I don't even know what the training data was.

584

: 00:49:34

So don't even quite know how to monitor drift properly and all of these types of things.

585

: 00:49:39

So you get excitement decreasing pretty, you know, seriously as a function of time.

586

: 00:49:47

So what we want to do is lift this entire curve up.

587

: 00:49:51

so people know how to solve these types of issues.

588

: 00:49:55

And the way we think about it is focusing on first principles.

589

: 00:50:00

And there are a few first principles.

590

: 00:50:01

Of course, we introduce a lot of tools to help, but tools, as we know, come and go, except

PyMC, of course.

591

: 00:50:10

But if you focus on first principles and learn how to build software in accordance with

these principles, we think that will take you a long way.

592

: 00:50:19

So the types of principles I'm talking about are that you've got API calls to vendor APIs

or your self-hosted models.

593

: 00:50:26

You've got inputs and outputs.

594

: 00:50:29

then got, you need structured outputs.

595

: 00:50:31

You have certain knobs you can tune such as temperature.

596

: 00:50:35

You have different types of context you can put in.

597

: 00:50:37

Context management is going to become incredibly

598

: 00:50:40

important.

599

: 00:50:41

Next principle is that LLMs are non-deterministic.

600

: 00:50:44

So if you understand this principle and the implications for testing, as we've already

discussed, you can have same inputs and different outputs.

601

: 00:50:52

That's not what we think of as software, right?

602

: 00:50:55

So we need to kind of get in the zen of dealing with non-deterministic outputs.

603

: 00:51:02

The other principles that we go through that I've already spoken to, but I think are

incredibly important, are logging, monitoring, and tracing.

604

: 00:51:08

So you could really

605

: 00:51:09

Pardon me.

606

: 00:51:11

Look at your data and then the process of evaluation at the different levels we spoke to

before and iteration.

607

: 00:51:17

So if you understand the non-determinism, you have your logging and monitoring in place,

you're able to evaluate and iterate.

608

: 00:51:25

These are the things that will get you out of proof of concept purgatory.

609

: 00:51:30

Hmm.

610

: 00:51:31

Okay.

611

: 00:51:32

This is very interesting.

612

: 00:51:34

That's first time I...

613

: 00:51:36

I hear about that concept and also how to solve it.

614

: 00:51:39

So yeah, that's really fun because yeah, I guess that's not an issue you have a lot in my

field where you do a lot of models and then a lot of these models are deployed.

615

: 00:51:51

So yeah, I see, but I definitely see what you mean where it's like, yeah, you have a lot

of park and then how do you actually make something happen from that?

616

: 00:52:04

I wonder how

617

: 00:52:06

you know how much it is a function of the of the recency of a field.

618

: 00:52:12

Maybe because LLM's are so recent in comparison to Bayesian models, for instance, where

you still have a lot to a lot of trees to cut if you want to really make a path through

619

: 00:52:23

the through the forest.

620

: 00:52:25

I think that's part of it.

621

: 00:52:26

I also think so.

622

: 00:52:29

We talk about deploying models that perform statistical inference or incorporate machine

learning.

623

: 00:52:35

That's not

624

: 00:52:36

quite the same as traditional software, right?

625

: 00:52:38

Because, yeah.

626

: 00:52:41

you have the entropy of the real world coming in through data.

627

: 00:52:44

you have monitoring challenges, which you may not have with traditional software, right?

628

: 00:52:48

You also have a changing skill set of a data centric skill set, a scientific skill set of

people who are perhaps accustomed to working in scientific notebooks and that type of

629

: 00:52:56

stuff more than deploying things to production.

630

: 00:52:59

So I think there are several changes that happen with statistical software and machine

learning powered software.

631

: 00:53:05

Now what happens with generative AI, I think is just next level because it's really

632

: 00:53:11

non-deterministic in a way that machine learning powered software isn't.

633

: 00:53:14

Machine learning powered software of course as I've said incorporates the uncertainty of

the real world but not in a like a consistently non-deterministic output manner.

634

: 00:53:24

On top of that man these things are generative and what I mean by that and I just want to

step back and say it's also stolen the term generative AI because generative modeling of

635

: 00:53:35

course refers to

636

: 00:53:38

specifically writing down the data generating process, which is Bayesian, but that perhaps

is for another day.

637

: 00:53:46

But the fact that these models...

638

: 00:53:49

express themselves, well I don't want to anthropomorphize too much, the fact that these

models, their output is like natural language, you can just generate lots and lots and

639

: 00:53:58

lots of stuff and there's a certain chaos that happens with the ability to generate that

much quote unquote content as well and data.

640

: 00:54:10

Yeah okay, yeah that makes sense.

641

: 00:54:12

I like that, that's really something I...

642

: 00:54:16

wasn't familiar with this so yeah thanks a lot for you know helping me become more more

aware of these these kind of things yeah and your course in general I really recommend

643

: 00:54:29

people to check it out at least the syllabus because that sounds like a lot of fun and in

the lightning sessions also if you have people like Ravin definitely recommend that

644

: 00:54:42

Exactly.

645

: 00:54:43

And I should mention, I'm teaching with an old friend of mine, Stefan Kraucic, who he's

been building ML platforms and that type of stuff for over a decade, including a long

646

: 00:54:55

tenure at Stitch Fix.

647

: 00:54:56

he's I think, yeah.

648

: 00:54:58

And now he's got his own company, Dagworks, where they work on open source software for

these types of things as well.

649

: 00:55:06

Nice.

650

: 00:55:07

Well, yeah.

651

: 00:55:08

So that's like you, students are in good hands.

652

: 00:55:11

For sure.

653

: 00:55:13

Actually, just, know, more broadly, I'm wondering with all the experience that you have,

also all the students you see, what guidance would you offer to aspiring data scientists

654

: 00:55:29

looking to enter the field right now or, or advance their careers?

655

: 00:55:35

My first piece of advice is focus on problems you want to solve.

656

: 00:55:40

People come to me and they're like, hey, I want to use deep learning.

657

: 00:55:44

How do I do that?

658

: 00:55:45

And I'm like, well, what type of problems are you interested in?

659

: 00:55:47

And they'll tell me something.

660

: 00:55:48

I'm like, oh, why don't you try a random forest or XGBoost?

661

: 00:55:52

Because you have tabular data.

662

: 00:55:54

And they say, but I...

663

: 00:55:55

want to do deep learning and I say, okay, maybe you can do some computer vision stuff and

they're like, well, I'm not interested in computer vision.

664

: 00:56:02

So I say focus on the problems that you want to solve and learn the tools around it.

665

: 00:56:08

So I think this is age old advice, right?

666

: 00:56:10

But be problem specific, not tool specific.

667

: 00:56:16

Be hungry.

668

: 00:56:18

wherever you work, wherever you are, go and talk with people about their problems and what

their needs are.

669

: 00:56:24

Embed yourself in different parts of the organization.

670

: 00:56:27

I can send you something actually, it's an essay, Eric Colson, who he used to be VP of

engineering, sorry, VP of, yeah, engineering and ML at Netflix and then was chief

671

: 00:56:41

algorithms officer at Stitch Fix.

672

: 00:56:42

He recently wrote something for O'Reilly saying,

673

: 00:56:47

So I'm going to paraphrase, but it's high data scientists for their ideas, not their

skills, right?

674

: 00:56:54

So you want to avoid data scientists becoming a service center where, you know, customer

success gives them tickets all the time.

675

: 00:57:01

And he gives the example of there was an organization he was at where there was a team

talking about the customer segmentation, which they'd done, which customers they'd done

676

: 00:57:10

based on a customer survey.

677

: 00:57:12

And they were like, look, this segmentation doesn't help us with anything.

678

: 00:57:16

It doesn't help us with recommendations.

679

: 00:57:17

doesn't help us selling product like this segmentation sucks essentially there was a data

scientist in the room who then when they heard this they went and did a cluster analysis

680

: 00:57:27

and they came up with entirely different segments which made sense to the domain experts

which allowed them to recommend more relevant content and product to the users there so

681

: 00:57:41

that's to say be that data scientist

682

: 00:57:45

or that technologist in the room with people with problems, because they don't know what

you can do.

683

: 00:57:50

Only you know what you can do.

684

: 00:57:51

So get in there and help people solve their problems.

685

: 00:57:57

I also think find quick paths to value generation.

686

: 00:58:02

So don't necessarily...

687

: 00:58:04

go in and be like, okay, I wanna build this LLM powered yada, yada, yada, yada, yada.

688

: 00:58:10

If someone is like, I need this information, maybe you can build them a dashboard in the

afternoon, then get their buy-in and then get their teams buy-in, then have more leeway to

689

: 00:58:20

explore more things.

690

: 00:58:22

And these of course are all like deeply cultural questions as well, right?

691

: 00:58:26

So recognize that technology doesn't exist in a vacuum.

692

: 00:58:31

So maybe to summarize that,

693

: 00:58:33

Focus on problems, not tools.

694

: 00:58:36

Get in there, be hungry and chat with people and help them solve their problems and find

quick paths to value generation.

695

: 00:58:45

Yeah, love that.

696

: 00:58:46

And I really resonate with those two, Fisher, especially when you've worked in industry,

like I've done all my career, definitely the focusing on the problem and the

697

: 00:59:03

product part is extremely, extremely valuable.

698

: 00:59:08

And I've been lucky enough to always work with problems where patient stats were, you

know, at least part of the answer.

699

: 00:59:20

So that's cool because I got to use the tools I love and I develop.

700

: 00:59:26

But yeah, always having basically, who is your client?

701

: 00:59:31

Who are you?

702

: 00:59:32

making the model for and how are they going to use the model is extremely important.

703

: 00:59:39

And then the other part where you always also need to develop your skills in our line of

work.

704

: 00:59:48

So the fun thing is, as you were saying, find something you really want to work on, you're

really curious about, and you're like, I'll do that for free because that's fun.

705

: 00:59:59

It's like a hobby.

706

: 01:00:01

For me, it started with electoral forecasting and then it was open source and it's still

easy, know, writing examples, writing tutorials, trying new methods that we develop at

707

: 01:00:13

PyMC and much more brilliant people than myself develop on the algorithm side of PyMC.

708

: 01:00:21

Then I get to use those and try to translate that for people, how they can use that in

their own models and use that in my own models at the

709

: 01:00:31

for the millions, you know, and before that for our clients at Pimsy Labs.

710

: 01:00:37

That's really important because you you need that sandbox, right?

711

: 01:00:43

That nerdy sandbox because you never know, you know, right?

712

: 01:00:48

Working today, learning today about Gaussian processes and how to make them faster and how

they can help in some, in a lot of problems.

713

: 01:00:57

Today, you may not use a Gaussian process, but

714

: 01:01:00

In a few months, you may end up having to use one and then it's going to benefit everyone,

not only you, but your employer that you actually studied all of that stuff a few months

715

: 01:01:11

ago because now you are already operational to try first GP model.

716

: 01:01:17

You're not an expert for sure, but you already passed the first barrier, which is I need

to document myself and read a lot about that stuff because that's not trivial.

717

: 01:01:30

So that's something you definitely need to do.

718

: 01:01:34

I would say that kind of the meta skill to keep doing that is be able to be comfortable

with being uncomfortable because that means you almost always feel like you don't know

719

: 01:01:47

enough.

720

: 01:01:48

You don't know anything because like you almost never look backwards, you know, and take

stock of what you learned.

721

: 01:01:55

You always almost look forward and I'm like, my God, but I

722

: 01:01:59

I don't know how to do that.

723

: 01:02:00

I've never done that.

724

: 01:02:02

I don't know that method.

725

: 01:02:03

you're like, always, you can always feel like you're, you know, chasing something that

never comes.

726

: 01:02:09

If you are not able to make peace with the fact that there will never be a point where you

know everything and you're like, yeah, I know, I know how to do that.

727

: 01:02:19

Each time.

728

: 01:02:22

So yeah, that's like something I always tell the students and people who ask me how to

start this.

729

: 01:02:30

Yeah, exactly that.

730

: 01:02:32

And so that's why also, you know, choosing at least on your open source project and

hobbies project that you're really passionate about, where it's like, I cannot not write

731

: 01:02:44

about that stuff.

732

: 01:02:46

That's going to help you a lot because when it gets tough and it will.

733

: 01:02:51

then you'll keep motivated because you really care about that thing.

734

: 01:02:54

Without a doubt.

735

: 01:02:59

actually, I'm like something I want to ask you now that like, I've already taken a lot of

your time and I know you had a, you had a dinner.

736

: 01:03:09

I'm here for it, man.

737

: 01:03:10

This is, I'm having a great time.

738

: 01:03:12

That's flying by.

739

: 01:03:14

Yeah, me too.

740

: 01:03:14

mean, I still have, I still have a lot of questions.

741

: 01:03:17

Don't worry about it.

742

: 01:03:18

I'll start, you know, closing us down.

743

: 01:03:21

but something I'm going to ask you is I want to

744

: 01:03:25

your brain about like it's really something I would ask you you know if we were not

recording and just having a maté somewhere in the world what do you see as the most

745

: 01:03:40

exciting trends or advancements in data science in AI in the coming years?

746

: 01:03:49

I have several answers to this.

747

: 01:03:52

One is like the unsexiest thing possible, right?

748

: 01:03:55

But I do think like you can do things that make the impossible possible, like build

transformers for the first time or something like that, or like GPT two, GPT three, GPT

749

: 01:04:08

four, these types of things.

750

: 01:04:10

And you can also make the possible widespread and it's

751

: 01:04:15

It's the latter which is far more interesting for me.

752

: 01:04:18

And I honestly think that if you took all the technology that came out the past several

years and nothing else, it will take us decades to figure out how to apply this

753

: 01:04:29

successfully, right?

754

: 01:04:30

And to get as many people as possible using it.

755

: 01:04:34

So.

756

: 01:04:39

That's all to say that I'm very, very excited about helping as many people as possible and

a movement to, I mean, it's a word that's thrown around too much these days, but to

757

: 01:04:51

democratize all of these tools and skills.

758

: 01:04:54

I think that's the most exciting thing.

759

: 01:04:56

Now, on the technology side, let's step back a bit and think.

760

: 01:05:03

You know last year was a real language model year, right?

761

: 01:05:08

We had a huge amount of people coming out with and companies and frameworks coming up with

all types of cool stuff from large language models to smaller language models to

762

: 01:05:20

you know, easier ways to fine tune to do information retrieval, all of these every week,

every day.

763

: 01:05:25

Sometimes it was like, Whoa, this is, this is intense.

764

: 01:05:28

Right now.

765

: 01:05:29

I think this year, 2024 has been an incredible year for, multimodality and we're starting

to see the fruits of that.

766

: 01:05:38

So I don't know you remember like a couple of months ago, llama released, meta released

llama 3.2 and it included like not only a three B

767

: 01:05:48

3 billion parameter language model, but like an 8 billion parameter vision model or

something like that, right?

768

: 01:05:53

Which is absolutely, absolutely wild.

769

: 01:05:56

So I think the future with getting all of these multimodal models working together is

incredibly rich.

770

: 01:06:03

On top of that, smaller models.

771

: 01:06:07

We've just had a model released on all open data.

772

: 01:06:11

It's the first model that's been released on, on,

773

: 01:06:15

Totally open data.

774

: 01:06:16

I'm just trying to find this actually.

775

: 01:06:18

It's play us P L E I A S and play us.fr.

776

: 01:06:25

They're French.

777

: 01:06:28

so I'll, I'll, I'll share that link and they've released,

778

: 01:06:32

Yeah, it's the Common Corpus.

779

: 01:06:33

It's the largest open multilingual dataset for LLM training and they've trained the model

on it.

780

: 01:06:40

So I think that's incredibly cool.

781

: 01:06:43

The ability to have self-hosted models, privacy-preserving models, incredibly interesting.

782

: 01:06:51

That's on one side of things.

783

: 01:06:54

On the other side, I'm just really excited with, I honestly think there are still only a

handful of organizations that do data science really well.

784

: 01:07:03

And I'm really excited for the next decade of, you know, a lot like the long tail of

companies that could be more data driven.

785

: 01:07:12

You know, the mom and pop stores, Ravn gave a great example in our podcast of using Gemma

to automate things that are bakery, right?

786

: 01:07:19

These types of things, incredibly exciting.

787

: 01:07:21

Don't even need large language models for this type.

788

: 01:07:24

thing.

789

: 01:07:27

I think the third thing and I actually this is I understand why it didn't happen but I

actually thought probabilistic programming would take off more than it that it has.

790

: 01:07:38

So I'm I'm slightly less bullish in the short term but in

791

: 01:07:44

the medium to long term, think a lot of data functions are still counting and if we can

bring in more causal inference, more probabilistic programming, these types of things,

792

: 01:07:54

that's something I'm very excited about as well.

793

: 01:07:58

Yeah, yeah, yeah, I see.

794

: 01:08:00

Damn, that's cool.

795

: 01:08:03

I really like that at least.

796

: 01:08:06

I could go on as well.

797

: 01:08:07

yeah, I just realized, yeah, this is one of the reasons I probably learn so much.

798

: 01:08:11

I just put two and two together myself.

799

: 01:08:13

I get excited about too many things.

800

: 01:08:16

Yeah, I mean, that's fantastic.

801

: 01:08:17

And that's why it's awesome to talk with you.

802

: 01:08:19

It's very energizing.

803

: 01:08:23

And actually, I'm wondering, know, and that's something I'm also wondering about Ravin.

804

: 01:08:28

You know, he does so many things that I don't know when he sleeps.

805

: 01:08:31

So, yeah, same for you.

806

: 01:08:32

You you're a consultant, an educator, a podcaster.

807

: 01:08:36

a learner, how do you balance your time and energy across these different roles?

808

: 01:08:44

To be honest, I don't think I'm particularly good at it.

809

: 01:08:48

And I probably don't sleep as much as one should, be honest.

810

: 01:08:55

But I think in all honesty, the past six months having

811

: 01:09:01

entered a freelance kind of workflow and building my own business, it really comes down

now to drastic...

812

: 01:09:10

prioritization.

813

: 01:09:12

And it's it's tough like because I have clients who I need to deliver in the short term

but I also have plans that I need to develop in the medium to long term as well.

814

: 01:09:22

So it's about becoming quite principled around my approaches to what I choose to work on

and the age-old learning to say no not only to other people but to myself as well.

815

: 01:09:34

So when something excites me going actually you don't have time for this right now put it

on your wish list

816

: 01:09:41

and then get going on what you're up to at the moment.

817

: 01:09:44

Hmm, hmm.

818

: 01:09:45

Okay, yeah.

819

: 01:09:46

But the other thing, I don't know where I got this.

820

: 01:09:50

I saw a tweet once, I know this is a horrible thing to say, but I saw a tweet once years

ago where someone wrote something like, do what you feel like doing and it works out most

821

: 01:10:03

of the time.

822

: 01:10:05

Now of course, if you have an addictive personality, maybe you don't wanna do...

823

: 01:10:09

what you feel like doing all the time.

824

: 01:10:10

Like sometimes I feel like I want to go and eat three cheeseburgers.

825

: 01:10:13

Well, that's not entirely true.

826

: 01:10:15

But yeah, maybe you don't do that.

827

: 01:10:16

And sometimes when you don't feel like working out, you do do that.

828

: 01:10:19

But I do think if you follow your intuition and know the biases around your intuition as

well, then that will kind of lead you in the right direction.

829

: 01:10:32

And so, know, just

830

: 01:10:37

Then based on all of that, you know, the thing you love, the thing you want to learn,

what's next for you?

831

: 01:10:45

You know, are there any upcoming projects or idea you're particularly excited about?

832

: 01:10:51

Yeah, actually, I'm pretty much, besides all the work I'm doing currently, in the near

future, this course on building LLM powered applications is the thing that's occupying a

833

: 01:11:01

lot of my thought and time.

834

: 01:11:05

But I'm very excited about teaching more next year and getting back to the basics of how

to build data science workflows, how to build machine learning in production, and how to

835

: 01:11:18

power all of these using foundation models.

836

: 01:11:21

and generative AI as well.

837

: 01:11:25

Yeah, damn.

838

: 01:11:27

very exciting, very exciting month ahead, right?

839

: 01:11:33

I'm curious how long does a course like the one we talked about, how long does it take to

develop?

840

: 01:11:41

My rule of thumb is that one hour of good content can take anywhere between

841

: 01:11:51

15 and 25 hours of work.

842

: 01:11:58

And it depends on how much you already have there.

843

: 01:12:01

Like, you know, maybe it's a bit less, but if it's from scratch, that's pretty, pretty,

pretty ballpark.

844

: 01:12:08

And I think so on the order of, you know, hundreds of hours.

845

: 01:12:14

Yeah.

846

: 01:12:15

Yeah.

847

: 01:12:16

What's your experience with creating the courses you've done?

848

: 01:12:20

Yeah.

849

: 01:12:21

So hundreds of hours and basically, so that translates to month because I was definitely

not doing that full time.

850

: 01:12:32

And I would say that was even harder at the time because I was working.

851

: 01:12:36

Well, I mean, like you, I was working freelance.

852

: 01:12:39

So the, one of the trade-offs when you're working freelance is you can work whenever you

want, but

853

: 01:12:48

When you don't work, you don't earn anything.

854

: 01:12:52

that's like, you know, so that's hard because anytime not passed, work, not building a

client is an opportunity cost.

855

: 01:13:00

Yep.

856

: 01:13:01

And so yeah, building a course is a really delayed gratification.

857

: 01:13:06

It's like a marshmallow test for grownups.

858

: 01:13:10

So you're absolutely right, because it does.

859

: 01:13:14

The thing is you are investing in yourself in a way that you're not when working for

clients as well, right?

860

: 01:13:21

Like you're building your material and own content, which perhaps you can leverage later

on as well.

861

: 01:13:27

Yeah, yeah, yeah.

862

: 01:13:28

No, exactly.

863

: 01:13:29

So yeah, that's definitely something to have in mind when you start building a course.

864

: 01:13:37

And to be clear, I mean, that's the one-off cost of building the first material.

865

: 01:13:41

Then each time you teach it, there's a certain amount of maintenance you need to do.

866

: 01:13:44

as you know, I've been teaching in-person classes with our mutual colleague and friend

Eric Ma for years on all of this Bayesian data science.

867

: 01:13:54

And we created the initial material years ago, and we update it every now and then.

868

: 01:13:59

Yeah.

869

: 01:14:00

When when they'd be but it isn't as though each time you teach it you need to you know do

a hundred hours work.

870

: 01:14:07

No no yeah for sure.

871

: 01:14:10

It's like it's like a good movie it's almost evergreen.

872

: 01:14:15

Exactly.

873

: 01:14:17

Awesome Hugo.

874

: 01:14:17

Well I know I need to let you go in a few minutes so let's wrap it up.

875

: 01:14:24

It's so I mean it's it's so fascinating to talk with you that day.

876

: 01:14:28

I could could easily do a three hours episode.

877

: 01:14:30

have no doubt about that.

878

: 01:14:33

We should do it sometime then, man.

879

: 01:14:35

Yeah, yeah, for sure.

880

: 01:14:36

You would break the record.

881

: 01:14:38

I think the record on my podcast is two hours, something like that.

882

: 01:14:44

I think you have it in you to break that record.

883

: 01:14:48

Well, I did a three hour podcast once as the interviewer with a panel of five people.

884

: 01:14:52

So.

885

: 01:14:53

wow.

886

: 01:14:54

Yeah.

887

: 01:14:54

Yeah.

888

: 01:14:55

Although five people is, you know, thin.

889

: 01:14:58

I would say if you do three hours just with me, that would be more impressive.

890

: 01:15:01

But have to do it then.

891

: 01:15:07

Okay, so, but of course, before letting you go, I have to ask you the last two questions I

have a guest at the end of the show.

892

: 01:15:14

So if you could have unlimited time and resources, which problem would you try to solve?

893

: 01:15:21

I'd try to make...

894

: 01:15:23

everything we're talking about data science, ML, actually accessible to everyone.

895

: 01:15:28

Now, I don't want to say too much more about that, but I do want to give an example.

896

: 01:15:32

I'm just thinking what if my mom

897

: 01:15:34

could fine tune a language model on her own medical data or easily build a rag system that

ingests emails and populates a calendar for her that doesn't rely on open AIs, API and

898

: 01:15:47

that type of stuff.

899

: 01:15:48

I think there is a future in which we're all building or fine tuning or interacting with

small models that are pertinent to us and our local communities.

900

: 01:15:58

It's totally a non-obvious design choice for a civilization to have like some companies in

some

901

: 01:16:05

you know niche area in Silicon Valley in Northern California that build things that the

entire world uses.

902

: 01:16:13

Different societies have different values and actually you mentioned your wife is

Argentinian when we were chatting before right?

903

: 01:16:19

On High Signal recently I had

904

: 01:16:23

Chilean man on who's at Stanford Gabrielle is at Stanford Business School, but he was he

was talking with Businessmen in Chile though.

905

: 01:16:32

Like why don't we have a Chilean, you know?

906

: 01:16:35

language model, which can help us with, you know, issues that are pertinent to our people

and these types of things.

907

: 01:16:42

ChatGPT can't help with that.

908

: 01:16:44

So that's something I would dedicate a lot of time to is figuring out how to enable people

to build all these technologies or leverage these technologies themselves and have

909

: 01:16:54

autonomy around their usage as well.

910

: 01:16:58

Okay.

911

: 01:16:59

Yeah.

912

: 01:16:59

Yeah, definitely.

913

: 01:17:00

That's, that's something I think a lot of people would benefit from.

914

: 01:17:05

So yeah, as a French speaking person, can definitely resonate with that.

915

: 01:17:11

There is even less, you know, corpus of French text to train the models than Spanish.

916

: 01:17:19

I Spanish is much, much more widespread than French, for instance.

917

: 01:17:23

Absolutely and to that point I mean I like as an Australian we're kind of Americanized in

in many ways but chat GBT is a bit like polite in an American way that doesn't resonate so

918

: 01:17:35

much with Australians I was in Germany recently and my German friends are like I wish it

had just Say I don't know or tell me to shut up or tell me I'm wrong if I'm wrong as

919

: 01:17:44

opposed to try to please me in this American way all the time So yeah, we have Every you

know one has their own cultures and it'd be lovely

920

: 01:17:53

to see that reflected in the technology that people are enabled to interact with and

build.

921

: 01:18:01

Don't say that though, I fear that if you do that for French Chachipiti, it will just go

to strike from time to time randomly, you you have to be I that though, maybe it should.

922

: 01:18:13

Maybe it should take the maybe it should take the open AI API to Versailles and chop its

head off, you know, maybe maybe we need the real the real revolution.

923

: 01:18:22

Yeah.

924

: 01:18:24

Be careful what you wish for.

925

: 01:18:25

Exactly.

926

: 01:18:26

No, it's dangerous times.

927

: 01:18:29

And the second question, Hugo, if you could have dinner with any great scientific mind

that alive or fictional, who would it be?

928

: 01:18:38

I'm going to cheat.

929

: 01:18:41

this question slightly.

930

: 01:18:43

I thought about this a lot and there are lots of lots of people I'd love to love to speak

with but what I'd actually like to do is be a fly on the wall in Jan Lacoon and Mark

931

: 01:18:54

Zuckerberg's one-on-ones.

932

: 01:18:57

I'd love to get signal into because Zuckerberg has done some very interesting things and

plays with open weight models these days like we're just talking about llama 3.2 right

933

: 01:19:09

I think a lot of that is driven by that relationship.

934

: 01:19:12

He's been very explicit about why it's important for Metta as well, which I find

fascinating.

935

: 01:19:17

Who would have thought Mark Zuckerberg would be, at least in some ways, a hero of openness

and these types of things as well?

936

: 01:19:23

Like what world do we live in when that happens?

937

: 01:19:25

But I do think, as kind of I made clear earlier, I actually have a very deep interest in

the intersection and coevolution of technology and commerce.

938

: 01:19:36

We think of technology as something in the abstract, but forget that a lot

939

: 01:19:39

of it is generated through commercial interests.

940

: 01:19:43

So having insight into their one-on-ones are so important for what is happening in the

world right now.

941

: 01:19:50

So I would love to just be out, not even have dinner with them, just listen and what.

942

: 01:19:54

So I know that was a slight cheat to your question, but I feel like it was close enough

to, I'll let you tell me if it's allowed, but.

943

: 01:20:03

Yeah, I like it.

944

: 01:20:04

I like it.

945

: 01:20:04

That's the first time I have it that way, you know.

946

: 01:20:08

Usually the way people cheat is they choose more than one guest.

947

: 01:20:12

So which you did actually.

948

: 01:20:14

Yeah, but one isn't a scientist, right?

949

: 01:20:16

So only one is a scientist.

950

: 01:20:19

Yeah.

951

: 01:20:20

But yeah, I love it.

952

: 01:20:22

That's good one.

953

: 01:20:23

But it's the first time someone says they would not like to be officially in the room,

which is like, it's a nice twist on the question.

954

: 01:20:34

And the question is definitely like deliberately broad.

955

: 01:20:37

So then.

956

: 01:20:38

people can actually be inventive in there what i care about these missing it tells you

about what is important to the guest at the moment you know so awesome well hugo let's

957

: 01:20:54

let's call it a show that was absolutely fantastic next time you have a course let me know

you'll be welcome back on the show and yeah like thank you for taking the time

958

: 01:21:08

These show notes are going to be big.

959

: 01:21:09

So folks, if you're interested in anything we mentioned today, show notes are going to be

Laplace to be, as we say, at Learning Vision Statistics.

960

: 01:21:20

That's hilarious.

961

: 01:21:21

Hugo, thanks again for taking the time and being on this show.

962

: 01:21:25

Well, thank you, Alex.

963

: 01:21:26

I really appreciate your time and the wonderful conversation and everything you do for the

community as well.

964

: 01:21:32

So thank you.

965

: 01:21:33

And anyone listening, if you want to connect on LinkedIn or something like that, please do

reach out.

966

: 01:21:37

We can include that in the show notes as well.

967

: 01:21:41

Yeah, it's already on the in the show notes because you know, like as every guest, I

stalked you, Hugo.

968

: 01:21:47

So yeah, I read your LinkedIn in the notes already.

969

: 01:21:51

Well, thank you once again, Alex.

970

: 01:21:53

Yeah, thank you.

971

: 01:21:58

This has been another episode of Learning Bayesian Statistics.

972

: 01:22:02

Be sure to rate, review and follow the show on your favorite podcatcher and visit

learnbaystats.com for more resources about today's topics as well as access to more

973

: 01:22:12

episodes to help you reach true Bayesian state of mind.

974

: 01:22:16

That's learnbaystats.com.

975

: 01:22:18

Our theme music is Good Bayesian by Baba Brinkman.

976

: 01:22:21

Fit MC Lass and Meghiraam.

977

: 01:22:23

Check out his awesome work at bababrinkman.com.

978

: 01:22:26

I'm your host.

979

: 01:22:27

Alex Andora.

980

: 01:22:28

can follow me on Twitter at Alex underscore Andora, like the country.

981

: 01:22:33

You can support the show and unlock exclusive benefits by visiting Patreon.com slash

LearnBasedDance.

982

: 01:22:40

Thank you so much for listening and for your support.

983

: 01:22:42

You're truly a good Bayesian.

984

: 01:22:45

Change your predictions after taking information in.

985

: 01:22:48

And if you're thinking I'll be less than amazing, let's adjust those expectations.

986

: 01:22:55

Let me show you how to be a good Bayesian Change calculations after taking fresh data in

Those predictions that your brain is making Let's get them on a solid foundation

Share Episode

Shownotes

Transcripts

Follow

Links

Chapters

Video

More from YouTube