Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!

• Intro to Bayes Course (first 2 lessons free)
• Advanced Regression Course (first 2 lessons free)

Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work!

Visit our Patreon page to unlock exclusive Bayesian swag ;)

Takeaways:

• The hype around AI in science often fails to deliver practical results.
• Bayesian deep learning combines the strengths of deep learning and Bayesian statistics.
• Fine-tuning LLMs with Bayesian methods improves prediction calibration.
• There is no single dominant library for Bayesian deep learning yet.
• Real-world applications of Bayesian deep learning exist in various fields.
• Prior knowledge is crucial for the effectiveness of Bayesian deep learning.
• Data efficiency in AI can be enhanced by incorporating prior knowledge.
• Generative AI and Bayesian deep learning can inform each other.
• The complexity of a problem influences the choice between Bayesian and traditional deep learning.
• Meta-learning enhances the efficiency of Bayesian models.
• PAC-Bayesian theory merges Bayesian and frequentist ideas.
• Laplace inference offers a cost-effective approximation.
• Subspace inference can optimize parameter efficiency.
• Bayesian deep learning is crucial for reliable predictions.
• Effective communication of uncertainty is essential.
• Realistic benchmarks are needed for Bayesian methods
• Collaboration and communication in the AI community are vital.

Chapters:

00:00 Introduction to Bayesian Deep Learning

04:24 Vincent Fortuin’s Journey to Bayesian Deep Learning

11:52 Understanding Bayesian Deep Learning

16:29 Current Landscape of Bayesian Libraries

21:11 Real-World Applications of Bayesian Deep Learning

23:33 When to Use Bayesian Deep Learning

28:22 Data Efficiency in AI and Generative Modeling

30:18 Integrating Bayesian Knowledge into Generative Models

31:44 The Role of Meta-Learning in Bayesian Deep Learning

34:06 Understanding Pack Bayesian Theory

37:55 Algorithms for Bayesian Deep Learning Models

42:10 Advancements in Efficient Inference Techniques

48:21 The Importance of Uncertainty in AI Models

51:41 Advice for Aspiring Researchers in AI

54:08 Future Directions in AI for Science

Thank you to my Patrons for making this episode possible!

Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor,, Chad Scherrer, Zwelithini Tunyiswa, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Ian Moran, Paul Oreto, Colin Caprani, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Michael Hankin, Cameron Smith, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, Robert Yolken, Or Duek, Pavel Dusek, Paul Cox, Andreas Kröpelin, Raphaël R, Nicolas Rode, Gabriel Stechschulte, Arkady, Kurt TeKolste, Gergely Juhasz, Marcus Nölke, Maggi Mackintosh, Grant Pezzolesi, Avram Aelony, Joshua Meehl, Javier Sabio, Kristian Higgins, Alex Jones, Gregorio Aguilar, Matt Rosinski, Bart Trudeau, Luis Fonseca, Dante Gates, Matt Niccolls, Maksim Kuznecov, Michael Thomas, Luke Gorrie, Cory Kiser, Julio, Edvin Saveljev, Frederick Ayala, Jeffrey Powell, Gal Kampel, Adan Romero, Will Geary, Blake Walters, Jonathan Morgan, Francesco Madrisotti, Ivy Huang, Gary Clarke, Robert Flannery, Rasmus Hindström, Stefan, Corey Abshire, Mike Loncaric, David McCormick, Ronald Legere, Sergio Dolia, Michael Cao, Yiğit Aşık and Suyog Chandramouli.

Links from the show:

• Vincent’s website: https://fortuin.github.io/
• Vincent on Linkedin: https://www.linkedin.com/in/vincent-fortuin-42426b134/
• Vincent on GitHub: https://github.com/fortuin
• Vincent on Medium: https://medium.com/@vincefort
• Vincent on BlueSky: https://bsky.app/profile/vincefort.bsky.social 
• LBS #107 Amortized Bayesian Inference with Deep Neural Networks, with Marvin Schmitt: https://learnbayesstats.com/episode/107-amortized-bayesian-inference-deep-neural-networks-marvin-schmitt/
• Position paper on Bayesian deep learning: https://proceedings.mlr.press/v235/papamarkou24b.html
• Position paper on generative AI: https://arxiv.org/abs/2403.00025
• BNN review paper: https://arxiv.org/abs/2309.16314
• Priors in BDL review paper: https://onlinelibrary.wiley.com/doi/10.1111/insr.12502
• BayesFlow: https://bayesflow.org/
• BayesianTorch: https://github.com/IntelLabs/bayesian-torch
• Laplace Torch: https://aleximmer.github.io/Laplace/
• TyXe: https://github.com/TyXe-BDL/TyXe 
• Introduction to PAC-Bayes: https://arxiv.org/abs/2110.11216
• Training GPT2 with Bayesian methods: https://proceedings.mlr.press/v235/shen24b.html
• Bayesian fine-tuning for LLMs: https://arxiv.org/abs/2405.03425 
• Try out NormalizingFlow initialization with Nutpie: https://discourse.pymc.io/t/new-experimental-sampling-algorithm-fisher-hmc-in-nutpie-for-pymc-and-stan-models/16114/5

Transcript

This is an automatic transcript and may therefore contain errors. Please get in touch if you're willing to correct them.

Today I am excited to host Vincent Fortwin, a leading researcher in Bayesian deep learning

and AI for science.

2

Vincent is a tenure-track research group leader at Helmholtz AI in Munich, where he leads

the Efficient Learning and Probabilistic Inference for Science group.

3

In this episode, we explore why traditional deep learning often struggles in scientific

applications

4

and how incorporating prior knowledge and uncertainty quantification can enhance model

reliability.

5

Vincent shares his insight on generative AI, meta-learning and inference techniques like

Laplace and subspace inference, explaining how they contribute to more efficient and

6

robust AI models.

7

We'll also discuss the current landscape of Bayesian deep learning libraries, the

challenges of real-world applications and the role of PAC Bayesian theory

8

providing generalization bounds.

9

Whether you're an AI researcher or someone interested in the intersection of deep learning

and science, this episode is packed with insights into the future of reliable and

10

data-efficient AI.

11

This is Learning Vision Statistics, episode 129, recorded November 22, 2024.

12

Welcome Bayesian Statistics, a podcast about Bayesian inference, the methods, the

projects, and the people who make it possible.

13

I'm your host, Alex Andorra.

14

You can follow me on Twitter at alex-underscore-andorra.

15

like the country.

16

For any info about the show, learnbasedats.com is Laplace to be.

17

Show notes, becoming a corporate sponsor, unlocking Bayesian Merge, supporting the show on

Patreon, everything is in there.

18

That's learnbasedats.com.

19

If you're interested in one-on-one mentorship, online courses, or statistical consulting,

feel free to reach out and book a call at topmate.io slash alex underscore and dora.

20

See you around, folks.

21

and best patient wishes to you all.

22

And if today's discussion sparked ideas for your business, well, our team at Pimc Labs can

help bring them to life.

23

Check us out at pimc-labs.com.

24

Hello my dear patients!

25

Well, I hope that you are doing well and before we dive into this episode, I wanna thank

some new patrons, namely Ejit Asik and Suyog Chandramouli.

26

I hope I am not butchering your names, guys.

27

But thank you so much for supporting the show on Patreon.

28

Well, not exactly Patreon, actually on YouTube.

29

You guys are the first ones

30

to support the show on YouTube on the good Beijing tier and above.

31

So well done guys and thank you so much.

32

Thank you so much not only for being the first one to support the show on YouTube, but to

support the show period.

33

Really as you know, this is exactly the support that makes the show possible.

34

I pay for editing, I pay for everything with your support editing.

35

Posting recording all that stuff that you don't see you that goes into producing the

episode.

36

Well, I pay for that Thanks to you guys support.

37

So thank you so much Make sure to link your YouTube account to your discord account and

that way you will be automatically added to the LBS discord server and well, I can't wait

38

to see you in there and

39

If other people are interested in supporting the show on YouTube, well, you can just go to

the YouTube channel, LBS, Learn Big Stats.

40

You look into that in YouTube and then you will see a membership tab and you'll have all

the info in there.

41

That should be super easy to set up with your YouTube account.

42

You just link then your Discord account and you're all done.

43

So on that note, thank you again, guys.

44

I will see you in the Discord.

45

And now, onto the show.

46

Vincent Fortoyn, welcome to Learning Vision Statistics.

47

Hi Alex, great to be here.

48

Yeah, thank you for taking the time.

49

How was my Dutch pronunciation?

50

Perfect, mean, impeccable.

51

Although, you know, I have to say like there's this weird phoneme in Dutch that like also

I personally can't actually pronounce that well because I grew up in Germany, but like

52

it's definitely as close as I would come.

53

So well done.

54

wants to contribute more to baseball and I keep trying but where keep getting in the

middle my secret wish is that at one point I will find a way to have to use amortized

55

patient inference for the Marlins and then I can actually contribute to base flow for my

job so that's like that's what I'm trying to do but right now I have to focus on other

56

priorities still

57

contributing to other open source packages.

58

Sorry about that, Marvin.

59

I'm trying.

60

I'm trying.

61

That's perfectly fine.

62

Yeah, sure.

63

So I didn't have the most straightforward path.

64

I started off studying biochemistry in my undergrad.

65

And I don't know if you've ever been to a biochemistry lab.

66

Essentially, it's this kind of place where you pipette little watery liquids into each

other, and it takes you a week.

67

And then you stick it into some machine that goes, merp, you did it wrong, start from

scratch.

68

And so I'm caricaturing, but like I guess I was just a bit too clumsy for the experiment,

so they never worked out and I found that a bit frustrating.

69

And I realized that I could handle computers much better than pipettes, so I moved on to

bioinformatics.

70

And when I did my masters in bioinformatics, it was just about the time when you would see

all these papers that claimed that all these algorithms that bioinformaticians had worked

71

on for decades.

72

were essentially being beaten by deep learning solutions, right?

73

So I wanted to be on the right side of history and moved into deep learning for my PhD.

74

But what I realized was that a lot of this hype in AI for science was not really

delivering on the promises they made because scientists really have a lot of prior

75

knowledge about their field that they wanted to get into these models.

76

And they also were really careful about

77

like the kind of predictions they made, right?

78

They didn't just want a high accuracy, but they want to have well calibrated predictions

that they can really generate insight out of.

79

And normal deep learning didn't quite fit the bill, right?

80

And so that's how I got into Bayesian deep learning, where then the hope is of course to

marry the expressive power of deep learning with all the promises that Bayesian statistics

81

usually gives us, which is like putting in prior knowledge into the models and getting

uncertainties and just like.

82

optimal decisions under some sense.

83

Okay, okay, I see.

84

That's super cool.

85

I really like the meandering path like that, know, illustration of randomness and a lot of

good things actually come from randomness.

86

that's great.

87

And so today, what are you focusing on?

88

And you know, what does it mean to be a researcher in patient deep learning?

89

Because to me, it sounds like, you know,

90

the conjunction of three extremely highly rated SEO keywords.

91

Yeah, for sure.

92

I mean, there's definitely like a lot of the work that we're doing in our lab that's still

on the method side and trying to be better at doing inference in these Bayesian models and

93

trying to, you know, like just make them more reliable and robust.

94

But we also look a lot into application areas like AI for science.

95

So as I said, like my background was in science originally, so I'm still trying to follow

through on that a little bit at least.

96

And a particularly interesting area that we're looking at right now is sequential

learning.

97

So in the context of Bayesian optimization or Bayesian experimental design.

98

So I know you had Desi on the podcast recently, so that kind of stuff.

99

and obviously like these days, if you do anything related to deep learning, you can't

ignore that we have these big foundation models and LLMs and these kinds of things now.

100

So some of the work we do is also trying to figure out how we can fit in into that space

and how we can make them more Bayesian in some way, which, probably doesn't make much

101

sense on the pre-training because you have more or less infinite data anyway.

102

But in the fine tuning, we've done some work that is quite interesting where typically if

you have your big GPT or whatever, LLM, and you want to fine tune it on a tiny data set in

103

your target domain, like this is where you really start caring about the uncertainties.

104

And then if you fine tune it in a way that's inspired by Bayesian updating, that usually

gives you much better calibration than if you do the standard fine tuning.

105

Okay, I see.

106

That's pretty cool.

107

I didn't know that.

108

So I should, we'll...

109

Let's dive a bit into that because I want to ask you a bit more about Bayesian deep

learning and so on, but I'm curious about that.

110

Like how would that work?

111

Like what would be the workflow here where you would use Bayesian statistics or I'm

guessing more prior knowledge and infuse that into the fine tuning of the audience?

112

Because I mean, I'm not surprised by that because if I'm like that last time I checked and

read more about these methods, fine tuning was

113

kind of a, human heavy element of the LLM workflow.

114

So I'm really curious to hear about that.

115

Yeah, definitely.

116

So I think like philosophically the way I think about normal fine tuning is also from a

somewhat Bayesian viewpoint, right?

117

So if you pre-train your LLM on the entire text on the internet, you can somehow view that

as a, as a way to encode all the prior knowledge that is on the internet into some.

118

into some condensed form, right?

119

Which now comes as a point estimate of parameters of some big transformer model.

120

And then what you typically do these days in fine tuning is this idea of parameter

efficient fine tuning, right?

121

So you actually keep this big model fixed, like the backbone, as people would say, and

then you add these low rank adapters like LoRa, or there's kind of more modern versions

122

that are called Vera or whatever.

123

But so what you end up doing is you have this big model with billions of parameters.

124

But then you have your small adapters with like just a few million parameters that you

fine tune.

125

And they actually guide the big model towards the task you care about.

126

And so what we did is essentially like within these small parameter efficient adapters to

then treat them in a Bayesian way, right?

127

So like the big model is still just a fixed backbone of a point estimate.

128

And that's essentially in some way our prior.

129

But then we use these small adapter layers to do Bayesian inference.

130

because they're

131

you know, like very small, I said, like you can do Bayesian inference actually quite

efficiently as opposed to the big neural network or the big transformer where you couldn't

132

do it.

133

Okay, I see.

134

This is awesome.

135

I didn't know that was the case yet.

136

So actually, could you now actually define, you know, Bayesian deep learning, you know,

maybe what's deep learning in comparison to machine learning, for instance, and what makes

137

it Bayesian?

138

Yeah, sure.

139

So, so, okay.

140

I'm going to give you the more traditional view first, and then a second one that I

prefer.

141

so in the traditional sense, like if you think about what deep learning is, like it's

essentially trying to learn functions that are parametrized by these artificial neural

142

networks, right?

143

So it's essentially, an architecture that has an input layer where you put your data and

then it propagates through several layers.

144

So that's where the deep comes from.

145

And at the end, there's an output layer that gives you the predictions, right?

146

So algebraically speaking, essentially, it's just a bunch of matrices, which are your

weights that you multiply with the input vector.

147

And then you apply some nonlinearity, which is the activation function of the neural

network.

148

Now, this is really a powerful way of learning functions because you can prove that if you

make your network big enough, it can approximate any function.

149

So that's what's called the universal approximation theorem.

150

And, you know, using techniques like backpropagation, we can actually do this quite

efficiently in GPUs, for instance, right?

151

So that's why this whole deep learning paradigm has become so popular because we just

happen to have the hardware that could do these matrix vector products quite efficiently.

152

And so now once you have this neural network idea, right, this deep learning, then the

classic idea to make it Bayesian is to say, instead of just learning a single setting of

153

the parameters,

154

you learn a distribution over parameters, right?

155

So then you don't just have one setting for your weights, but you have, for instance, a

Gaussian distribution over weights, which is then defined by a mean and a covariance

156

matrix.

157

And then you have to figure out how to get to that distribution.

158

And one intuitive way is to do it via Bayesian inference, right?

159

So you write down some distribution that is your prior, then you observe your data, you

use Bayesian inference to update, and then you get a posterior.

160

And then from that posterior, can sample different parameters for the network, which will

then give you different predictions.

161

So each sample of the parameters gives you a different set of predictions.

162

And then you can use that to quantify the uncertainty in your prediction space.

163

So this is kind of the classic textbook view of what Bayesian deep learning is, right?

164

The way I personally like to rather think about it is slightly the other way around.

165

And I think this is where

166

Like the difference between statistics and machine learning comes in, right?

167

So in statistics, you really care about the parameters of the model, right?

168

So if you build a model for how well different players and the model ends up performing,

then these are actually like interpretable parameters.

169

Like, you know, which parameter is which player and you infer a posterior and that tells

you something about the real world.

170

In this neural network setting, of course, these parameters are just arbitrary, right?

171

We just like build this big model.

172

We could have built it twice as big or half, and then these parameters would be different.

173

But ultimately what we care about is the function we're learning.

174

So we're trying to learn a function that maps from inputs to outputs.

175

And we just use this neural network as a convenient shape of that function because we know

that it can approximate things well.

176

And so that's how I kind of like to think about Bayesian deep learning rather as Bayesian

inference in the function space.

177

So very similar to how a Gaussian process would work, right?

178

So like in a Gaussian process, you essentially also have a distribution over functions,

but it's a very restricted one because it's Gaussian, right?

179

And in the Bayesian deep learning sense, you could say we have a very flexible

distribution over functions because we know that these neural networks can essentially fit

180

any function we want.

181

And then the main thing we have to care about is like, how does our posterior and function

space look like?

182

And we don't really care about the parameters.

183

That's just the means towards the end of getting a distribution over functions that fits

our data well.

184

Hey, very interesting, because I was going to ask you, so we've talked already on the show

several times about the thing that actually neural networks converge to Gaussian

185

processes.

186

and the way he's right and one of the extremely close to what caution processes are doing

so as gonna see what the difference is between well the deep neural network and a motion

187

process he seems like they are very close to each other anyways, that's equal to two here

and in a person's for someone who would like to

188

Which library would you recommend looking at?

189

Yeah, so that's a good question.

190

It's a bit of a pain point maybe.

191

like right now, I guess we have this issue that there isn't like one library that rules

them all kind of, right?

192

So I think that's why you mentioning the base flow, right?

193

Before, I think this is a great effort to try to...

194

put everyone on one ship.

195

So in Bayesian deep learning, we currently don't have that.

196

So we have different libraries for different types of inference.

197

So there's a Laplace library in PyTorch that does Laplace inference quite well.

198

That's something that I've worked with quite a lot.

199

But there's also one that is called Tihi, which is doing MCMC inference.

200

And there's one that's Bayesian Torch, which does variational inference.

201

And all these libraries are essentially maintained by different people and need slightly

different ways of defining your model.

202

And the problem is really that a priori, you don't actually know which is the right

inference, right?

203

So like, if you really want to do stuff, you probably have to actually install all the

three libraries and like try all of them.

204

And of course in practice, like most practitioners don't want to do this.

205

So I think that's one of the main problems why, you know, we have a lot of papers.

206

where we show academically that it can really make a difference, but then in the real

world, people just don't want to go through that hassle of having to figure out what's the

207

right library and stuff.

208

And if they can just use normal deep learning and two lines of code, they don't want to

spend more than that on Bayesian methods.

209

So I think this is really something where the community still has to come together a bit

more and build tools that maybe have a joint API that can then talk to all these other

210

libraries.

211

and that makes sense also because that's really at the frontier of you know of research so

it's like yeah the time for the research to trickle down to tools you can actually use is

212

quite normal you know that's what happens with time c for instance which i'm one of the

core developers of is like for instance if you take hs gp so helmer space the composition

213

of gps i think the the the then the time you know that

214

we read the paper, we understand it deeply, we implement our first version, we deploy it

into Prime C, we make sure everything works and that doesn't break anything.

215

It takes time, so it's like, it's already quite fast, it always take quite some time.

216

But that's already, like, doesn't mean the alternative would be faster.

217

yeah.

218

But yeah, so I'll link to these.

219

to these three libraries that you've named in the show notes anyways, for people who are

interested, for sure having to understand which method to use before, that's a pain point

220

for practitioners because I'm guessing most of them are not specialized.

221

So they don't know, know, like it's not that more than they don't want to, I'm guessing

it's that they don't absolutely don't know how to do that.

222

So that's for sure.

223

And then the classic deep learning libraries, I guess these are PyTorch, TensorFlow, and

always forget the third one, you remember?

224

Jacks, I guess, is one of the big ones.

225

yeah, of course, Jacks.

226

How can I forget Jacks?

227

I use it all the time with PyMC.

228

Yeah, so yeah.

229

Anyways, so these are good references for people to try out if they want to.

230

You don't have to...

231

stuff with that, but that's already something to get familiar with neural networks.

232

Yeah, I mean definitely if people from the more open source software engineering community

want to get involved, I think there's a lot of open problems we could need help with

233

because we're more like these academic types.

234

We write our little GitHub repo and link it on your paper, but then just as you said,

putting it into production on the level of PyMC is a whole other...

235

problem for which you also need qualified people that actually know how to do this well.

236

Yeah, for sure.

237

Good point.

238

And actually, you have...

239

So I know you're more on the algorithm side, but I'm wondering if you have some real world

applications where patient deep learning has significantly improved outcomes?

240

Yeah, that's a great question.

241

So I think for the reason that I mentioned before that like somehow we

242

use it a lot in research but it hasn't really made it easy enough for people to use.

243

I think people still have this preconception that Bayesian deep learning just doesn't work

because they don't see it used very much.

244

But if you actually look into the papers that people write, there's all kinds of

applications like healthcare, drug discovery, astrophysics, climate science, robotics,

245

autonomous driving and so on, where it actually can make a difference.

246

You know, lot of these are projects where then you have some domain experts working with

someone like me, right?

247

Like some researchers from Basin Deep Learning.

248

And then we can show on a project by project level like that it actually has a positive

impact.

249

But of course, yeah, for the wider impact, then we would need to make it more usable for

people.

250

But I think it's really very promising to see that in all these different areas, have been

attempts to.

251

to use it and that it actually has made a difference there.

252

And just for people that want to read a bit more about what the pros and cons are and how

it's being used in all these fields, maybe I kind of shamelessly plug a little position

253

paper that we wrote and we published at ICML this year, which was co-written with a whole

bunch of very, very good co-authors from different institutions.

254

And essentially like there, we try to make an argument why really like today we still need

to use base in deep learning, like probably more than ever, just because AI is so

255

pervasive in the world and to make it more reliable and trustworthy.

256

That's one way of doing it.

257

Yeah, I love that.

258

Yeah, for sure.

259

And I totally agree with your point that, yeah, seeing more real world applications is

definitely something that's going to inspire people to use these kinds of methods much

260

more.

261

In addition to...

262

the whole workflow convenience and package convenience that we just mentioned before.

263

I'm actually curious, in which cases would you recommend people look at deep learning or

patient deep learning in particular, and in which cases you think, no, that won't be

264

useful here or that's an overkill?

265

Yeah, yeah, that's a great question.

266

So I think the main

267

The main properties that I think make a problem interesting for Bayesian deep learning is

if you have some kind of prior knowledge, right?

268

So typically in a lot of sciences, that's the case.

269

Then secondly, if you have certain decisions that you want to make with the predictions

that depend on your uncertainty, right?

270

like, and I mean, medicine is a classic example, right?

271

Like if you have a diagnostic machine learning system, like you probably care about

whether.

272

The system is 99.9 % certain that the patient has a certain disease versus like 70 %

because that might change how you treat them immediately or run another test or something.

273

Right.

274

And I guess the third thing is like if your data are kind of expensive to generate, right.

275

So typically in sciences, again, like often you.

276

I don't know if you have a chemical experiment and it costs you like a few thousand bucks

to run each experiment, then you can't generate billions of data points, but like you're,

277

you're quite limited in how much data you can generate.

278

and that really helps you then to really get the most out of your data to, to be Bayesian.

279

Like on the, on the Contra side, again, like if you want to pre-train a language model on

like 15 trillion tokens that you scrape from the internet, like probably you don't need to

280

be Bayesian because you have.

281

large data set and you probably don't have any better prior knowledge than what's written

on the internet anyway.

282

So maybe there it's fine to just use normal deep learning.

283

Okay, yeah.

284

So basically, if you have prior knowledge, patient deep learning, if you don't, but have

and or have a lot of data, then classic deep learning will be useful to you.

285

And we all like, how much data is enough data for classic deep learning?

286

I would say?

287

Yeah, that's that's really, I mean, super hard question.

288

I think it really depends on your problem, right?

289

Like, I mean, first, it depends on the

290

dimensionality, obviously, if you have a one dimensional problem, then probably if you

have 100 data points, there's already like a lot in one dimension.

291

But if you have a problem that's a million dimensional, and you need, you know, more data

points, and then it depends how complex your your problem is, like if it's just a binary

292

classification, maybe you can do with quite a few data points to fit some decision

boundary.

293

But if you want to do 1000 class classification, like like an image net or something.

294

then you might need more data points, right?

295

So I think it's very hard to give any like, you know, off the mill like number, but yeah,

I definitely think like if you look at your data and you randomly sample data points and

296

they start looking very similar, then probably you have a lot of data, right?

297

Like if you sample data points and they all look very different, then you're probably in

the low data regime to some extent.

298

And then maybe there it helps you more to be based in.

299

And also, from what I understood from talking with Marvin and the Baseflow team, something

that's very important for you to be able to apply at least amortized patient inference, I

300

don't know for the other methods, is the number of parameters, as you were saying, and

dimensions in your model.

301

For instance, most of my data is very hierarchical and my models have tons of parameters

and dimensions.

302

And in these cases, it seems like it's not very useful to use amortized Bayesian inference

because then if understood correctly, the neural network will be very hard to learn.

303

Whereas if you have, as you were saying, something that's less dimensional, but a lot of

data, well then here that's a clearer use case.

304

Am I summarizing that well?

305

Yeah, I mean, I think for normal deep learning, that's definitely true.

306

I guess in the basic deep learning, it's always a question, like how good your prior is,

right?

307

So like in the, you know, let's kind of for the sake of argument, let's assume you

actually already know what the solution is and you can write this down as a prior.

308

Then you don't need any training data and you already have a good posterior, right?

309

so it also always depends on this.

310

Like if you, if your problem is very hard and high dimensional, as you say, but you

actually already kind of know what the right solution probably is.

311

And you just want to fine tune it a little bit to like.

312

fit the last wiggles, then you can do that quite well.

313

But yeah, if you don't know much a priori, then of course the more complex the problem is,

harder it will be to learn.

314

Okay.

315

Yeah.

316

Yeah.

317

I see.

318

I see.

319

That's interesting for me to really understand that.

320

In something you work on quite a lot is also something that's called data efficient AI.

321

I'm wondering what that is mainly, you know,

322

and if you could discuss your work in these and especially if I understood correctly,

there is a relationship between deep generative modeling and data efficient AI.

323

Yeah, definitely.

324

yeah, mean, the data efficiency is really like what comes from this idea of having prior

knowledge, right?

325

As I just said, essentially, like if you have a perfect prior, then you're a maximally

data efficient because you don't need any data and you already solve your problem.

326

And this is really what in many scientific applications is quite useful, right?

327

Like if, know, as I said, in chemistry, it's very expensive to generate data, but on the

other hand, you have a whole library full of chemistry books that tell you a lot about how

328

that field works.

329

Then the hope would be that you can extract some of that prior knowledge, put it in your

model, and then you don't need to see as many data points to make progress.

330

So the connection to generative AI is also quite interesting because

331

In some level, like Bayesian deep learning and generative AI are quite related in that

they both model joint distributions.

332

Now in the Bayesian deep learning case, you have a joint distribution between your

parameters of the model and the predictions, right?

333

While in the generative AI, you typically model a joint distribution over the data itself.

334

So between inputs and outputs.

335

But, you know, like because they both care about modeling joint distributions over

different things.

336

You can quite fruitfully exchange ideas between the two.

337

like typically the one way would be to say like we can actually use generative AI tools to

do the Bayesian inference better.

338

And that's maybe along the lines where people might say these days we have these powerful

diffusion models for generative modeling and we can use diffusion models now to model

339

posteriors of Bayesian neural networks, for instance.

340

And the other way is obviously the other way around where you can say like, let's take one

of these big

341

generative models and try to infuse it with some Bayesian prior knowledge to make it more

data efficient, right?

342

And so this is a bit like saying, yeah, if we already know what kind of antibiotics look

like, and I want to build a diffusion model that can produce new target molecules that

343

look like antibiotics, like maybe I can put some prior knowledge in there so it doesn't

have to see as many of them to learn how to model them.

344

Yeah, it's fascinating.

345

really like that.

346

And it's all intertwined in everything you're doing.

347

that's so cool.

348

And is that related to some- thought about, like, I should maybe also mention, like, I did

write another position paper on generative AI.

349

So maybe we'll also put that in the show notes if people are interested.

350

yeah.

351

Yeah, yeah, for sure, for sure.

352

That's going to be super interesting.

353

Yeah, yeah.

354

And how is that related to another interest of yours that's meta-learning?

355

How does that interact with your research and-

356

What advancement have you observed in this area?

357

Yeah, that's another good question.

358

Yeah.

359

So, so essentially, as I said, like one of the main things in Bayesian deep learning is to

have a good prior, right?

360

So like the better your prior is, the better your whole like model is going to be, and

it's going to be more data efficient and hopefully more calibrated.

361

But writing down priors by hand can sometimes be challenging, right?

362

So sometimes if, if you go to a medical doctor and you tell them, like we're trying to

build this model to predict some disease, like what's your prior.

363

they might not be able to actually tell you that much that you can put in there.

364

So one way to get these priors is to use meta-learning, which is essentially some way to

look at other tasks that you've solved before that are similar to the problem you care

365

about.

366

And then you use the knowledge from those related tasks to make the performance in your

target task better.

367

So you can actually view meta-learning as a hierarchical Bayesian model where you have

essentially like a distributional.

368

tasks at the top level.

369

And then you have like the different tasks as like different Bayesian inference problems,

but you use the previous task knowledge to inform the prior on every new task you see.

370

And so this is kind of how, how, yeah, meta-learning can really help you get better

priors.

371

And then of course, there's the other way around where you can actually say, we can also

use meta-learning as a tool to learn how to do Bayesian inference.

372

And then that's

373

you know, the amortized kind of inference idea that you mentioned that Marvin and others

are working on, where you really meta-learn how to do the Bayesian inference, so you don't

374

actually have to run the whole Bayesian inference routine, but you can use some like

meta-learned neural network or something to do that inference for you.

375

And for instance, one class of models that do something like that on neural processes,

which are kind of a way of framing, you know, Gaussian processes with neural networks.

376

So that comes back to what we talked about earlier.

377

And that's one of these meta learning frameworks for Bayesian inference.

378

And we're also quite interested in, my group.

379

Okay.

380

I see.

381

So it's like, so meta learning would be learning about the models themselves.

382

Am I, am I understand understanding that right?

383

Yeah.

384

So meta learning is really like, you kind of try to try to look at some previous task and

then say like, okay, now, now that I've solved this task, like, what can I do better next

385

time?

386

Right.

387

So I think in the normal world, you might think about, you know, if you learn several

languages, right?

388

Like I know you speak, you speak different languages.

389

Like I think every, every new language gets a little bit easier because you have all these

previous ones and you can kind of reuse some of that knowledge to have a better prior of

390

what the next one might be like, right?

391

Okay.

392

Okay.

393

I see.

394

Interesting.

395

And so is that related to pack Bayesian theory?

396

which is another thing you're doing in your lab.

397

So can you explain what that is and yeah, basically why that's useful, why it's relevant

to your work?

398

Yeah.

399

Yeah, yeah.

400

So, so it's not just moving on from meta-learning.

401

Like there is a relationship and I actually have a paper on using pack-based in theory for

meta-learning.

402

It's not necessarily like directly related.

403

It's just something that I happen to do, but maybe on a, on a higher, more general level,

I guess the idea of pack-based.

404

is it's one of these things where people try to marry like Bayesian and frequentist ideas,

right?

405

So I think in the past, there was kind of like in the early 20th century, like there were

these kind of fights essentially between the frequentists and the Bayesians and they were

406

like solidly on different sides of statistics and arguing about things.

407

But these days, I think it's actually like more, more ecumenical, right?

408

Like people really try to use ideas from Bayesian statistics and

409

printed statistics and make them work where they work and use the others otherwise.

410

PackBase is a great example to combine these two.

411

So PackBase essentially is a way to give you generalization bounds.

412

All right.

413

So you essentially have a model that you trained, could be a Bayesian model, could

actually be something else.

414

And you try to ask the question based on the test error that you have, that you care, like

what's, what's kind of a bound, how bad that could be.

415

Right.

416

So like how well will your model do on unseen data?

417

And typically the form that these pack-based bounds take is to say with a high probability

over possible test data you might observe, the expected test error under your posterior

418

will not be much larger than the expected train error under the posterior.

419

So if you have your base posterior and you evaluate it on your training set and you get

some numbers, so on MNIST maybe you get 1 % or something.

420

then the pack-based bound might tell you with like 95 % probability your test error on

unseen data might not be worse than 1 % plus x.

421

So it might be 2 % or 3 % or something.

422

And then like how much slack there is, right?

423

Like how much between the test error bound and your actually trained error that depends on

how many data points you've observed, how high you want the probability to be, right?

424

So if you want it to be 99 instead of 95%, it will get a bit looser.

425

usually on the KL divergence between your prior and posterior in some way.

426

So this is where this kind of pack-based idea comes in that you really have this prior and

you compute the KL divergence.

427

And I guess some of your listeners might now think, okay, if I say train error plus

something like KL divergence, that sounds a bit like the elbow, right?

428

The evidence lower bound and variational inference.

429

And indeed you can.

430

you can see it in a very similar way.

431

So you can optimize a pack-based bound in the same way that you optimize the elbow and use

it for model selection or to actually derive posterior measures.

432

And whereas the elbow, like if you optimize it, will essentially give you the proper base

posterior, with these pack-based bounds, you can get something like pseudo-posteriors that

433

also gives measures, right?

434

So they have very similar mathematical form as a base posterior, but they might be more

robust in certain ways because they deviate slightly.

435

We've talked a bit about that, algorithms for the Bayesian deep learning models.

436

Can you give us a rundown of these algorithms and when they are useful for which cases?

437

Yeah, definitely.

438

So there's essentially a spectrum of trade-off between how expensive the algorithm is and

how good it is, right?

439

Like how well you fit the posterior.

440

So on one side, we have things like Laplace inference, which I've been working on quite a

lot recently, which is very cheap.

441

So there essentially you just train your model as you would normally, right?

442

So you optimize your log posterior typically.

443

So you get a map estimate and you have a point estimate for your parameters.

444

That's your neural network.

445

And this is going to be your mean for the posterior.

446

And then to get some distribution around it, you essentially have to approximate your

Hessian.

447

So you have to...

448

compute a second order derivative on the loss function.

449

And that Hessian then gives you the covariance for your Gaussian approximation, right?

450

So you just wrap a Gaussian around your optimized point estimate.

451

And that sounds very crude, right?

452

So like this clearly doesn't fit the entire posterior, but it turns out that it works

quite okay, right?

453

So like for how cheap it is, it's quite a decent approximation.

454

And with the modern algebra frameworks, you can do this Hessian approximation quite fast.

455

So this is something that these days people have actually successfully done even on GPT-2

or something, right?

456

So you can really scale this up quite a lot.

457

Now, if you want a slightly better posterior, you can do things like variational

inference.

458

So there you can choose a bit more freely what your posterior shape might be.

459

So doesn't have to be Gaussian.

460

You can use some...

461

other distribution and then just optimize the elbow between your true posterior and your

approximate one.

462

And you can also do things like mixtures, like if you believe that your posterior might be

multimodal, you can actually have a mixture distribution that you optimize.

463

And maybe subset of that mixture variational inference is these kind of particle-based

approaches where the mixture is actually a mixture of Dirac measures.

464

So you actually just have some.

465

some point masses that you move around.

466

And you can show that you can move them around in a way that they cover the posterior

quite nicely.

467

So this is actually an elegant way of saying, typically we care about drawing samples

anyway, right?

468

And so if I said from my posterior, I would naturally draw 50 samples.

469

And instead of first approximating the whole posterior and then drawing 50 samples, I can

just start with 50 samples and then just move them around so they look like they were

470

drawn from the posterior and then I'm done.

471

And that's quite related to deep ensembles, which is also a slightly non-Basian bass line

that people often use in practice because it's easy to implement and easy to use.

472

And so there's some kind cool connections between how, like if you take a normal deep

ensemble and you add some certain repulsive force between the ensemble members, then if

473

you do that in the right way, it essentially recovers some bass posterior.

474

And then of course the more...

475

Expensive ones are all these MCMC approaches, so Markov chain Monte Carlo approaches like

stochastic gradient Langevin dynamics and Hamiltonian Monte Carlo that, you know, like the

476

longer you run the chain at some point it mixes and I mean, I guess you know all that from

PiMC.

477

So this is the best way to get actual samples that have some guarantees, but it's also

very expensive.

478

So you might have to pay a huge amount of extra compute.

479

And it really depends like how much compute you're willing to spend in your particular

problem, right?

480

Like if you pre-train a language model, like already one training run costs you millions

of dollars.

481

You probably don't want to spend any extra compute on that.

482

But if you're a medical researcher and you spend already half a year generating your

dataset, then maybe you don't care whether your neural network training now takes one day

483

or five days or something, right?

484

Like this is not the bottleneck in your project.

485

So it really depends on what the application is.

486

And sometimes if you can afford running more compute, then of course you should run the

better inference.

487

Yeah.

488

Yeah, for sure.

489

That makes sense.

490

And what are the latest advancements when it comes to, well, more efficient inference

techniques for patient deep learning?

491

Yeah.

492

Yeah.

493

I mean, a lot of them are still being developed.

494

Like the Laplace stuff I talked about, we definitely had a lot of papers recently, also

some of them that I was involved in.

495

that have kind of pushed the scalability by using all kinds of clever tricks from

matrix-free linear algebra and all these things that you can do these days in cool

496

frameworks.

497

I think a general idea that's quite interesting is the idea of subspace inference.

498

So that's essentially the insight that you don't have to treat every single parameter of

your neural network probabilistically in order to get a good enough

499

posterior over functions, right?

500

So this kind of comes back to what I talked about earlier.

501

Like if you think about the Bayesian neural networks as just like a neural network that's

now Bayesian and you make the parameters a distribution, then it sounds like you might

502

need to do this of all parameters.

503

But if you think about it just from the lens of saying we want to have a posterior over

functions that makes sense, then it becomes obvious that, you know, in your millions of

504

parameters, maybe there's a subset that

505

is enough to be random to actually introduce enough randomness in the functions that are

being implemented.

506

And so there's a lot of work that, for instance, just takes the last layer of the neural

network or just the first layer or some sub network inside the bigger neural network.

507

And as I told you before, like in the case of language models, for instance, you can take

the backbone and leave it fixed and frozen as a point estimate and then just have these

508

small parameter efficient adapters that you treat as a Bayesian.

509

So I think this is really like where you get a lot of efficiency gains is to figure out

like out of your huge neural network, like which subset of parameters do you need to treat

510

in a Bayesian way to then make the function space posterior fit what your function should

be.

511

And then most of the other parameters you can just leave as point estimates.

512

Okay.

513

Okay.

514

I see.

515

Very interesting.

516

So yeah, if you have any, any link to dive deeper into that for, for listeners and for

myself, myself, honestly.

517

Yeah, leave that in the show notes because I'm always curious to see these latest

developments.

518

Something I'm very excited about is the inter-twanning, I don't know if that's the word in

English, of initialization of the MCMC chains with neural networks.

519

So normalizing flows or the pathfinder algorithm from Bob Carpenter.

520

where you would use basically draws from pathfinder or normalizing flows as the

initialization of the MCMC chains and that that makes your sampling not necessarily faster

521

because you still have to train a neural network but sometimes when like if you have

really a lot of data and MCMC is really slow then that's definitely useful especially if

522

you can have access to GPU

523

And that also makes basically, especially the pathfinder option basically makes

variational inference much more practical because it usually gives you much better

524

answers.

525

And then there is the normalizing for initialization option that here if you have GPU,

that's definitely extremely helpful.

526

And actually there are ongoing efforts right now going on the PIMC side to add

PathfinderVI to PIMC as an initialization option to MCMC, which would be super efficient

527

because that's also when Bob Carpenter created that for us.

528

Basically running Pathfinder using some draws.

529

initialize MCMC and then you can just run MCMC for a few iterations on an out of chains

and that's a much faster convergence than pure MCMC but that should also be

530

much more reliable than the classic VI that we have right now in PIMC.

531

And then there is also ongoing effort by Adrian Zeebelt, in particular on the NutPy side,

where he just added the ability to use normalizing flows as initialization for the MCMC

532

inference on NutPy.

533

So you can already use that in your PIMC or STAND models to try that.

534

So definitely I'll put the links in the show notes of that option because the pathfinder

option on PIMC is still under development so it's still a pull request.

535

So not very useful to link to but the nutpy thing with normalizing flow that's definitely

implemented already.

536

So I'll put a link in the show notes to a discourse post that Adrian posted.

537

recently to explain how you can use that.

538

And so I definitely encourage people to check that out.

539

Also report to Adrian on any GitHub issue on Notify if there are any problems because

that's renew so...

540

In this case, that's extremely useful for open source developers to hear from early

adopters to fine tune the details.

541

Keep in mind, and you'll see that in the discourse post that definitely GPU will help you

a here because you need to fit the neural network first.

542

yeah, don't expect any model to run in less than 10 minutes.

543

But then if your model is already

544

beer than that and takes much more time than that, then that could be a viable option.

545

cool.

546

That sounds very good.

547

that's really awesome.

548

And I'll definitely definitely link to that.

549

also another question I have for you actually is related to be to that.

550

But what advancements do you foresee and maybe also do you wish for in making these AI

models?

551

more reliable and data efficient.

552

Yeah.

553

mean, I mean, definitely as, as you probably guessed, like I'm personally hoping that

Bayesian deep learning can, play a role in that.

554

Right.

555

and I guess in our position paper that I mentioned before, like we really tried to make

this argument quite strongly.

556

I think what's really, I think an issue is often we don't do a good job in communicating

to people like what they need these uncertainties for.

557

Right.

558

So I don't know if you have that.

559

in your consultancy and stuff, right?

560

But like I often talk to scientists and then they, you know, they're like, we need this,

this model to solve our task.

561

And I'm like, okay, what if we make it Bayesian?

562

I can also give you uncertainties.

563

And then they're like, but what should I, what should I get uncertainties for?

564

I don't care.

565

I just want like a good performance.

566

And I think it's a bit unfortunate that in the communication, don't make them understand

that well, that it's really about downstream decisions, right?

567

So like we don't.

568

care about uncertainties for their own sake, but we care about making good decisions in

the real world.

569

And as I said before, like often you really need the uncertainty to make the decision,

right?

570

Like if you're a doctor and you have a patient, like you need to understand whether my

algorithm is giving you like a 99.9 % accurate prediction or whether it's just like a 80,

571

20 kind of guess, in which case, like in the letter, you probably want to do another test

or something, right?

572

And, you know, the problem is

573

that as long as we don't communicate this, then people also won't have an intrinsic

motivation to try our methods.

574

So I think this is really something that maybe as a community, we could be a bit more kind

of like better at communicating, right?

575

That we really don't care about uncertainties just because we want to have nice likelihood

numbers in our tables, what we care about making good decisions in the real world.

576

And as long as people care about this, then I mean, I'm also happy if they find other ways

to.

577

to serve that purpose, right?

578

Like, I said before, like frequentist methods can also work quite well.

579

And if people want to use conformal prediction for some specific task where that works

well, then I'm not going to force them to do a Bayesian thing, right?

580

I think as long as people actually start thinking more about like, why, why do I need

reliable predictions?

581

What do I use them for?

582

And how expensive is my data?

583

Can I maybe get more out of this small data set that I paid a lot of money for?

584

Then I think.

585

you know, like that I'm happy.

586

if they, you know, I hope that most of them will find that that base is a good way of

doing this.

587

But some of them might find a different way.

588

And that's also fine.

589

Yeah, in the end, whatever works, right?

590

That's important.

591

It's better to have a good enough model than that no model at all.

592

Yeah, for sure.

593

So to play yourself, I'm wondering if you have any advice to offer to

594

those looking to pursue a career in topics you're working on?

595

So whether deep learning or probabilistic inference?

596

Yeah, definitely.

597

mean, I think, you know, one of the big advice that I always give my students is to really

focus on the foundations.

598

So I like to tell this anecdote when I started my PhD, like everyone was crazy about GANs.

599

I don't know if you remember that.

600

So like these generative adversarial networks.

601

was the time where, you know, they were everywhere and there were like hundreds of papers

about them.

602

And so when I started my PhD, some people are like, you should really get into GANs,

right?

603

You should like learn about that in detail and learn all the tricks to train them and

whatever, which I didn't do.

604

I'm guessing if I had done like none of this would be relevant anymore, right?

605

Because like these days people don't use GANs anymore.

606

People use diffusion models and you know, that's going to be the next next thing, right?

607

But I think in contrast to that, if you look at things like Bayesian inference, that's

been used for 200 years and people still use it and it's still an important thing to

608

understand.

609

So I think if people focus more on these big ideas, these foundational things rather than

the latest trends and fads, I think that serves a much better purpose.

610

And I understand that these days a lot of people are quite excited about language models

and stuff, but who knows how long.

611

we still care about them in this particular way, right?

612

Like maybe next year someone comes along and develops some cool new architecture that's

very different from a transformer.

613

And suddenly, like everyone does language models differently, or people stop doing

autoregressive language modeling and use diffusion for language or whatever.

614

And then suddenly, like if you've only learned about this particular thing because it was

cool right now, then your knowledge will be obsolete, right?

615

So I think that's maybe the main.

616

the main idea.

617

Then maybe another advice that I always give people is to just talk to as many people as

possible, right?

618

And I think what's really nice in our community is that people are quite open.

619

Like if you go to any machine learning conference, you can just talk to anyone and people

are happy to have a chat.

620

Like nobody's going to turn you down.

621

And just to get an idea of the diversity of research that's being done and to get an idea

that not everyone just works on LLMs, but there's actually like tons of interesting ideas

622

that people work on.

623

And yeah, it's quite an exciting time to be in that field really.

624

Yeah, yeah, I second everything you just say.

625

Extremely welcoming community.

626

Feel free to ask questions.

627

Always politely, of course, you know.

628

But yeah, like extremely welcoming community.

629

And if you're persistent and really want to help people out and be active, I don't think

you're going to have any problem getting a foot in the door, let's say.

630

Yeah, yeah, it's a good question.

631

mean, since I started my own research group, I feel like it's been diffusing around the

edges a bit because suddenly you have PhD students and they obviously have their own ideas

632

as they definitely should, right?

633

So yeah, there's a lot of things we're looking into.

634

mean, as I said before, I think AI for science applications are still something that I

find really exciting.

635

And I feel like we should, you know, like now is the time to show the world that

636

the kind of algorithms we've developed over the last 10 years or whatever actually make a

difference.

637

And particularly like looking into how we can also benchmark them better, right?

638

So I think that's a bit of an issue right now that we end up using a lot of benchmarks

that other people have developed for their particular models to make them look good,

639

right?

640

So typically if you look at these deep learning benchmarks like MNIST and CIFAR and

whatever,

641

They're really like very highly curated data sets that are very clean.

642

Like there's essentially no uncertainty about these digits.

643

Right.

644

So like if you, if you train a normal neural network on MNIST that works perfectly fine.

645

So arguably you don't need to make it Bayesian.

646

And then if we try to benchmark on this and you don't see much of a benefit and then

people will say, there's not much of a benefit to be Bayesian.

647

Like, why would you do this in the first place?

648

But it's because this data set is just not interesting for that purpose.

649

Right.

650

Similarly on the Bayesian side, a lot of the data sets that people use are very small and

low dimensional because that's where traditional Bayesian methods work well.

651

Right?

652

Like if you just have a Gaussian process, then you want to run on like some little, you

know, five dimensional regression thing.

653

so I think what we're lacking kind of right now are these benchmarks that are realistic

type of data from the real world that is both high dimensional.

654

that deep learning is useful.

655

but also has all this like complicated noise structure or some uncertainty.

656

And so I think this is something that we're also like looking into a bit more now to find

these kind of the right niche for our product.

657

Like it's a bit like, obviously then people might say, if you have a hammer, you try to

make everything look like a nail.

658

So I don't want to be that person, but I still believe that there are nails in the world

and I want to find them so I can use my hammer.

659

And I don't want to hammer on everything that's not a nail.

660

like, I think that's something that...

661

that we're definitely looking into right now.

662

Cool, yeah, that's very exciting.

663

Yeah, love that.

664

Come back on the show as soon as you have something to tell us about that.

665

Yeah, definitely.

666

Yeah, that sounds fascinating.

667

Awesome, well Vincent, I know you have a lot to do and I have to let you go because you

have a hard stop, so I would still ask you a ton of questions, but let's call it a show.

668

I think we covered a lot of ground.

669

learned a lot of things and yeah I really like it because lots of things are clear to me

now that we have talked that before the show so that's awesome that's also why I do that

670

show but before letting you go I'm gonna ask you the last two questions I ask every guest

at the end of the show first one if you had unlimited time and resources which problem

671

would you try to solve

672

question.

673

So I don't know if they use that slogan anymore but DeepMind used to have that slogan

where they said their goal was to solve intelligence and then use it to solve everything

674

else.

675

So I might make that, you know, solve Bayesian inference and then use it to solve

everything else.

676

But jokes aside, I think like there's so many interesting problems in AI for science right

now, you know, from healthcare to material science to climate.

677

And so really hope that we can have some impact there by essentially building AI methods

that scientists can use, which are strongly founded on Bayesian principles and are

678

therefore more reliable, more robust and more trustworthy.

679

Yeah.

680

I feel thinking about this.

681

don't think I have any more creative answer than all your other guests sent before me.

682

I do think having dinner with like Bayes or Laplace would be fun, right?

683

Although for the latter, I might have to brush up on my French a little bit.

684

Otherwise, one thing I'm quite sad about personally is that I never actually got to meet

Dave McKay before he passed away.

685

So I think that might also be very nice to meet him for dinner one time.

686

Yeah, for sure.

687

Well, Vincent, a pleasure to meet you, a pleasure to have you the show.

688

Come back any time.

689

And as usual, I'll put a link to...

690

website, your socials and and all the papers for this episode, a lot of packages.

691

So that's great.

692

The show notes already very big.

693

So feel free to add anything in the show notes for for those who want to dig deeper.

694

Thank you again, Vincent, for taking your time and maybe on the next show.

695

Thanks, Alex.

696

was great fun.

697

This has been another episode of Learning Bayesian Statistics.

698

Be sure to rate, review, and follow the show on your favorite podcatcher, and visit

learnbayestats.com for more resources about today's topics, as well as access to more

699

episodes to help you reach true Bayesian state of mind.

700

That's learnbayestats.com.

701

Our theme music is Good Bayesian by Baba Brinkman, fit MC Lance and Meghiraan.

702

Check out his awesome work at bababrinkman.com.

703

I'm your host.

704

Alex and Dora.

705

can follow me on Twitter at Alex underscore and Dora like the country.

706

You can support the show and unlock exclusive benefits by visiting Patreon.com slash

LearnBasedDance.

707

Thank you so much for listening and for your support.

708

You're truly a good Bayesian.

709

Change your predictions after taking information.

710

And if you're thinking I'll be less than amazing.

711

Let's adjust those expectations.

712

Let me show you how to be a good Bayesian Change calculations after taking fresh data in

Those predictions that your brain is making Let's get them on a solid foundation