Hello everyone and welcome to the new episode of the High SK Student podcast.
Speaker A:
My name is Sira and together we are looking for exciting stories and personalities.
Speaker B:
From the High School.
Speaker A:
There is certainly no lack of relevance in today's episode.
Speaker A:
The hype surrounding Artificial intelligence has grown significantly in recent years and everyone is talking about it.
Speaker A:
Our guest today is a true AI expert who can definitely.
Speaker A:
Shed some light on the subject.
Speaker A:
Damian Both is Professor of Artificial Intelligence and Machine Learning and Director of the Institute of Computer Science at the University of St.
Speaker A:
Gallen.
Speaker A:
He is also a board member of various organizations and institutes dealing with data, AI, large language models and technology.
Speaker A:
So I am very excited about this conversation, because precisely this hype surrounding AI means there is a great danger of getting lost in buzzwords today.
Speaker A:
We want to counteract that.
Speaker A:
Welcome Damian.
Speaker A:
It's great to have you here.
Speaker B:
Yeah, thanks for having me.
Speaker B:
I'm very much looking forward to our session.
Speaker A:
Me too as usual.
Speaker A:
I have to ask the question in the beginning, if you had to introduce yourself in one word only, what word would you choose and why As a.
Speaker B:
Researcher I would say curious, right?
Speaker B:
That's what drives me?
Speaker B:
I hope that drives the people that are working in my group and we are in a very lucky position to be in the middle of the thing that is currently happening, that is connected to artificial intelligence, where everybody has a very well-educated feeling that things.
Speaker B:
Are changing and I have the feeling they're very close in understanding when things are happening before they reach the and that makes me very excited and that makes me very curious in which direction the journey will go.
Speaker A:
That's super interesting.
Speaker A:
So you say, many things are changing.
Speaker A:
Could you maybe give us a short impression of what is going on right now?
Speaker A:
In this whole world of artificial intelligence.
Speaker B:
We have an increased accelerated pace of announcements, developments and research breakthroughs.
Speaker B:
And in particular in artificial intelligence we had this what I call a second spring after the Genai break through twenty twenty two in December with ChatGPT people in research and in those big labs like Google, Meta and others are racing towards developing new better methods, larger models and thinking differently about how to use those models.
Speaker B:
And this propagates not only in like not only stays.
Speaker B:
In the academic field, but propagates very much into commercial interests, exciting ideas with startups, new companies being founded.
Speaker B:
So it's now not about going for millions in funding, but rather billions.
Speaker B:
And this is becoming on the one side pretty crazy and on the other side also pretty exciting because of the velocity that is happening, fueled obviously by conferences and announcements of those big corporations.
Speaker A:
And do you have the impression that you know what is going on and you know about all the trends that are currently evolving because you're an expert?
Speaker A:
Or do you think the broad public also has a clear overview on what is happening when it comes to AI?
Speaker A:
What is your impression here?
Speaker B:
The impression is very much that it's a crazy time because of that velocity that I mentioned before and given that I sometimes have the feeling that I do not need to look into all the details of every announcement, because they're going to be a better larger announcement next week, which is a bit sad.
Speaker B:
But on the other side there are paths.
Speaker B:
So if you are inside of the community and you understand a little bit the technology underneath you see and research roadmaps and connecting them helps to put those announcements in order such that you know, okay, that's the missing puzzle piece for reaching that goal and that's the missing puzzle piece for reaching that goal and this announcement is a commercial announcement to attack this company.
Speaker B:
So being I think very close to this helps me to order things and I think that might be something that is missing in the public where all those announcements are happening potentially independent in the perception and therefore it might feel very overwhel.
Speaker B:
It is for me still overwhelming, but there are patterns that you can follow and it kind of also has this kind of funny element of when you expect an announcement, but something happened and then it comes and you say okay, now it's coming.
Speaker A:
I also feel like, especially because of this very high velocity, many people can feel a bit overwhelmed as you already said and that they just stick to what they know.
Speaker A:
So I already mentioned the buzzwords so many medias for media, newspapers, whatever channel you would like to They also use these buzzwords and often the information is more or less the same.
Speaker A:
So for me, if I for example don't focus at all on what is actually going on in this whole field, I will focus on what tackles me in my daily life maybe so of course, what is going on with ChatGPT?
Speaker A:
What is going on with the three or four big players maybe?
Speaker A:
And what are potential risks so very superficial I'd say, because maybe that's also then the safety mechanism of people to just focus on what they already know and how this is developing and maybe not really realizing what is happening behind.
Speaker B:
Or under the surface very much agree and there's always this game of what the technology is doing and what commercial product people develop from that technology and the product names are different than the technology.
Speaker B:
So the Classical ChatGPT Being the Tool.
Speaker B:
That everybody knows is a chatbot that is getting more and more capabilities which comes from the technology being a language model which is underneath a neural network.
Speaker B:
So people invent new terms because of framing the ground for commercial product.
Speaker B:
We do the same in academia.
Speaker B:
We invent new terms because we want to also claim our area and domain in our discipline.
Speaker B:
So that's very natural.
Speaker B:
There are however some terms that help to structure this work world.
Speaker B:
So everybody speaks about artificial intelligence, that's the umbrella term.
Speaker B:
So this is having machines that are as intelligent or even more better or with smarter and more intelligent than what we humans are underneath.
Speaker B:
There's many, many disciplines.
Speaker B:
One that is very successful is machine learning.
Speaker B:
So using statistical means and large scale data to teach machines from the data patterns and within that domain, there's deep learning the deep neural networks.
Speaker B:
And this is for me kind of a maybe too oversimplified view on this world, but it helps to put those puzzle pieces in the right boxes to organize what happens outside and it works very well.
Speaker B:
And therefore, whatever happens and the media writes about AI, deep neural networks are underneath.
Speaker B:
So I would say, like deep neural networks are the engine of modern AI and currently it is more true than it was even ten years ago.
Speaker A:
Of course, when I hear neural networks, I would first associate that with the brain, the human brain.
Speaker A:
So do these neural or deep neural networks function in a similar way or why do you call them?
Speaker B:
Like that these networks are inspired by the human brain.
Speaker B:
So in our brain neuroscientists believe that we have, they know we have neurons and the neurons are firing and if this happens all in your brain and my brain, and you know we have perception and thoughts and actions that are derived from that.
Speaker B:
So taking this as an inspiration and approximating that mathematically with, to be honest, very simple high school math and dog product and a nonlinear activation function.
Speaker B:
And we can build those artificial neurons, put them in networks and make them very, very complex.
Speaker B:
And then suddenly they're able to do all those interesting, exciting things like recognizing cats and dogs in pictures or understanding language, translating or even generating content pixels or text from.
Speaker B:
A very simple basic unit being inspired by human brain.
Speaker B:
The one thing is what we have with artificial neuronator.
Speaker B:
The other thing is what we have with the brain and we still you know the neuroscience discipline is ST discovering a lot about.
Speaker B:
The brain learns a lot.
Speaker B:
So I would look at those as two different things that are connected by motivation inspiration.
Speaker B:
But you know, the artificial neural networks are not really what happens in our brain and maybe we will learn something about our brain that we can then put into the artificial neural networks to make them better.
Speaker B:
But yeah, the biological inspiration, the link is there as inspiration having a human brain and building machines that are as intelligent as humans.
Speaker B:
I think that's a nice story, right?
Speaker A:
It is, I actually wrote a paper in university.
Speaker A:
I mean, of course I'm not an expert.
Speaker A:
So from a student's perspective about the question whether AI could develop a consciousness and so there I also focus on maybe what has not been found out about human brain and what maybe makes humans still different from artificial intelligence.
Speaker A:
So for me personally, that's a very, very interesting question.
Speaker B:
That's also one question I always ask my students when I have a lecture.
Speaker B:
There's this point where I'm asking, do you think we will be able to build machines that are as we are, whatever that means right?
Speaker B:
Being self conscious, being sentient, being aware of their own existence and then you know, people are raising hands and I'm discussing that.
Speaker B:
It's kind of interesting because there are some people who say sure, why not?
Speaker B:
Yeah, also my opinion, my opinion is, the brain is machine and runs with water and then some electricity, little multivoltage and you know we need only food to eat and we can keep it up.
Speaker B:
So there's nothing magical in that no quantum effect so far.
Speaker B:
That we discovered.
Speaker B:
And there's some people who say well, I don't think this will be possible because there's something special.
Speaker B:
There is something that we cannot grasp.
Speaker B:
I don't have a nice debate about that.
Speaker B:
I think that if we teach those machines to understand more and more data, more and more about our world, then it might happen that we perceive them as being sentient, self conscious And then the question is not so much, is this really true or not?
Speaker B:
But if we perceive them as self conscious, should we treat them also with the respect of a self conscious being.
Speaker B:
That's a good question.
Speaker B:
I had one student that raised it.
Speaker B:
I found it very interesting, because testing if a machine is truly self conscious, is very difficult.
Speaker B:
The Turing test is in nineteen fifties.
Speaker B:
Idea of Alan Turing, how you could do it, but it's already obsolete, because you would never know is this machine truly or is this AI truly conscious?
Speaker B:
Or is it acting or as being conscious, but not really giving you the impression that you perceive it as intelligent, but not truly being self conscious.
Speaker B:
So it's quite tricky and you know, Hollywood has a bunch of nice movies about that problem.
Speaker B:
But I found the other idea very interesting if we perceive it as being self conscious, which is treated self conscious.
Speaker B:
I have a good friend and colleague Veronica Barassi, who is.
Speaker B:
We had this discussion and she saw when I was using ChatGPT and I always say thank you to ChatGPT, right?
Speaker B:
And she's like Damien, that's a machine.
Speaker B:
What are you doing that?
Speaker B:
And I'm like no, they will remember me that I'm the nice guy once they take over.
Speaker B:
And that was obviously a joke.
Speaker B:
I'm not afraid of this being happening, but kind of an interesting idea.
Speaker B:
And I think a good analogy would be animals, right throughout the history animals were perceived also as feeling pain and having some kind of expressiveness.
Speaker B:
And therefore we also introduced as society animal rights.
Speaker B:
Maybe we'll walk along a similar path.
Speaker A:
With machines who knows, okay, so maybe artificial intelligent rights are something in that direction.
Speaker A:
And of course this gets also very relevant when we talk about the question of responsibility, maybe, because before we dive deeper into already these somehow philosophical or very groundbreaking questions, just also for our listeners to put it simply you already mentioned machine learning, deep learning neural networks.
Speaker A:
But now how does AI work?
Speaker A:
If I ask this very that's a very simple question.
Speaker B:
Very simple question, but a very complex answer.
Speaker B:
Let's try to give an impression like how are those language modeling or this large language models work because this is the technology that's behind ChatGPT, all the other agents or agentic AI tools that are popping up nowadays.
Speaker B:
So we have a big neural network with big, I mean many layers, many neurons, many parameters.
Speaker B:
Those are the things that we have to fit during learning.
Speaker B:
And we train those neural networks from data entirely from data.
Speaker B:
There's nothing hand coded in those neural networks.
Speaker B:
And we take as an example Internet scale datasets every sentence.
Speaker B:
In every language that is written down on the Internet, that we can have.
Speaker B:
We can grasp digitally and the training is actually very simple.
Speaker B:
You take a sentence and you block one word out of the sentence and the neural network has to predict what the blocked word is like my name is bond.
Speaker B:
James and the Neural network predicts in the right work and if this is true, then you know nothing happens.
Speaker B:
If this is wrong, we start the learning algorithm to change the weights of the neural network a little bit in the right direction, so that next time this happens hopefully the right word will be predicted and we do this with all the words and all the sentences and billions and trillions of tokens that's the unit to this language models process and then.
Speaker B:
The neural network can predict every word on the Internet.
Speaker B:
So now there's this big debate.
Speaker B:
Is this learning by heart?
Speaker B:
Or is this then truly, is this machine truly understanding grammar and language and the semantics behind it?
Speaker B:
It's a very classical question that we at the university also ask when we design tests.
Speaker B:
Are the students really understanding the subject?
Speaker B:
Or are they learned that content by heart and just are repeating what they read?
Speaker B:
So this theory is called the stochastic parrot theory.
Speaker B:
Are those language models just thing like stochastic parrots?
Speaker B:
That they just repeat what they saw on the Internet and therefore not able to have any original thoughts or are they able to create original thoughts based on what they saw on the Internet?
Speaker B:
So you can simply say, are they just repeating the data that they saw?
Speaker B:
Or are they able to interpolate and create new data?
Speaker B:
Which is by the way a very important question with regard to copyright, you know yeah, what's happening currently?
Speaker B:
And there are some people who say these are stochastic parents, they just learn by heart and others say yeah well, but they're able to do something interesting because they autocomplete and reason and take the answers that they have back as the question and think about the answer and improve it.
Speaker B:
And there's a big debate about that and maybe what we see in the next couple years.
Speaker B:
Which of those groups is closer to the truth?
Speaker A:
Very fascinating question also to ask but you've been into this whole field for quite some years now.
Speaker A:
And I wonder what actually was it that first drew you into the whole world of artificial intelligence.
Speaker A:
Did you have for example a specific moment when you realized like this is actually what I want to research and what I want to focus on?
Speaker B:
Yeah, I mean boo.
Speaker B:
That's a long time ago already.
Speaker B:
So I'm in this particular field since two thousand seven in computer vision.
Speaker B:
And then language more and more language.
Speaker B:
But the moment I really felt in love with AI was and I think that was I was on a semester abroad at UC Santa Barbara and I had two lectures.
Speaker B:
One lecture was Matthew Turk and that's the gentleman who invented face detection, the first paper eigenfaces.
Speaker B:
And that was very impressive, because he totally was not focusing on that in the lecture.
Speaker B:
He was very proud that he was he was the first passenger of an autonomous vehicle and he was very proud about that, not so much about the face detection, which was more important.
Speaker B:
And even one question on the exam with a bonus point was who was the first passenger in an autonomous vehicle?
Speaker B:
And then you had to write Matthew Turks.
Speaker B:
So that was funny.
Speaker B:
And the second lecture was with Edward Cheng and Edward Cheng, he did computer vision at that time and he had a startup.
Speaker B:
So it was this connection of AI and entrepreneurship that fascinated me, which was very different to what I at that time experienced in Germany at the university and that changed my entire perception of you can learn something new technology and use this in a product to commercial for commercial purpose.
Speaker B:
And you know, Google was at that time big and Second Valley was close to Google and was kind of fascinating.
Speaker B:
I guess that was the first time I thought about maybe that's something that was Two thousand three.
Speaker B:
Maybe that's something that is so interesting that I want to spend my life on that.
Speaker A:
Did you ever have the doubts whether this field is actually going to develop in an interesting way or has it always been sure for you that this is going to be super relevant in the next ten, twenty thirty forty years.
Speaker B:
Of course, No, seriously, I was always thinking if we are able to get to the point that the community Artificial intelligence is a research field, is super old, right nineteen fifty six the term was coined at the Darthmouth workshop.
Speaker B:
So I always knew if we were able to reach that this would change a lot, because science fiction was very well able to plot.
Speaker B:
The scenarios.
Speaker B:
However, at The time from two thousand three I was a student, two thousand seven, when I started with the master thesis and then PhD, we were fighting hard to get a little bit of improvement.
Speaker B:
We were really really putting a lot of effort, a lot of thinking, a lot of compute and then we got like two percent better and we.
Speaker B:
Were super happy.
Speaker B:
It was far away from anything that people would want to use in the real world.
Speaker B:
And then twenty twelve alexnet happened and there was this Canadian research group, Alexitskova and Jeff Hinton, and they published this neural network and submitted it to this competition called ImageNet.
Speaker B:
One million images and one thousand concepts.
Speaker B:
One of those concepts by The way was Appenzeller but I will.
Speaker B:
Open.
Speaker B:
We keep it open if it's the liquor, the dog or the cheese.
Speaker B:
And suddenly those Canadian they submitted this neural network and to the surprise of everybody, it became number one and outperformed all the other AI's and nobody was believing me, because I myself being in the middle of my PhD, my PhD advisor Thomas Breuil at that time, he was chasing me with neural network.
Speaker B:
He said like Damien, that's the future go for it.
Speaker B:
And I was like I don't think they work.
Speaker B:
I want to stick with my old methods like support Vector Machines, right?
Speaker B:
They're nice, I know them.
Speaker B:
So I was wrong and he was right.
Speaker B:
So sometimes makes sense to listen to your advisor, right?
Speaker B:
And then it totally changed.
Speaker B:
After twenty twelve the first ten years of great achievements party a lot of startups founded and then in twenty twenty two we have now.
Speaker B:
The second phase of that.
Speaker B:
So that was the flipping moment where I saw that working hard getting a couple of percentage point improvement versus suddenly.
Speaker B:
We have this leap forward and everybody's excited and everybody is kind of moving fast.
Speaker B:
And now we have this new time where even everything is faster and the velocity of breakfast is even faster.
Speaker B:
So I'm now in this already for more than ten years and I'm looking forward to have maybe a little bit of slowdown.
Speaker A:
I see, yeah, but often, also with innovations, it's always the question not only is that for example the technology ready, but also are the people ready and sometimes the people are not ready yet, so you could have a great solution.
Speaker A:
But yeah, society isn't prepared for that.
Speaker B:
No, it's even worse.
Speaker B:
What I saw along the way is that researchers, they very often develop technology and then they want to go out and say everybody needs AI.
Speaker B:
But what people from research very few realize that people don't want to buy technology, they want to buy products and services and moving technology to a product and service is a lot of work.
Speaker B:
Having a language model is great academically, but building chatgpt on top of that is a lot of work and getting this fit to what a client potentially needs the product market fit, that's a lot of work and a lot of people in academia, what I experience in Germany are far away from that.
Speaker B:
When I was in California, in California and Berkeley and Santa Barbara, people are much closer to this.
Speaker B:
They're much more understanding how to turn those things into something that others might want to use.
Speaker B:
And then there's this moment where people are also willing to use and then exciting things happen right.
Speaker A:
You just mentioned it around twenty-Fourteen you were at.
Speaker A:
UC Berkeley when deep learning really started to also take off, can you maybe share a bit what that time was like and also how it shaped the fields that we can see today.
Speaker B:
Yeah, so we had the breakthrough twenty-twelve December was always like the imagenet announcements and then twenty-thirteen it slowly moved.
Speaker B:
Into the labs.
Speaker B:
I started very early twenty fourteen, and lucky punch, I ended up at the lab around Trevor Darrell, that they developed that time keep learning and they tried.
Speaker B:
To replicate the experiments that the group around Alexander did and they worked the whole summer on getting this done.
Speaker B:
And as a side effect they created a software tool called CAFE.
Speaker B:
It was the first tool to train neural networks that are very deep and they open sourced it because that's.
Speaker B:
What academics do right.
Speaker B:
I mean, they just open source it.
Speaker B:
And given that, because this first tool was open sourced, every other tool was now also open sourced, including TensorFlow, PyTorch, Keras.
Speaker B:
So we don't pay for AI Software, you pay for the training and the models, right?
Speaker B:
So that was a very big shift in the thinking of that.
Speaker B:
And being there in twenty fourteen was really exciting, because all those people, there were those people who did object detection and they thought about the architectures.
Speaker B:
There were those people who were doing classification of cats and dogs.
Speaker B:
They were thinking about that, there were people who were thinking about language.
Speaker B:
Everybody was excited, this energy was really remarkable and you knew something is happening and you knew that everybody of those people will be successful and it was a great time and it was a busy time, but it was the right time being at that spot and that was very important what I learned.
Speaker B:
There is everybody needs one thing to move forward and these are Graphic Cars, GPU's, Graphic Processing Units from Nvidia.
Speaker B:
So everybody was trying to get those to train their models.
Speaker B:
And I was like what the heck are they doing?
Speaker B:
Because why is this graphic card that is supposed to render your monitor screen so important then.
Speaker B:
You learning that was very important, because twenty fifteen I came back then to Germany, to the German Research Center of Artificial Intelligence, DFKI, where I then figured out, okay, the first thing we need to do is to get some graphic cards, because if you want to take this technology and move forward, we need the hardware to train that.
Speaker B:
And that leds then to moving to the community.
Speaker B:
Going to the Nvidia GDC and Lucky Punch.
Speaker B:
Meeting Jensen, who I'm the CEO of Nvidia and convincing him for getting a beer and getting us the first DG Xbox.
Speaker B:
So there were two ships to two DGX boxes shipped to Europe, one to Schmidt Hubers Level and one to my deep learning competence center.
Speaker B:
And this was was the first what Nvidia called deep learning supercomputer.
Speaker B:
The very first produced, the very first DGX produce we got our three months later.
Speaker B:
The very first DG Xbox produce went to OpenAI and Elon Musk received them and designed that.
Speaker A:
Interesting.
Speaker A:
And I mean of course Nvidia is a big player now still.
Speaker A:
And I feel like most people heard of the company maybe not, everyone knows exactly what they're doing, but these GPU's, they are a source of power somehow.
Speaker A:
So I have to ask you just also mentioned you met the CEO Jensen Huang, what makes him stand out as a person?
Speaker B:
Maybe he is super knowledgeable, I'm surprised.
Speaker B:
He has a breadth of knowledge and a depth of knowledge that is remarkable and he's a founding CEO, so he knows how his company is working, he knows what and this was very impressive.
Speaker B:
He knows he has a very good understanding about markets and where to go first so he really has this like this gut feeling what he has to address.
Speaker B:
And the most important decision was, hey, we have those crazy ResearcherS, they need GPU's, they do something with that, We don't understand what they do, let's give them more GPU's for free.
Speaker B:
No.
Speaker B:
Can you name some European CEO's who would do the same?
Speaker B:
I don't know any who would do that right in Germany?
Speaker B:
If you would misuse a product of a company for a secondary use, you probably would get some letters from the lawyers of the company, right?
Speaker B:
And not like, oh great, here you have more GPU's do whatever you want, we're learning right how you're using that.
Speaker B:
And Nvidia turned at that time this kind of secondary revenue stream into the primary revenue stream and a four point five trillion dollars company, which is crazy.
Speaker B:
And Jensen is Jensen is very dominant, he's a very dominant person, he has this kind of visibility and this kind of he convinced you he's able to convince you to work very hard.
Speaker B:
So he's not putting pressure, but he's kind of catching you and moving you in this excitement in this area.
Speaker B:
And therefore it can be very exhausting to work with him what I heard, what I heard on a daily basis, because he's so demanding and so exciting and then so knowledgeable, so it's hard to keep track with him that's what I heard, but he's a very nice guy and I very much appreciate his understanding of markets and the capability of the technology and getting the right people.
Speaker B:
And Nvidia is not only a graphic cards manufacturer.
Speaker B:
They have the entire ecosystem, so they have the hardware and the entire software stack.
Speaker B:
And therefore people ask me what happens if this other hardware company builds a superchip or some kind of replacement for Nvidia.
Speaker B:
I say always, okay, they might build that that, but they Don't have the Software Stack and the Developers.
Speaker B:
And the community around that.
Speaker B:
So that's something that they built on.
Speaker A:
Top in this fast paced surrounding.
Speaker A:
It might also be a necessity to really be flexible to the agile and to just go with whatever is going on right now.
Speaker A:
So that is super interesting.
Speaker A:
However, I already mentioned the hype in the very beginning of this episode and you once said in an interview that you hope that the hype around AI decreases in the future and so I wonder what makes maybe a hype dangerous when it comes to for example artificial intelligence.
Speaker B:
Yeah, so I mean, having this momentum that is, you could definitely say it's a hype is great, right?
Speaker B:
Suddenly, the topic that I'm working with is interesting for many, many people.
Speaker B:
I mean, that's great.
Speaker B:
Not every researcher can claim this.
Speaker B:
It's also very demanding, because a lot of male people want to have having questions which is great.
Speaker B:
So it's our task to move forward to explain this technology to others.
Speaker B:
So this hype is really really great.
Speaker B:
This hype or this momentum attracts capital.
Speaker B:
And because it's a new technology and everybody wants to participate, everybody wants to fund companies and venture capital is pouring a lot of money into it.
Speaker B:
And then always, when you look at the history of artificial intelligence, we had this great moment these highs and then the lows and the lows are the so called AI winters and these AI winters.
Speaker B:
Too much promise.
Speaker B:
Not everything was achieved that was promised by the people.
Speaker B:
Funding was cut at bad times.
Speaker B:
This is something we don't want to have happening in the next couple of years and this already happened like two or three times, depending on to whom you're listening.
Speaker B:
And this happens usually every ten years because of venture capital funds are set.
Speaker B:
Up for ten years with exits.
Speaker B:
And if all those startups that promise a lot are not achieving that people are disappointed and it moves from private equity to public.
Speaker B:
And then we have have, we have maybe a bubble if the hype is really, really large and then a disappointment, a bubble bursting and this is something that we observe nowadays and everybody talks about that.
Speaker B:
I started actually to talk about it last year.
Speaker B:
Last year I was in sabbatical in the US, in the University of Washington, Seattle.
Speaker B:
So first people were a little bit concerned.
Speaker B:
What we see is, there's a lot of money in the market, a lot of expectations, a lot of promises.
Speaker B:
I mean companies are funded, reflection AI, the most recent one with two billion in funding.
Speaker B:
I mean this is two thousand million.
Speaker B:
I mean this is crazy.
Speaker B:
People are moving and being hired by Mark Zuckerberg for his superintelligence lab, two Hundred Million Up to Billions.
Speaker B:
Stock Options for People.
Speaker B:
Moving from one company to another.
Speaker B:
So those are numbers that are beyond what the community knows and this is moving into NBA level salaries, soccer salaries.
Speaker B:
It's crazy.
Speaker B:
So the question is, how sustainable is that?
Speaker B:
And is this an indication and an early warning sign of a bubble bursting?
Speaker B:
So I fully agree, we are inside of a huge bubble.
Speaker B:
I would not agree that that bubble might burst because we still believe.
Speaker B:
The majority of people believe that we are maybe able to reach some moment where we have an AI that is as good as we are.
Speaker B:
We call this artificial general intelligence, AGI.
Speaker B:
And if we're able to maybe get at this pace, maybe We will get AI's a little bit smarter than us, AI super intelligent.
Speaker B:
So maybe they will be able to solve all the problems that we have, like fusion and climate change and cancer.
Speaker B:
And you know, and the very moment we're there, we will have a great time, so it's worth investing.
Speaker B:
And therefore, I think it's not only a commercial question that we're asking if we are inside of a bubble that might burst, but it's turning to become a national security question.
Speaker B:
And in national security the question is from the US point of view are.
Speaker B:
We able to be faster than the Chinese?
Speaker B:
And from Chinese point of view are we able to be faster than the US.
Speaker B:
And whoever sits in the middle Euro is nicely watching the rest of the world doing that kind of arm raise the fear of missing out that the others are faster, is fueling this.
Speaker B:
And as long as we believe that we will be able to reach AGI in three months, six months, nine months, one year, this will move on.
Speaker B:
And the very moment where the first person will stop believing that we're reaching that point, then I'm getting a little bit nervous Because then it might be, you know, the first indications of a.
Speaker A:
Bubble bursting but trying to.
Speaker A:
Be the fastest when it comes to developing AI can also be very dangerous, right?
Speaker A:
Of course, I always think then about the disrace to the bottom, maybe some concerns are being ignored, just because it makes you quicker than your opponents.
Speaker B:
Yeah, or competitives or whatsoever.
Speaker B:
Of course, when you want to be fast and be first, there might be things that slow you down that you're not doing right?
Speaker B:
Things that might be important to not doing it.
Speaker B:
And we're talking about all the safety and trustworthiness of AI systems and there are.
Speaker B:
Concerns about those things and those big labs they are addressing them.
Speaker B:
But it's kind of, it's a secondary thing, not the primary thing that they're trying to achieve.
Speaker B:
They want to be first and therefore, that might be not considered equally important.
Speaker B:
And therefore, now the situation is, how much of a government do we want to use with regulations to enforce or put these concerns that are there in society on the agenda of those companies and kindly ask them through regulation to take them as serious as being first right.
Speaker B:
And then if you have the reaction that others say well, then innovation or regulation is innovation.
Speaker B:
So should we have regulation or should we have less regulation?
Speaker B:
And you know, move fast and break things, Like Mark Zuckerberg said for Facebook, this is not true anymore, because it's there are more important things that can happen here with self driving cars, with machines making decisions on top of people, because you know they make financial decisions on granting loans or not health.
Speaker B:
Insurer insurance is making.
Speaker B:
Those decisions.
Speaker B:
So there are many disciplines where we would love to have more regulation and some where this is not happening currently.
Speaker A:
Interesting.
Speaker A:
But do you think that international frameworks or regulations could be a solution?
Speaker B:
Well, we can, I mean, we can sit together in Europe and have beautiful discussions and think about everything that we think is important and raise the fingers and we can all maybe agree on one common sense regulation.
Speaker B:
What will happen?
Speaker B:
I mean, the others are still racing right?
Speaker B:
So nobody will care.
Speaker B:
So it's it's kind of interesting we might protect in the European Union or the European Market, which will actually be a strong statement, because it's a big market.
Speaker B:
But then what we also do, we kind of deattach the European companies from the latest technologies.
Speaker B:
There are currently situations where in Europe you're not getting the latest technology, although in US it's available because of those regulatory burdens.
Speaker B:
So it's a very kind of slippery slope and you have to keep some balance.
Speaker B:
I personally believe that we in Europe with our values.
Speaker B:
We could actually use regulation as fueling our innovations.
Speaker B:
So I don't think it's something that is mutually exclusive.
Speaker B:
I think that if we would be able to certify artificial intelligence systems to guarantee their behavior to make them more trustworthy, then we might become the gatekeepers to AI and others will build them.
Speaker B:
I mean, that's kind of a fair deal, right?
Speaker B:
But this means we have to build tools that are making those certifications and give us those guarantees and not just excel sheets with checkboxes where people go and say, okay, there is a ISO standard or whatsoever.
Speaker B:
And this is hard, this is technology.
Speaker B:
So this is something where we have to invest much, much stronger than we do now.
Speaker B:
They're great companies.
Speaker B:
Let's slow is one to mention, call another in Europe.
Speaker B:
And I think it would be interesting in particular for Switzerland with its mutual understanding and perception to become the number one hub for this type of technology, which I definitely believe every human on this planet would appreciate at some moment in time.
Speaker A:
Trustworthiness of AI is also a research focus of yours, So how close are we in your view to truly trustworthy AI systems?
Speaker B:
Well, trust is a funny thing, right?
Speaker B:
It takes a long time to trust something and then it can be destroyed very easily, right?
Speaker B:
I'm flying quite frequently.
Speaker B:
So when I'm flying under, thinking about I'm flying in an airplane, when there was this hiccups and accidents with Boeing, I suddenly was more aware am I flying an Airbus or a Boeing machine, right?
Speaker B:
And then after some time I didn't care anymore.
Speaker B:
And if there is no trust in something, then you need an artifact.
Speaker B:
Some kind of stamp of a third party saying it's certified and then you're borrowing the trust from the third party.
Speaker B:
But if you're using that after a while you're forgetting that.
Speaker B:
So that's also something that I think will happen with AI.
Speaker B:
It's you.
Speaker B:
So therefore we need this kind of trust.
Speaker B:
And once we use this and it's gonna be in our daily lives, even without us perceiving is that this is AI, We're going to forget this AI and it's going to be the same like Electricity, right?
Speaker B:
Electricity can be very dangerous.
Speaker B:
You know, touching that I would not recommend, but it's everywhere.
Speaker B:
And without this we would be having a very different life.
Speaker B:
There are rules around how to build electricity networks and know how did you live that?
Speaker B:
So I think we will have to move we already moving the direction where AI would be a technology surrounding us and probably not even be perceived as AI anymore in five or ten years.
Speaker B:
It's going to be just there helping us to deal with our life and you know in corporate environments and in private environments and therefore hopefully a little bit more boring, right?
Speaker A:
Who knows?
Speaker A:
You just mentioned electricity, which reminded me of the huge amounts of power that modern AI systems actually require.
Speaker A:
And that is something maybe people don't think about often enough, because, also when we talk together, there's like gigawatts, data centers and the amounts of data and energy power in general is huge.
Speaker A:
You cannot imagine that now, how sustainable is this growth?
Speaker A:
And could the energy demand maybe also become a limitation?
Speaker B:
Oh, it is.
Speaker B:
It is already a limitation.
Speaker B:
It is one of the biggest limitations that we have.
Speaker B:
So the first ten years we understood we need more trips, more GPU's.
Speaker B:
Nvidia is able to produce them.
Speaker B:
There's only one place on this planet who can produce them.
Speaker B:
This is TSMC in Taiwan, but they were able to do so.
Speaker B:
The current limitation is the electricity and the power grid.
Speaker B:
So you have data centers and you can put a lot of chips inside of this data center.
Speaker B:
But the data center is only one electricity input and the power grid is the limiting factor.
Speaker B:
You cannot just double the amount of chips in the data center, even if you have physical space, because you have to put the energy and the electricity in and the cooling and the water out, right?
Speaker B:
So that's a limiting factor.
Speaker B:
And you know, there are a lot of people who are saying that OpenAI currently serving eight.
Speaker B:
Hundred million people, so ten percent of the population.
Speaker B:
They could easily increase that, but they are not able to do so, because they don't have the data centers and the power to do so.
Speaker B:
This does.
Speaker B:
And it is growing exponentially.
Speaker B:
The second exponential growth that we experience is that with ChatGPT.
Speaker B:
It was very easy.
Speaker B:
I asked a question.
Speaker B:
ChatGPT gives an answer, asked a question, gives an answer.
Speaker B:
It's like ping pong.
Speaker B:
But now when you ask ChatGPT an answer, it reasons and it thinks so suddenly it needs ten times or one hundred times more opacity, electricity, power and compute for the reasoning.
Speaker B:
So each token, each unit of reasoning will be cheaper, but it will use them ten times more and one hundred times more.
Speaker B:
And the longer it thinks, the better the results.
Speaker B:
So that's the second exponential that we have have.
Speaker B:
And we see that on every different direction, the supply chain, not only from the semiconductor industry, but from the data center producers.
Speaker B:
The guys who are doing the cooling, the guys who are doing the cables, which is moving now from copper to fiber optics, because it doesn't produce heat, right?
Speaker B:
And you can have more dense installations.
Speaker B:
This is currently the one, the growing area and we'll have more data centers tomorrow than today, that's guaranteed.
Speaker B:
The question is now, where should this electricity come from?
Speaker B:
That's an open question in the US.
Speaker B:
There's this group of people who say nuclear power no co two is good for climate.
Speaker B:
We have the other problems with nuclear power.
Speaker B:
So there are now companies that are producing small nuclear power plants and not the big ones and trying to pitch that as a startup.
Speaker B:
There are other people who say, why not put in those data centers somewhere where.
Speaker B:
We have a bunch of renewable energies like Norway, where there's all this hydropower.
Speaker B:
They could actually fuel the entire electricity demand for Europe, if we would extend that the true how happy people from Norway are about that.
Speaker B:
But there are interesting ideas how to move that.
Speaker B:
And we have on the other side also fusion that is moving forward and we just said this week or last week, good fellow with deep mind having a breakthrough in fusion with Commonwealth fusion and company on the east coast in the US.
Speaker B:
So maybe when we have those big neural networks that are able to calculate how to keep those plasma stable, we might be able to make some progress in that direction and maybe move towards at some time clean energy that is abundant, right?
Speaker B:
So different directions.
Speaker B:
We probably should not go and Fuel.
Speaker B:
The AI Revolution with Fossil Fuel.
Speaker B:
Being coal or oil.
Speaker B:
I think this is is something we shouldn't do.
Speaker A:
Yeah, a lot is going on, also in the field of power supply.
Speaker A:
When it comes to artificial intelligence, it's.
Speaker B:
An exciting area for investment currently.
Speaker B:
And there are a lot of players that are unknown and that are moving in that direction.
Speaker B:
Yes, agree.
Speaker A:
Now, if you, if you had to name one development in AI, that you were personally most excited, or maybe also most concerned about right now concerned what.
Speaker B:
Would it be so excited and concerned.
Speaker B:
And there are two different things.
Speaker B:
So what we see currently is that this is one rule.
Speaker B:
Bigger is better, bigger models are getting better performance and there is this observation since last fall that this rule, bigger models are better, models is slowing down, which is kind of easy, right?
Speaker B:
If you need to build a bigger model to get better, you just need to buy some GPU's of the next generation and you can build those bigger models better GPU's bigger models.
Speaker B:
This is slowing down, this is called the neural scaling law and this is saturating.
Speaker B:
So if this neural scaling law is slowing down, there will be no need to buy a new GPU next year from Nvidia and then the chain reaction of a potentially bursting bubble will happen.
Speaker B:
So that's the concerning moment.
Speaker B:
There is a second thing that development where people are moving to smaller models, lightweight neural network, There is currently an exciting breakthrough with the ARK Challenge.
Speaker B:
And constantly one of my former PhD students, as an indea, the startup who runs that, and one of the collaborators and the roommates developed a very small model that outperformed like seven million parameters, super small, outperformed some of the billion parameter models on this ARC challenge.
Speaker B:
So that's interesting.
Speaker B:
And we saw already that there are other types of models, like the entire deep seq story that might be.
Speaker B:
Able to to actually outperform this previously established knowledge how to build neural network.
Speaker B:
So there's a lot of movement in that it's obviously no you have to put more intellectual capacity into that than just buying with the money that you're getting bigger GPU's.
Speaker B:
So that's kind of exciting for me what I think what we actually doing currently, We are creating millions of neural networks and once they are trained, they have a particular purpose and then there you know another model is trained, which is better and this model is kind of, it's not used anymore, so it's wasted.
Speaker B:
So if you think about you have a neural network model that is encapsulating this amount of compute and this amount of co two, it would be great to use this knowledge somehow.
Speaker B:
So we have up to two million models that are uploaded to a website called Hugging Face for Different Purposes.
Speaker B:
And we're not using this knowledge, but we're training those models from scratch or maybe from a pre trend model.
Speaker B:
So how could we use the entire knowledge of all of those models somehow in making the new models train faster and better and all that.
Speaker B:
And this is something that we do research on, we call this weight space learning and this is what excites me the most.
Speaker B:
Because it would be actually amazing to use all this knowledge of those two million models as a foundry idea, not the factory, but a foundry idea.
Speaker B:
And then to train new models much faster, maybe only needing fifty percent of the actual time or twenty percent of the actual time time and therefore be.
Speaker B:
Much more agile and sampling your models as you need and just do some fine tuning.
Speaker B:
And this is some research that we do and this excites me the most, because it's a very simple idea, but nobody had this idea and we started four years ago and it's moving faster and faster and it's working surprisingly well.
Speaker A:
So we should stay tuned for what your research is about to bring to the surface in the next yeah, months, weeks, years.
Speaker A:
Who knows?
Speaker B:
Who knows?
Speaker A:
It's absolutely interesting.
Speaker A:
Wow.
Speaker A:
And I'm sure we could talk for hours about the different topics, because the whole field is so broad and it's huge and there's so much going on, which also just shows once again how relevant it is, that people are actually, also doing research in this topic, that people are connecting, that there is conversation happening internationally.
Speaker A:
But I think that today's conversation already showed that AI isn't just about data and models, but rather also about curiosity, ethics and how we choose to shape the future.
Speaker A:
So thank you very much Damian, for being here and sharing your perspective.
Speaker B:
Yeah, thanks for the discussion.
Speaker B:
Really interesting and let's see what the next eighteen or twenty four months will bring.
Speaker A:
I'm very curious for that.
Speaker A:
And to all listeners, perhaps this episode has given you a bit to think about.
Speaker A:
But at the very least you should now have a better overview of the broad field of AI.
Speaker A:
Never forget to use your own mind, but be open to new developments.
Speaker A:
Thank you for listening and see you next time on the HSG student content.
