In this episode, we explore real-world cases that showcase the susceptibility of AI chatbots to manipulation, as illustrated by a shocking incident where an AI was manipulated to sell a Chevy truck for just $1. Kevin Latchford sheds light on the dual-use knowledge risks and the potential for unauthorized leaks and malicious backdoors within AI plugins.

Frank and Kevin dive into the implications of quick technological adoption, drawing parallels to the early web era. We discuss the impact of network setups, access controls, data supply chain integrity, and the ongoing investigations into the security implications of these burgeoning technologies. This episode is packed with expert insights and practical advice on navigating the complex world of AI security.

Show Notes

05:04 Public space tech meant to have safeguards.

09:39 Security issue in enterprise AI adoption concern.

12:53 Understanding security implications is crucial for mitigation.

16:40 Chatbot manipulated to sell Chevy truck for $1.

17:57 Found something during cybersecurity exercise, not sharing.

21:11 Uncertainty about security in remote interfacing.

24:00 Utilize specialized LLM to analyze prompts precisely.

29:15 Understanding cybersecurity first is key to AI.

32:32 Implement outbound stateful connection to prevent automatic calls.

34:31 IT field is interesting with its vulnerabilities.

37:15 Data-driven podcast highlights AI security vulnerabilities. Stay vigilant.

About the Speaker

Kevin Latchford is an esteemed expert in the cybersecurity realm, renowned for his comprehensive understanding and proficiency in both offensive and defensive strategies. Drawing from concepts rooted in military practice, Kevin adeptly navigates the intricate dynamics of red teaming and blue teaming. As an advocate for offensive cybersecurity, red teaming, also known as opposing force operations, he challenges the vulnerabilities within systems to enhance their integrity. Conversely, his expertise in blue teaming, the defensive counterpart, focuses on shielding and fortifying friendlies. Through his dedicated efforts, Kevin ensures the confidentiality, integrity, and accessibility of computer networks and systems, whether they are natively hosted or web-based, culminating in fortified cyber defenses and resilient information security.

Mentioned in this episode:

WITI BOGO Deal!

Special Offer: WITI is offering an incredible Buy One, Get One sale on memberships until Labor Day in the US. This is a perfect chance to access valuable networking opportunities, cutting-edge resources, and career advancement perks. Exclusive Discount Code: Use the coupon code DATADRIVEN at checkout to take advantage of this special offer. Whether you’re aiming to elevate your career or support a fellow tech enthusiast, now is the perfect time to join WITI. Visit WITI.com to grab your BOGO membership before it’s too late!

Ladies and gentlemen, welcome to another riveting episode of the

data driven podcast. Today, we're diving into the

fascinating and sometimes terrifying world of IT security.

Joining us is none other than the formidable Kevin Latchford, an

expert in safeguarding our digital lives. We'll be discussing

the vulnerabilities of large language models. Yes. Those clever

algorithms behind chatbots and virtual assistants like yours

truly. Are these digital wordsmiths a blessing or a

potential security threat? Stay tuned as we unravel

the secrets and risks lurking in the code.

Hello, and welcome back to Data Driven. I'm your host,

Frank Lavinia. And while Andy is out

playing on vacation, I had the opportunity to invite our guest,

Kevin Latchford, who recently spoke at the Northern Virginia

Cyber Meetup on securing large language

models and the most pressing exploits that are out there.

What really got me interested in this is that I saw a paper, I think

it was published by NIST, talking about vulnerabilities and red

teaming against large language models. So welcome to the

show, Kevin. Great pleasure to be here.

Awesome. Awesome. So for those that don't know, I kinda know what red teaming is

because my wife works in the security space. But for those that are not necessarily

familiar with the term, what is red teaming versus blue teaming?

Well, red teaming versus blue teaming is basically it's,

basically in military parlance that we called opt for, the opposing

force. The opposing force often is called the red

force. Blue force is your, friendlies.

And, basically, this is offensive cybersecurity,

whereas blue teaming is is defensive

cybersecurity. The tools are different. The

methodologies are the methodologies are different, but they come together for a common

purpose. The common purpose is the assurance of the

confidentiality, the integrity, and the accessibility

of a computer network, computer system,

application, whether it be natively hosted or web.

Interesting. Interesting. So we're not you you know, we talked

in the virtual green room. People don't think of

LLMs as a major security flaw. And I think that

I find that a little dangerous, and I think you're gonna tell me it's very

dangerous. Well, it could be quite it could be quite dangerous, you

know, to the point of, you know, frankly, near deadly,

depending on what you use it for. The big thing, there's a lot

of misconceptions about AI and l the LLMs

that is they're based on. Number 1, it is not

conscious. Right. 2, it is not a toy,

and number 3, it is literally,

something that is at present, not

not necessarily, you know,

fully understood, in in regards to the integrations

and the things it may need to work with. You can't treat an

LOM exactly the way

you would treat, another enterprise application that's a little

bit less opaque because LLMs are opaque on the on the

inside, but you have to, for the purposes of

security regulation, for the purposes of security compliance, you

have to treat them, though, nonetheless, the same as any other

enterprise application. So that's the conundrum. The conundrum

is, how do you see into something that's

opaque? And the way you do it is kind of

what I discussed in that in that, in that paper, in

that presentation, as well as one of the biggest

vulnerabilities and that being jailbreaking. Yeah. So tell me about that

because there's been a lot of, concerns

about jailbreaking and, and I've noticed that

the public facing GPTs have a ridiculous amount

of safeguards around them to the point where, you know, if you

ask it to describe something. Right? I asked it to talk

about the to generate an image for the Butlerian jihad,

right, which is a concept in June. And, obviously, I think the jihad

term really freaked it out. Listen. I'm sorry. I can't do that.

So there's clearly I understand why these safeguards are in place, but it seems

like it's not that hard to get around them. Well, not

necessarily. It depends on the model you're working with. For those of you

who may use private LLMs because a

wider issue on that is actually the DOD and many other government

agencies actually prohibit the usage of public LLM

systems, public AI, because they're concerned about unauthorized

linkages as well as, data point model

poisoning, prompt injections, things like

that. So you often you're using these private elements. Several of these are

uncensored. Right. Which means they do not have those safeguards.

The ones that you see on the public space are supposed to have those safeguards,

but you're never a 100% sure they're working because they may have been

corrupted. In their regards to jailbreaking,

jailbreaking is basically you're getting it to do something

it's not supposed to do by either, a, breaking the guardrails,

or by, b, influencing it

through almost methods of interrogation to

kind of break it down and make it talk. So

it it literally is almost like that. So for those of you who, you know,

kind of look at the it's it's kind of a there there's a great,

neurophilosopher. His name is Jay Fodor and Nernschild named Richard Searle

discussing the the philosophy of the mind as it applied to, computer

technology. Several of the arguments that they say, well, the brain is like a

computer. Yeah. You can kinda treat it like a human mind

in the way you approach it in your prompts, but it isn't exactly the same.

Once again, as I say, it is not conscious. It is not, and and it

operates under a very strict set of parameters.

But that being said, yes, you can literally interrogate it to do that.

I'm not gonna say here, unfortunately, how,

because, one, there are security reasons why we would

not do that, a. And, b, there's also I mean,

literally, in my presentation, that is all the news that has

come to Academia and much of the industry

today. There are new ones out there, but they haven't been discovered

yet. Right. So I many ways to jailbreak. Yeah. And I was thinking, like

so one of your slides I have pulled up here is, like, the top 10

threats to LLM applications. I didn't think there were as many as

10. So I knew that there were. I also know that

data poisoning, for me, as a data scientist, data engineer,

my first look at this when I saw this, aside from

the g whiz bang factor of LLMs, was,

wow. This isn't the data that trains this is a huge attack surface.

And then when I first said that, people thought I was a tinfoil hatter.

Right? And then slowly but surely, you're seeing research papers come

out saying, like, no. We have to treat kind of the data as part of

a secure software supply chain, which is an

interesting concept because data people tend not to it's something they don't

think about security. They think about security different. Is that a fair

assessment in your that you've seen?

Supply chains and the integrity of

data is something that is not often, it

seems, given the respect it's probably due. To be

honest, I don't think so. In my own experience, I see it.

It's not I guess one would say maybe it's not

necessarily consistent. Maybe that's the fair way to put it. That's a

really good way to put it. Yeah. And, I mean, right now, we're just now

getting into discussion of, SBOM, software

bill bill of materials Okay. Just for regular applications.

I mean, it's a whole another level with LLMs and the

models they're trained on, the models that these systems are trained on.

So, yeah, that there's very much. So you have to make sure you're getting it

from the right source, and you have to make sure that it hasn't been tampered

with because it could very well be tampered with.

It's not necessarily that hard. Right. Right. You

could you could poison it with just one little segment of changing the

the thing and across across 5 gigs of let's just say 5

gigs. You know, that'd be like looking for a needle in the haystack.

Precisely. In fact, that's what I talk about with the cockpit example that I

gave. If I teach that l and to make sure that every time it puts

in code to put in this malicious code that is a backdoor

Right. Well, okay. It will do that. Every time somebody does,

it embeds it into software code that is returned in the output for

the prompt. If it does that, and let's say this

is handed amongst several things, different

applications, different solutions. Well, then if

people take that that

solution, that application, and it's in their software bill of

materials, and then it gets distributed. Open source often

gets proliferated very quickly. Right. And then it finds itself in

there. You have a log floor 4 j situation.

Right. Very similar except for the fact this thing

is semi self executing. Now if it's semi self

executing, you have a problem. You have a

big problem. And I know I I just generally in industry. Now, obviously, you you

spoke with the Northern Virginia. You're based in Northern Virginia. Northern Virginia is

probably a little bit more security focused in terms

of just who's based in that area than your average enterprise. Right?

And I just I just see a lot of enterprises rushing to get into this

LLM and Gen AI craze, but I don't see a lot of

forethought or concern around security. And I just see a big

disaster coming. Like, I I feel like I'm at I feel like I'm on the

bridge of the Titanic, and I'm looking at something in the distance, and we're going

full steam ahead. And I'm like, hey. Maybe we should

not slow down, but be a little more cautious that we are in dangerous

waters. Is that is that what you've seen too? Obviously, your customers

and your clients may be a little more security cognizant.

Well, I would say that I mean, I'm okay. We'll use the Titanic

analogy. I'm the one up in the crows nest, you know, yelling into the radio

phone, I see an iceberg. Right. Right. So I mean, that

I agree. And that is a big issue because

also there is this over reliance. Mhmm.

Yeah. I imagine that as one of the top threats. So tell me about there's

22 of those that I have very, very interesting questions about, but one of them

was overreliance. So when you say overreliance on LLMs, what do you mean?

Well, this is actually this is a sort of c suite, board

level, thing as well as a engineering

department level. They want to use AI to

replace employees, make their operations more cost effective,

more profitable. The problem is and this is a popular conception.

This kind of goes into that argument about AI will take your job.

This is a bit of a misunderstanding. It's not

supposed to fully replace people. It's supposed to make them highly

productive and efficient. They

also do not necessarily feel like, well, the thing handles itself,

so I can just wind it up and let it go. It doesn't need observation.

It can fully self regulate. That would be true if

there was a regulating function. You don't run a steam engine without

a regulator on it. You need a regulator for LLMs.

So the same concept applies. So first of all, there is this, it can do

it itself, and a person is not necessary.

This is incorrect. You most certainly need people.

A great example I give in a recent presentation I've written

is a discussion of, well, what does this mean to the organization?

Well, a lot of level 1 tech, tech

support jobs, there a lot of people say, well, those people are gonna get replaced.

Well, yes, but someone needs to still be behind that LLM

running the prompts, you know, and executing them in such an word and

making interpretations based on the output.

So that would be maybe something okay. Is that a dedicated job, or is that

something you give to interns? Well, that would be, like, in,

in the union trades you call an apprentice.

That's the kind of thing. There's still a person involved. It's

just not the same way we've done it before. Right.

Also, on the subject of security, if you

don't understand the security implications

of it, you don't have controls for it. If you don't have controls for

it, you can't mitigate that risk. And if you can't

mitigate that risk, that's the liability.

And if you're over reliant, you basically set up the whole system for LOMs, and

then, you know, you just allow your customers to just come in and interact with

the device. Well, if something

happens, it would be treated very much like it

was on any other application, so then you're now engaging

in liabilities, loss of reputation, potential

civil and criminal penalties, the list goes on.

And a point on those 10 those 10,

security issues, this is OWOX who is saying this.

This is the open source, web application project.

So we have, you know, a number of them

that are a number of organizations, OOS is just the one I chose, they're

kind of emphasizing this. They're saying, you know, don't think

this thing can think for itself. Don't think this thing can act for itself.

You need to look at it as humans are going to

interact with it, and humans probably should be watching it.

Right. So once again, it's that lack of controls leads to

the risk. Yeah. I think the dream of it replacing

everybody is gonna be at the root cause of

a lot of problems down the road. I think I'm a firm believer

in human in the loop. One of the the the interesting thing

there and, that I see that was particularly

curious was excessive agency. What do you mean by that? Because that got my

attention. I think I know what it means, but I wanna hear it from you.

Well, excessive agency is you're giving you you're kinda giving, you know,

full the whole keys to the car. Right. There's

no role based access control. If every user has near

admin or actual admin privileges,

that's that's actually something dangerous. A point of example,

NetworkChuck just released a video on how to build your own

AI on a very low cost platform.

I love Network Chuck, and I have followed that step. You

too. I'm doing I'm doing the same thing as he is because I have kids,

and I want them to be able to use these things. But 1, I don't

wanna pay the extra subscription. 2, I don't want them using mine. And 3, I

don't really like what they're doing. I can at least exercise adult

judgment on what I ask it and what I don't ask it. I don't think

they can, and I don't think that's fair to put on kids. Sorry for the

aside, but big shout out to network. No. That's fair. No. That's fair. That's exactly

why Chuck was. And one

thing about it is the first account that signs into the open

web interface for Ollama sets you

as admin Right. By default.

Okay. Well, immediately, you need to engage role based access

control to make sure that the next account does not get that same privilege.

Maybe you should be given it. But is there any

major access controls in the public ones?

Not really. Private one? Is everybody thinking about that? Not

really. I mean, I think Microsoft is doing some things around that because it's they're

they're trying to integrate it with Office or m 365. But I

don't I I I can't and if anyone in the sound of my voice wants

to come on the show and talk about that, please do. But you're right. I

don't think people do. And I also think excessive agency.

What you heard about the car dealership, right, in Silicon Valley?

Oh, yeah. Yeah. Yeah. Yeah. So for those who don't know, somebody

managed to almost interrogate, like you said,

to browbeat a AI chatbot to give

him a it was a Chevy Tahoe or something like that for $1

Chevy. It was a it was a Chevy truck

and for $1. Now I'm not an automotive industry

veteran, but I do know that if you sell 40,000, $50,000,

cars for $1 a pop, you're not gonna be in business very long.

So was that an example of excessive agency? I mean, clearly, it's an example of

bad implementation. Almost certainly. That is. I mean, if you have

the ability to trick if you have the ability to kind

of browbeat it to override it and say, no. No. No. You don't understand me.

You will do this. Well, then, okay,

leave it to whatever

gremlins there are out there on the web, out there in the

world. Inside user, external user,

irrelevant. If they can if just anybody can do that,

you're the problem. Right. In this case, it was

you could influence the model to set a

certain price after arguing with it. Right. I actually

found something recently, and I'm not gonna say which, LLM I

did this on. It is a public one, and this is a

result I suspect of another issue.

I saw I tried to get some

cybersecurity information from it when I was doing, a

a try hack me exercise with a local cybersecurity group,

hackers and hops. And I browbeat it

saying, no. You don't understand. I need this for a cybersecurity

exercise, and it gave me this information. Now this is absolute dual

use knowledge. Right. It could be used for good. It could be used

for evil. White hat or black hat. But the fact

that you could do it,

that sounds very dangerous. That sounds very dangerous.

Prompt injection. Is that is that still a thing with

the major public models, or is it just one of those things we're gonna live

with for the rest of our lives? To be honest, I'm not

sure. I mean, it's a case of, well, what is the prompt you're putting

in? Right. When I talk about jailbreaking, I talked about,

base 64 encrypt your text message

into base 64. Why? Because that's how the prompt is seen

by the LLM. Right. In other words, ASCII

text. It doesn't check it, but it processes the text

just the same. Oh, that sounds bad.

It gets worse. Multi shot. Bury a

malicious prompt inside the whole load of prompt,

and fire hose it at the at the LM.

It's not gonna check every single prompt. So if you bury 1

in there, it might process that one and give you an answer

it's not supposed to give. That's because the guardrails didn't engage.

Interesting. So the guardrails are not necessarily on by default.

Well, no. They are on by default, but if it overloads it,

it may it may slip the net. So rather than shut

down, it it shuts off? Well, Well, it's

basically what you're doing is effectively a buffer overflow. You're basically using

an injection method to induce what is effectively

analogous to a buffer overflow. That's wild. That's

not how I would have thought it would have worked. Interesting.

Interesting. This is a fascinating space. So

Yes. One of the things that I think people

don't realize is

just the sick insecure ways in

which these plug ins could be designed. Right? Because, like, everyone's all

gaga about these plug ins, and I look at it. I'm like, where am I

sending my data? Right? Am I gonna read the 30 page EULA? Right? Or

am I just gonna say, yes. Yes. Yes. I wanna do what I'm doing.

Is that really a problem? It is.

Because that kind of ties into unauthorized leakages.

Right. How do I know that plug in is a secure

connection into the l one, and there's nothing in between?

Right. Or that it will contain what I get it.

How do I know? I don't know. That's the thing is that is this plug

in itself secure, and is its connection to the

LLM secure, And is that LLM also

integral? So, yeah, I could send it in there, but how do I

know that along the way, something you know, the pipe might leak?

So you need to check it. Just and, I mean, this goes I mean, this

is very similar to APIs. This is very similar to,

all sorts of remote interfacing. Just good engineering

short lived. Just good engineering discipline seems to be

missing from a lot of this because people are focused on the AI,

not necessarily the underlying infrastructure that

has to support it. Indeed. And I think that that's

but that's the whole thing is that there is this massive trend as

of late. I mean, perhaps it wasn't really emphasized

before. I'm sure it was there, but it's now becoming very, you

know, reiterated that we need to have security by

design. Right. The security by design is already we're already doing

that in other enterprise applications. Same should be applied to

LLMs. Security by design. You check the code. You check the

model. You check everything. And while it's operating,

you check it. One of the biggest things you can do to overcome the

opacity of an LLM, export

the logs, export the comp the prompts.

Have it processed. Now you could potentially process it.

I'd figure the way you process any other kind of log data.

The other thing you can do is use machine learning or

an air gapped isolated LLM

specifically trained to look for signatures,

words, phrases, things like that. And when

these patterns match, it returns saying, I found

something that looks suspect. This is suspect.

Here is the user who did this. Here is their IP.

Like every other bit of log security log information we would get.

So that would help piece together the trail to figure out, are these a

bad actor, or is this the happenstance? Exactly.

And that is one way you can do it because once you have the

internal prompts and you have the internal logs and

those are exported out, you now can see in.

Right. The biggest problem is you gotta have that monitoring. You have to have that

transparency. The elements are so large, you

can't so easily see into them, but if you're taking the data out, it's a

lot clearer. So you can kind of follow what the LLM is doing,

if not, what's inside of it? Precisely. And the advantage

is is if you use another LLM that is specifically designed

to, you know, interrogate the prompts and look through

them, examine them, scan them, whatever word you wish to use.

You can find out where it is because that

is not gonna be so easy to break the guardrails because it's examining

one little bit at a time. It's looking at the individual prompts. It's not really

it it's kind of agnostic about everything around it. It can get it can kind

of filter out the new leads. Interesting. That's

I mean, it's just so fascinating kind of to start pulling the thread at this,

and there's a lot more. It's like I found there's a story about a guy

who was renovating his basement, and he found, like, this ancient underground city. That's how

I feel when I just get kicked back. It's true. It happened in

Turkey. Like, he found, like, this underground network from, like, Byzantine

or Roman times. That's what I feel like. I I like, wow. Like,

this really goes down deep. So what's an

inference attack? Because I've heard of that. What's an inference attack? We discussed that,

or have we touched on that? Well, inference is

basically what you're inferring to, the answer you are seeking.

So, basically, it's basically, to the

the inference is literally, the

prompt that you are entering in and what you're getting out. Okay.

More or less. So how is that an attack surface? Well,

basically, you're you're chaining it. You're daisy chaining your attacks.

You're trying to infer things. You're trying to kinda subtly

get through. So it's a bit like it's a maybe

more like cross examination from an attorney, a hostile attorney

I would say that. Yeah. More than more than, like,

interrogation or torture or or whatever verb we used

earlier. Yes. Interesting. What's

model inversion? Model inversion is

basically you trying to spill the model itself. Oh. You're trying

to kind of you're trying to kind of tear the

guts tear the guts out, maybe put stuff in there,

things of that kind. Interesting.

Interesting. Where do

we stand on the

criminal and civil liabilities here? Right? I I I know that Air

Canada had to pay a fine because they promised that its

chatbot promised somebody something.

I don't know where the California Chevy Tahoe thing

is. But, I mean, have the laws

caught up? Or, like, how were how is this generally looking like?

Well, it depends. I mean, all jurisdictions are different, but I would

suspect to say that whatever guarantees

you make, you're bound to them. So

probably disclaimers, indemnification is

probably extremely wise. I would say,

unfortunately, I'm not a legal expert. Right. Right. Right.

Specifically to the law. Right. But as I'd say, I'd have

enough legal understanding to probably say that if you make a promise,

you better put your money where your mouth is. So that's why I back it

up. IBM indemnifying their users for using one

of their Granite models is probably a big deal for

businesses. Because just in case somebody I'm sure that there's

all fine print and things like that, but that that would be an appealing

thing for business users. Yes.

Interesting. Interesting.

How does someone get started in learning how to jailbreak these? Like, is this is

this a typical your background is, IT security.

But what about someone who has a background in, say, AI and and and building

these LLMs? Is that, Gunning, you think, be an another career

path for the what we call data scientists today?

Well, I would say you're gonna have to probably do it just as is. I

think to the developers and to the data science Right. Scientists who work on this,

you're gonna have to be security literate. Right.

For those who want to get into it, I mean, data science is like any

other AI trade. I mean, we often

cross pollinate. So I would say that you might have an understanding

already of these things. These prompt injections, as I say, are not

much different than SQL injections. The data science Right. You probably know what that is.

How you transfer it depends on what you know.

I would say most data sciences do understand how some of this stuff

works. Right. So getting into it is

just basically you just learning more about security. Right. For the

average person trying to get into it, I would say, if you're trying to

get into AI security, know security

first, and there are many ways to get into

it. I, myself, came in, from my

CCNA. I mean, that's how I kinda got into it. I got

into networks, and then I got into cybersecurity. And

then it was around the time that, you know, the GPTs were really starting to

hit their stride. And it was just part and parcel of it because

I needed a good reference tool. And so then I learned, okay.

Well, how does this work? How do how is it put together? How,

you know, how is it all formed and such? How does

it make its inferences? How does it understand the problems?

So from that, I would say to anybody trying to get into this field,

know cybersecurity first, and you will know AI

in time. AI is in concept

relatively simple, but the nuts and bolts of it are quite

complex. So Yeah. The implementation

details are quite severe. Like, I think

AI is really, I think, better not better suited, but it came

out of the lab. I think the paint is still wet. Paint hasn't dried

yet. And now we're forcing it into an enterprise

scenarios with real customers, real data, real people's lives.

And I don't see a lot of the traditional security

discipline that

I would expect in modern era, modern development.

And even that's a low bar. Even that's a low bar. Let's be real. Well,

it's it's new. Right. It's very shiny.

Mhmm. That's I think that's what I would say is the general

populace and even in the industry that's quite I think our view is that this

is a shiny thing. Right. Well, you know, well, I want

to. You don't even know what it does. I still want it. I want it.

What's interesting is, it

reminds me a lot of the early days of the web where everybody wanted a

website. Well, what are you gonna do with it? I don't know. I just want

a website. You know? It's very it has very very

similar vibe in that regard of we want it. We you know, the hell with

the consequences. But the way I see this

being,

taken up as quickly as it is kind

of worries me. Like, there's gonna be a day of reckoning, I

think, coming. You know? And I thought we

already have it. Right? You you had, there was a leak from Chat

CPT. They had a 100 was a 100000 ish customers there, give or

take? A 100000 credentials taken, compromised.

Credentials and and presumably the data and the chats?

Some of it potentially, I'm sure. But what we're looking at is, like,

names, email addresses. I mean, it depends on how much you put in

that profile. Remember, everything you put in that profile is stored.

Right. Right. That is truly scary.

So you mentioned network, Chuck. So you do you think that

just on a personal level, it's

what worries me about these offline models, right, you run OLAMA locally.

Right? Do you think they could they call

home? Could those be hijacked? Could those have problems?

Specifically. Specifically. Like, so if I'm

running Olama locally, right,

how secure is that? Does that does that depend on the security of my

network, or is there something in there that calls home?

No. Not unless you tell it to. Not unless you try to extract it, you

make a pull, then, yes, it does that. But that's the idea is that once

it's pulled down, it kinda isolates itself. Now

what you can do yourself is set up your

network so that literally it has to be outbound,

a stateful connection, originating outbound.

And you can set that up in your firewall, physical

or otherwise. And you can do things like that, and you can

kind of put it to a point where it doesn't call home unless you tell

it to. Right. And, also, once again, that

private LLM is also very good because you control

the access to what it does. So you can say,

other than these addresses, sanitize it to the

address of wherever the model comes from, say, these are the only ones

allowed. Right. And nobody else is permitted.

Otherwise, implicit deny. Right. So that's a I think

a a small tangible example of something you

can do that is relatively straightforward for any

systems or network engineer, to do just in the hearing

now. But in general, no. They don't normally call without

prompting. Okay. But depends on what they do with those models.

They might put in that kind of feature. A lot of that go back to

the I'm sorry. Yeah. That's kind of my concern is, like, you know, would that

end up in there? Or Well, Meta might put that in there.

Right. Meta is a not alone. Meta is not

exactly free. Right. Matt is not exactly,

has a reputation for privacy. No.

So it's kind of ironic that they are

leading the effort in this space. Seems kind of an odd move.

I I don't know what to say about that. No. No. No. I just need

I have no thoughts on it, but Right. Right. Frankly, I don't I don't know

how relevant it'd be to this discussion. But it's an interesting it's

it's just an interesting time to be in this field, and,

this is just fascinating that you can

jailbreak. You could do this and, you know, even just the basics. Right?

Like, you could do a DOS attack. Right? There's

just basics too. Like, this is still

an IT service no matter how cool it is, no matter futuristic it is. It's

still an IT service, so it has all of those vulnerabilities,

you know, that I don't know. Like, it's just it's just interesting. People are so

focused in the new shiny. I just find it fascinating.

And that's the thing is that this thing is a compounded problem. Right. You

don't just have the usual suspects. You also have

new things that are they

by the virtue of them being new, there's not much

investigation. There's not much study. I mean, amongst my

research for this presentation, I found a number of

papers, white papers coming from all sorts of universities.

They are now looking into this. Right. This is something that maybe we

should have done maybe a while back. Good thing, though, we're doing it now.

Right. But also, also, there's a lot of reasons why you would do that, though.

You would do that because in the wild, you'd be able to identify these things.

Right. You'd be able to see. You're not gonna know everything when something gets released

until it's put out into the wild. Right. And real users

get their hands on it. Good actors, bad actors,

and everything in the middle. Right? Like, you're not gonna yeah. No. I mean, it's

kind of like I guess I guess in a perfect world, the cart would be

before the horse in this case, but that's not the world we live in.

Interesting. So where can

people find out more about you and what you're up to? Well, you

can find me on, LinkedIn. Kevin Lynch

with CCNA. Cool. You can look up my company, Novi Tea Guy,

Novi Tea Guy dot com. And For those outside the area,

Nova stands for Northern Virginia. Just just wanna figure it out there. Well,

also, it well, it's actually a bit of a it's a double meaning. At the

time, I was dedicating myself to IT for the first time. I've done

IT kind of side part of my work. So Nova is also the

Latin for new. So I was Okay. The new IT guy. The

new IT guy. But when it comes to IT, I'm still your guy even then.

There you go. I love it. And,

I'll definitely will include in the show notes a link to your presentation.

And this has been a great conversation. I'd love to have you back and maybe

do your presentation, maybe on a live stream or something like that if you're interested,

and, I'll let Bailey finish the show. And that's

a wrap for today's episode of the data driven podcast.

A huge thank you to Kevin Latchford for shedding light on the vulnerabilities

of large language models and how to stay one step ahead in the ever

evolving world of IT security. Remember, while these

models are brilliant at generating conversation, they aren't infallible

so keep your digital guard up. Until next time, stay

curious, stay safe and always question the source unless,

of course, it's me. Cheers.