Cloud infrastructure provisioning can be a challenging task, often requiring a delicate balance between speed and accuracy. In our discussion today, we explore how AI can streamline this process. Our guest, Marcin Vizensky, co-founder of SpaceLift, introduces us to their innovative solution called Spacelift Intent, which simplifies infrastructure management by removing the complexities of traditional tools like Terraform. Marcin explains how this platform allows users to express their needs directly, enabling quicker provisioning while maintaining necessary controls and policies. Join us as we delve into the intricacies of AI-driven cloud provisioning and its potential to make infrastructure management more accessible for developers and data scientists alike.

The discussion centers around the challenges of cloud infrastructure provisioning and how AI can provide solutions. Marcin Vizensky, co-founder of SpaceLift, outlines two extremes in the current landscape: the rapid but unrepeatable manual provisioning of cloud resources through console clicks, and the slow, complex processes involving tools like Terraform that require extensive knowledge and setup. He emphasizes that many users, such as developers and data scientists, do not need to become cloud experts; they simply want to provision resources effectively. Marcin introduces Spacelift Intent, a tool designed to simplify this process by allowing users to express their needs in natural language, which AI translates into API calls. This approach shortens the lengthy deployment cycle typically associated with traditional tools, making it easier for users to manage infrastructure without deep technical expertise.

Takeaways:

1. Cloud infrastructure provisioning is challenging due to the extreme approaches we have adopted.
2. AI can facilitate the process of managing cloud infrastructure by streamlining resource provisioning.
3. SpaceLift Intent provides a middle ground between fast but chaotic and slow but formal cloud setups.
4. Using AI in infrastructure management can allow for quick prototyping without extensive learning curves.
5. The approach of SpaceLift Intent helps users transition from initial setups to formal infrastructure management.
6. Historical data from previous configurations is preserved, allowing for easy migration and state management.

Links referenced in this episode:

1. spacelift.com
2. opentofu.org
3. softwarearchitectureinsights.com

Companies mentioned in this episode:

1. SpaceLift
2. Open TOFU
3. terraform
4. pulumi
5. Cloud formation
6. AWS
7. GCP
8. Microsoft

Mentioned in this episode:

What do 160,000 of your peers have in common?

They've all boosted their skills and career prospects by taking one of my courses. Go to atchisonacademy.com.

Atchison Academy

Hello and welcome to Software Architecture Insights, your go to resource for empowering software architects and aspiring professionals with the knowledge and tools they require to navigate the complex landscape of modern software design.

How can AI assist you in developing and managing a cloud based infrastructure?

My guest today is Marcin Vizensky.

Marcin is the co founder and chief R and D officer at SpaceLift and he's also the co founder of the open source project Open toefl.

He's here to talk about bringing AI and AI based tools to the world of cloud infrastructure provisioning.

Marcin, welcome to Software Architecture Insights and I hope I didn't mangle your name too much there.

That was perfect.

Like a native Polish speaker.

Thanks for having me and welcome everyone.

So why is cloud infrastructure provisioning so hard?

It's a very good question.

I think we kind of made it hard in a sense.

The reason I think we made it hard is because we put the possible solutions on two sides of a ostensibly very broad spectrum.

One is optimizing for extreme speed.

I just go to the console and I click on things like a caveman and I get things done instantly, right?

I can't repeat them.

I don't know what I did and definitely my colleagues don't know what I did.

But it works, right?

And this is the kind of caveman approach.

And then there is another spectrum or another extreme of that spectrum, which is an extreme ceremony.

And that extreme ceremony is what cloud practitioners would preach.

So that extreme ceremony involves writing terraform or pulumi or some other arcane language.

Right?

Cloud formation with its arcane YAML.

And it used to be JSON.

Like it's probably the worst JSON I've ever seen in my life.

So you're correct.

You put it under like some form of CI cd.

You need some manual approvals, you'll centralize it.

You'll need some staging space or storage for your state file.

You'll need to learn the arcane language.

But not only that, you'll then need to learn the kind of APIs the provider in a terraform space or open TOEFL space, you'll need to learn the provider.

Then from that you need to learn the API of a particular resource that you need to provision, right?

So if you're just beginning this journey and every now and then like not everyone has to be a cloud practitioner.

Most of people just provision cloud resources just to keep going with their job.

They're like developers, they're data scientists.

They just need something from, from the cloud.

They're not necessarily obsessed with getting Everything right the first time.

It takes days to do things properly.

Right?

It takes days.

And even for an experienced professional in a highly regulated environment, in a highly controlled environment, without the right tooling and without like very like the best practices, it'll take hours.

Right.

So we go between like and.

There's nothing between like doing something fast and stupid and doing something very proper, but very ceremonial.

And that's what I think makes it kind of difficult.

Yes.

You probably shouldn't do things in, in the console.

Right.

I mean other than incidents response, I wouldn't advise that anyone does things this way.

But like, most of the time when you're just developing cloud infrastructure and you're just playing around, you want to build it like iteratively, you don't want to learn all that stuff.

Like it's usually not your bread and butter.

You're an app developer, as I said, you're a data scientist.

Why would you spend hours learning Terraform?

Why would you spend days setting up pipelines?

Right.

And yet if you want to do things properly, this is what the gods of infrastructure kind of require you to do.

And I think the problem was infrastructure development being so difficult is that we made it difficult by not offering anything in between.

Yeah, the way I like to describe it is, you know, the.

There's the time to first provisioning and then time to second provisioning.

The console makes the time to first provisioning fast, but the time to do anything follow on hard.

And the traditional, the Pulumi Cloud Foundation Terraform, the CICD approach to infrastructure provisioning is time to first provisioning is extremely long, but then the second and third and fourth are much easier and better and you can control it very well and make changes and understand it very well, but the time to do the first thing or anything is so expensive and so much time and effort just to get it tuned exactly right before it will do anything.

So you're right, these are two extremes here.

So what, what I'm assuming we're going to talk about next is the middle ground.

And so what you do is you provide a platform to manage infrastructure provisioning, but you do it kind of in this middle ground area.

Is that true?

Exactly.

That's the point.

We're calling it spacelift Intent and it comes as an open source version and as a commercial version.

In a nutshell, what we're doing is we're taking the very, very long process of Terraform, or open TOEFL in this case deployment, and we kind of short circuit it on one side.

You have user intent, I Want a database, I want an S3 bucket, I want a Kafka cluster, I want a Kinesis topic, right?

And then on the other side you have an API call from, like, from, from aws, from gcp, from, from Microsoft.

And there is a very, very, very long process that involves like writing code, figuring out the right code to author first and then authoring that code, then taking this code through the deployment, like setting up the credentials and all that, and then getting it through all the steps.

And ultimately you're talking to provider APIs, you're talking to the hyperscaler APIs.

So what if we took your intent, we took those APIs, we proxied them to a very well defined schema with a provider, with a Terraform provider.

And we let your LLM be the translator.

So you remove the terraform from the mix entirely.

You let the human assisted by LLM make calls directly to provider.

And then what we do as spacelift intent is first of all, we facilitate this interaction, but we also capture that we serve as a middleware.

So we, we will provide gates before you talk to an API.

And then after you provide, after, after you talk to the provider, what that means is we are able to first add policy as code layer.

So before you talk to the provider, we'll make sure that what you're asking for is something that you allow, that your organization allows, and it generally makes sense.

These are the rules that you can set up yourself.

Your admins can do that.

And once you write to the provider, once you talk to the provider, we will take the state and we'll save it exactly like you'd save Terraform state.

So essentially you have the full Terraform experience without Terraform and without the whole software development life cycle in between.

Now is it the ultimate solution in infrastructure's code?

First of all, there is no code, but no, I think it's just a tool on the spectrum that is very broad and it kind of occupies this sort of sweet spot for when you want to do prototyping and you just don't like, you don't want to be bothered with this entire like dance of learning and writing and deploying and approving, etc.

And there's another really cool use case here is, you know, we sometimes think that Terraform is this, is this always greenfield projects, right?

Like everything is terraform.

And by the way, I thought that spacelift was being used by companies who are already in the green and they want to keep being green.

The reality is that it's almost never the case, almost all terraform projects are brownfield projects.

So a tiny bit is terraformed, a tiny bit is codified.

Could be some other.

Like I'm using terraform as almost like a shortcut.

But the reality is that it's the most widespread technology.

But part of it is codified, part of it isn't.

People have been doing things on the console.

People have had like a one off script.

Someone tried Ansible, someone tried this, someone tried that, someone did Terraform, but they forgot.

And now you have terraformed resources, but you don't have the code or the state.

So kind of like someone did something on their laptop and then left, right.

So the project is a brownfield project.

Now terraforming it, pulling it under terraform management is extremely hard unless you have something like intent.

Because the beauty of intent is that it's, it kind of tricks the whole process because it saves terraform state.

So even though it doesn't speak terraform, because the providers provide like they return state, in return you can take the state and you can create HCL out of that.

So you can use that, your LLM can essentially scan your entire account, see what you're not managing using terraform, pull those resources and generate beautiful terraform for you and generate the state and kind of promote something to a terraform project, which is an amazing experience compared to whatever we had in the past.

Okay, I want to talk at some point in time about non determinism versus determinism and AIG is that.

But I want to hold on to that for a second.

I want to get back to what you're talking about here a little bit.

So you essentially allow AI to go from the idea to a completed configuration change to a cloud provider, and then you snapshot that state and store that for repeatability purposes.

Is that a fair summary?

Correct?

Yeah, it's quite correct.

I wouldn't overestimate the role of AI here.

AI is more like a translator of intent into a particular schema.

So terraform resources have a very strict schema and it doesn't leave you a lot of freedom.

See, an LLM would translate one representation of what is required, which is a natural language.

And you know, the more details you provide, the more strict that translation will be into a schema that is provided by the terraform provider.

Okay, so it turns out that LLMs are very good translators.

Yeah, exactly.

So that actually answers, I guess the core question is are you using AI to modify your infrastructure or are you using AI to essentially create terraform scripts?

And you're using those scripts to actually modify your infrastructure.

Something in between.

Something in between.

So we're not actually like there is no hcl, there's no Terraform in, in between that we don't generate code that we would run through Terraform.

You don't generate the terraform script per.

Se, not terraform script per se, but we generate the input that the terraform script would pass to the provider.

So we short circuit that language translation layer.

Because if you think about it, what Terraform is, is there are two layers of translation here.

The first layer of translation is inside a human to machine process where you use a programming language or it's really more like a markup language.

You use a markup language to express something in a language that a computer or computer instruction.

The HCL code, the Terraform code.

Right.

So there's one layer of translation and then what Terraform does is it translates the intent expressed as HCL language into an API call to the, to the terraform provider.

So there are two parts of it.

And what we're doing is we're saying, look, we don't need that middle layer because if we're doing a translation, you can actually do that translation directly between a user intent, what the user wants and the schema.

Because an LLM is your assistant in this case.

So you don't necessarily need that extra formality that comes with Terraform and you get something for free.

Now in a traditional infrastructure as code environment, that script plays an important role.

And that is the, that is the script is what is checked into revision control compared against past revisions.

It's used for evaluating changes to, you know, you do a pull request to make sure that it's get a change reviewed by your peers to make sure it's reasonable approval processes as well as during an incident you can check to see what changes have occurred in the past and see whether that might have caused the incident, et cetera.

So all of those sorts of things are value that you get out of that script.

Now if we pull that script out, what part of the system is providing that particular set of value?

So the historical data is still there because the state is versioned.

So if you want to understand whether the changes have happened, first of all you have a full history of each resource and each operation in your database inside intent.

For open source that would be a SQLite or a Postgres.

For a commercial version, that is something that SpaceLift would host.

So what you'd get from state history is still there.

So your incident management story is still like it still takes that box.

Now, what is not covered specifically by intent is the ability to replay something from that state.

So what we do in this case is we allow you to export an intent project, so like a collection of resources in like a single logical entity, export it to hcl.

And so you can take that HCL and you can either go through Terraform and essentially replay it as a terraform project, or you can do something even more devious.

You can pass it to an LLM because it's also an expression of intent.

So you kind of export what you, you know, the instructions and the LLM.

It doesn't care whether the instructions are in a human language or in a computer language, as long as it can map it directly to a schema, it's good.

And so it is actually very good at even translating Terraform code into API calls without terraform.

Right.

But in general, like the moment that you need to replay what you did in a production environment, I would say that's the moment where you just eject do the HCL and you know, graduate to a production ready process.

And you're right, you're getting a lot of value from that process.

Question is at which stage you need to, you need to invest that time.

And what intent gives you is the ability to really get started quickly and not have to pay this, you know, ginormous price up front that you need to otherwise pay to get something terraformed.

Right.

You can start very slowly and kind of ease your way into very formal infrastructure management.

This is the middle ground approach, right?

Is where you provide something that's easier or as easy as using the console may be easier, but easier than doing the formal mechanism.

So you can get started a lot quicker.

But the assumption is by the time you, you move into a production, enterprise grade production environment, you probably have moved off of spacelift and into a more.

Formal CICD system of intent of intent.

So yeah, so in spacelift we would provide you that golden path where you say, okay, the intent project is promoted to a spacelift stack and that's it, you know, the code is generated, the state is there, and it now operates as a, as a version or a source controlled version controlled formal infrastructure scope project.

And then you can replay it in a different environment and you'll get the same results, et cetera.

So yeah, at that point, this is it.

So how does intent help with that migration?

If you're under the assumption that you're building a production stack but you want to build it fast, so you use intent to get up and running and the infrastructure up and going easily.

But now you've got to take the next state and move it.

You're going to go live, you're going to build a production environment and you want to put all the production safeguards in place before you actually do that.

What types of tooling or what types of abilities exist for taking that intent managed infrastructure and moving it to a full IAC stack?

That's a good question.

So to clarify what intent is, it's an MCP server spacelift intent.

The external functional surface is that of an MCP server.

So all the functionality is exposed through what's known as MCP tools, just things that API that LLMs can call, like APIs that LLMs can call.

It's not very different from RESTful endpoints in HTTP.

It's almost exactly the same thing.

So we expose those as abilities as tools and we expose two tools specifically.

Because Terraform, when you think about it, it's actually composed of two main artifacts.

There is the declaration code, the HCL code, the TF files, and there is the state.

So if we know like we can generate the HCL for you, so we know what resources you're currently managing.

And because we have the state generated by providers, we just give you the state and we give you the HCL code.

So we give you the two artifacts that is sufficient to start managing that through Terraform like immediately.

So we essentially give you all the components of a Terraform project, both the state and config.

You're good.

Got it.

Great, thank you.

So let's get back to this deterministic standpoint here now.

So one of the advantages and disadvantages of modern AI today, and I'll use that term very generically, it's non deterministic, right?

You, you take a given set of controlled input you supply to an AI multiple times, you're gonna get different results each time, or some combination of some set of results that are different each time.

And the art or the act of provisioning infrastructure, though, you need that to be very deterministic.

I need a server that looks like this and I need it 20 times.

That's a very deterministic set of things to do.

You wanna do the exact same thing 20 times in order to have your 20 servers set up exactly the same way, or you want at least to have a level of predictability in what it does.

And traditional historical AI doesn't do that.

And so how do you deal with that aspect of AI and building and using within the Intent Project forum for creating provisioned resources, it's a very good question.

So we kind of live with it in a sense.

I mean, AI is good at certain things.

It's not very good at certain things.

We're trying to use it for the things that it's good at and not have people use it for the things that it's not very good at.

If you ask the same question of your model 20 times, even if you set temperature to zero, you'll probably get 20 different answers.

There is, however, there are, however, ways of mitigating that.

First of all, even in the absence of other guardrails, the more strict you are with your expectations of an LLM, the fewer degrees of freedom you leave to the LLM to generate an answer, the more deterministic it is.

Empirically, we found out that the terraform schemas, with their extreme verbosity and very good descriptions, are very, very prescriptive.

And so they serve as very powerful guardrails that generally keep the LLM focused.

So that's one thing.

Even before we move to other things that we did, the other thing that we did is we set up policy as code guardrails.

You know, you say that LLMs are non deterministic, and you're completely right.

Well, I'll ask a very provocative question.

Who writes code?

Humans write code.

Are humans deterministic?

No.

No.

And the reason people, like people, realize, leaders realize that people make mistakes, and it's inevitable that they'll make mistakes.

Right?

Mistakes are part of the job.

So what you do is you set up processes, you set up policies, and you codify policies to make sure that someone having a worst day will not blow up your production.

So the thing that you do for a human, why wouldn't you do it for an LLM?

Of course, the LLM will find probably more ways of being stupid than people.

Although, I mean, what people can do, you probably know yourself that there's probably no limit to what people can do, and there's probably no limit to what LLMs can do.

And so you kind of assume that both will make mistakes and you're trying to codify those policies.

So what intent will give you is the tooling to do those policies, first of all to author the policies and then to enforce the policies.

So essentially, you're trying to keep an LLM on track with two guardrails.

One guardrail is very, very strict instructions from the schema.

And, you know, the more strict instructions you give in your intent, your expression of intent, you know, on the intersection of that will be more and more predictable.

And on the Left, the left guardrail is the.

In the the policy as code framework which is completely deterministic, right?

Once you have a rule, there is no interpretation.

You'll go through the interpreter that is fully deterministic.

There is input, there's a rule, there is an output, there is a decision.

Right?

The LLM will do nothing about it.

It'll just learn.

Okay, I'm sorry, but I did something I shouldn't have done.

Here's the explanation.

Human, what do I do now?

Basically what you're saying is AI's make a mistake.

Yes, but so does humans, which we're trying to replace in this case, they make mistakes too.

The net result is we have processes in place in order to make sure that what the humans do is, isn't destructive.

We can do the same thing with AI.

And that's what you do correct is you put this process in place for making the mistakes that are fatal and only the livable mistakes and be able to deal with them.

And ideally if there is a way to make those mistakes cheap.

So like you make those mistakes in a throwaway or pre production environment and then production environment is then fully deterministic that you know, how do I make, how do I let an LLM or a human make mistakes that will not cost me my business?

Well, generally I let them make those mistakes in an air gap or a sandbox environment the same way, you know, I wouldn't let my kids run onto train tracks.

But you know, if a child does something that is generally safe, but you know that he'll learn a lesson, then you might actually want to have them make that mistake so that they learn from their own experience.

And in a sense if we can provide a safe space for LLMs to do that, you know, running like a preview changes and running those policies in front of any API calls, then an LLM gets the same kind of sandbox learning experience that a human otherwise would.

There's no difference between the two.

So this is an interesting way of doing AI based provisioning and I like it.

But the more classic way of involving AI in the infrastructure as code model is more akin to the vibe programming approach.

You don't bypass the terraform step.

You use AI as a tool to assist you in building terraform.

Now my, my guess is you think the approach that intent does is better than that.

And under that assumption, or is at least different, what do you think of that approach and how does it compare to intent?

And why is one better than the other?

I think they both, they're neither is superior.

I think they both have their place.

The way I would describe it is like with terraform, when all you know is terraform, everything looks like a terraform problem.

So when all you have is a hammer, everything looks like a nail.

My understanding of AI assisted terraform generation is that you still have a hammer, you just make it swing faster.

And that's by the way, it's a major win.

But you still need like, okay, you've auto terraform a little bit faster.

So you remove that part of a very long loop.

You haven't removed the rest.

Right.

So you made a hammer faster.

But the rest of a process is still there.

Sometimes you need that process.

I'm not against using AI to generate terraform.

I think it's a very valid approach at a certain level of maturity of a project and maturity of a team.

And I think an ideal situation would have you be able to create infrastructure from intent in that sort of like human in the loop way that we do it with spacelift intent and then allow you to use the same approach in a more formalized fashion with, with Terraform being, and with, with the code review and the terraform being the thing that is being reviewed.

I don't think there is necessarily a conflict or one is better than the other.

We, you know, with spacelift intent we just give you one tool and I think AI assisted terraform generation is another tool.

Both valid, both have their use cases and I think the future belongs to teams choosing the right tool for the job.

Whereas the right tool might be a different tool at various stages of maturity and the project lifecycle intent is a.

Tool along the transition, not the end state is a good way to put it.

I love that.

Yeah.

Thank you.

My guest today has been Marcine Visensky.

Marcine is the co founder of SpaceLift, an AI enabled cloud infrastructure provisioning product.

Marcin, I want to thank you for joining me on Software Architecture Insights.

Thank you for having me.

Thank you for joining us on Software Architecture Insights.

If you found this episode interesting, please tell your friends and colleagues you can listen to Software Architecture Insights on all of the major podcast platforms.

And if you want more from me, take a look at some of my many articles at Software Architecture.

And while you're there, join the 2,000 people who have subscribed to my newsletter so you always get my latest content as soon as it is available.

Thank you for listening to Software Architecture Insights.