Igor Debatur, the CEO and co-founder of Uploadcare, joins us to discuss the complexities and challenges of file management in modern applications. Uploadcare specializes in making the file upload experience seamless for users. Igor shares how their journey started from being an agency that built products for clients to creating a comprehensive solution for file management that addresses the needs of engineers today. We delve into the intricacies of file uploads, which may seem straightforward but are fraught with challenges such as security, compliance, and scalability. Igor emphasizes the growing demand for efficient file handling as applications increasingly require users to upload larger files, including images and videos. He explains how Uploadcare's services help developers avoid the pitfalls of building file management systems from scratch, allowing them to focus on their core business rather than dealing with tedious technical issues. This episode provides valuable insights into why many companies should consider using a managed solution like Uploadcare instead of attempting to build their own file management solutions, especially when they can save time and resources in the process.

Takeaways:

• Uploadcare helps engineers manage file uploads, making the process more efficient and reliable.
• The growing demand for video and image uploads means tools like Uploadcare are essential for developers.
• Igor emphasized that building file management systems from scratch is often not practical for companies.
• Uploadcare offers a modular file uploader that integrates easily with various JavaScript frameworks like React and Angular.
• The service provides real-time image transformations and document conversions to streamline user experiences.
• Using Uploadcare saves companies valuable engineering time, allowing them to focus on their core business needs.

Links referenced in this episode:

• uploadcare.com
• softwarearchitectureinsights.com

Companies mentioned in this episode:

• Upload Care
• Zillow

Please Donate

Please consider donating in order to keep this podcast...and other content by Lee Atchison...arriving in your inbox.

Please Donate Here

Hello and welcome to Software Architecture Insights, your go to resource for empowering software architects and aspiring professionals with the knowledge and tools they require to navigate the complex landscape of modern software design. My guest today is Igor de Igor, the CEO and co founder of Upload Care.

Upload Care helps engineers to collect, store, secure, manage, optimize and deliver images, videos and documents. Igor, thank you so much for joining me on Software Architecture Insights.

Thanks for inviting me.

So your company creates and builds tools to help make the file upload experience, image processing experience, video processing experience, etc. Try and make that a better experience experience for the customer. Is that a good short summary or is there more to it?

It's a good summary, definitely good enough. It took us a while to explain what we actually do. We started as an agency, we built a lot of products, we built a lot of platforms for our customers.

And file uploads, they were tricky all the time. They look easy, they looked easy 10 years ago, they look easy now.

But in overall it combines a lot of things that you need to glue together and then you need to store everything somewhere. You need to make it safe, reliable, scalable, compliant, et cetera, et cetera. Then you need to deliver them, to transform them from format to format.

So we help companies to build new products where people need to upload something and then to deliver it at large scale.

It seems like nowadays almost all apps have at least an avatar or something, but most also have other image upload requirements. Videos becoming very popular, of course, video, et cetera.

So I imagine that the need for a product like yours has grown a lot or grown significantly over the last several years.

Yeah, the need is growing, so more products are being built and now more people can build some products too. And traffic volumes, they increase all the time because new devices, they record images, they record videos and shoot photos in higher resolutions.

So demand is growing, great, but it's not becoming easier. And another thing is that previously many people tried to build everything from scratch. It started to change a while ago.

Now most people, they're looking for some ready made solutions instead of building something from scratch.

I imagine cloud computing has helped make that a lot more consistent and so made the use of tools like you much more prevalent and much more acceptable, maybe is the better term.

It's kind of became a default to try to find some solution before building it from scratch. And we like to joke that our not to joke. I think it's a fact that our best customer is a CTO who tried to build file management before.

They don't Want to build it anymore?

You were an agency, you were doing this all for your customers. That got tedious very quickly. So you built some standard tooling and you kind of grew from that.

Do you want to talk a little bit about that experience?

Yeah, it was a fun story. So we thought that someone might have solved it better than ourselves.

So we went to Hacker News and just asked if there is a better solution or if we should build something for everyone to solve file uploads. And there wasn't any better solution, so we just built it ourselves.

Are you bringing in people who have a serious problem they need to fix, or are you bringing in people who, hey, this is something new I can add to my app that gives me additional value?

No, I wouldn't say it's something that they want to add, just to add it. It's file management. Usually it's boring and it's required. So they have their product managers requesting some advanced feature.

For example, let's let our users to upload 50 gigabyte files. It's complicated enough to try to find some solution to do this because it's not just a file upload field, it won't work.

And then you have to make sure that the link to the file is secured so no one who shouldn't have the access, they won't be able to download it.

And then if you allow people to upload 50 gigs files, what if someone will upload something malicious or some dark kind of content that you don't want.

To distribute, hurt your application, not just the customer. Right.

And then someone uploads something explicit and then you have to deal with legal in your country.

What I wanted to do is go into a little bit more detail of specifically, you know, why is file upload hard? Why is it a problem?

I would say that it's the problem that looks very easy when you look at the tip of the iceberg. But when you start building it, you start remembering about edge cases. And there are a lot of edge cases.

If you want to allow file uploads up to one megabyte per file and just one file each time, it's easy. Then if you want to allow larger files, you have to make sure that you store them somewhere. Okay, S3 is an easy option.

You can just put everything to S3, but then you need to deliver this file. So everyone uploads images up to 10 megabytes each. And then you deliver them at scale.

You start delivering a terrific amount of data and then in non optimized quality. So you are getting Problems with your Lighthouse score. Then you have to optimize all the images.

You can find some library that will optimize them, but then you have to save all the derivatives. Then you have to save all the versions of these images and you know that you have to generate for example, 10 versions.

But what if your designers will change the website layout? You have to regenerate everything and you already have multitude of files.

And it's just about simplest use case file uploading for just images and delivery, but then you have to deliver them in optimized formats. What about Avif? Avif is tricky.

We spent more than three months just to find proper configuration to optimize awif and we had all the experience in the world to work on this. I would say not every developer can do the same, so even if they implement it, it won't be as good. What if someone.

What if you need to secure file uploads? So you need to generate some signatures for every upload. It's another tricky thing. And then some people will ask you, are you GDPR compliant?

It's another pain. So plenty of things to think about.

So in a nutshell, there's file size issues, the scaling issues, there's image processing and there's. I'll just lump together version management as a kind of a general category, but that's. That encompasses lots of things.

Probably could even put GDPR into version management. Those are kind of the core reasons why this is a hard problem. So given that, what do you do to make those easier?

We created and we call it Image cdn. So people can generate any kind of derivatives from any image in real time without waiting in milliseconds.

And it's very useful because you can change design templates and you can get any resized images right now without waiting, without regenerating them. You store only sources and then you generate every version on the fly. It's the first thing that we have built.

Then we thought that people deliver not just images, they deliver documents. And we created document transformations so people can request to convert Docx to PDF, for example, and to generate a thumbnail.

It's very convenient and it's something that you don't want to build, you just want to have it, just in case. Then we started, then we introduced video encoding.

So now people work with different kinds of media content and they also want to allow people to upload videos. You have to deliver videos, you have to stream them, you have to encode them to different qualities.

We also do this on the fly with our newest file Video encoding. And we also decided that if you have to show videos, you want to get some ready made video player. That's something that we also started doing.

We provide a video player that can stream all the videos uploaded to upload care. So I think the DNA is still the same.

We were an agency, we built a lot of different products, marketplaces, e commerce websites, website builders, plenty of them.

And we wanted to build a solution that helps to build similar products just to make developers lives easier and to help them to focus on their main business, not at something that should just work.

So so far I think all of the things you've mentioned have been part of the download process.

You've got files that you're storing for the customer and you're providing versions of them available for the application to consume for their customers. So you've mentioned image transformations, document transformations, video encoding, those sorts of things.

But you also help the, the upload process itself, that itself can be problematic in many respects. What, what do you do specifically to help developers build the upload process?

We built a very simple yet advanced product. It's a JavaScript file uploader that can work any library that you use. It can work with React, Next, whatever, Vue, Angular.

It's easy to integrate, it's modular. So if you need a drag and drop field you can just get it.

If you want to have a file uploader that is an overlay with a drag and drop field, Dropbox connected, Facebook connected in a native ux, it's also easy to get, it can be configured, it has a built in image editor. So basically it's a modular file uploader that works.

And unlike some open source solutions you don't have to think about it because no product fits all the use cases.

And if you use some open source you have to make some modifications and then you have to maintain these modifications every time they update the code, you have to maintain all the changes. With upload keyer, you don't have to do that.

So basically you provided a JavaScript tool to help with the upload process, then backend processing in order to do all of the different transformations and management necessary to get all the versions that customers may want of their image or their video or their document etc. Able to be downloaded.

So you're doing all aspects of the file management including by the way, the storage and the, the, the coordination and the global distribution of that, the typical CDN sort of actions, you're managing all of that. One thought with an application like yours is the files you're transferring around can be quite large.

Is there a downside to not having those images closer to the application itself? And how do you deal with that?

Some customers, they want to process files by themselves and we started supporting custom S3 storages so they can use their own S3 buckets and put files there.

So in this case we can also process them, but files that are kind of closer to the customer and then we definitely allow them to copy files to their bucket, it's a different feature. So if they want they can still use Applotge's own storage, but they can also copy files to their storage.

Okay.

So if they need to access them for some period of time via an API command that you'll copy them to their own buckets, they can do whatever they want to there and you can copy them back.

And that's right? Yes.

Yeah. And as far as efficiency of how that happens, are you using S3 in the back end or are you using your own service? You are.

So they're a long term AWS partner.

Got it, got it.

So things like export content costs and things like that tend not to be an issue when using you because you're not moving content off of S3, you're moving it from one managed bucket to another managed bucket and back and forth. So let's talk about monitoring and analytics. What type of analytics do you provide to your customers?

Yes, it's very important for our customers to understand what's going on and if there are any traffic spikes and what could have been the reason. So we provide traffic volume, number of requests, most frequently served files. It usually helps to understand if there was a traffic spike.

It's usually some large file was downloaded millions of times, could have been some DDoS attack or something, some mistake on their end. Then we showed them recommendations for reducing traffic and improving load times and overall user experience.

Talk to me about how you save the customer's ongoing operational and management costs by them choosing to use a managed solution like you versus rolling their own system.

So in overall the most the largest savings that they get when they're in the beginning of their businesses or if they're mid sized businesses, it's engineering costs because engineering time it costs a lot. And while traffic is getting just cheaper and operations are also doesn't cost much, engineering times, engineers are costly everywhere.

So with AppletCare and they just save months of time and tens of hours of time of engineers to support a solution, they just don't have to pay this money. They are getting something that works for Larger customers. We provide multiple discounts for traffic, for storage, for operations.

So we are a large customer of CDNs. We don't have our own barebones servers, we work with hiddens like Fastly, Akamai, Cloudfront and we buy everything in bulk.

So we definitely can provide better prices even for smaller customers, prices that they just cannot get themselves just because we are a large buyer of traffic.

I'd like to talk a little bit more detail about scaling, but I want to talk about scaling in two different dimensions. The first is as an individual customer of upload care and my needs for scaling change as time goes on. How does upload care help with that scaling?

And the second one is I want to talk about how you as a company are scaling based on number of customers, number of demands, et cetera, and how that increases over time and how you're able to scale that. So I want to keep those two separate. Let's talk about the first one first.

How do you help customers provide a scalable solution and allow them to scale their applications using upload care?

From customers perspective, the answer is really simple and concise. The beauty of upload care that they don't have to think about scaling at all. So everything scales automatically.

They can throw any amount of files, hundreds of thousands of files per day, and we will just handle them.

So from the customer standpoint, your goal is to make it so they don't even have to worry about scalability. And in fact that makes a lot of sense because their normal application scalability they deal with normally.

But they don't have to think about the file management aspects because you're doing all of the storage management, all the upload and all the transformation on your Systems as a SaaS service. So nothing has to happen locally, nothing has to happen on the servers associated with the application. All of that is done by you.

So as long as you're scaling, fine. The customer can use you as much as they need to, as much as their customers are demanding, and it just magically works.

The true value of SaaS is the ability to do that.

Exactly.

Great. So that's great. But then let's talk about the second side of that, which is how are you scaling? And that is.

So as your customers individually grow, plus you're growing a number of customers, you have some tremendous scaling challenges. How do you manage the scaling of your application and your customers data within your system?

So we definitely, as AWS partners, we rely on, AWS has to provide the most cutting edge features they have. We use all of them and then we are horizontally infinitely scalable.

So when we see an increase in the load, we just launch more and more instances and then we can downsize. So it's kind of living organisms that can increase in size and then downsize depending on the season.

Because November and December, for example, they're highly loaded seasons. Because people start to invest in marketing E commerce segments that are blooming in the end of the year.

And we don't have to keep an eye on the servers just because we know that everything is already configured in the way that platform will scale itself.

So basically you rely on the infrastructure as a service offerings of your cloud provider to do that. There's been kind of a push recently by a lot of customers to say that cloud services are too expensive and they're great for what they do.

But there's been a pullback in using of cloud services recently by customers who just want to save money or can't afford the scaling costs associated with building applications in the cloud. They're building your own infrastructure back in their own data centers again.

And that's coming more back into reality than actually even people like me thought would happen. Does that affect you? And if so, if it doesn't, why not?

It might affect us in a way that we just cannot see. Because customers of that scale, they are just not considering platforms like ourselves.

I think they're too large and I think it's fair just to recognize this and to confirm that yeah, there are customers who are too large for upload care, or upload care is too costly for them, so they're just not a good fit.

So what's your ideal target customer?

I would say both SMBs and enterprises. If you're speaking about, I don't know, streaming all the CNN videos. No, it's not something that we cannot do.

But even for enterprise size customers, we are maybe good fit for some of their products and a worse fit for some other products. We have large companies using upload care. Zillow for example, is our customer.

So there might have been a chance that you browsed some images that were served by apploadcare without knowing it or some E signing platforms, they use upload care in their core and it's hard to find out for a user that upload care is there. So yeah, SMBs, enterprises, even indie developers, many of them, they use upload care just to launch faster.

They are ready to pay some extra just to be able to save time. I think the hardest question for us to address is the build versus buy.

It took us a while to start explaining like why it doesn't make sense to try to build file management by yourself. And as I mentioned, the best example of Upload Care's customer is a CTO who already tried it.

So yeah, I would say that time is the essence and if you can use some ready made solution to solve something that is boring, just do that. Spend time building something beautiful that you want to build.

One of the things I always tell people is be an expert in what you want to be an expert in and don't be an expert in everything else. And that's kind of the core here is do you want to be an expert in file upload or do you want to do your business logic?

And in your case, your product is designed to help those people, which are most people who don't care whether they're an expert in file upload or not. They might have to be without you, but with you they can avoid that expertise altogether.

Yeah, well said.

My guest today is Igor Debatur. Igor is the CEO and co founder, Upload Care. Igor, thank you so much for being on Software Architecture Insights.

Thank you and have a good one.

Thank you for joining us on Software Architecture Insights.

If you found this episode interesting, please tell your friends and colleagues you can listen to Software Architecture Insights on all of the major podcast platforms.

And if you want more from me, take a look at some of my many articles@softwarearchitectureinsights.com and while you're there, join the 2,000 people who have subscribed to my newsletter so you always get my latest content as soon as it is available. Thank you for listening to Software Architecture Insights.