Industrial Talk is onsite at OMG, Q1 Meeting and talking to Justin Wilson and Erik Hendriks about "Data Distribution Service (DDS) the middleware that enables the free flow of data".
Scott MacKenzie hosts a podcast celebrating industrial professionals and their innovations. At the OMG SQ1 meeting in Reston, Virginia, he interviews Erik Hendriks and Justin Wilson about DDS (Data Distribution System). Erik, a senior architect at ZettaScale, has been involved with DDS for 20 years. Justin, a principal software engineer at Unity Foundation, explains the open-source implementation of DDS. DDS enables efficient data transfer without the need for applications to know the transport details. It supports a publish-subscribe model, ensuring data reliability and scalability. The conversation highlights DDS's role in modern data-intensive applications and its future focus on scalability, big data, and security.
Action Items
[ ] Explore the open DDS foundation website (open-dds.org) to learn more about the technology.
[ ] Connect with Erik Hendriks on LinkedIn to discuss DDS and ZettaScale's other products.
Outline
Introduction and Purpose of the Podcast
Scott MacKenzie thanks listeners for their support and celebrates industrial professionals for their bold and innovative work.
The podcast is broadcasting from the OMG Q1 meeting in Reston, Virginia, focusing on solving today's challenges.
Scott MacKenzie introduces Erik Hendriks and Justin Wilson, who will discuss DDS (Data Distribution System).
Background of Eric and Justin
Erik Hendriks has been involved with DDS for 20 years and joined OMG during the standardization of DDS.
Justin Wilson is attending his first OMG meeting and has a background in IoT applications involving DDS.
Erik Hendricks is a senior architect at ZettaScale, fully involved in DDS for the last 20 years.
Justin Wilson is a principal software engineer at Unity Foundation and Open DDS Foundation, previously involved with Object Computing.
Explanation of DDS (Data Distribution System)
DDS stands for Data Distribution System, which helps get data from its source to its destination efficiently.
DDS decouples applications to such an extent that they don't need to know about each other's existence.
DDS is unique compared to other middlewares due to its ability to efficiently and reliably distribute data.
Justin Wilson explains the open-source implementation of DDS and its benefits, including efficiency and productivity.
How DDS Works and Its Benefits
DDS uses a data-centric approach, allowing applications to specify what data they need without knowing the source.
DDS middleware handles the discovery and delivery of data, making applications simpler and more efficient.
DDS can handle both unicast and multicast data delivery, optimizing data transmission based on the number of subscribers.
DDS contracts specify the quality and reliability of data, ensuring consistent and efficient data flow.
Real-World Applications and Implementation
DDS is used in various industries, including manufacturing, where it simplifies data collection and distribution.
DDS can be integrated into existing systems, reducing the need for custom solutions and ongoing maintenance.
DDS offers flexibility in data collection, allowing for easy integration of new data sources and applications.
DDS is often used in standards for specific industries, such as automotive and transportation, providing interoperability and flexibility.
Future of DDS and Security Considerations
Scalability and big data are key focus areas for the future of DDS, ensuring systems can handle growing data volumes.
DDS is evolving to support higher-level programming languages like Python, making it more accessible to developers.
Security is a critical aspect of DDS, ensuring data is only accessible to authorized users and not intercepted during transmission.
DDS is growing to meet the demands of modern data-intensive applications, providing robust and secure data distribution solutions.
Conclusion and Contact Information
Scott MacKenzie thanks Erik and Justin for the insightful conversation and highlights the importance of DDS in modern data collection.
Justin Wilson provides information on how to contact the Open DDS Foundation through their website.
Erik Hendricks can be found on LinkedIn under Erik Hendricks (without CK) at ZettaScale.
Scott MacKenzie encourages listeners to engage with OMG and connect with industry professionals like Erik and Justin.
If interested in being on the Industrial Talk show, simply contact us and let's have a quick conversation.
Finally, get your exclusive free access to the Industrial Academy and a series on “Marketing Process Course” for Greater Success in 2024. All links designed for keeping you current in this rapidly changing Industrial Market. Learn! Grow! Enjoy!
Industrial Academy (One Month Free Access And One Free License For Future Industrial Leader):
Business Beatitude the Book
Do you desire a more joy-filled, deeply-enduring sense of accomplishment and success? Live your business the way you want to live with the BUSINESS BEATITUDES...The Bridge connecting sacrifice to success. YOU NEED THE BUSINESS BEATITUDES!
TAP INTO YOUR INDUSTRIAL SOUL, RESERVE YOUR COPY NOW! BE BOLD. BE BRAVE. DARE GREATLY AND CHANGE THE WORLD. GET THE BUSINESS BEATITUDES!
Industrial Talk, OMG meeting, DDS system, data distribution, data centric, publish-subscribe, open source, data reliability, scalability issues, big data, Python bindings, security concerns, manufacturing data, interoperability, future of DDS
00:00
Scott. Welcome to the Industrial Talk Podcast with Scott. MacKenzie. Scott is a passionate industry professional dedicated to transferring cutting edge industry focused innovations and trends while highlighting the men and women who keep the world moving. So put on your hard hat, grab your work boots, and let's go all
00:21
right, once again, thank you very much for joining Industrial Talk, and thank you for your continued support of a platform that is dedicated to industrial professionals all around the world. You are bold, you are brave, you dare greatly, you innovate, you collaborate, you solving problems each and every day. That's why we celebrate you on this platform Industrial Talk. Thank you once again for your continued support. We are broadcasting at OMG S q1 meeting here in Reston, Virginia, and it is again, a collection of incredible professionals focused on solving today's challenges and making the world a better place. That's why we are here, and we are interviewing some of the best leaders in the industry, and in the hot seat, speaking of in leaders, we've got two Erik, Erik Hendriks and Justin Wilson. We're gonna be talking about DDS. And you're saying to yourself, Scott, I don't know what that is. Well, you will stay tuned. It's coming your way. Hey guys, how you
01:22
doing? Thank you for having us. Pleasure
01:24
is mine. Thank you for your flexibility again, because we were running late on the last conversation, and you know you guys were very kind and flexible. Appreciate that the conference, how long you guys have been a part of, OMG, or the and I know they got concerned. It just, it's easier for me to just say, OMG, how long Justin,
01:45
this is actually my first year Shut the front door. Is attending an OMG meeting before that, one my colleagues attended for, you know, a number of years, and has been involved, I don't know how long, but he's kind of transitioning out, and I'm transitioning in. So this is kind of my first, first year getting my feet wet. What
02:03
about you, Erik,
02:05
you may remember yesterday that we did the cake cutting for 20 years of DDS, I actually joined, OMG, right then, when we were standardizing DDS, and that's why we joined. OMG,
02:16
yeah, well, you tell it Justin to say, Don't do that. Do this. Go over there. Don't go over there. Well,
02:21
I don't need to. He's, he's a good professional. I can see his experience and in the matter. So before we
02:29
get going, let's, let's talk a little bit about background. Erik, give us a little background on who you are at outside of the fact that you've been part of this for 20 years. That's a long time.
02:38
Yeah, I've, I'm a senior architect at ZetaScale. I'm fully involved in DDS. Been working on DDS, really, for the last 20 years. What
02:53
does DDS? The acronym stand for DDS, stands for
02:57
data distribution system. Good. Got it, and to try and give a little bit context to it, data distribution services about how to get data from its source to its destination. And you there's many ways to do that, but you want to do it in a way where the application that needs the data or that is providing the data doesn't need to know about anything involved in the transport. You try to decouple applications to such an extent that they don't even know of each other's existence. And so it's sort of a magical box I put something in, and everyone that needs it will get it. And I'm not, don't need to be concerned at how it's done, but I can be sure that if it's done, it's done in the most efficient way, and that makes DDS unique when you compare it to other middlewares.
04:08
We're going to riff on that Justin a little background on who you are.
04:12
I'm with currently unity foundation and open DDS foundation jointly. I'm a principal software engineer there. Before that, I was at a company called Object computing, which was a member of the OMG before. And while I was there, I was heavily involved in IoT applications that involved DDS.
04:32
See, this is interesting, because in this today's world, everybody wants to be connected. Everybody wants that data. Everybody wants that data efficiently and accurately, and everybody wants to be able to take that historical data and clean it up and scrub it so that we can create a picture of where the business and then run simulations and doing all the stuff that is in its and it's happening now. But see, many people don't realize and recognize. Of how challenging, the the ability to be able to get the data from one place to another into that area, so that that whole, that whole process, I remember when I long ago, long ago, when we were installing systems, back years, years, years ago, it was always, what data do you want? Okay, we want that data. Well, that data resides over here. How do we get that data over there? That conversation happened all the time. How do we get that going? Justin for that sake. Just just help us understand a little bit about the foundation, what it's all about, and then, and how you're facilitating that, that effort, right? So,
05:46
open DDS foundation is for the open source an open source implementation of DDS. There are proprietary vendors our, our particular implementation is open source, which means you can download it and use it for free. The the, there's another foundation involved here, called DDS Foundation, which is really the consortium of all the vendors. And its main purpose is to just advocate DDS, educate people about what it is, so that people start using it, start benefiting from it, benefiting from, you know, the decoupling that it gives you so that you can write applications more efficiently and you can just be more productive.
06:29
How does that happen? Erik, with that said, how does it make it simpler, easier, stronger, faster, whatever you want to say, how? Let's say I'm a manufacturer, and I'll just I'm a guy who owns this manufacturing process, and I'm interested in getting the data. Somebody says, I need to be connected. I have to have these devices out there. I got to do all of this stuff. Tell me how, how this helps me.
06:53
So what is important when you talk about DDS is that it's very different from how a lot of people are educated. Most people are educated in the client server model. So you're a client, and everything starts with, you need to go to the server. And you already started with, I need to get my data. So that means you need to know where to get it from. So DDS sort of flips it around. DDS says you shouldn't need to know all this. You should specify what you're interested in, and we'll bring it to you. So it's what we call the data centric approach, where, basically, if you if you look at the history of any distributed system, systems evolve over time, and so applications that we were running 20 years ago don't look at all like what applications look right now. But if you look at the core of what a system does? I have a background in naval combat systems, and you look at, let's say, 100 years ago, and you look at the current combat systems. What is the essence of a combat system? Well, there's these plots that you're following, and they have coordinates, and they have a speed, and there's a sensor that detects the plot, and there is a processor that needs to do calculation where it's going. And if you look at a system like that and you say, what's the information flow that goes through the system that is actually very stable. It's very a very domain specific information set that hardly changes over time, even though applications from now can do very different things than applications in the past, the information that's being exchanged is actually very stable. So why not go away from the focus on the application and go to the data that is being exchanged, and then from the application perspective, I can just say, you know, I need a plot. I don't care who is giving me the information about this plot. It could be a radar, it could be a camera, it could be anything, but I need coordinates and speed, like it needs to be XYZ or it needs to be another coordinate system, but I need to specify what I want, and I need to specify what the quality should be, what publishing frequency should I expect, what and so I sort of create this contract like, this is my need. This is what I want. And somebody that is offering information like this will have a contract that looks very similarly like this is what I'm offering. This is my quality. I'm publishing with this frequency. I'm reliable. If something gets dropped during transmission, I'll try to retransmit. And so now the middleware is the middleman. It discovers all these contracts, it compares the contracts and it determines. Hey, I see that you're interested in some data that this guy is offering. Okay, you guys have a match. I'm going to make sure that what you publish is pushed to that destination. And if I'm a censor and there's like, one note that's interested, I can unicast it. If there's more than one node, I can try to multicast. I can try to find a more efficient transport that I don't need to repetitively put the same data on the bus or on the wire. So DDS solves a lot of things that typically applications try to solve. But in this case, the applications are just very simple. I have something to offer the system. So I have a call to write it, that's it, and somebody else needs information, he has a call to read it and to be notified when it's there. But that's the essence of DDS that applications can be very simple, and all the stuff that you typically would solve in applications, you solve by specifying what you want to do in a contract, and DDS then takes care of it for you.
11:07
. I don't care. All I want is:
11:46
picture. So we use the term publish and subscribe. You've got applications that are publishing data. They they're in control of it, they own it, and they're offering it to whoever. It doesn't matter, that's right, okay? And the subscriber, on the other hand, is expressing an interest be right? And part of that is this contract that he's talking about as to, you know, do you want it to be reliable? Do you want to see data that's from the past, all of these things now, the publisher and the subscriber kind of have to agree on that contract, but in principle, that's the essence of it, is that you can stand up what we call a topic, which is this, this collection of data, you as an application, can subscribe to that topic, I, as a publisher, can publish that topic, and you'll receive that data. Now, another application comes along and wants the same data. So what do you do? You don't have to rewrite me. You don't have to affect the publisher. You just have to write a new subscriber that accesses the data the same way, and it's the DDS middlewares responsibility to make sure that both subscribers, or all subscribers, are satisfied.
12:47
Because I want the data. I want data from here, there, over there. I want data various systems. I'm not even going to throw a spreadsheet in. I don't want a spreadsheet, but I want data from all over the place brought into exactly to paint that
13:05
picture. So, so compare it to a relational database. Yes, right? You know what type of information you you want, so you need to know the table in which that information is at. That is given, if you say, I want a quarterly numbers for this financial thing that in a relational database would be a certain table. So that's the knowledge you need. But what you don't need is, where is this database residing? Like? Is it in one location? Is it multiple locations, and it just happens. It just happens. So the only thing that you need is the source of information, which would be in a relational database, would be the table. And in DDS terms, you would call it the topic, and you could give in your contract, like, what quality you want this information with, if it's information that is updated with a very high frequency, you might say, well, if a sample is lost during transmission, and I would need to wait until it's retransmitted, I'm actually holding up, like Everything that would come afterward. I've sort of have a temporary stop, and then I have to catch up again. Maybe it's just better to say, you know, let that sample be the next one is already coming up. Just so that would be one of the things I could specify in my contract. Don't bother retransmitting samples lost during transmission, because the next one is already around the corner. On the other way around, if it's low frequency data, I could say, No, I really need every update, so it has to be reliable, and so both the publisher and the subscriber would have this in their contract. And so if I say, I have to go. At my data reliably. I want to make sure I'm not missing any sample, but a publisher is saying, Well, you know, I'm not, I'm not going to retransmit. Then we have incompatible contracts. We're talking about the same data, but the way in which I want to have guarantees about my data is not compatible with that publisher. And so what DDS does in that case is say, Well, you have a mismatch in contracts. So I'm not going to make you communicate, because I would potentially violate somebody's contract, but I am going to notify both of you, hey, you have a interest that is compatible, but the quality is not compatible. So I notify both of you that there is there's this incompatibility, and then the application can determine what to do. In that case, they could re evaluate their contract and say, well, maybe I would change my contract to comply with what is available, or I just wait for somebody else to offer the same data, but with a contract that is compatible to mine.
15:59
How do I set this up. Let's say I'm a company. I've got multiple systems, because I've been in business forever, and so I definitely have multiple sources of data, applications, whatever you how. How do you approach a company to say, hey, there's a better way of getting this data that maybe you need to be aware of. How do you set that up? Justin, so
16:24
the the way that I've kind of seen it in the past is that a lot of companies evolve their own homegrown publish, subscribe system, yep, and when you when you come in, you can approach them and saying a lot of that effort that you exerted that had nothing to do with your application. It was all about, well, how do I move this data from here to here? How do I retransmit we say we can take care of all of that for you. We can eliminate that effort, that and that ongoing maintenance effort, and you guys can just focus on your applications. So if they're already set up for publish, subscribe, it's usually a pretty light lift to say you take. You've got applications that are publishing. Use this API from DDS, you can start publishing your data same way on the receiving side, that's usually a relatively light white lift for other applications. Then it largely depends just on how they're structured as to if DDS is going to be the right solution for them or not. But it's largely thinking about, how do I decouple How do I think in terms of just data that I can publish for the benefit of my organization, and then data that's there for me to consume for the benefit of whatever I need to do. It's interesting. It's a change in mindset. It is
17:47
I'm really, I'm not. I can't say that I'm struggling. I'm just, I know that I have from a historical perspective, I know that it's been data. Was everything, and especially today, right this market, this, this where we are today. Everybody wants to get data, because we have solutions and technology in such a way that can truly benefit from the accurate collection of data, absolutely. And we can talk about frequency, we can talk about all of that, but it has to data set right, and, and there is the necessity to be flexible and nimble enough, because maybe somebody else wants more data over here, or whatever, something new, something attached, something and. And
18:33
one of the things I'll say is that you may be using DDS, and you might not even realize it, yeah, because it's usually layered in something else. There's automotive standards, there's transportation standards that are all built on top of DDS that's using it as that interoperable way of communicating, but then they then extend it or clarify it, or restrict it in some way to be specific to the application that they want to solve, controlling trains or having an automobile interact with a transportation system or something like that. So you may already be there, but it provides this great flexibility where I can now tap into this piece of information. I can tap into that piece of information. It's already there and ready for me to go. I don't have to contact the vendor of that piece of equipment or something to have them make a special modification for me, it's just there. You just have to go get it. I just have to write my application and find it.
19:28
Erik, I always ask the question of the future, where do you see DDS going? Because everybody wants data. Everybody, I mean, I just what's sort of that next thing within the world of PBS,
19:43
I think scalability is a big issue. What you see is systems growing in size over time. There's more sources where data needs to come from. There's. Are bigger volumes of data, and so often what you see is that there are scalability issues, that if you grow the number of nodes that participate in a system, then the performance just goes down and down and down and down. So one of the things that we're really trying to address is in DDS, is to make sure that whatever approach you take, it should be scalable. And another thing that I think is more and more coming to us is like big data. And so what you see already in people that are processing big data is that they're they're no longer the What will you call it, the programmers that know every bit every they're writing in much higher level languages like Python and and so they would benefit from higher level languages and higher level constructs. And I think for something like DDS, we would, we would try to also cater to them, for example, by offering Python language bindings and and then the third part that I think is really important
21:31
is security. So
21:35
some information is publicly available. Everyone should be able to access it, but some other information should definitely not be available to everyone. And it's not only that I don't want to give information to some people that are not authorized to get it, but I also want to make sure that nobody's snooping on the wire to see what I'm doing. And that's also something that DDS is growing in.
22:01
You guys are great. That was a great conversation. I really enjoyed that one. I got it anything to make data collection easier and painless and frictionless is I'm all in big thumbs up with what you guys are doing. Justin, how do people get a hold of you?
22:18
Best way is open eds.org is where the open EDS foundation. I'm a Luddite man. I work with technology, but I'm a Luddite. Wow.
22:33
We should just clip your ear. Oh my gosh. How about you, Erik.
22:39
You can find me on LinkedIn if you look Erik Hendriks, but it's important, it's not with CK. Erik is just a K. Hendrix is just Ks, yes, Erik Hendriks and ZetaScale. And you should find my areas LinkedIn, and there's some information about our company and our other products that we also have.
22:56
You guys are wonderful. Sorry. I'm going to be posting our pictures, but you're not going to see them, because, no, you're not on LinkedIn. All right, thank you guys very much. All right, we're gonna have all the contact information for these two gents out on Industrial Talk. Thank you very much. Connect with them. You will not be disappointed, because they know what they're talking you need Sherpas like these two gents, big time. All right, we're broadcasting from OMG, go out to omg.org. Get engaged. Be a part of the ever expanding ecosystem of problem solvers that exist at OMG. All right, stay tuned. We will be right back.
23:32
You're listening to the Industrial Talk Podcast Network.
23:42
How about that for a conversation, there is so much happening in the world of industry, so many things that we just don't know taking place. It's, it's an exciting time. Gosh, it's just have all the respect that I could possibly have for individuals like Justin and Erik that give up their time to do to create the standards necessary what DDS, data distribution services, we just take it for granted that data goes from one system to the next, and it just happens and but we don't see what's in the middle, and that's what they do. That's what everybody does at OMG, they just give up their time and effort and their treasure, and it's, it's an amazing organization you need to go out to omg.org, and see how you can get engaged. Because you'll, you'll come with contact with some really incredible people like Erik and Justin. All right, you have a podcast. Put it out on Industrial Talk. You want to be on a podcast. Put it out on Industrial Talk. We're gonna have another great conversation shortly. So stay tuned. We will be rebel.
