Using Artificial Intelligence To Combat The COVID-19 Pandemic With Mike Blakeman And Trevor Chandler
18th June 2020 • Business Leaders Podcast • Bob Roark
00:00:00
00:41:08
Shownotes
The COVID-19 pandemic has forced everyone to take a good, long look at how things were being run once upon a time and see how things can change for the ultimate benefit of public health. One of these fields is the technology sector, where every day, companies find new uses for existing technologies to help the world adapt to the crisis. Mike Blakeman and Trevor Chandler are, respectively, the CEO and CTO of Visual Globe. They sit down with Bob Roark to discuss how artificial intelligence can be used to combat the spread of the COVID-19 pandemic. The time to act about the crisis is now, and with these technologies on hand, we might just be able to make the full recovery we need to.
What we're going to go through here is there's a benefit from this company, first in the utility space pole and equipment and monitoring and creating intelligence from the data extraction. We were issued the COVID pandemic. Data extraction and looking at heat signatures and so on map nicely over to the software and the potential needs that are coming from Visual Globe as it relates to distancing, face masks, and temperature reading. Trevor, please proceed.
Bob, we were extracting insights from pixel-based media images of power poles, transformers, and creating all kinds of important insights for these companies. When the pandemic hit, we went to the core of our technology and started thinking, “What can we do to help people get back to their jobs? What can we do to help keep people safe? What can we contribute to the humanitarian side, have some value here, and see what we can do to help out?” Our technology did transfer quite well for us to determine if a transformer on a power pole is within a particular heat or to determine if a person has a fever, for example.
Those are very similar things. We are already doing the location-based type of insights so we could track locations of where things are happening and whatnot. We're very good at grabbing information without violating the privacy, but still being able to use that information for helpful things. For the pandemic, it's all about making sure people are social distancing. It's about seeing who has symptoms and not just when they walk into a building to check their fever but to check their fever in different areas across different locations to understand what areas they work in. If somebody starts exhibiting symptoms, we can detect it before too many people get exposed and we can even tell what areas the person was in. Without going into any privacy issues, we can alert in certain areas to let them know that there may have been exposed.
[bctt tweet="As for the pandemic, it's all about making sure people continue social distancing." username=""]
We have a whole set of these types of useful features. The core of it is doing computer vision, artificial intelligence with RGB cameras, regular camera images to detect people where they are. Thermal sensors to detect their temperatures. We also can take data from any data source like text-based logging from USB or remote infrared thermometers. We can listen to coughing, sneezing. We can do a wide variety of these types of insights stored all on a map and we can create triggers. If this person was within this distance of another person, then send an email, beep a speaker, send a text, play an audio file, etc. I'd say the main features are thermal detection for temperature. We do facial recognition to detect the face. On multiple points, we go ahead and grab the temperatures and then average across all of the points to make sure that we're very accurate.
Don't forget to mention the mass detection.
That's one of the core ones as well. Face covering, mask, that type of thing so we can discover a face. We're very good at detecting eyewear, mouth, facial expressions. In fact, we do full-blown emotion detection for intent recognition and things of that nature. For this particular situation with the COVID, thermal then sounds, tracking where people go without identifying particular people and breaking privacy laws or concerns. Face coverings, we can even track eye coverings like eyewear. The COVID is getting spread largely through the nose. It's put in people's nose and the second cause is rubbing or particles of the air getting into the eyes. We see eyecare and masks both as being important.
I was thinking about its application. I'm the business owner and I'm listening, going like, “I've got my normal cameras for security in my building. I don't know if I have a thermal scanner.” I would imagine if they don't know, they don't. For the business owner that's tuning in and go, “I'm going to try to go back to take care of my employees as they return to work.” Are you in the camera business? Are you in the data business?
We are the data business and we can utilize any camera they have like RGB, regular cameras. If they don't have thermal, it's best if they get it. If they don't get it, we can use other forums like remote IR thermometers. Maybe they post their security guard and they do remote thermometer readings or maybe you give each employee a USB data logging temperature stick. We configure all those devices. We gather all that information.
We want to focus on the data side of things. We do not want to be in the hardware. We do not want to be in the installing. We want to be agnostic towards the types of sensors, RGB cameras, security cameras, and thermo sensors. We want to be completely independent of those.
When I talked to my father-in-law and he talks about background noise and his hearing aids. He goes, “If you get in a loud restaurant or somewhere that’s got background noise, that’s a mess.” For you, if you were taking and doing work either in a casino or in a meatpacking plant, I think about the chips, the noise, and any of the other. Can you discriminate between the background noise and the signature noises you're looking for?
Absolutely. We have many different ways to do this. We could do software-based depth detection inside of a camera. We could tell how far everything is. One technique we'll do is a depth-sensing software run on an image or frames in a video. We could do this in real-time. We'll turn it into a color map and then we'll use different color detection techniques. That is one of the ways that we split the distance. Another thing we do is we take objects that are in the scene and then we perform distance calculations, distance to pixel, take into account the distance of the camera from the objects they're taking.
In essence, we create a bunch of little rulers of objects. We identify a lot of objects in the scene that have a known size that we calculate based on the layers, the distance, and then we could use those objects as rulers to having very precise measurements, sometimes down to millimeters. This is something we do on power poles all the time. We detect wires, their gauge, what type of connector they're inside of. It translates quite well to being very accurate in real-time, even dealing with multiple types of sensors in very busy environments.
What type of companies owns utility poles? What benefit do they derive from this data gathering and discrimination that you do for the utility pole business?
The utility poles are owned by various types of companies, whether it's telecom or power, and even government-owned poles. What we're doing there is simply reaching out to those companies and explaining that there's a better way to information, be able to identify features that have never been done before. We're learning now that the companies are coming to us saying, “No one's doing what you're doing and you're the first.” We're certainly in a great position, but what we want to do is see our technology apply to those who want to get back to work and we see it as opening up doors.
The biggest threat is getting people back to their offices, one. You're going to have the mass transit issues where there's going to be large gatherings, whether it's a transportation here or a large gathering such as church and other types of gathering that occur. We want to get people back to work and be able to help these clients on the backend in terms of the workflow process. What is it they want to accomplish during this sequence of events? We know we can gather the data, connect to the data and pull information now. As a company owner, what do you want to gain from this?
The thing is if the owners of companies, they're going to look back and tell their employees, “We're making your place safe. We're giving you a sense of security. We're looking out for you.” There's some kudos or some goodwill going on with the employees when they see themselves going back to work, not being at risk. I also see the opportunity to sporting events, whether it’s getting them into facilities so they can practice, getting people back to work in manufacturing, pharmaceutical companies getting their people back to work to manufacture and distribute. We see a lot of that type of opportunity. Right now, we're seeing people come forward to us and say, “I want people back in their offices, and here are my requirements to do so.”
In the power pole business, the insights we gather, there are situations constantly where there are attachments on a power pole that nobody knows about and money is owed to somebody. Either somebody isn't paying or somebody hasn't collected. When you're dealing with hundreds and hundreds and thousands or millions of these poles, one simple mistake sometimes can result in tens of millions of dollars. That's one side of it. We come in, we identify who's using what poles, what services are being used. We notify companies that didn't even know that they forgot to collect $15 million over here.
The other side of that is violations. They're violations that due to how difficult it is to get these insights from so many poles, the phone companies can't get in compliance fast enough. The government comes in and these people get fined. The companies get fine millions and millions of dollars and they get deadlines, “You have two weeks to fix this. If you don't, that's another $5 million.” They can't. It goes over and it repeats. They get stuck in these vicious cycles. We come in. We find the violations in thousands of images. We can run probably a million images in 8 or 9 hours. We find all the violations in advance. The companies can take care of them and ultimately, they save huge amounts of money. The whole thing, in general, when you take all the extra money they get by properly billing their joint use on their poles if they're forgetting to bill or don't know about and you eliminate the violations that we've discovered for them now before the government does, it's a huge profit and violation cost reduction.
That's exactly what I was heading toward is I think about, why is the company interested? If there was a 5G rollout, Mike, you and I talked about that once before, which poles qualify, which poles don't. The thing that strikes me that's in the news a lot is supply chain issues in the food and meatpacking plants. There's a great deal of fear with the employees in meat packing plants because they're concerned about the environment that they're all going to take in and share whether asymptomatic or symptomatic. How would you envision your technology addressing that potential issue in the packing plants?
That's a fantastic use case for us. In that situation, we would use thermal sensors to make sure the heat at the right time was at the right temperature. When the correct heat is achieved during the preparation of particular foods, that's when the virus is killed. That's one major area in the preparation of the food that we can have a huge effect. We can also from a regular camera check the consistency and the characteristics of the meat or whatever food it is and make sure that it's not contaminated. We can do this with actual size imagery, but we can also do this microscopic. We could even put a microscope and we use these microscopes that are handheld. You put them on a little stand, on the conveyor belt and they can magnify up to 2000x. We can detect a huge variety of contaminants at that level.
[bctt tweet="You may not have a fever when you get to work but develop symptoms later in the day." username=""]
To detect them and to save people from getting infected, that's one huge part of it. The other part is we could even implement inside of the manufacturing and have an alert, hook it to quality control where that gets flagged when it detects contamination. Right then and there, it gets addressed and it gets routed to the right conveyor belt or whatever it may be. We have an open API with our capabilities. That's why we could go from power poles to COVID, making sure people in the workplace can go back to work. They don't have fevers or whatever and then we could switch to make sure that the food in the manufacturing is also safe and integrate into with that type of thing as well.
I think about the employees that are coming and they're so concerned. There's a plan up Nebraska right now. I've got this mental image of the entrance coming into the packing plant where all the employees typically come and go. There would be some level of imagery and data that are collected at the entrance. A quantity of the employees are asymptomatic and they have no temperature. They're all the things that you look for. They're not coughing and they don't have a runny nose. They're all wearing their mask and all that stuff, yet you have the 1 or 2 people that didn't know. They have it, but they don't know it. How do you take and identify those people and then create a notification that they're not allowed to join the other pool of workers? How does that work?
One way we talked about is using that USB temperature reading. Everybody at home would have a thermometer that they would check their temperature before they left home to go to work. Why drive all that distance when you get a high temperature and show up to work and then have to drive back home? They're very inexpensive temperature readings that you can plug into a USB port and be able to track your temperature and prevent even being exposed outside.
Bob, in addition to that, certainly if we can get them taken care of before they come into work, then we've won. That's the best-case scenario. If they do make it closer or into the building, we could track people. We could have an anonymous list face tied to badge number and we can integrate into any of the badge APIs. We could stop the door from opening. We could flash a red light. We could send a notification to the security that somebody needs to approach that badge number and tell them not to come in. There's any level we could be as open as sending me an email every day reporting the people that probably shouldn't have gone in. We can be every bit as strict as plugging into the badge system and not allowing the door to open. It depends on the owner, what they want to do, and what's appropriate for privacy, federal and state law, and all of that. From a technology perspective, we can get in there at any level they want and enforce it, automate it, hooking into other systems or whatever we need to do there.
I was thinking about these people coming through. You may not have a fever when you get there and you may develop symptoms later in the day. There was a restaurant I heard of out of Chicago. They instituted a rule where every employee had to wash their hands once an hour. That's how it is. If you didn't want to do that, you couldn't come to work. Let's say that there was a certain protocol that you established in the company. You said, “Bob, I want you to take and check your temperature every three hours. I want you to wash your hands every hour.”
---
Watch the episode here:
[embed]https://youtu.be/z6Wu4OZRGr0[/embed]Using Artificial Intelligence To Combat The COVID-19 Pandemic With Mike Blakeman And Trevor Chandler
My name is Mike Blakeman. I’m the CTO of Visual Globe. With me is Trevor Chandler, our CTO for our technology and what we're doing now. I know, Bob, you wanted us to share our vision and what we're trying to do with our experience and solutions. We’ll be talking more about that technology here. The thing I wanted to do, which I typically do, is you guys have created an ability to solve some particular problems. You were working in the utility space and then we had the pandemic come along. You took, adjusted, and adapted to also solve this problem. If you would talk about what problems or benefits you bring to the utility space and then also talk about the benefits that you're bringing to the safety of the community in the COVID timeframe. I’ll give a brief overview of what the company's doing now and what we're about to do with the wellness program. I’ll allow Trevor to go into the details of the technology and his experience with artificial intelligence. Our company is strictly focused on data and information that we receive from our customers regardless of the market we're serving. Now we're serving power and telecom, but the information that we receive, it's all about taking raw data and converting that to information. We convert that to put that into insight and actionable items that they can react to. This has been on extracting information out of images that we collect on the distribution poles for telecom power and the ability to go out to the field and capture transformers with infrared technology to be able to determine if there those assets are healthy. As a company, Visual Globe is focused on physical assets that you can see on earth and to determine their wellness, condition, and maintenance and what needs to be handled and what the decisions need to be made in order for something to be reacted to or supported by others in the field. Right now, our contracts are simply going out and doing auditing at poles on the transformers and those who are attaching their equipment to the poles and be able to pull that information right out of the images based upon artificial intelligence and the automated feature extraction. Let me move it over to Trevor. He’ll talk more about the technology as it relates to what we're doing and then we'll move into what we see as a great opportunity for people to get back to work using our technology.
What we're going to go through here is there's a benefit from this company, first in the utility space pole and equipment and monitoring and creating intelligence from the data extraction. We were issued the COVID pandemic. Data extraction and looking at heat signatures and so on map nicely over to the software and the potential needs that are coming from Visual Globe as it relates to distancing, face masks, and temperature reading. Trevor, please proceed.
Bob, we were extracting insights from pixel-based media images of power poles, transformers, and creating all kinds of important insights for these companies. When the pandemic hit, we went to the core of our technology and started thinking, “What can we do to help people get back to their jobs? What can we do to help keep people safe? What can we contribute to the humanitarian side, have some value here, and see what we can do to help out?” Our technology did transfer quite well for us to determine if a transformer on a power pole is within a particular heat or to determine if a person has a fever, for example.
Those are very similar things. We are already doing the location-based type of insights so we could track locations of where things are happening and whatnot. We're very good at grabbing information without violating the privacy, but still being able to use that information for helpful things. For the pandemic, it's all about making sure people are social distancing. It's about seeing who has symptoms and not just when they walk into a building to check their fever but to check their fever in different areas across different locations to understand what areas they work in. If somebody starts exhibiting symptoms, we can detect it before too many people get exposed and we can even tell what areas the person was in. Without going into any privacy issues, we can alert in certain areas to let them know that there may have been exposed.
[bctt tweet="As for the pandemic, it's all about making sure people continue social distancing." username=""]
We have a whole set of these types of useful features. The core of it is doing computer vision, artificial intelligence with RGB cameras, regular camera images to detect people where they are. Thermal sensors to detect their temperatures. We also can take data from any data source like text-based logging from USB or remote infrared thermometers. We can listen to coughing, sneezing. We can do a wide variety of these types of insights stored all on a map and we can create triggers. If this person was within this distance of another person, then send an email, beep a speaker, send a text, play an audio file, etc. I'd say the main features are thermal detection for temperature. We do facial recognition to detect the face. On multiple points, we go ahead and grab the temperatures and then average across all of the points to make sure that we're very accurate.
Don't forget to mention the mass detection.
That's one of the core ones as well. Face covering, mask, that type of thing so we can discover a face. We're very good at detecting eyewear, mouth, facial expressions. In fact, we do full-blown emotion detection for intent recognition and things of that nature. For this particular situation with the COVID, thermal then sounds, tracking where people go without identifying particular people and breaking privacy laws or concerns. Face coverings, we can even track eye coverings like eyewear. The COVID is getting spread largely through the nose. It's put in people's nose and the second cause is rubbing or particles of the air getting into the eyes. We see eyecare and masks both as being important.
I was thinking about its application. I'm the business owner and I'm listening, going like, “I've got my normal cameras for security in my building. I don't know if I have a thermal scanner.” I would imagine if they don't know, they don't. For the business owner that's tuning in and go, “I'm going to try to go back to take care of my employees as they return to work.” Are you in the camera business? Are you in the data business?
We are the data business and we can utilize any camera they have like RGB, regular cameras. If they don't have thermal, it's best if they get it. If they don't get it, we can use other forums like remote IR thermometers. Maybe they post their security guard and they do remote thermometer readings or maybe you give each employee a USB data logging temperature stick. We configure all those devices. We gather all that information.
We want to focus on the data side of things. We do not want to be in the hardware. We do not want to be in the installing. We want to be agnostic towards the types of sensors, RGB cameras, security cameras, and thermo sensors. We want to be completely independent of those.
When I talked to my father-in-law and he talks about background noise and his hearing aids. He goes, “If you get in a loud restaurant or somewhere that’s got background noise, that’s a mess.” For you, if you were taking and doing work either in a casino or in a meatpacking plant, I think about the chips, the noise, and any of the other. Can you discriminate between the background noise and the signature noises you're looking for?
Absolutely. We have many different ways to do this. We could do software-based depth detection inside of a camera. We could tell how far everything is. One technique we'll do is a depth-sensing software run on an image or frames in a video. We could do this in real-time. We'll turn it into a color map and then we'll use different color detection techniques. That is one of the ways that we split the distance. Another thing we do is we take objects that are in the scene and then we perform distance calculations, distance to pixel, take into account the distance of the camera from the objects they're taking.
In essence, we create a bunch of little rulers of objects. We identify a lot of objects in the scene that have a known size that we calculate based on the layers, the distance, and then we could use those objects as rulers to having very precise measurements, sometimes down to millimeters. This is something we do on power poles all the time. We detect wires, their gauge, what type of connector they're inside of. It translates quite well to being very accurate in real-time, even dealing with multiple types of sensors in very busy environments.
What type of companies owns utility poles? What benefit do they derive from this data gathering and discrimination that you do for the utility pole business?
The utility poles are owned by various types of companies, whether it's telecom or power, and even government-owned poles. What we're doing there is simply reaching out to those companies and explaining that there's a better way to information, be able to identify features that have never been done before. We're learning now that the companies are coming to us saying, “No one's doing what you're doing and you're the first.” We're certainly in a great position, but what we want to do is see our technology apply to those who want to get back to work and we see it as opening up doors.
The biggest threat is getting people back to their offices, one. You're going to have the mass transit issues where there's going to be large gatherings, whether it's a transportation here or a large gathering such as church and other types of gathering that occur. We want to get people back to work and be able to help these clients on the backend in terms of the workflow process. What is it they want to accomplish during this sequence of events? We know we can gather the data, connect to the data and pull information now. As a company owner, what do you want to gain from this?
The thing is if the owners of companies, they're going to look back and tell their employees, “We're making your place safe. We're giving you a sense of security. We're looking out for you.” There's some kudos or some goodwill going on with the employees when they see themselves going back to work, not being at risk. I also see the opportunity to sporting events, whether it’s getting them into facilities so they can practice, getting people back to work in manufacturing, pharmaceutical companies getting their people back to work to manufacture and distribute. We see a lot of that type of opportunity. Right now, we're seeing people come forward to us and say, “I want people back in their offices, and here are my requirements to do so.”
In the power pole business, the insights we gather, there are situations constantly where there are attachments on a power pole that nobody knows about and money is owed to somebody. Either somebody isn't paying or somebody hasn't collected. When you're dealing with hundreds and hundreds and thousands or millions of these poles, one simple mistake sometimes can result in tens of millions of dollars. That's one side of it. We come in, we identify who's using what poles, what services are being used. We notify companies that didn't even know that they forgot to collect $15 million over here.
The other side of that is violations. They're violations that due to how difficult it is to get these insights from so many poles, the phone companies can't get in compliance fast enough. The government comes in and these people get fined. The companies get fine millions and millions of dollars and they get deadlines, “You have two weeks to fix this. If you don't, that's another $5 million.” They can't. It goes over and it repeats. They get stuck in these vicious cycles. We come in. We find the violations in thousands of images. We can run probably a million images in 8 or 9 hours. We find all the violations in advance. The companies can take care of them and ultimately, they save huge amounts of money. The whole thing, in general, when you take all the extra money they get by properly billing their joint use on their poles if they're forgetting to bill or don't know about and you eliminate the violations that we've discovered for them now before the government does, it's a huge profit and violation cost reduction.
That's exactly what I was heading toward is I think about, why is the company interested? If there was a 5G rollout, Mike, you and I talked about that once before, which poles qualify, which poles don't. The thing that strikes me that's in the news a lot is supply chain issues in the food and meatpacking plants. There's a great deal of fear with the employees in meat packing plants because they're concerned about the environment that they're all going to take in and share whether asymptomatic or symptomatic. How would you envision your technology addressing that potential issue in the packing plants?
That's a fantastic use case for us. In that situation, we would use thermal sensors to make sure the heat at the right time was at the right temperature. When the correct heat is achieved during the preparation of particular foods, that's when the virus is killed. That's one major area in the preparation of the food that we can have a huge effect. We can also from a regular camera check the consistency and the characteristics of the meat or whatever food it is and make sure that it's not contaminated. We can do this with actual size imagery, but we can also do this microscopic. We could even put a microscope and we use these microscopes that are handheld. You put them on a little stand, on the conveyor belt and they can magnify up to 2000x. We can detect a huge variety of contaminants at that level.
[bctt tweet="You may not have a fever when you get to work but develop symptoms later in the day." username=""]
To detect them and to save people from getting infected, that's one huge part of it. The other part is we could even implement inside of the manufacturing and have an alert, hook it to quality control where that gets flagged when it detects contamination. Right then and there, it gets addressed and it gets routed to the right conveyor belt or whatever it may be. We have an open API with our capabilities. That's why we could go from power poles to COVID, making sure people in the workplace can go back to work. They don't have fevers or whatever and then we could switch to make sure that the food in the manufacturing is also safe and integrate into with that type of thing as well.
I think about the employees that are coming and they're so concerned. There's a plan up Nebraska right now. I've got this mental image of the entrance coming into the packing plant where all the employees typically come and go. There would be some level of imagery and data that are collected at the entrance. A quantity of the employees are asymptomatic and they have no temperature. They're all the things that you look for. They're not coughing and they don't have a runny nose. They're all wearing their mask and all that stuff, yet you have the 1 or 2 people that didn't know. They have it, but they don't know it. How do you take and identify those people and then create a notification that they're not allowed to join the other pool of workers? How does that work?
One way we talked about is using that USB temperature reading. Everybody at home would have a thermometer that they would check their temperature before they left home to go to work. Why drive all that distance when you get a high temperature and show up to work and then have to drive back home? They're very inexpensive temperature readings that you can plug into a USB port and be able to track your temperature and prevent even being exposed outside.
Bob, in addition to that, certainly if we can get them taken care of before they come into work, then we've won. That's the best-case scenario. If they do make it closer or into the building, we could track people. We could have an anonymous list face tied to badge number and we can integrate into any of the badge APIs. We could stop the door from opening. We could flash a red light. We could send a notification to the security that somebody needs to approach that badge number and tell them not to come in. There's any level we could be as open as sending me an email every day reporting the people that probably shouldn't have gone in. We can be every bit as strict as plugging into the badge system and not allowing the door to open. It depends on the owner, what they want to do, and what's appropriate for privacy, federal and state law, and all of that. From a technology perspective, we can get in there at any level they want and enforce it, automate it, hooking into other systems or whatever we need to do there.
I was thinking about these people coming through. You may not have a fever when you get there and you may develop symptoms later in the day. There was a restaurant I heard of out of Chicago. They instituted a rule where every employee had to wash their hands once an hour. That's how it is. If you didn't want to do that, you couldn't come to work. Let's say that there was a certain protocol that you established in the company. You said, “Bob, I want you to take and check your temperature every three hours. I want you to wash your hands every hour.”