Sign up for the 451 Research and Esper webinar: https://www.esper.io/webinar/digital-transformation-for-dedicated-devices

Ever wondered how Android can just listen and identify a song at any time? Or how your phone can tell you've been in a car crash? Using an obscure Android resource that interacts with ultra low power sensors, Google has created powerful always-on integrations. But why aren't other Android devices taking advantage of this?

Android Bytes is hosted by Mishaal Rahman, Senior Technical Editor, and David Ruddock, Editor in Chief, of Esper.

• Mishaal's Twitter
• David's Twitter

Esper enables next-gen device management for company-owned and managed tablets, kiosks, smart phones, IoT edge devices, and more.

For more about Esper:

• Esper Blog
• Mobile Device Management (MDM) Guide
• Android MDM Guide
• iOS MDM Guide
• MDM Solutions
• MDM APIs & SDK

New from the Esper blog:

• Edge AI
• Edge devices & edge computing

This week I wanted to talk a bit about a topic that many people do

not know about because it involves.

Underlying hardware that doesn't run an operating system you're probably

familiar with and it involves API APIs that are not generally

accessible to third-party developers.

But I do have a third-party developer here who is spin, trying to basically hack

his way into accessing those API APIs.

I'd like to welcome to the show.

Karen Quinn, who is a developer with a company in the UK.

And he's also a well known third party app developer on the XDA forums and

online who basically tries to hack his way into getting Google's applications,

working on any device in any way he can.

So welcome to the show.

Karen,

thank you very much, but like you say, Pretty much what I intend

to do to get as many things working on as many devices as possible.

And tap, tap, like you say, is no exception to.

Yeah.

And so for, for background here, what we're talking about this obscure thing

is the Android this has a name that I'm going to have to look up because I've

just forgotten it, the the context hub.

So this is like, if you, if you want.

If you have some computing background, like this is a, like,

this is a chip on most smartphones.

It usually manifests in the form of a dedicated little part of

the DSP digital signal processor.

And this is branded variously by companies like Qualcomm and

BD attacking Google and Samsung.

And so it runs a tiny little real-time operating system.

Very, very lightweight, designed to be extremely low power usage.

And Android, you know, has had support for this for a while, but usage and

adoption of it is kind of limited.

And I think that's part of what we'll get into today, but maybe we should really

start by explaining what this hub can do.

Right.

So to step back a bit you know, your slab rectangular Android

smartphone is filled to the brim.

A plethora of sensors.

For example, any modern flagship phone will include sensors like an

accelerometer, gyroscope, an ambient light sensor, a proximity sensor,

and so on, and mostly sensors do exactly what their names say they do.

They measure the acceleration in the case of the accelerometer, the

magnetic field, in case of a maximum.

The ambient light in the case of an ambient light sensor.

And so, you know, these, these basic sensors there

you'll find them pretty much.

Every smartphone, a few low end smartphones might lack one sensor here

or there, but in flagships, you'll find almost all of these sensors.

And because there are so many sensors and there's so little room to put

them in, into a smartphone package.

A lot of vendors, what they do is they pack these physical sensors

together into a single package.

So for example, an IMU chip, that's an international sorry, I forgot the

acronym stands for, but it combines an accelerometer and gyroscope into

a single chip on the software side.

Android can combine the sensors, not physically, but in software to create

what's called a composite sensor that basically collects data from two or

more sensors and fuses them together to where it can detect certain functions.

So for example, there's a rotation vector composite sensor, which

combines data from an accelerometer might need a meter and a gyroscope.

So basic sensors themselves, like the accelerometer can also do multiple things.

They don't have to be combined with another center to be useful.

The accelerometer, for example, it can be used for basic step counter

detection, or it can also be used for the significant motion detection.

So for example, like if you were to start running or going on a bike, if

it detect that your act doing those activities, or it can also be used to

protective, just basically walking.

So it has a lot of sensitivity there.

So there's all sorts of sensors.

You can find on a modern, smart form, but I wanted to know from you guys, are

there any sensors that you wish were packed into more smartphones these days?

Samsung, for example, used to have a heart rate sensor in their flagship galaxy

S series, but they stopped including them with the galaxy S 10 launch.

Do you think more smartphones should have dedicated health sensors like that?

Or do you think another approach is more suited?

I liked the idea of having a heart rate sensor in the phone.

But I do question how much separating the cry at any moment, close off the

plethora of smartwatches that have the same sensor in them, and that the many,

many health apps that are associated with that, even the really cheap

smartphones nowadays have sensors in them.

In terms of having a dedicated one, as opposed to using the camera system.

It is a, it's a nice to have, but you do, you do have to then

consider where would you put it?

So yeah, on the back of the phone, they used to be sometimes

integrated into fingerprint sensors.

So that made sense.

That's why you think it usually was.

If you can do it with a camera.

And if the phone is set up in such a way, I have used a very similar API with an

app at work that I won't go into that used the camera other than the flashlights,

in order to detect the heart rate.

I, one of the problems we've had is that various different phones of the

flashlight in different positions, and then phones, these days have so

many different cameras on them and in different arrays that you have to have

your finger in a very specific position.

So it would have been really nice if we could say, well, it works.

The long list of Samsung smartphones.

That's still included them after the S 10, but obviously they

dropped them and we ended up just looking down the roots of all.

Can we integrate it into watches or anything like that?

We never got that far, but that is one of the things that we, we prefer to do.

So handing off this idea of, of doing heart rate sensors to the

watch is probably a better way of doing it nowadays with them being

the kind of dedicated health system.

But other senses, I don't know, I've been out to you guys.

I think that if I were to choose a sensor to make more common on a phone and

only one phone ever did it would, it would be radar solely on the pixel for while

it was very limited in what it could do.

The.

Possibilities of that system were really intriguing.

And I think that you could implement that kind of technology across

a wide variety of form factors.

And that's something we've seen because Google is trying to

genericize solely to be used in other applications by other businesses.

And you probably see solely popping up and things like

what is the Google telepresence project star link, not star link.

That's Elon Musk.

I'm.

Star line.

That's what it is it star line.

So when I look at something like radar, I think that holds a lot of, a lot of

promise because it allows your device to see in a totally different way and also to

do so, like in a way that can recognize.

Motion in a very fine grain kind of manner.

So maybe not a phone necessarily is the best use case for that, but something

that's stationary where someone's sitting there or standing there.

And that radar can determine an intent basically potentially using, you know,

this low this low power sensor hub as part of like a wake series of commands,

basically where like there's a basic presence detection that trips a low power

sensor, and that wakes up the radar.

So.

So, yeah, I think that radar is one that I'm, I'm really intrigued by person.

Yeah.

And Google only included the solely radar sensor in a single smartphone

series that pixel for, but before the pixel four launch, they actually

in their patent applications, they showed off various video.

That showed solely radars inside a wearable.

But we haven't seen that use case emerged yet.

We have seen them equip their second generation nest hub smart

displays with solely radar.

And I believe they use that for in, in conjunction with

the sleep detection feature.

But yes, that would be a fascinating example to see more deployment of.

So I'm speaking of underutilized sensors you know, there's a lot of sensors

that have been around for a long time.

Intelligent use cases of the sensors goes pretty far back

to the early days of Android.

In fact, I think I'd say it's one of the key ways that one of

the biggest smartphone makers, Motorola used to differentiate its

smartphones from the competition.

If you'll recall way back in the day, the company had a suite of actions

called Moto actions that would let you do things like turning the flashlight

on and off with a chopping motion by basically raising your hand.

Moving it up and down quickly.

They used to call that chop chop.

And I think that I remember seeing a lot of commercials about that too.

It was a pretty popular gesture.

They had other gestures like launching the camera up, twisting

your wrist, or like putting your phone face down and silence calls.

These features.

Aren't really all that special nowadays because well, they basically

being copied all over, but they weren't common to find way back then.

Personally, I think a lot of these emotion-based gestures are kind of silly

cause I don't really see myself waving my hand around to do things with my phone.

But what do you both think?

Did you find any of these gestures particularly useful?

I can't say I've ever used the camera on either.

I'm much.

I'm much.

I'm much prefer the the double tap and triple tap.

The the more recent integration equivalent for

it, especially with Snapchat.

I can say a lot of people use them the face down to silence face down

to turn the screen up, especially if you don't have a number display.

So yeah, I can see, they use personally not a huge fan of

the Australian gestures either.

They look great on paper and they probably look great in commercials

and, and to board members, books.

Yeah, I don't think a lot of people would be using that kind of thing

in their daily life to be on.

And I think that's right, that they demo really well.

And the chop, chop, Michelle, you are literally in commercials and

the Modo chop, they really marketed that what's interesting to me is

that we never saw this really take off much with wearables and Kieron.

Maybe know a little bit about this, but like, I, I always assumed this was a data

noise issue where the wearable is getting so much accelerometer and gyro data that.

Finding an action that can consistently be recognized with

accuracy would be a real challenge.

I would imagine that is probably the Keisha right there, but

wearables also have the unique or the more unique type of thing where they

have a much smaller battery as well.

So even if you do have a low power CPU, you still have to be considered,

considered the battery aspect of it.

And I'm not sure.

I don't know if you guys know.

Whether the old wearable chips actually even had these lower power CPU's in them.

That might be an open question.

Do you know whether they

did or not that I don't know.

Michelle, do you have any insight?

I know

that over time these wearable chips has happened adding more and more.

Low power subsystems that components.

But I think that is one of the biggest differentiators between the wearable

chip sets and the chip sets for bigger devices like smartphones is that

wearable chip sets don't have enough of the low power, like machine learning

chords, for example, to offload a lot of that processing to which is why you

can see things like super low power efficient voice processing on smartphone

devices, but not on like wearables.

That's, you know, it is a good point.

The power envelope, there is so much tighter.

You do have to be very judicious with it.

And I do wonder if even where O S would make something like that really practical

for a developer, but that kind of gets into, you know, what can you do with the

sensors, especially in the context of like a smartphone where these are really

becoming very, very complex system.

Yeah.

So over the years, you know, we've seen a mobile SOC has become incredibly

powerful, but we've also seen behind the scenes, these sensors they've

become incredibly more advanced the data they're able to process as much more

fine-grained smartphones themselves.

Getting getting bigger so that they can pack more and more sensors inside of them.

And as I mentioned before, machine learning is, you know, something that's

really taken off in the recent years because of all of these innovations,

we've seen software companies use sensor data in really innovative

ways, ways that we've never really considered were possible many years ago.

So I like to pose the question to both of you.

What in your view has been the most impressive feature to come

out of sensor applications?

I think car crash detection is quite quite a big one, but also just,

just in general use of sensor data for me, it's, it's, it's got to be the now

playing feature on pixels because that is even if it doesn't go through this

low power stop as such, or at least I'm not aware if it does complete.

The fact that they have managed to get that to work offline and with such a

big database is it's just insane to me.

I've looked at all the stuff behind it.

I still still boggles the mind when you look at it and it just works like you, you

barely have a song playing in the distance and it will recognize it sometimes.

So that always listening capability, I think is the top.

I think I'd have to agree aside from car crash detection, which, I

mean, we have it in the outline here.

So I think we're all being drawn to it because it is very cool.

But the music detection feature is obviously in evolution of hot

word detection, which became table stakes on smart phones in the last

six years, I guess, about that.

How'd that long that's been around, something like that.

So I do think that that feature is a great example of the convergence of

machine learning and AI, and then using these ultra low power kind of sensor hubs

to take a very, what is really a very.

Data's signal, right?

You are taking one piece of input and you were then feeding that into this massive

dataset and coming out with, and you know, something actionable for the user.

So that's in that sense.

I do think that the.

The music detection feature is more impressive than car crash detection,

because you are doing something with like so little contacts.

That's, what's impressive about it.

The car crash detection feature, which Michelle, you can explain

a little bit how that works.

Really is impressive because it's able to fuse data from

multiple sources into another.

Yeah, car crash detection.

You know, it's like David said, fuses data from multiple sensors, including a

gyroscope accelerometer and a microphone.

And basically because you know, it's such a life-saving event and needs

to be processing data continuously from all these sensors, because you

know, it can't miss a single beat.

It can't, it can't miss a car crash event that would be catastrophic, potentially

devastate life devastating to the user.

And not as detrimental to the user, but the now playing feature that Karen

mentioned, you know, that needs to continuously process microphone data

in the background because it needs to pick up on audio cues and mashed up.

Fingerprint from a, from a database that's stored on device.

And because of this continuous processing requirement for the microphone for now

playing and the continuous requirement for the gyroscope, accelerometer

and microphone for the car crash detection, you know, companies like

Google that are making devices with limited batteries in a smartphone, you

know, they have to consider how do we implement these features with that?

Destroying the battery life on a device.

And you know, if you were to keep the main applications, processor on

a device, such as the Google tensor chip in the pixel, six awake at all

times in order to process that data, it probably destroy the battery life.

So instead the solution is to not do that.

Don't wake the application's processor at all times.

Instead use something else, something much more low powered something called a

sensor hub, which is what David alluded to in the beginning of this episode

sensor hub, which is also called a context hub in other contexts is a low power

processor that exists solely to process.

Data from sensors and then wake the main applications processor,

whenever something needs to be done.

We've seen sensor hubs used in devices going back all the

way to the Motorola example.

They've they use a sensor hub.

That's apparently a arm powered microcontroller.

Sensor hubs can also exist on the diet.

In the form of an island that runs his own operating system,

you'll find it on Qualcomm chips.

They have what's called an SLPI or a sensor low power island, which

is a part of the hexagon DSP.

Google devices have also had sensor hubs since the days of

the nexus five X and six feet.

And of course their latest smartphones also include a sensor hub, which

they call an always on compute.

So the challenge with implementing a sensor hub is that it's a

different kind of platform.

The software running on it is very different.

It's not Android.

Most of the times as go, as David mentioned earlier, it's a real-time

operating system and many central hubs often run different operating systems.

You'll have solutions like free our costs or Zephyr OSTP, and then

you'll have proprietary ones such as the ones that many Silicon vendors.

Because of this variability in the software platforms on sensor hubs,

Google decided to create a standardized framework called the context hub

runtime environment, which is a software environment to execute small and native

applications written in C or C plus.

These native apps, which are called, and I know apps because they are small and they

are native only do basically three things.

They start collecting data, they stopped collecting data and they handle events.

And under the C H R E API, Android is able to interface with these nano apps.

So that's a whole bunch of context and I'd like to ask Karen.

A question about his research into nano apps, because he's been looking into

these nano apps and context hubs, et cetera, while he was digging into how

the pixel six, his back tap feature works.

So, Karen, can you tell us a bit about the pixel six is backpack feature

otherwise known as quick tap as well as your research into how it makes use of

context hubs and the nano application.

Yeah, so little bit of background, the feature use there's

accelerometer and gyroscope data.

And basically just feeds that into machine learning algorithms to

figure out whether the user is tapped on the back of the device once.

And then does that within a certain period of time to detect whether

they've done it as a double tap.

That's as far as the future goes on pixel devices.

Previously, when we'd seen it in Android 11, it was done in Italian app.

So it was all visible code, except obviously the student learning

algorithm, which is through density.

So that was very high power.

And that is previously what people were using.

And the original version of that that's happened.

That's what was draining the battery for a lot of people.

Even if you run it on a device that uploaded some of the processing of

TensorFlow stuff onto low power CPU.

Having the processor online all the time, and therefore you're draining a lot,

your battery, but an Android 12, we saw the feature disappeared from the code

from the, the app code, but the works.

So I looked further into it and then discovered that actually there is now

a nano app called Columbus Columbus being the name of the feature in

term in Google, which does all of the processing of the gyroscope accelerometer

data and also the time-based data.

And just to mix these events for a double-tap.

The way that sets up interfaces with this is a little bit of a hack.

So already, as well as the permissions framework, you have to get around the

fact that the nano app is emitting just the double-tap event, which is great.

If you only want double tap or touch up, or is that triple tap?

So the way I get around that is it also a mitzvah events for logging

purposes for when a single tap happens.

So I then do the triple tap detection.

In the app itself.

So that does use a little bit more power, but it's still tapping into the,

the low power ability of the, of these nano apps, unfortunately, and something

that people have asked me a lot.

They aren't portable because you can't build them without having the source code.

And they are very specific.

They are specific to the base and in some cases from why, so you can't just

take one that's been built on, on pixel sex and then check it on a one plus

four and hope it will work because.

Even if the OnePlus phone had a context of available, which, which it doesn't.

So no, it can't be parted, but yes, it is a great feature to use.

If it is available, there are also other nano apps available.

So as well as Columbus, there also one for detecting like ambient background stuff.

So just noise levels, that sort of thing, which I think feeds into.

Activity recognition on, on Google maps and that sort of thing, which is itself

an extra nano ultimate that Michelle, the dimension is that they are quite modular.

So one app will use another app to do some of its recognition

and check I would across them.

So you've got one for activity.

There's the car crush one, obviously, which, which takes in a lot of data.

And then for metadata, Information on when a car crash happens,

but it's extremely specific.

It has to happen.

It has to pass a lot of checks because obviously arguably a a number of false

positives will be worse in terms of PR hopefully not for the person, but it'd be

worse in terms of PR not triggering at all, because that might be recoverable.

But if you end up with the feature of interview disabled, because you.

I'd state with the authorities then, but that's much, much worse with PR.

So they've been very careful with that one.

Interestingly, there's also things like geo-fencing.

So if you're not aware of geo-fencing, what that is is basically the

device has an app on it that says, I want to be notified when

the device was in this location.

So it's using things like Google pay.

When you go into a store to recommend you use your cards and that sort of thing.

That that needs to be done low power.

Cause you don't want the, the CPS be on all the time.

That's processing all of the location data so that that's

done at a low power CPU as well.

But we've also found that there's this strange ones, like

calibration stuff for rest sensors.

So there's, there's obviously a lot of stuff going on in the background

or that Google they're trying to optimize to, to stop it from draining

the battery, making use of their.

Tends to CPE, which probably has a lot of processing for this sort of thing compared

to maybe some other CPS, the others, the loss of nano apps that you don't even know

are happening behind the scenes, because it is completely transparent to the users.

Super interesting platform to look at.

And it's just a bit of a shame that it isn't opened up to more developers,

which we'll be going into in a moment.

I'm sure.

Yeah.

And I think maybe we could skip ahead here to really why certain device makers are

using this and why they aren't, because that seems to be the bigger issue is

that Google's framework for using this.

These hubs has very low adoption.

Kieron, why do you think that is

the partially due to what I've just said?

Y Yes, as a third party developer.

So there is less incentive for them to put it in their devices because there

aren't like loads of apps using it.

So they're like, oh, this doesn't work on a Samsung phone is literally the case

that if, if Samsung came along and said, we want to implement the CHR, a chances

are they've actually already got under the framework somewhere in the system, but

he's doing the same thing for their apps.

So there is little incentive for them to integrate the standard.

If it's not being used by lots of third party system, I'm not sure how it

works for their kind of like geo-fence like, I was just saying an activity

tracking, whether they run on a different framework on Samsung phones or on other

OEMs that have different processes implemented or whether they simply

run on the CPO and use not battery.

I don't know.

But that, that will be interesting to find out because that will

probably be one of the big reasons why it's not been implemented.

Other than that, it's probably also due to lack of resources.

So it's smaller.

May not have the resource or the ability from their developers to

implement this sorts of thing.

It's very, very low level.

When you look at some of the how it interacts with the system.

It's I think it's, I think from the presentation that the docent was

running and C plus plus, so it's, while the platform itself can be integrated

within Java, you need a C plus plus developers be able to do that as well.

Slightly different platform than, than most of Android.

And obviously a bit of in the current level book or Williams may not have the,

the capacity for that sort of thing.

Yeah.

I think those two reasons probably combined for on the whole,

that's probably why they're not.

So I'd like to bring up one major downside to the fact that many OEMs

apart from Google haven't implemented

And it's a fact that as Karen alluded, alluded to earlier, some

Google applications make you.

Fi framework, they have nano apps for things like activity

recognition or geo-fencing.

And those Google applications are actually found on pretty much every entry device,

Google play services, for example, implements activity recognition as an

API that other devices can subscribe to.

So if you're looking to detect, say when a user is.

Walking or biking or running, you could use Google plays.

The Google play services activity API to basically implement those into your

app into your application, but because very few devices support or have

nano apps that Google play services can run except for on pixel devices.

Only pixel devices will be able to have that activity recognition being

incredibly powerful and running continuously in the background.

Karen, what do you think about that?

Do you think it's something that I'm like, what do you think about this situation?

Why?

Because I th I think it's, it's a shame that it's not been

implemented on, on other devices.

It is possible for a device for, so for an application to load a,

a nano app at runtime, that is a possibility, but it has to be able to.

Interacts with it.

So it needs certain permissions and it needs to be signed by the system.

And also none of apps have their own security layer.

So they need to start with a certain certificates in order

to be able to do certain things.

So it's possible that if, how we can support with better, that

they will be able to implement more of these things properly.

But it, it does just leap back to this whole idea that there's been little

incentive for them to do it at least.

What shows to the user, especially if they have their own system to do it.

Play services is a complete black box with no idea how most of it works.

So there is, there's a decent possibility that in there somewhere there is the

equivalent of this that is processing on, I dunno, the ex-con CPU or Qualcomm CPU's

or similar things on, on Samsung devices that have their own implementation.

So on that level, the ma the may be.

Things that we aren't aware of that are alternatives to that.

Sure.

And that makes sense, because when you're talking about, especially anything,

that's getting ML models involved too, you're going to have different

ML blocks across chip set vendors.

You're going to have different implementations across generations

of Silicon because it's still evolving pretty rapidly.

And so probably a lot of these more sophisticated use cases like

Google's music scanning require, obviously like you said, Kieron, they

require a very narrowly tailored.

Nano app that is very specific to the use case and is only really going to be

useful to the one device being targeted.

So having a broadly accessible platform for building on this hub or this con this

contextual hub, probably like you said, it doesn't have much appeal to the vendors.

They're already using the tools they want to use and the

frameworks they want to use.

But I do think that still there's, there's a whole lot of potential

here in terms of what the CRH can do and what it can see.

And Michelle, there's a, you have a great list here of kind

of the stuff it can gather up.

A lot of which I don't think is even being used by anyone.

Right.

Yeah.

So looking at the documentation, the CHR three implementation

actually supports multiple sensors, including the basic ones, like the

accelerometer, gyroscope, ambulance sensor, proximity sensor, and it also

has API APIs to request location data.

Scan for wifi networks, get cellular ID information and process

batches of data from a microphone.

So I think that's probably the big one.

We probably haven't seen many use cases from the audio data processing.

Google does.

Have a few audio related features that seem to make use of this

such as the now playing feature.

But as Karen mentioned, it, doesn't, doesn't look like now playing actually

has a nano app from what we can tell, but yeah, there's clearly

a lot that can be done with this framework and sensor hubs in general.

But I wanted to ask both of you, what do you think would be the next big

feature to make use of a sensor hub?

It was opened out.

Go

on.

Oh, no, go ahead, Kiran.

I

said, if it was opened out to third-party developers, it would be sleep

tracking apps because they process audio data, movement data, especially if the

device had a radar sensor on it as well.

I don't know.

I think some of them are recommend, but you can put your

phone physically next to you.

So they'll probably also use gyroscope and accelerometer data.

So something like that would be nice to have a nano up that exposed

some data to an app on the system.

So that, that would be a nice feature to have, which when you'd have to market it

to users as a health feature, probably.

But other than that, it's hard to tell until they come out with.

Because all of the ideas that have used nano app so far and

the no, no play gesture stuff.

They've before that they've always be, oh, it would be a nice feature, but it

uses internet or it uses too much power.

So until the OEMs come up with these ideas and you think, oh

yeah, that's a really good idea.

It's, it's hard to think ahead of time what they are going.

I think that one for me, and this is something that Google experimented

with for a long time with a smart lock was the pocket detection mode for

phones, which I don't think still exists.

Michelle, is that, is that still work?

That it was deprecated a while ago?

I thought like, basically it seems like it's.

Yeah, the texts in your pocket.

I'm not sure if it's still, I think smart lock removed a lot

of the like unlocking abilities.

Like you can still have your device unlocked if it's connected to your

Bluetooth smartwatch, but I don't

know about anything else.

Right.

So I think that I could see a future for the CHRs being able to do some,

some fusion with kind of basically.

A lower level of trust around personal authentication.

So not quite biometrics, but something that helps the phone

realize, okay, I'm in your pocket.

You know, like I know how you walk.

I know how you sound.

I know like if you've left a place And use that to provide some more trust

for like, you know, seeing content without explicit unlocking, because

on Android, we're seeing that, you know, facial ID is either too expensive

or too form factor compromising for a lot of the OEMs to adopt.

Even Google got rid of it and fingerprint scanners are quite spoofed bubble.

So I, I could see some security stuff going on there.

And I imagine Google is probably already doing some of this with

like deciding when the device needs to be unlocked manual.

I like the idea of walking detection, because

everybody walks slightly do.

Yeah, I would be really interested to see what the sensor data would

look like there because Google also, I remember, no, this was Qualcomm.

I heard years ago.

They said that if you have a good, good enough radar system, you can actually

identify somebody's radar signature.

Because everybody's radar signature is a little bit different.

And so they had a proof of concept where they had radar.

Wifi points deploy throughout a home and they could tell who was in what room.

So that's less of a low power issue because these are stationary devices,

but you could see a mobility case for something like that, potentially.

Yeah.

And speaking of walking detection, actually the digital wellbeing

application on that's developed by Google, they actually have like a feature called.

That the texts when you're walking and using your phone at the same time and

like tries to warn you just cut it out because you know, that's dangerous.

And that's, that's an example of a feature that I never would have

thought of to implement before.

And I'm sure processing data in a sensor hub, the accelerometer data to detect

footsteps would be very power efficient and allow that application, that

feature to be continuously monitoring for, you know, heads up moments.

Yeah.

What was, what was going to say as well as if so your walking detection, you could

use for accessibility purposes as well.

So if you're walking around a space that somebody isn't familiar

with in the partial sighted, then you could use it to vibrate.

If there is something unexpected in front of them, say that would take them lots of

sensor data, but where things like radar and Sullivan serving and that sort of.

It's a possibility I would imagine.

So basically your phone would become your navigation system

for somebody to get around.

It might be done in the future.

Yeah.

Where we're going to probably conclude here is getting into, okay.

Well, the we've talked about smartphones.

We've talked about what it looks like in some consumer implementations

we've seen, but in our world, when we're thinking about like dedicated

devices and enterprise and business and industry sensors are really

commonly in use in all of these cases, especially in places like factory floors.

So for example, on a factory floor, you have tons and tons of.

Devices these days often communicating over Bluetooth, low energy or wifi you

know, not necessarily very sophisticated, but there is a lot of wireless activity.

There are a lot of computers crunching, a lot of data.

And so imagine an environment like a factory.

If you have people walking around.

You, for example, if you use low power geo-fencing to detect like,

Hey, you're entering a hardhat only zone, your watch is going to vibrate

and say, Hey, are you wearing the right safety equipment right now?

Or even go so far to say, Hey, you're entering a restricted zone.

You're not authorized to be here.

It could also be something like apples, fall detection on the apple watch.

So if you have a safety incident at a workplace.

Wearable can tell you, Hey, you have an employee who probably fell down that

can help you respond much more quickly to an incident call paramedics, get

somebody onsite if that's necessary.

So there are a lot of ways you could be using something like this.

And probably that aren't necessarily like sophisticated from a sensor

data standpoint, especially in a really controlled setting.

Like a factory floor, like an office building, or like, What's

another location, like a hotel or a restaurant where the context is

relatively fixed and most of the computing assets are fixed too.

So it could be that you have employees wearing the device.

It could be that you have employees using a handheld one example that we've

already seen, and that apple is probably.

The most famous for doing on the consumer side?

Not that they were the first is using like super high frequency, no meter wave

to do basically echolocation of things like air tags or your apple watch.

So these things are, you know, they're becoming more and more

common devices are emitting more and more types of signals and looking

for more and more types of signals.

So it only tracks that we're going to come up with ways to fuse that data usefully.

So it's hard for me to.

A future in which this doesn't start to get more attention, maybe not as

the CHR three, I can't really speak to the viability of that as a platform

or why it appeals specifically.

But as a overall concept, I don't see sensor data becoming less important.

I see it becoming way more important in more and more use cases in context.

Yeah.

I think just touching on the idea of location data as well.

The fact that in the last two years, the idea of precise location tracking from a

wearable device or a handheld device using things, both of that and high power GPS

that is going to feed into this as well.

So we've got a lot of Bluetooth tracking research that's been done for.

So some trucking and and trace and that sort of thing.

So that has, I know from experience that I've all go into that has fed into little

wearable devices that do this sort of thing that are able to track somebody's

location on the factory floor that are using systems or the van GPS, or other

than Wi-Fi in sensitive areas that that some things aren't allow with all

that you're not allowed your personal phone on you or that sort of thing.

So.

All this data and all of the new research that's been done the last few years

made cause a huge increase of, of this sort of thing in the next few years.

So it'll be very interesting to see the direction that this.

And I guess before we wrap up, you know, maybe one more thing to

think about in that context of why, why Android makes sense as the platform

for this is the power efficiency.

That we were talking about earlier, and that I think is why the wearable and

mobility use case like these are new for a lot of these businesses and industries.

And so there's just starting to learn how this stuff can be used to gather

data and, you know, meaningfully improve processes or employee safety.

Or whatever their goal may be.

So you have tons of companies coming now over from windows machines and

who are trying to use, like trying to learn this mobility landscape.

And Android is kind of like, you know, CHR may not be very well known at all in the

enterprise world at this point, but the fact that Android has that extensibility

built in at the OEM level Undoubtedly there will be exploration there.

And because I don't think that, could you build this on iOS?

Could you build this on windows?

Almost?

Definitely.

No.

Could you build it on Linux?

Well, sure.

If you had all the time and money and resources in the world, but Android is

the only one, the only platform that seems like primed for this change.

Absolutely agreed on that.

Been there, the only other prevalent light you say will

be sunsetted Linux environment.

And I think from research that was done for this, the Google

have been looking at implementing it on a, on a different system.

So it wouldn't be on Android, but perhaps something that doesn't require

the power to run the OS itself, which maybe would be useful on a smart screen

or something or small wearable device.

But Android is, is really the way to go for.

Handheld devices that use something like this.

Yeah.

And if you look at Google's work in particular, they're particularly invested

in promoting a R toss called Zephyr OSPF.

They've been contributing a lot of development effort to it.

They've also recently started to port the CHR framework onto it, so

that Enterprises or developers that are building embedded controllers

and are seeking an operating system can use that for LS and

implement Google CHRs framework.

And who knows?

We might see an uptake of chips with these embedded operating systems

and interfacing with a high-level operating system like Android.

Android is not competing with fees.

Our costs it's working in conjunction with them.

And I think Android as both David and Karen mentioned as

a perfect platform for that.

Yeah.

One of the things that was mentioned by the guy during the presentation,

which I'm probably linked at some point from this, but he mentioned that you

could in theory have, because CHR is a.

Framework you could in theory, have modules that sit on top of it that can

be shared between different devices.

So you could have a module that you can just import, but it does all of your

location tracking for you, which saves you on lots of time compared to implementing

it yourself on a Linux-based system.

So with something like that, we could really see like a huge uptake in this sort

of use for, for lots of small electric.

Because it removes some lots of work that they have to do to program this.

All right.

Well, I think this is probably going to be the most exhaustive resource

on CHRs and Android sensor hubs online, at least audio resource.

So Kieron, thanks so much for joining us.

This is a really esoteric and honestly, quite fascinating topic because.

You know, the fusion of hardware and, you know, accessibility in terms of the

operating system is really interesting.

And an area where Android has been uniquely equipped from the beginning to

really capitalize pretty well in ways that legacy platforms are mitigated ecosystems.

Like apples just don't really have, even when Google does try to make

it hard to play with its toys.

So Kiran, where can folks find you and what you're working on

and anything you'd like to do?

On the topic of this, I mean, I've got to have some taps out cause it's it

took a good few months to get the CHR.

We still found that on the update.

To a point that I was happy with, unfortunately, looks like you will

make no fight in Andrea 13 when that comes out with it, it still

be accessible with roots at least.

But yeah, tap tap is available on my hope, which is gethub.com/ Kevin Quinn.

I'm also on Twitter at Quinny 8, 9, 8.

So you're welcome to follow that if you want to follow with.

And Michelle and I are with Esper.

And if you found this show interesting, because you're trying to build at

Android device, whether from the ground up or using a operating system,

distributed by an OEM, come talk to us.

This is the kind of discussion we'd love to have, because we want to

know why you want to use Android.

What you're trying to enable with it.

It could be anything from a kiosk.

It could be a smartwatch.

It could be a television size display, really anything where you're trying

to do something very specific with Android and probably in a, either

business or customer contacts where somebody is interacting with this

machine to do a specific sort of thing.

Asper is really good at this.

We build our own distro of Android that is designed for these use cases.

That's really hard and.

Able to handle like a lot of, you know, a lot of updates, really,

really easy to implement overall.

And we work on a pretty wide variety of hardware platforms, including x86.

If you'd like to talk to us about that.

We're at esper.io and this has been Android bites.

Thank you for joining us, everyone.