Virtual Reality promises great things. Headline often read along the lines of "VR will make any world possible". Is this true?

For more information on Aleph Insights visit our website https://alephinsights.com or to get in touch about our podcast email [email protected]

Hello and welcome to the Cognitive Engineering Podcast, produced by me, Fraser McGruer, for Aleph Insights. In this series of podcasts, we take a look at interesting topics and discuss what we think they tell us about analysis and decision-making. I'm here with Nick Hare and Peter Coghill of Aleph Insights, and this week we're discussing, is anything impossible with virtual reality? Peter, is anything impossible with virtual reality?

Well, I was reading a blog post or a news article about virtual reality and all the massive promises that it potentially brings, and you get lots of headlines like, VR will make worlds where anything is possible, and VR is going to change everything. And I started idly thinking, well, is that true? And I thought it'd be fun to discuss what are the limits of virtual reality? Are there any? If there are, what are they? Because it occurs to me that virtual reality is just another way of overlaying on what you would normally see with a box you put on your face and maybe some sensors you put on your body and some earphones you stick in your ears. It just overlays other stuff that you would normally perceive. And there are inherent limits in our capacity to perceive things. So what are those limits? And so assuming there is a limit, it's not all possible. What is possible? And what, you know, where are the limits? Nick?

Well, this is, I think, is a brilliant question and surprisingly hard to know how to go about answering because, you know, of course, our interaction with the world is so fundamental, we don't notice it. And what we understand about the way we process information is really based on us, how we are now and how we normally process information. You know, so it's very hard to say how our brains and our perceptions might adapt to, you know, perhaps really re-rigging the kind of sensory information we get. But I'm going to immediately bring bats into this. So here's my question, of course, is could we, there's a very famous philosophy essay by Thomas Nagel called What is it like to be a bat? Where he discusses the fact that we cannot ever really know what it's like to be a bat because we simply don't, there's a whole set of experiences that bats have that we don't have. And so I think that's a good place to start, a good test case. Now, could VR, I'm going to, and I'm going to extend it and say, well, let's just start with conceiving of VR as essentially giving us sensory information through the sensory organs we have. We can talk maybe in a bit about what, how we might be able to, you know, use new sensory organs, create new sensory organs, but just stick with, let's just say we've got VR, we've got kind of, you know, 3D immersive visual and let's say auditory and, you know, possibly tactile kind of sensation. So can it, can it show us what it's like to be a bat? Now, I think in sense one, can it show us what it's like to have sonar? No, we are not, all it will do is translate in some sense, the experience of sonar into a visual image. And then we'll just know what it's like to be a human seeing what, you know, what a bat's sonar would look like if it was turned into a visual image, which isn't at all the same as knowing what it's like to actually have sonar. But if you think about what it is that we actually experience, we don't experience visual images. You're not sitting there watching a visual image like you might watch a TV. You're sitting there being in the world, a world that your brain is constructed from the information it's got. Agreed? Yeah. So, so the thing is that maybe, careful Fraser, it's going to get deep. Yeah. Maybe, maybe, maybe virtual reality could give us an insight into what it's like to be a bat in the sense that it can show us the world that the bat perceives. It can show us the bits of the world that the bat sees. And it might be that it, you know, what we, what you would end up doing if you're wearing that virtual reality, seeing a bat's, you know, sonar view of the world, that you would see a great big, you know, it will be dominated by little insects that were very close to your face and, you know, only tangentially show you a few bits of where the walls were. Very unlike our own view of the world, which is really hugely about environment and quite a lot about humans and so on. And so I think, you know, taking that bat case, I feel like the answer is, well, VR is never going, never going to tell us what it's like to have bat sonar, but it can definitely give us an insight into the world that a bat perceives. And that's, that's a step forward, at least. It's definitely new. I mean, we, you know, we can't do that without, without that sort of technology. So we get kind of bat

light or sonar light. Yeah, you get, you get a sort of, you get an interpretation, you get a sort of interpret bat, where you get an idea of what it's kind of like to be a bat, but you don't get the inherent sense as if you'd been born with that, that set of sensory capabilities, as opposed to your own sensory capabilities. You're kind of mapping one onto the other in some sort of way.

There are, Peter, I'm sure you know of other examples of this, but I know, I know that there is a wearable you can get, a wearable, it's like a belt, which has motorised magnet-y things on it, where they vibrate. The one that's north vibrates. So the magnet that's pointing, the thing round your waist, which is pointing the north, vibrates, so that your body sort of knows where north is. And as I understand it, people who wear this all day, eventually get an intuitive sense that that's north. This is brilliant. And their sense of direction, and their sense of direction improves dramatically. Right. So they, they have now, on one hand, you might say, well, they're only experiencing touch. But I think it might be more accurate to say, no, they have a new sense, a technology enabled sense, because it, they, as far as I understand from, well, my interpretation of what people have said who've done this, is that it isn't that they're feeling a kind of vibration around that side of their body and thinking, well, that way's north. What they're thinking is, oh, well, that way's north. Yeah. It's coming straight through all of that. You're sort of, you're

sort of, you're hacking a bit of the sensory organ, like the tactile skin sensors. And your, the brain is finding a utility in that being consistent enough to be useful. And it just reprograms itself to go, this is a new thing I can use to tell me something useful about the world.

Because what I mean, what I was going to say, we wouldn't be, is there a way, you know, let's say we wanted the real bat, you know, full fat bat, right? That's what we wanted. As opposed to bat light is, is we would need to evolve. Okay. And I don't know if there's a way using, not necessarily VR, but I don't know, some sort of computer jiggery. I mean, as much as we do much as one, I think one of the reasons why we experiment on fruit flies, for example, is because they pass through generations really quickly, right? Because they've got such short lives. Is there, is there?

This is going a kind of whole breeding program. We don't experiment on fruit flies, but people do. A breeding program wasn't what I had in mind. I was, but I was, but I think, you know, there is the prospect of, well, okay. The question is, are our brains plastic enough that this is, and I think this is, I mean, I don't know any way we could answer this particularly, but are they plastic enough that have we, you know, have we evolved into a design for a brain that's, that's so general that if we happened, if we took that exact brain and we plopped it into a body, which had totally different limbs, didn't have eyes, didn't have ears, didn't have any of those things, would it still be able to create a coherent model of the world? That's the question. Now, I, I honestly think the jury's out. I don't think, although we tend to think of our brain, our brain is very specialized towards the human experience. Perhaps the way that humans have specialized, it might be the case that the specialization is generalization and that what we have really done is evolve a brain that actually, if we could add new sensory organs, we would just get on fine with that and we'd get on with it and we'd understand it and we would use that to simply enhance our

picture of the world. And there is sort of evidence for that. If people who, um, amputees or people who are blinded late in life or people who have a stroke and their brain learns to use the brain and the sensory organs readapt and to allow them to function at a similar level that they were before. So it's extremely reprogrammable, even within the life of one,

one, one, one, one specimen. Yeah. So, and that reminds me of another experiment where fitting people with special glasses, with mirrors that make the world look upside down. And after, you know, in a fairly short space of time, um, they adjust to it and it sort of feels normal and then you take them off and it feels weird for a while. Like with, like with sunnies.

If you're wearing, wearing the old aviator sunnies, like, like all of us do when we're out in Ibiza raving it up, then, you know, when you take them off, the world looks, looks, the colors change. Right. And you could pretty quickly, you adapt. Um, I think, so look. Well, hold on, before you continue. No, as long as you're not going to take it in a boring direction. No, I'm not. I'm not. I'm not going to take it in an interesting direction. Well,

I'm going to say, so, so far we're actually erring on the side of, um, you know, this is going back to the original question. Does VR, is anything impossible? Is nothing impossible with...

I'm glad you said that because that's where I was going, right? So well, well, let me just put... But we're erring on the side of it's all possible. No, let me set, because what we talked about is sensory information now, but what we haven't touched on really so much is the characteristics of the world. And this is, this is where it gets more fundamental, even than that. So, so now we, so what is our world like in fundamental terms? The things that we experience, things that we're capable of experiencing are basically require things like a world where there's physical objects and sound and light. Um, and three dimensions, particularly, that's one that we don't tend to even question, but three dimensions, three is a totally random number. It's perfectly possible to have a consistent four dimensional, 10 dimensional or a hundred dimensional world. Those things are completely mathematically possible. We just happen to have three. Okay. There's nothing special about three. Um, and, uh, you know, in a direction of time. Um, so there were, there were things that seemed to be, I mean, now it gets harder to imagine what it might be like to, um, to experience what it'd be like if there was no time or if there were no objects or if there were five dimension, but take the example of time. There's a game called braid, which is a two dimensional light platformer, but it's, it, it, it buggers around with the way that time works in a really clever way, you know, so that in some levels, as you move from left to right, time goes forward. And as you move the other way, time goes backwards and you can replay time in some of the levels and it does all these things with time. And at the end, you, you actually start to think about time a different way. And, and, uh, there's also a book, uh, which you guys might have read the time's arrow by Martin Amis. Uh, everyone should read it. It's only about 150 pages long. One of my favorites. Amazing. And when I read it, I had an, I, I found that after, you know, once I'd got really into it, I was listening to a story on the radio that someone was telling just to like a phone in show and someone was describing something that had happened to them. And I was mentally translating it into the other way round, you know, that the cause was happening after the effect. Um, and, and then I switched back very quickly. But the point is that if that kind of experience can fiddle with those very fundamental elements of our experience, because we think that's the way the world, the world is, well, maybe we can, maybe VR does offer the prospect of fiddling with those really fundamental elements. Yeah. Maybe we could

imagine experiencing VR. But I suspect just to bring about a sort of, uh, a kind of purist interpretation of the question. It's not, not everything is possible to comprehend because there'll be, there are limitations. So you still need, even in the game where time messes around or the ability to thinking about movies where they use time travel quite creatively, uh, it's, there is a certain limit at which point it becomes just too much, too difficult, too, too less fun. And you're going to take the VR headset off, you know, like Alice in Wonderland, imagine that on acid, just where everything, there's just no, there's just no sense or reason to anything. And it's just a, just a random bunch of flashing lights and colors and shapes and senses from all, from all different directions. That's not going to be any fun, is it? You may have some inherent, uh, logic and reason behind it for doing that, but there's a limit to your capacity to comprehend all of this, that it's just going to cause you to give up. And I think the, the, the three,

the dimensional limit is particularly, um, uh, hard. I don't think, I don't think it's possible because of the fact that we have all of our senses in particular, well, our visual sensory information is two-dimensional. And from that two-dimensional information, we construct a three, what we feel like is a three-dimensional view of the world. But all it really is, is flat pictures, flat pictures that we construct, the idea of three dimensions, because we appear to live in a three-dimensional world, but it won't be possible, I don't think, to get to a, to have an intuitive understanding of what it'd be like to have three-dimensional eyes perceiving a four-dimensional world. And there might be listeners out there. Well, we do have a bit of 3D perception because we've got two eyes. I know, but no part of that is, so a three-dimensional eye would accept, would accept light coming in from four-dimensional objects in, um, in three directions. Whereas we accept light coming in from the 2D surfaces of three-dimensional objects coming in, in, uh, essentially two directions. Yeah, but in near field, you've got 2D, you've got 3D perception. Well, no, no, it's faux 3D perception. What it is is pretend just like 3D cinema with the two, yeah. So what it is, it doesn't show you what's behind the object you're looking at. Right, so a genuine perception of 3D, if I wanted to perceive a 3D object like Fraser, I would see all of his insides at once. And that wouldn't be an advantage in anyone's language. The point is that, you know, we don't have, we just simply do not have the apparatus to process three-dimensional data about a four-dimensional world, let alone 99 dimensional data about a hundredth dimension. But,

just a point of correction, I'm beautiful inside as well as outside. Yeah, mainly, mainly. Well,

I see you're more beautiful inside than you are outside. So the readers can read in a lot about you, but, but, but given the, the general, uh, the general flexibility of the human brain, is there a way that like the belt that you wear to tell you which way is north? Is there a way of hijacking the senses to give you that capacity to perceive four dimensions that we've just not tried?

Now, indeed, and I, and, but, you know, my, my only thought is that the processing capability of a, of a four-dimensional brain would be necessarily just so vastly bigger that, you know, we would imagine, imagine taking a slice, a two-dimensional slice of a human brain and living in a two dimensional world and asking, well, what, how much can we get this two-dimensional brain to process? It's tiny, uh, compared to the amount, it's not just, you know, some fraction, it's a tight, it's almost, it cuts off almost everything that we can do because of the way that our neurons can ride around each other and, and go off to lots of different directions at once. And, and in a two-dimensional brain, you can't cross over stuff. You can't go from, you know, you can't go through other connections, whereas with three dimensions, you can. And, um, so, you know, a, a genuinely four-dimensional brain would be, uh, would give us, if, you know, any being with a four-dimensional brain would have an inordinately vastly greater ability to experience things. And we simply can't do that for the same reason we can't, we can't experience what it's like to

be more intelligent than we are. If we, if we relax the constraint that we're using only the processing power variables of the brain and we out, we outsource some of that processing capability. So there's, so, um, there's a, some sort of computer there that's look, analyzing what you are, what your current field of view is in this four, this higher dimensional space and only processing and giving you access to that stuff. So whichever way you pan in four dimensions still is consistent. Potentially you could, you could bootstrap that four-dimensional perception.

And possibly, yeah. I mean, again, uh, this is a surprisingly hard question to answer because you would think the more fundamental something is, the easier it would be to think about, but actually we're really, uh, delving into the, the very sort of most fundamental elements of human experience here. And luckily we've, we've nailed that now. Well, we have, yeah. Finished, done.

Lots of unknown unknowns. Um, okay. All of this is, we're talking about technology being potentially mind-blowing and, you know, VR, wonderful things that it can do, right. Um, or will do. Um, and there, what's exciting about this is there will be within our lifespans, um, there will be stuff, um, happening with VR and, and other technology, which will just blow our minds. It's wonderful, right? Hopefully not literally blow our minds. Hopefully not. However, I'd like you to cast your mind back and think about a point. See, can you tell me a point in the past when your mind has been blown by something cool that's technological? Um, I mean, and just to start you off, I remember, uh, being about, I don't know, 20 years old perhaps. Um, and there was, who was the Sega hedgehog bloke, Sonic the Hedgehog, right? And I think it was the second iteration of that where he goes on this kind of 3d journey through in, along a tube that goes around.

Yeah. He's collecting rings with his, with his mate, uh, that, that, that squirrel or something. Yeah. Well, he always did that in profile, but then I don't know what the sort of the name of I think these are different levels on the same Sega, Sega Mega drive. Yeah. Yeah. So you've got the 2d thing where they're running along tails, I think the guy was called and, uh, You're right. It was the same, but

Yeah. And then he goes into the third dimension. Right. Well, I remember seeing that when I was about 20, I must admit, I was probably slightly under the influence. 20? Yeah. Well, this is, that was probably about Were you like 55 or something? I'm 45. So this would have been about 92, something like that. Yeah. Um, yeah. Why is this surprising? But, um, but I remember that I was just, it just freaked me out. I'd never seen anything like that. And I was just, I was just so, um, breathtaking. It was amazing.

Peter? Uh, so I don't know. I, I've, I've kept up with technology unlike Grazer and Nick here. So I've not had these kind of revelation epiphanies about, wow, that's amazing. So I kind of grown up with it, you know, it's been very incremental for me. I've not sort of seen these big changes because I've just been sort of, the changes have been incremental in my perception.

Fair enough. Uh, I mean something I, I mean, I noticed recently is in my daily work, I use, um, uh, I mean a Skype, it's not Skype, but a Skype like platform every day, all the time, five, six, seven times a day. Uh, and we could be, I mean, I say Skype, but it could be Google Hangouts could be whatever. Um, and actually, you know, half a decade ago, a decade ago, that was still quite, um, okay. It wasn't impossible, but it was still relatively unusual.

Yeah. I suppose if I look, I mean, if I sort of take that retrospective view, then some of the things we do now just routinely day to day, like being able to talk to anybody anywhere in the world for nothing, any time of day is totally incredible. Um, uh, and yeah, so the, uh, but, but maybe another podcast, why do some really annoying things like email still persist so much? Um, but yeah, but yeah, so retrospectively, yeah, there's some pretty

amazing things that we do and take for granted now. If you'd have shown us talking as we do on Google Hangouts amongst each other, if you'd have shown that to our seven-year-old selves. When we're carefully scripting this podcast. Exactly. Uh, if you'd have shown that to our seven-year-old selves, we'd have been going, great, the future's coming. Here it comes.

This is what I'm saying to my kids all the time. You're living in the future. You know, we are in the future. There's nothing more that we need really. I mean, we've got space travel, we've got space X. What do you want? Free energy? That's coming. They've got all these solar panels everywhere now. It's just, nothing's missing really. There's almost nothing from sci-fi we haven't already got. It's incredible. Well, so on that sort of note, um,

you've heard it from Aleph Insights. The future is now. The future is here. Everyone stop innovating

right now. Yeah, that's it. We're done. Otherwise we'll go too far. Yeah. We'll tip it over the

edge. Okay. Uh, we'll stop there. So thank you as always, uh, for listening to the Cognitive Engineering Podcast. I'm Fraser McGruer. I've been here with Peter Coghill and Nick Hare of Aleph Insights. Thanks for listening until next time. Goodbye.