Merging Worlds: How AI is Transforming Virtual Reality

John Gauntt [00:00:06] This is Culture & Code, a podcast about creativity and artificial intelligence. I’m John Gauntt. Culture & Code explores innovation across storytelling, technology and audiences to help professional creators collaborate better with AI and each other. Support this podcast by subscribing to our newsletter. Visit our website https://cultureandcode.io.

John Gauntt [00:00:32] Okay, so I’m speaking with Rylan Pozniak Daniels, who is a student in Southern California and also a VR and AI developer. So Rylan, thank you so much for speaking with us today. Okay, let’s start a bit with more of your particular focus and background. Right now.

Rylan Pozniak Daniels [00:00:52] I’m a VR AR operator and a futurist. I’ve been involved in like making new kinds of media experiences in VR/AR since 2015. That’s almost nine years. And currently I’m also working on launching three new VR and AR VR experiences in the next three months. So one new app every 30 days and use of this and each VR experience will incorporate generative AI in a significant way. For example, you’re talking to a generative AI avatar character and this is all built on my, you know, experiences doing VR/AR working, not just like NASA and Apple and Niantic. And I’m trying to see how we can push the boundaries of immersive experience through virtual yet augmented reality headsets like Apple Vision Pro. And Meta Quest  to make A.I. the tool that empowers them. I had launched the first Apple Vision Pro app to come out on day one of the App store called AR Lingo, which was an AI character driven language learning to occur where you can speak to AI characters that walk train you through how to speak different languages. And so I’m building on that to make new kind of journey experiences with VR.

John Gauntt [00:02:06] And you said you’ve been involved with it for nine years. So tell me, well, what was it? What was the initial draw where the first time you were exposed to something like VR/AR, for which I know you and I are two, nine years ago is gone. But what grabbed you in the gut and you said, I’ve got to do this?

Rylan Pozniak Daniels [00:02:23] Well, I think what’s so exciting is that in my opinion, VR/AR is like the ultimate form of human expression and showcasing human creativity. You look at other isolated technologies like even, you know, a machine learning models by themselves or even blockchain or stuff like that, but they’re all really about the kind of data and the machine behind it, but with AR and VR. These are very, you know, visual. Auditory is sensory experiences that, you know, I’m always inspired by worlds building and, you know, building stories and worlds that can give people all new kinds of experiences. And so that’s what really attracted me to that.

John Gauntt [00:03:03] You also mentioned that you’re now incorporating generative A.I. into your work. You know, it goes without saying that VR ARre are very advanced technologies to begin with, but what is it specific about generative sites where you’d like I want to put these together, you know, peanut butter and chocolate.

Rylan Pozniak Daniels [00:03:22] Well, I think what’s so interesting about what I do know with the intersection of AI and VR, AR also known as XR, is that I think augmented reality and virtual reality are sort of like the the best user interface for experiencing A.I. In other words, we don’t have the popular Chat GPT interface where it’s like a text based interface or interaction with the agent. But when the AR VR glasses that we wear, you know, all day will become democratized and accessible to the masses. I think apps and experiences and even games that incorporate like human computer interaction, meaning you’re either speaking to an AI agent or you’re even maybe talking to or even, you know, you could even touch an AI character in a virtual world or even place an arbitrary character in your physical living room through augmented reality. I think those are going to be sort of the ultimate ways to interact with an AI agent in your physical or digital realities. And I would but work now to enable those kind of use cases.

John Gauntt [00:04:31] What are some of the still the gating factors that you see in hardware design that we need to overcome so that AR, VR that’s also enhanced by AI becomes as common and ubiquitous as, let’s say, video on your smartphone.

Rylan Pozniak Daniels [00:04:48] I think that one of the things is the form factor that we can perhaps just like we take our phones out to, you know, as we go about to the world, in our schools, in our workplaces. I would say even just on the street, I think, by the way, that those are some of the most transformative use cases for XR and AI is impacting education and health care, productivity and even social experiences. I think that’s what the hardware sort of has to catch up to. Our vision that developers you can can already, you know, illustrate and create through, through apps and experiences, but at the hardware sort of needs to catch up to that form factor like enable those experiences. So a form factor could be like for example you could have just a normal looking glasses that actually have an augmented reality capability. Little bit like what’s a matter I recently showed at there I’m a I’m at the Connect conference. Where are the Orion glasses that just looked like a very close to a regular sunglasses but actually have full augmented reality visuals that can actually place digital characters and digital content in your physical environment. And you know, I think the use cases for education, health care, creativity are very profound. I’ll give you a recent example of a hackathon I did at the Stanford XR hackathon where my team and I built this A.I. agents that you put on the MEata Quest headset and you could actually talk to historical figures from ancient times, for example, like a pharaoh from from ancient Egypt or Napoleon. And these are historical figures are manifested in the form of 3D avatars that you could see in virtual reality. And you could use your voice to ask them any questions. What was the kind of the dream of any  you could see, like ask, you know, a Napoleon? What was the secret to to a winning the war so that you did you could ask the Pharaoh, how did you actually build the pyramids? And I think that’s an example of a use case that AI and XR together could combine, be able to do some kind of contactor in the headset.

John Gauntt [00:07:01] So so let’s turn to, you know, how we met, which was at L.A. Tech Week. They had they basically kicked it off with this hackathon that was happening in Santa Monica and was called Cerebral Beach. This was put on. There were like 400-500 hackers there and we came together into a team. First question I got for you is what attracted you to this event in general? Because I could see from your website you’ve been a multiple hackathon, so you know what? What attracted you to Cerebral Beach?

Rylan Pozniak Daniels [00:07:30] Well, I had done a lot of hackathons, maybe. At this point, I don’t know if it’s something like maybe 40 or 30 hackathons in many different areas. We’re more like excited. We are focused on more just a general stance. But I mean, I’ve done a little like but you know, like a Stanford hackathon, as I mentioned to MIT. See a reality how less we will be together. And that was interesting because it was really focused just like Gen AI. And it’s actually it’s really good timing for this hackathon because we’re really like right now, this year is when finally the tools for making A.I. content, including generative video content, are, I think, finally sort of at the minimum point where you can highlight like a significant level of, you know, control and expression, not just it just being your fidelity and the quality of what you can create and is, is finally at the sort of minimum, of course it’s going to get even better than that. Yeah, seven years to come. But at least this was a really good turning point and inflection point where now that AI generation schools are like mentally runway and many others are all coming together before, you know, I think this hackathon was a good opportunity to experiment with and execute on a vision fora generative project. That was a really good opportunity to do that.

John Gauntt [00:08:54] At the hackathon, they had multiple tracks that, you know, that range from, you know, media entertainment all the way to I think they called it 0 to 1. But what what attracted you specifically to the media and entertainment track?

Rylan Pozniak Daniels [00:09:08] You know, like again, were also originally attracting me to to build in technology and your I know initially you know nine years ago was the intersection of the arts and humanities with the technology and that still is is the driving force for what I think is kind of the ultimate potential for using technologies like AR/VR. And then into the AI is that aspect of showcasing a human’s creation and vision through like enabled through a tool of technology and through a medium of new technologies. It’s not just the tool, but it’s also new kinds of mediums to experience the new visions. You know, we had books from Gutenberg historically to, you know, the invention of cinema. And these are all evolving technologies that enhance the medium of storytelling. And now that AI is in the fold, I really wanted to experiment with how can I be this medium for not only making stories but experiencing stories?

John Gauntt [00:10:06] Well, in some of my research, I also saw that the you’re a musician, musical training and also musical expression is also tightly tied into a lot of kind of the problem solving that we do in general. I mean, one of the things when we talk about the democratization of tools is not just simply, Hey there, available, go play kids, but it’s also that idea that you don’t need to ask permission.

Rylan Pozniak Daniels [00:10:34] Right now, right?

John Gauntt [00:10:35] You can come in and just say, Hey, it’s all new, let me try something. So one thing I’m curious about is that if you were to look, you said you’ve been to like 40 different hackathons. If you were to look at sort of the problem solving mentality that you take to a hackathon and the problem solving mentality that you take to composing a piece of music, do you see what parallels do you see?

Rylan Pozniak Daniels [00:10:58] I think there’s actually a lot of parallels, I think. First of all, they both have all types of creative problem solving in the sense that they’re well, but they involve a lot of iteration at prototyping. And what I mean by that is so with a piece of music, you might reiterate the same themes of the same sort of parts of the musical score multiple times. You kind of try it out. Maybe you could. You know, I actually when I was a boy, I had to have little practice sessions with the musicians there to to sort of see how it would actually sound in a real life instrument. Because I was writing sheet music both on like paper and the computer. I could only imagine a little bit either playing a little bit on the piano or kind of the computer simulated how that would be. But it’s sometimes different when you’re actually experiencing it with live musicians. And so that did require a lot of iteration and sort of like a reforming of the same melodies until I got it right on the musicians. And I think similarly with Hackathon, of course you’re very limited by time, but as much as possible you may sort of rework and, and iterate on the idea and say, you know, 24 hours or 40 hours or however long it is until you get some sort of a sort of a MVP or a minimum viable product, which is what they often call in startups, like kind of first sort of usable version of the product or the output and results. I think another aspect is the collaboration aspect. So with no as I saw the L.A. Phil and working with musicians, conductors and all kinds of people, and in the orchestra there, there’s so much, you know, different kinds of roles that have to be accounted for, different only different musical specialties and instruments, but even different, you know, genres and different perspectives of different people who have different musical backgrounds. Same with a hackathon. You have to bring together a team sometimes of people you’ve never worked with before.

John Gauntt [00:13:08] Like us.

Rylan Pozniak Daniels [00:13:08] It’s like, yeah, a designer, the coder, or perhaps a marketing person, a kind of, you know, a visionary person. Like, like it takes a lot of collaboration to make this kind of stuff work.

John Gauntt [00:13:21] Yeah, I mean, turning turning back to Cerebral Beach, we met at a mixer on a Saturday morning and then very quickly developed a team. There was you, there was me. There was a woman who was a dev. There were two other people, one of which I believe was working in the ad tech space. You know, again, there were no there were no professional filmmakers.

Rylan Pozniak Daniels [00:13:44] On our.

John Gauntt [00:13:45] On our team. And we very quickly scoped a story and then we just basically went off on it. So with that, with with that set up, let’s talk about your experience not only in this hackathon but in general and these hackathons that What did you find about Cerebral Beach that was similar to other events you’ve been to and what do you find unique about it?

Rylan Pozniak Daniels [00:14:13] Yeah. So. You know what? The energy was so interesting around the kind of excitement of these new, AI Technologies at Cerebral Beach, which I think that’s especially coming sort of as like the introduction to LA Tech Week, which itself is a profound gathering of creative minds and technologists who to, you know, have interactions around the future of technology and, and, and, you know, human creation through making technologies, right. I think that was a profound atmosphere for sure. A cerebral beach for me, the previous hackathons I’ve been to. It’s also a very a very profound atmosphere around, you know, sharing ideas, interacting with, you know, sometimes of like minded people who shared the same interest in what they want to create, perhaps coming from different backgrounds, as we said earlier. That’s kind of the magic is the kind of interactions, the interdisciplinary interaction with design technology and the business all together.

John Gauntt [00:15:18] And then additionally, with most hackathons, I’m assuming or like Cerebral Beach, you’d have some industry sponsors who are going to say, you know, not only are we supporting the hackathon, but also we’re supplying the tools. And there was a same thing with Culver Cup. And usually there is there’s a requirement in your submission that you use a particular set of tools?

John Gauntt [00:15:42] Which, you know, to me is a transparent kind of easy social contract to understand. Well, what do you find in general about hackathons? Doesn’t need to be cerebral beach, but in hackathons kind of good, bad and ugly about having, you know, these are the tools you must use and you can use other types if you want.

Rylan Pozniak Daniels [00:16:01] Yeah, well, I actually found that sometimes. You know, of course, the principle of what is imagination and creativity is sort of, you know, boundless, unbounded freedom to create anything. But I found to my experience during during hackathons that. You know, the constraints actually fuel creativity, in fact, because you have to sort of maybe think outside the box to be very resourceful with what you’ve got, you know, not only with the requirements of what technologies or tools or even themes to build with, Right. Because a lot of hackathons sort of asked you to build around a certain core idea or topic the seem in this case in the media entertainment, but also the time constraint, right? You may only have 24 hours, maybe 48 hours and depending on the hackathon. And so the tiering tools, all that is a constraint. But having said that, many that could actually get you faster to a very clear solution because sometimes the, you know, perhaps simpler, but more focused ideas end up being a stronger end result than maybe the more ambitious. But. More, you know, scattered plenty for ideas that sort of try to do so many things moderately well versus, you know, focusing on the one particular thing quite well, I think for for the hackathon we did about AI,  I think we really were able to to focus in on, you know, telling a carefully crafted story through generative AI visuals and, you know, even making that one together. I think that there is a beginning, the middle and an end. There were different types of shots from, you know, wide angles. The closeups even had a consistent characters that were consistent throughout the the course of the short film. So I think that was certainly successful. And just, you know, instead of just playing around with a lot of tools and just kind of, you know, mixing things together to hopefully come out with the result, I think we were very intentional beginning about what we wanted to to create.

John Gauntt [00:18:18] Yeah, I find that effective because it does seem that and I think I put it in my feedback to them is that it seemed like we were combining a sprint and a marathon. You know, you, you sprint form your team, you sprint to to come up with the idea that you’re going to execute and then it’s a marathon to make that idea actionable. Right? It’s kind of you got to you got to know how to toggle between those between those two areas. So you know you know, going forward, what are some of the things that you’re looking for was say, over the next year or two that are going to tell you that we’re tracking towards better democratization of the tools, meaning that people who come from outside traditional industries and that doesn’t mean that they’re often left field, they’re usually in adjacent industries can start using these tools to actually make professional level content that can give them a living, not just demos, that to say, Hey, this is what the tool can do. What are some of the things you’re looking for in the news cycle that are saying, Hey, we’re advancing on hardware, There are potential customers, there are different types of partners, what are the things you’re looking for?

Rylan Pozniak Daniels [00:19:35] Well, one of the things we’re looking for is where are the sort of sectors or even types of technologies that technically are available today but may not necessarily be completely widespread until maybe a few years from now? I think one of those things is in fact VR headsets. So I think that there are you know, I think it’s been in the millions of handsets, especially from the likes that are being being sold to to to people. But it’s not yet in the billions. It’s not yet obviously as as used and and as much as the phone particularly also there’s a phenomenon right now where most of the content out there in VR and AR is a gaming focused. So it’s it’s the video games and entertainment experiences, which, by the way, I’m a huge fan of. But in order for that to really become democratized, the kind of experiences should basically have a broad range of appeal to things like education, health care, productivity, social experience. And I think that’s come in and that also. The type of thing I’m working on. And I think the AI piece is very interesting because as I said before, I think I think A.I. is the, the technology that will where AR and VR is the ultimate way to experience AI and to interact with AI.  I think that that’s one piece is the Air VR to be the way you experience A.I., but also that A.I. is dealing with tool to make. AR VR experiences that incorporate AI. So like you might start with this workflow that you generate assets and even code of certain cases that can be used for a VR AR experience. And then some of that your code and assets can actually enable an interactive AI avatar agent was present in 3D that you can talk to and interact with through a VR and AR experience. So that’s some of the things I’m most working on, you know, But the first app I’m making actually, or the three months holograms or by making new app every few months is this Dungeons and Dragons type of role play panacea? Some ways for those uninitiated like D&D is a type of way to your friends, get together and tell a story and then usually it’s like a medieval fantasy environment. And then the story continues to evolve as the players actually choose what they want to happen next. And I sort of consider that to be a perfect use case or for AI actually for an application, because I can be very creative, right? I mean, people sometimes complain that it hallucinates, you know, it doesn’t give actual information about or something like. D&D , but isn’t that absolutely what you want have to happen, Right?

John Gauntt [00:22:25] Yeah, that’s a feature.

Rylan Pozniak Daniels [00:22:27] That also going back to constraints, right? Looking at what are the constraints today. And I and I’m try to think, well, what if we can actually go what if I actually use that constraint? The fact that AI is very imaginative sometimes actually use that as the feature and the problem why you’d want something like a Dungeons & Dragons immersive experience. And so I think about that of the way I kind of think about like a, a creativity these days.

John Gauntt [00:22:53] Just to wrap up first, how do people find you feel.

Rylan Pozniak Daniels [00:22:56] Free to to connect on social media by LinkedIn, by Twitter, by search you know Rylan Pozniak Daniels R-Y-L-A-N P-O-Z-N-I-A-K D-A-N-I-E-L-S, I also started a new discord server where I’m actually posting daily updates on these three apps in three months including this first one, the Dungeons and Dragons generated thing. You can find the link to that on social media as well. And yeah, let’s build together.

John Gauntt [00:23:27] Outstanding. So, Rylan Pozniak Daniels, thank you so much for speaking with us today.

Rylan Pozniak Daniels [00:23:33] Of course. Thank you for having me.

John Gauntt [00:23:37] You’ve been listening to the Culture & Code podcast, Creativity and Artificial Intelligence. You can find us on all major podcasting platforms and our website culture & code.io. That’s one word cultureandcode.io To support this podcast. Subscribe to our email newsletter for a weekly round up of analysis news plus AI training and growth opportunities for creative professionals. It’s all there at cultureandcode.io Culture and Code is a creative studio and professional education provider that’s part of the Augmented City LLC. I’m John Gauntt. Inviting you to the next episode of the Culture and Code Podcast. Thanks for listening.