Magical Diegesis

A Theory of Design for Virtual Reality.

19 min readFeb 23, 2021

Virtual Reality is the next step of the internet and immersive technologies. Print, video, web, mobile, desktop; each technology has differing qualities and limitations based on its physical components and digital frameworks. VR is still new and being actively developed in both its physical and digital components. It is malleable and shifting, I want to design in VR but how do you do that, especially now? What are control inputs are you working with, what space are you working with, what is accessibility like, are people sitting or standing, what is VR skeuomorphism, and a thousand more questions?

I have so many questions about VR design, so I started to try and find them for myself by experimenting in VR. It took me about a dozen designs before I realized that they were bad and that something I was doing was wrong. This is my process figuring that out and developing a conceptual model and theory of design for virtual reality.

VR, A Magically Mould-able Digital Space

VR exists in an odd, new space between existing mediums. It’s neither fully physical or digital, but exists in both planes, creating a whole host of interesting and difficult problems to solve. As a result of this dual existence, when I started to experience VR and subsequently tried to design in it, I was pulling from the wrong designs and patterns. I tried to pull from web and industrial design respectively; one a digital space and the other a physical; it only made sense at the time. I haphazardly pulled patterns from each field’s design ethos and mental models. This path led me to a series of VR designs with which I was thoroughly unhappy. The designs lacked… focus, they were a Frankensteined monster of both worlds while not actually existing in VR.

From those failed designs, I realized that it’s not enough to just pull patterns from other fields. At first I thought industrial design patterns would work because they exist in a physical 3D space, I tried to apply concepts like ergonomics but the designs ended up feeling wrong and lacking in an understanding of the not-fully-physical but pseudo-physicalness of VR. In the same way, web design patterns felt like they would work in VR, but trying to apply buttons and infinite scrolling patterns that were developed for use on flat screens inside web pages just didn’t work.

When experimenting I realized that I had jumped too directly into the weeds, so I took a step back. I realized that for the other mediums I work in I have mental models, experience and philosophies that help guide my work. The reason I jumped to translating patterns from other mediums is that patterns are established standard practices, but they were developed for those mediums using a mental framework specific for that medium. I realized I had to make my own framework if I was going to be able to design in VR.

Enter Magical Diegesis.

Magical Diegesis; Two Aspects of a Whole

Magical Diegesis is a theory of design in VR. In the hierarchy of knowledge it sits above concepts like Gestalt, Human Centered Design and accessibility but below things like web standards, material sciences, typography etc. Its intent is to be a mental framework in how to design in a specific medium and how that medium, that context, alters how you design. This theory comprises two foundational tenets that stem from my VR research, established design theory, and experimentation.

The Magics and Diegetic’s of Magical Diegesis

My goal is to create a mental framework, a set of tenets from which I can do work in VR from. I have chosen two of Dieter Rams 10 tenets that are especially relevant to VR design and then two more tenets that shall respectively form the interpretations of the terms magical and diegesis. First, the Dieter Rams principles that are relevant:

1 — Good design is Innovative

The first principle from Rams, Good design is innovative, is particularly prescient here as we are discussing an innovative technology.

“…innovative design always develops in tandem with innovative technology, and can never be an end in itself.”

While I have espoused the wonders of VR it is not so unique as to depart from the established design canon nor is slapping VR into your product or design a solution unto itself.

4 — Good design makes a product understandable

The next principle that pertains to VR is the 4th principle; Good design makes a product understandable.

“It clarifies the product’s structure. Better still, it can make the product talk. At best, it is self-explanatory.”

This principle could almost encompass all of the UX profession. In VR this principle provides strong guidance for what is lacking in. A medium is defined as much by what it can do as what it can’t. In VR the can’t currently means physical spatial issues and haptic feedback. By holding up this principle I am reminded that much of the basic design affordances I am solving for are making designs understandable.

The interpretation

The previous two principles I have taken as stock as they are important to keep in mind, but the next two I am going to interpret in a different fashion to fulfill the needs of Magical Diegesis. My interpretations are not meant to trump the original principles but are additions to each principle.

Good Design is Unobtrusive.

“Products fulfilling a purpose are like tools. They are neither decorative objects nor works of art. Their design should therefore be both neutral and restrained…”

Dieter Rams is talking about physical products here and how designers create objects which become tools that then become extensions of users, their purpose becoming the users intent. In the same way, a UX interaction can be thought of as a tool, one that should be both neutral and restrained. Design is unobtrusive, both in an aesthetic sense and an Interactive one.

If we keep following this analysis, a design’s goal becomes interactive immersion. This holds generally true and even more relevant in VR. The great thing about this is also that theater and film already have terminology for immersion that fits exactly what I’m looking for; Diegetic. Diegetic means “in-world”. In a movie when you hear the soundtrack but the characters don’t, that’s a non-diegetic sound. But if the music is playing in the movie and the character turns it off by turning off a radio, that’s a diegetic sound.

Whether you realize it or not you’ve experienced diegetic design and non-diegetic design, let me give a common example. Our world is still in the transition from print to digital, have you ever went to look at a restaurant’s menu online and it was just an image of their menu. I would consider this design non-diegetic, the menu was designed to be viewed printed out not viewed on a phone. And sure you can still use the menu but it does create some problems and dissonance.

Good design is contextual; the pamphlet could have been the best design in the world, but on a phone it doesn’t matter. This isn’t a new concept, I was taught to design to my medium in school and most designers get to a point where they do this subconsciously. But when designing for such a new medium it’s vital that we re-address the philosophical specifics so that we can construct a framework to begin to understand why good design is diegetic and why that’s so specifically important to VR

In VR we have control over the world we see unlike ever before, this is exciting but also very dangerous. We did not evolve with the ability to teleport yet in VR it happens all the time. VR is only now becoming a consumer viable product because people have worked for decades to improve hardware and software so that basic issues like resolution, latency, etc don’t cause people immediate distress. But the psychological effects of VR are still very much unknown.

A break in immersion is a disorienting experience in the physical world, but in VR it can be dangerous. In designing worlds in VR we are creating new realities to experience, realities where physics are malleable. We’ve evolved to believe what we see, and our brains’ ability to remodel our established mental models is much more malleable than you realize. It may not take long to become accustomed to the ability to teleport in VR but when you leave VR you can’t teleport. This dissociation can be extremely uncomfortable and even painful.

I have had short but severe experiences with dissociation after several longer VR sessions, and I can tell you it’s not something I ever want to experience again. I’ve also experienced several games in VR that made me dizzy and nauseous. I think VR has much to offer but anyone creating in the medium needs to understand that there are very real physical and psychological safety concerns that need to be taken into account. The ability to create users’ realities is incredibly powerful and should be handled intentionally by designers.

Good design is unobtrusive because good design is diegetic. Diegetic design is design that is contextual, that takes advantage of the inherent qualities of the medium. It’s not a design that is translated from one medium to another, but that is instead created for the medium it exists in. When a design has become decontextualized it is no longer diegetic to its medium, it’s no longer good design. Designers have the responsibility to build immersion through the intentional creation of a design or interaction.

Good design is Honest

“It does not make a product more innovative, powerful or valuable than it really is. It does not attempt to manipulate the consumer with promises that cannot be kept.”

Rams intends this principle as a guard against false advertising and feature creep. The 10 principles don’t talk about material constraints in the context of design except in an environmental way. I’m going to make an interpretation of good design is honest from a material’s interpretation. Rams’s principles attempt to encompass design into a series of 10 agnostic statements, much of my early design education attempted the same abstract and agnostic language. Lots of “form follows function” being beaten into me. The problem with teaching in this abstracted way is that if form follows function is only an abstraction, material form often gets ignored.

When concrete was introduced as a widespread building material at the start of the industrial revolution it was considered a “Modern” material. Concrete can support far more weight and in a different manner than any materials that had been previously available. Initially concrete was used by architects to make thinner arches and supports, all in the same style as they had been making them, just thinner and stronger with concrete. Soon though it began being experimented with by curious architects who realized that with concrete you could build forms previously not possible. From these experiments the most wondrous and weird structures got made, out of concrete. Then once its qualities were understood better, it was finally utilized for structures and forms that only concrete could accomplish by the wider profession. The understanding of a material, and it’s intrinsic qualities spawned a revolution of design and innovation.

This is how the landscape of VR feels right now. There are plenty of people simply building better arches in VR, but then there are people who are making wild and wonderful experiences. As impressive as it is to have a car or plane simulator in VR, it’s more impressive to do something that you can only do in VR. An experience that can only exist in VR because it’s taking advantage of the medium’s unique intrinsic qualities, something like a spaceship fighter simulator.

This is why bad design and good design are contextual, and a design must be honest to the qualities of its medium. When form follows function, it’s usually a question being applied to an established problem and in that way it’s a very valid guideline. Because from that line of questioning comes the decision of which material to use, dictated by the function that the form needs to follow. But here we are trying to ascertain qualities intrinsic to VR design, so we must work solely around form, as we have no particular function in mind. And when discussing solely form we can only talk about the qualities of the medium in which we are discussing, Virtual Reality.

What are the qualities of VR?

VR has all the malleable qualities of the digital world, but instead of constrained by being viewed through a 2D screen, you are situated to interact with it in a spatial 3D manner. Currently, with some VR setups you can move inside a small area, and you usually have controllers of some kind that can give haptic feedback but do not provide a solid resistance to press against. This is the current state of VR, but the industry is constantly pushing the boundaries in two directions; upgrading the visual immersion qualities of VR through better hardware and software, and trying to enhance sensory feedback. The qualities of VR will adjust as technology continues to progress but the core of the medium has already emerged, the joining of physical and digital realities in full immersion.

VR is a digital-physical space with limited means of haptic feedback and interaction. This is an incredibly malleable medium. With digital tools, from websites to video games to apps, there exists the ability to create any world one can imagine, and now with VR you can exist in those worlds physically and interact with them. Not only does VR mean world building but also the augmentation of human abilities: in VR you can fly, go to space, control time, visit the center of the earth, be a giant, be an ant. This is the amazing part of VR, the inherent creativity and malleability of the digital medium paired with presence and immersion of the physical creates a true extension of the human senses. It is for lack of a better word, magical.

Arthur C. Clarke’s quote about sufficiently advanced technology being indistinguishable from magic might be trite but I find it to be very true here. VR has gotten to the point where technology is fast enough and advanced enough that when you come to a cliff edge in VR, you pause. You pause because the instinctual part of your brain very much believes that cliff to be real, because everything you are seeing about it aligns with how our eyes have evolved to process visual information. As far as the more basic instinctual parts of our brains are concerned, that cliff you are seeing in VR is as real as the edge of the Grand Canyon. That in itself is sufficiently advanced to be magic, but we also have the ability in digital technology with code to not just replicate the physical world, but extend it in ways we can’t yet imagine, and that can truly create magic.

Good design is unobtrusive — Diegetic

Good design is contextual and immersive. In VR this means creating designs for VR and not porting designs or design patterns blindly from other mediums.

Good design is Honest — Magic

Good design utilizes the intrinsic qualities of the medium it’s being designed for. In VR this means taking advantage of pseudo-physical space and digital malleability.

A Wild and Wonderful Lack of Consistency

Now that we have a framework let’s return to my failed design attempts. I realized I needed more context, so I began to research and analyze the current VR space. I was hoping to see what standards and patterns might be starting to emerge; what I found was a wild and wonderful lack of consistency. There are a lot of fascinating and great design patterns emerging in VR right now, but also plenty of bad ones. I realized that lots of people were having the same problems I was and were solving it in all kinds of different ways. This is great, a new medium is a playground for the creative and curious and all these different solutions showed me what a flexible space VR is going to be. I’d imagine it’s what designers felt like during the early days of the web.

I focused on the games (VR is mostly games right now) and experiences that were “good.” My criteria being: “Does the game feel natural, is it a seamless and delightful interaction?”. The first time I realized you could fly in Richie’s Plank experience is a prime example of good: I actually felt like I was flying — it was amazing — and using the flying controls felt like second nature. This doesn’t necessarily mean that I understood the interactions with no instruction, but they were quickly learnable and felt frictionless.

Games that Felt Correct
(not a comprehensive list)

From here, I wanted to know what separated the bad design from the good design. The bad design in VR varies from simply not being good to physically making you ill. My definition of good in VR was at this point essentially immersive but I wanted a better way to qualify that so I put together a series of questions.

Cosmic Trip is a game that hits it out of the park on what I was looking for. The game feels natural, with all interactions part of a cohesive whole with logical consistency. At the same time the interactions are unique, they take advantage of the qualities of VR in a way that you can’t replicate in other mediums. The interactions are quickly learned and don’t cause undue stress. The trick of VR is that your eyes are seeing things and then telling your brain that they are real, but they aren’t, at least not physically. Then when you go to press that button and your hand phases through it, you have a problem. When this happens you experience such dissonance as to break immersion. Immersion is a fundamental experience of an interaction and cognitive dissonance is almost never desirable.

A few more extrapolations of good VR design.

Richie’s Plank Experience: When Flying in VR, you are usually vertical which can feel weird, but Richie’s gives you such subtle controls simply based on the direction of your controllers and the trigger sensitivity that it is instantly intuitive. In addition, the timing of the music with your flight is brilliant.
Fantastic Contraptions’ interface is built off of being bodily accessible, quick and seamless. Every interaction is within literal reach and your play area, which is designed to be the same size as your real room. Every aspect of this game feels like the creators asked themselves, “how would this exist in VR?” And as a result, excellent innovative interfaces abound everywhere in this game.
SuperHot, is a great example of a game that could only exist the way it does in VR. The fact that your body movement so perfectly becomes a way to interface with the world in such a non traditional way is pure thrill.
Rec Rooms’ aesthetic and social interactions are so good. Physical interactions like fist bumps and to form parties, spatial audio and simple object physics interactions create the groundwork for great social interaction to emerge.

Through my explorations, I started to notice two qualifiers that were emerging that I hadn’t given much thought to in other mediums. The interesting part was that these traits were less about separating the explicitly bad from good and more about the distinction from an otherwise reasonable experience to one that felt like a true VR experience. Plenty of decent vr experiences account for the normal gamut of UX considerations; affordances, user control, user intent, consistency, accessibility, etc. But those weren’t the qualities that made an experience feel true to form in VR, they were just the baseline. What made an experience feel natural and immersive in VR were two qualities of a designed interaction; magic and diegesis.

The march of technological progress is one of communication, knowledge sharing and the increase of individual ability. Each new technological medium making it easier to talk to one another, to learn, to build. The discovery and creation of the previous ages bringing about entirely new modes of experience, from the Bronze Age to the Information Age. With VR we have the beginning of a new Cyber Age that will be as fundamental to the human experience as the internet and mobile phones are to us now. And the best way I’ve found to describe the most unique quality of VR is Magical Diegesis.

Further Considerations

I have laid out the core framework of Magical Diegesis but there are still some clarifications and important considerations that need addressed.

The Dangers of Designing Immersion

In any medium obtrusive non-diegetic designs are bad; at best they are an annoyance and at worst they are accessibility hazards and dark patterns. But in VR the stakes are extra high. At a base level a non-diegetic design will break a users mental model creating low levels of cognitive dissonance. But in VR you aren’t simply staring at your phone or punching buttons on an ATM, in VR you have strapped a computer to your face and are now in a digital world inside the physical one. This presents two new dangers, the first is bodily harm, if you can’t see the outside world you need to have systems to protect users’ physical bodies. The second is the user’s psyche.

Immersion is a goal of interaction but as previously stated it is also an incredibly powerful and dangerous tool, especially for those who would look to abuse it. The tools of psychology and human perception that designers use to create good design are the same tools that designers used to create intentionally bad design, to subvert users will through dark patterns.

As long as design is beholden to monetary incentives, dark patterns will continue to be used by those who wish to take advantage of people to turn a profit. Each new medium brings with it new levels of immersion, print, digital, mobile and now VR. These new technologies have undoubtedly brought about massive societal progress but they have also shown that as their scope of immersion and amount of data they intake increases so does their ability to be abused, by both individuals, companies and governments alike.

I am warning of this here because making someone think a cliff is real is the tip of the iceberg when it comes to the senses that can be tricked in VR. Designers will undoubtedly face situations in which they are pushed by hierarchical, structural, and monetary forces to implement these perceptual tricks to subvert and trick users. Not only are dark patterns morally wrong as they foist another’s will upon the user but dark patterns are contrary to the immersive qualities of diegesis and the honesty of good design.

The Line Between Skeuomorphic Diegesis and Magical Diegesis

What does skeuomorphism in VR mean? And does it fit into Magical Diegesis? The answer is yes and no. Skeuomorphism is when designs created in one medium pull obsolete or non-functional elements from another medium.

This is often for aesthetic purposes but it can also help people unfamiliar with a new medium understand it. You take nonfunctional signifiers from the old medium and re-incorporate them in a familiar way so as to give the user familiarity and affordances that help them do what they need to. A classic example of skeuomorphism is Apple’s original iPhone notes app. The app looked like it had a leather top, shadows and yellow lined pages. This was all non-functional and technically unnecessary, but it did help many people who were experiencing a digital note taking app for the first time understand what they were looking at.

Skeuomorphism has some benefits but they are largely only relevant at the introduction of a new medium. When a user is in unfamiliar territory some skeuomorphism is helpful, but skeuomorphism has a very small half life. Look at the current iPhone notes app, can you find any skeuomorphism anywhere in the most recent versions of iOS? You can, but it’s not the same kind as yesteryears glass icons and leather notepads. Modern skeuomorphism is what you would call Neo-Skeuomorphism; which has done away with the use of more overt visual textures and instead uses more physics affordances. Translucent textures, shadows that show distance and animations that show personality and movement are much more common affordances that help users understand an interaction’s intent and flow.

VR has already experienced a bout of skeuomorphism, specifically I’ve seen several instances of diegetic skeuomorphism. The game Job Simulator is a good example of a game that uses lots of diegetic skeuomorphism to great effect. Job Simulator is a game set in a simulation of what an office in the 21st century was like, it’s goofy and has a bunch of interactions that leverage interactions and affordances that most people playing will be familiar with; using a printer, logging into a computer, making coffee.

The game makes things larger and easier to interact with since finesse in VR is a bit difficult right now. The majority of interactions are skeuomorphic and that’s fine, but I already know how to login to a computer in the real world and while the absurdity and goofiness of the game makes some of those interactions fun, there was a specific kind of interaction that I remember very distinctly. The printer in the game will make a perfect copy of not just papers but any object you can get on top of it. This was super fun, it led to lots of goofy shenanigans and is the interaction I remember the strongest.

This interaction wasn’t just diegetically skeuomorphic like most of the others, it had an element to it that you can’t do in another medium. You can’t sit at a copier in real life and infinitely copy coffee mugs. The 3D copier was an interaction that felt magical and was very diegetic. Job Simulator is an early VR game, I would hand it to someone who had never been in VR and they would have very little trouble playing it. Onboarding is very important but the parts I remember were the ones that weren’t purely skeuomorphic.

Closing

Designing in a new medium can often be both parts exciting and frustrating. You push the boundaries to find out the limitations of the medium while gaining the ability to create in ways that you previously weren’t able to. In most other mediums, that process is augmented with teachers and writings from the countless people who have come before you. But for as much information about VR I consumed and for the countless hours of VR I experienced, when I started to try and design in VR, I hit more walls than I wanted to.

So as a designer I did what I do best, I deconstructed the problem and questioned it from every angle I could conceive of. My solution to that problem, to my problem, is this theory. I hope that anyone else in search of design theory on VR finds this and that it helps them or they find reason to harshly critique it.

Thanks to my editors Ben, Alex, Paul and Spencer for putting up with all my rewrites.

This article has taken me several years to fully synthesize and during that time my involvement with VR has been off and on. Now that I have this working theory I am excited to put it to the test soon. Look for more evolutions of Magical Diegesis and other writings on VR from me in the future.