Volumetric filmmaking is an emerging art form. Right now that means it is being defined at technology companies and based on old standards and practices.

We need the weird, the bad, the queer and the feminine to help bring in the new. We need artists from outside mediums to show us what they know. We need you, but the journey from start to finish is often unclear and it’s hard to know where to start.

This article aims to help by sharing a process, taking you from a vague assemblage of ideas and intentions to a solid foundation to build on.

“Volumetric Filmmaking is a growing movement in immersive content characterized by interactive experiences created using predominantly 3D scanned imagery […] A true hybrid of video games and cinema.”

From "What is Volumetric Filmmaking", Scatter

I’m coming to this two years into my own volumetric journey to create a “labyrinth of interior worlds where sci-fi, self-help, and messy teenage yearning collide” known as The Goodbye Room. As I slowly come to accept that I might know what I am doing, I thought it’d be a good time to share how I got here.

This work can straddle performance, theater, video games, and probably even things we haven’t put words to yet. You can make virtual reality experiences, augmented reality applications, or use the footage for visual effects in things like music videos. For now I’ll focus on those first critical creative challenges at the start of any creatively driven volumetric project.

‍

Taking out the tech

The technology involved in capturing a volumetric performance can be a distracting dark hole to get lost in. Depthkit makes it simple, but it can still be tempting to get trapped in the realm of digits and pixels without first building a solid conceptual, aesthetic, or narrative foundation. Let’s explore what a volumetric experience consists of without all those distracting technical choices by exploring some lenses and questions to start your project on solid ground.

Intentions
Why are you making it?

Concept
What is it about?

Narrative
What happens?

Aesthetics
What does it look like?

Interactivity
What do you do?

These are hard questions. The answers will start as hunches and come into focus over a long period of time. This is normal. People may want to know exactly what it is early in development, and it’s okay not to know, or to only have vague directions to wave your hands in. It might be best to share your instincts early and watch how people’s facial expressions shift in response. Or it could be better to wait until “you know you know.” Then again, you could cop to not being 100% on anything and trust you’ll figure it out. Just remember that some people might not get it until it’s staring them right in the face. Ideas are tender and delicate and you can’t let them be validated or dismissed by anyone. You are the only expert in what you are making. With that in mind…

‍

Why are you making it?

Making volumetric content is a challenging and oftentimes frustrating effort, so you should be clear in your intentions, define realistic outcomes, and figure out your appetite for risk from the get-go. I’ve seen people use volumetric film to advance their commercial careers, for the challenge of it, to showcase an underrepresented community or convey a particular perspective on an issue. Whatever your reason, you have to really want it, believe in what you’re making, and prepare yourself emotionally and practically for a long road of self-sustenance. There are few organizations to turn to for support and engaging with the community often means untangling a nested web of conflicting business, personal and artistic interests. Still, by sheer virtue of persisting and being here, you get to play a pioneering role in shaping an entirely new medium that could redefine the way we engage with stories for decades to come.

‍

What’s it about?

All ideas, for me, start with an image. What is an image? Lynda Barry wrote a whole comic book slash memoir about it. In it, she defines an image as “the formless thing that gives things form”.

‍

“An image is a place. Not a picture of a place, but a place in and of itself. You can move in it. It seems not invented but there for you to find.”
‍
— Lynda Barry

Perhaps more practically, think of an image as a scene in your head that you can anchor your creative thoughts around. Maybe it’s from a memory, or a thought, or something you saw in a television commercial or a dream. If you’re making a non-fiction piece, maybe you’ve actually been there. Where are you? Who is there with you? What’s behind you? What’s ahead of you? What’s above you? Answering these questions can help give you some initial direction.

– An old woman standing outside of her house at the end of the cul-de-sac
– A 5,000-year-old gourmet food spread at an ancient, abandoned blockchain convention
– A cat gazing wistfully out at a stack of tuna in a warehouse

What do these things all have in common? They’re images.

I find that laying out disparate images and other material in a place where I can see them from a bird’s eye view is useful to form connections. Scrivener is a piece of software that lets you create collages of text and media to do just this. My Scrivener project for The Goodbye Room included screenshots of movies, emails from a decade ago, a bunch of bad teenage poetry I found on DeviantArt, and snippets of plays that captured a style I wanted to emulate.

‍

‍

If you want some structure in this process, I recommend the urtext on personal creative gestation, The Artist’s Way. Spoiler alert: it’s a 272-page text just telling you to wake up each morning and immediately and un-self-consciously fill a page with writing. I can’t recommend the practice highly enough, but it might get weird. We often can’t help what comes to us, and it can take a kind of courageous arrogance to believe in the importance of our ideas long enough to develop them into their highest potential.

It’s completely valid to start from a mushy place of images and feelings and slowly make decisions until it becomes solid. In the world of commercial design thinking this is the concept of “divergence and convergence”. Giving yourself room to explore is essential, but in the words of performance artist Dynasty Handbag — sometimes you have to just “pick a road and party down that road.” In her class on developing performances, she tells a story of being stuck in writing her version of Homer’s Odyssey (Homo’s Odyssey) while at a writing retreat. To clear her mind she goes for a walk and, near a lake, is suddenly chased by a large, white bird. When she returns to write she adds a bird chasing her to her performance, opening up new possibilities that allow her to finish her piece. Sometimes you just need to pick a direction and see where it takes you.

While you’re forming ideas, you may be tempted to ask yourself, “Is this good?” or “Does this suck?” It might not be good, and it might suck, for a very long time — and that’s okay. The secret is to stay with it, have fun, and be gentle on yourself while what you’re making starts to take form and you can whittle it down. Being concerned with those two questions will stop your creativity in its tracks.

‍

What happens?

In a volumetric film, nothing really has to happen at all. You’re presenting a moment, and the narrative can exist entirely in that moment. But in case you decide that a story is what you’re after, here are some suggestions.‍

Start small
It might be tempting to start with a big story outline and fill in the scenes from there. If it works, great, but if it feels intimidating and limiting, I suggest you start with writing small scenes.

“How do the tiny pieces stay alive when they’re aware that they are supposed to be gigantic? For me they tend to die in the process. I can totally identify with seeking the comfort of a big established outline but I have found it doesn’t make things feel alive.”

— Khaela Maricich, narrative advisor on The Goodbye Room

Writing the script in earnest began with an abstract conversation between a player character and their ex in a bedroom. It then expanded to include a variety of other scenes that originally came as lone images on their own — the player character and their ex in an aerobics class, a family dinner, going to a show together. Images developed into scenes that had their own emotional logic and weren’t super concerned with how they fit in the bigger picture. It was only when I had a bunch of these scenes figured out that I could take a step back and see how they connected and formed a story. I needed to have parts in order to begin to understand the whole.

‍Pop Songs vs. Ballads
Another bit of feedback given to me by Khaela was that I was writing a ballad when I needed to be coming up with a pop song. This came at a time when I’d set out to make something that felt more recognizable and familiar as a traditional narrative. It involved coming up with a backstory for The Goodbye Room and all of these characters to support it and reasons for it to exist. It felt weighed down by all the infrastructure when the essential parts could exist unexplained.

Consider, what is the most boiled-down essence of what you’re trying to say? What is the fun part of your story, the sexy part, the hook? How do you get to that immediately? What actually matters? When you tell someone about what you’re making, what do you say that feels the most alive as you’re describing it?

The answer for The Goodbye Room was simply the experience of being stuck in a room with your ex, just as you remember them. Who cares how you got there, what are you going to do now that you’re back together?

Narrative Structure
In the process of looking at the fragments I’d developed and seeing how they all fit, it was helpful to have at the back of my mind some general ideas about story structure and narrative theory. For that, I recommend checking out playwright Lauren Gunderson’s workshops on The How of Theater.

At other times, I got more out of non-traditionalists like Richard Foreman, Mac Wellman or Tina Satter who all write plays with narratives that follow elliptical or similarly unusual shapes, sometimes described as “a moving collage of events.” For some perspective on experimental theater and narrative, I highly recommend Foreman’s book Unbalancing Acts.

Finally, I encourage you to explore non-volumetric interactive experiences (read: “walking simulators”) like Tacoma and Everybody’s Gone to the Rapture for additional inspiration on how to tell a story across a space.

‍

What does it look like?

During the formative periods of a piece it’s important to put on some blinders as to the feasibility of rendering what you’re calling for, but there will be a time when you’re forced to confront what the thing will actually look like. Using the smallest, simplest form of representation can often work as powerfully as the most elaborate and complicated. What do you need to actually show? How important is realism? What conveys a feeling of presence? Is it how a thing looks or how it behaves? I found referencing theatrical stage designs to be useful in discovering elegant solutions under production constraints. Mood boards, storyboards, color palettes, all can help in defining an aesthetic tone to drive decisions as the experience develops.

‍

‍

‍

What do you do?

One way to immerse people in your volumetric experience is to provide a mode of interaction. This can be as simple as walking around a space to reveal additional content (Vestige), or rummaging through items in the back of a car (Queerskins). Interaction can give agency or just provide feedback to help the world feel more “real.” Watching people as a voyeuristic ghost in the room can easily become boring, and people in headsets quickly get antsy without anything to do.

For The Goodbye Room, we came up with several interactions in the form of gestures that reinforce the central themes of the story and allow for a way clear, constrained way to interact with the characters. Primarily you can “Push” and “Pull”, a stand-in for the emotional push and pull in an insecure relationship. You can also “Escape” inwards to spelunk ever-deeper interior worlds and continue the story.

‍

‍

Of all the things we tried, these verbs stuck around because they re-enforced the central themes of the game, provided opportunities to highlight and reveal emotional layers, contributed to the underlying metaphysical lore, and gave the illusion of player agency in a largely linear experience.

Try to find a unique angle to compliment your story. Indie games have led the way in interaction and conceptual synergy. Two great examples of this are If Found…, a game about finding closure that has you literally wiping away the past, or Little Inferno, a game about consumerism that has you tossing an endless supply of junk into a magical fireplace.

‍

And then you get to actually make it!

You know what you’re making, or at least have enough of an idea, and now it’s time to get your hands dirty.

I recommend breaking your experience down into the tiniest pieces for implementation. These can be higher-risk pieces that will take some time to prototype and get just right, or they might be larger pieces that, while relatively easy to implement, give an encouraging sense of progress. Some examples include a custom interaction or a unique visual effect. If you’re tackling this by yourself or with a small team, work in discrete phases to achieve goals and maintain momentum. For example, a phase might be to build all the environments, or do a previsualization pass without your volumetric footage, or get all the interactions working.

When it comes to shooting the volumetric footage for your project, see what resources exist in your community by checking out Peerspace for studio space and Backstage for casting. I’ve also had luck casting through Facebook Groups for actors and improv.

Visit community development resources for when you get stuck. Some great ones are the Unity Discord, the Volumetric Filmmakers Facebook group, and the Depthkit Slack. There are lots of community members like myself who are eager to help out and brainstorm solutions.

‍

Raising money

As someone who has been hugely unsuccessful at raising money, I only feel qualified to tell you what I don’t recommend doing. Unless you’re on your second or third title, I don’t recommend pitching your piece until you have an extremely specific vision of what it is, several polished key art assets, and a playable prototype representative of the quality of the finished piece. I’ve found that first impressions often count the most, and you might want to be intentional with the moment you decide to share it with someone with money or access.

The entities funding this content right now can be counted on one hand, so be willing to get creative with funding solutions and use tools like Kickstarter and Patreon. The current size of the industry results in funding outcomes that feel like cliquish popularity contests. Try not to take it personally.

Finally, scope appropriately. I would not recommend starting with a full-length interactive story with multiple environments, characters, and custom interactions because you could end up like me, two years in with something very cool in the works but little to show. Instead, might I suggest building up to that with a series of short pieces that serve to gradually establish career currency and reputation. That way you become a lot easier to give money to, have a portfolio, and can be sustained by the emotional feedback loop of releasing new work.

‍

Outcomes

After you’ve made your piece, where does it go? Depending on how you’d like to contextualize it, it can immediately launch on a platform like Steam and Viveport or premiere at a film festival with an emerging media section like Tribeca or Sundance. The latter route might be more useful for getting fancy eyeballs on it, but factor in timing, submission fees, travel costs, and the emotional toll this will take on you. For most people, these events are primarily for professional networking, garnering press, and gaining visibility as there is not a robust distribution market for this kind of content. Smaller festivals can be useful for getting free playtesting.

You might be able to get similar mainstream press coverage simply by emailing your experience to VR sites like UploadVR, RoadToVR, or pitching journalists directly at outlets like The Verge.

The community as a whole is still pushing to make paying for this work an accepted norm, so if you distribute it on a storefront please charge something for it. Prices range from $2.99 for a short piece to $14.99 for a high-end, hour-length experience. Talk to others who have done this before to get a realistic expectation of sales.

‍

It takes time

Creators in VR often come from technology or agency backgrounds where certain tasks can be project managed to take a predictable amount of time. The cold, hard, devastating reality is that creative concepting and writing doesn’t work that way. It takes time to create layers, have revelations on the walk to the corner store, and figure out what you’re making is truly about. Give yourself the gift of that time, be patient with yourself, and find joy in the process. There have been many moments in which I could’ve locked the design and script before it was ready. It would’ve been a nightmare. You’ll know it when it’s right. Believe in yourself and keep pushing until you get there. You will.