When I was approached by the Sound Designer and Creator of the new VR film “Gullfiskteorien” (The Goldfish Theory), Thomas Pape, I was intrigued and excited to score something different.
I have worked on two VR Escape Rooms, last year, for the company Sandbox VR. Deadwood Mansion is a multiplayer Zombie house where you and your friends have to fight your way to survival. It has become very popular alongside the Curse of Davy Jones, by the same company, with the same gameplay style, but on a pirate story setting.
Although Thomas Pape’s project was a film, it contained certain immersive elements to it, in the shape of decisions by the viewer, like most role-playing games have.
The story is based on an experiment by two scientists around a teleportation theory based on a goldfish. The viewer takes center-stage as the goldfish swimming in its bowl, watching the story develop and as the most important part of the teleport machine.
I believe the music, alongside with the sound design and where he positioned them on the virtual environment mix became very important aspects of the story, as it would unconsciously make the viewer look to the direction of the sound.
So, I decided to interview the creator and sound designer of the film, as a way of showing more people how VR can be used to create a new genre of films that could not only be enjoyed on a VR headset and headphones, but possibly in new 360 degree movie theaters that could benefit from content.
Hi Thomas! It’s great to have you and thank you for taking the time! I was wondering, what was your first interaction with VR?
I was introduced to VR by a composer having a workshop on music for games, at the Norwegian Film School, and I told him that I was very intrigued by games like “The Stanley Parable” where you have a strong non-linear narrative and he showed me “Accounting”. I was blown away by the humor and crazy worlds!
Was that interaction an inspiration to create “Guldfiskteorien”?
Sure, in a way I guess it was! I remember at first I was playing a lot of different games. And I was completely spellbound! Shooting zombies, exploring strange planets, drawing in thin air, but I wanted to travel to a place that wasn’t CGI. I was hunting for movies, but had a hard time finding something interesting that wasn’t just pretty 360 landscapes or roller coasters. I was wondering why there weren't more cinematic stories told in live action stereoscopic 360? So I started experimenting with filming a lot of 360 footage, and found it both exciting and challenging! Coming from the world of cinema, I found that I had to re-learn a lot of my skills to apply them to VR, simply because it is a completely different medium. I think this might be why so few people are doing live action VR, because you might THINK it’s the same as shooting movies. It’s just a 360 degree movie, right? But it isn’t. And I think a lot of people might have been discouraged by that. Sure a lot of the tools and tricks of the trade are the same when shooting “flat” movies and 360, but it’s like comparing stage acting to screen acting.
When you watch a flat film, you watch the movie from the outside. When you watch a VR movie, you are IN the movie! And that makes all the difference.
I think the inspiration from that first encounter with “Accounting” kept bringing me back to the idea of that small world where you are the unspeaking centerpiece of the story.
That’s a very interesting point! Now, can you tell me, how did you come by the idea of using a goldfish in a bowl as the camera for the viewers?
I don’t remember if it was the fisheye lenses of the camera, or if it was just the feeling I got when watching my own early material that spawned the goldfish idea. But when I was searching for a plausible and comical main character, who couldn’t speak, couldn’t move around, but only look around and observe, the goldfish in a bowl was obvious to me.
I am also a big fan of great thinkers like Einstein and Stephen Hawking, and the fact that they both like to use the goldfish for different explanations, made it obvious to me, that if I was going to brew up a fake scientific theory about time and space travel, this was the way to go!
The idea that the scientist in the story relate to the goldfish called Bent directly throughout their travels, gave me the key to let the audience be part of the story, even though they can’t talk back. At different point Bent is talked to as a colleague, sometimes a pet, or an enemy, but he always plays a central role in the scene.
I think that when telling stories in VR is very important that you have a reason to be in the world. It can be small like in “Pearl” where you always sit in the passenger seat of the car that literally drives the story. Or big like in “Accounting” where you are the epicenter of the story.
The problem with live action VR is, of course, that the film is already shot, and you can’t interact directly with the world you are in. But I wanted to give the audience just a bit of control, so they don’t just observe. So inspired by the story of the mice in “The Hitchhiker's Guide to the Galaxy” (who does experiments on the humans, though they let them think its the other way around) I gave the goldfish the power to choose between timelines. So at certain points in the story everything freezes, and you get to choose how to proceed. This way you become somewhat of an interactive viewer.
I remember you telling me the story of how hard it was to shoot in 360 degrees when you have to be hidden and can’t really see the actor’s performance well. Can you tell us about other challenges you faced shooting the film?
Hehe, the crew and especially the cast have certainly been tested to the limits when we shot this movie! When I started recruiting people I told them that this was going to be an experiment, and that we would have to invent how we should work as we went along. Luckily everyone was completely onboard, and really exceeded my expectations in numerous ways.
I had to write the script with both choreography and timing in mind since the scenes are shot as one-takes with no editing. Then I worked with the actors to move them around the 360 space in a way that gives the audience a cinematic viewing. For example when I wanted a close-up, the actor had to move closer to the camera, and if I wanted a two shot they had to stand relatively close to each other. If I wanted something that comes close to a standard cinematic shot- reverse-shot, they had to be at an angle that forces the audience to turn their head to look at them individually.
The set designers build an entire laboratory in the studio from very specific measurements relating to the fact that the actors couldn’t get too close to the camera because it causes stitching problems, and they can’t get too far away, because then the limited resolution in the camera would make us lose contact with them. It was a bit of a trial and error, were we had to first act out the script with tape markings on the floor, and then when we had the walls in place, we moved them back and forth a bit to get the dimensions right.
Then I had two cinematographers light our laboratory set from the outside, without having the lights visible in the shot. We had to work without a ceiling to bounce in the light, and they rigged a few black cloth contraptions to create contrast and simulate light fixtures. The roof was then added later as CGI.
The outdoors scenes were especially challenging for the cast, because I wanted to show some of the more rugged and beautiful parts of nature. So we filmed on a mountain top near Jotunheimen, Norway, which turned out to be -34 degrees celsius (-29.2 degrees fahrenheit) on the day of shooting! Both the cast, crew and equipment had to have these little heating packs taped to different parts, to keep them from freezing!
And then we were in a completely different scenario when we traveled to Israel to shoot the 3000 year old Oasis. The temperature there was +35 and in the middle of the desert. Unfortunately we couldn’t drive to the location, but had to hike with all our equipment for about an hour, and partly scale a crevasse with climbing gear, to get to the spot where I wanted to film. But it definitely paid off, and the 360 view is breathtaking!
Creating the audio of the Goldfish theory has also been quite an experiment, since conventional methods weren’t possible. I had to use a combination of hidden lav mics on the actors, and an ambisonics microphone at the camera to capture the performance, and then when editing it, I had to place the sound sources in a virtual 3D soundscape to simulate reality. I had already written a lot of audio cues into the manuscript, and worked with the actors to use sound to draw the audience's attention, but when I started sound designing and mixing it, I realized that I had immense control over the audience's attention. I could literally predict (to a certain degree) how people would look around, based on audio exciters planted at timed intervals in the story. It felt like I was harnessing the power of an editor in conventional flat film.
Now that the film was premiered, I feel like because it is a VR film, it is very hard to promote it with shots from the film and teasers, as if you change its format to be watched on a phone, or screen, it would lose the value that it has. What are your thoughts on that?
I think the best way to promote it is to talk about it, because you simply have to experience it. The moment you convert it to a format for phone or make a flat teaser, you lose the magic.
When you watch it in VR, the video is stereoscopic, so you see everything in 3D, and you feel like you are there. The same is true with the audio. It’s converted from an ambisonics master to binaural sound that correlates to your head movements, and fools your brain into believing that what you hear is real. And the moment both your visual and auditory senses tell you that the world you are experiencing is real, then it becomes real.
People have been telling me that VR experiences shouldn’t last for more than 5-10 minutes, because people get nauseous, or over-stimulated, or stressed out, or the medium simply can’t handle stories longer than 10 minutes. When people watch the Goldfish Theory they stay in that world for 30 minutes, and still come out with a smile.
Yes! That’s for sure! I remember watching the final product for the first time and it was very immersive and never really got me nauseous or sick in any way. Now, what about the future? Do you intend to work on new VR Films and dive more into this new industry?
Yes, indeed! I am going to keep exploring how to tell stores in VR. I think one of the greatest challenges I’m facing is that there is no conventional funding programs for VR movies, and so far It’s been very challenging for people to create a viable income. It’s also very expensive to work with the material and equipment since most of it is still experimental. I’m so excited that people all over the world are pushing the technological limits for what can be achieved in VR, and it is the exciting new frontier of storytelling. I want to participate in the development as much as I can.
Great! So, if anyone is interested in “screening” the film or adding it to their festival, how should they reach out to you?
Preferably they can get in touch with me through email on tpape@plainmomentum.com or call me on +45 30 26 66 03, and I’ll be happy to talk about how we can make it happen.
Alright! Thank you very much and once again, this film was a great experience for me and I’m sure it was for you as well. I hope to have more people checking it out and talking about it. VR films could become a new genre to create new original stories.