I created Zooming Away with my friend Dillon Zhang in May 2020. It was our final project for an MIT theatre class: Interactive Design and Projection for Live Performance (21M.737).
The Idea
Normally, the class would have us create interactive media for a physical space on campus. However, after the COVID-19 outbreak moved our classes online, we found ourselves with a new, unique challenge: creating an interactive live performance from the confines of our homes, to be shared with our instructor and classmates over video call.
Rather than create a traditional performance and "virtualize" it, we decided to embrace the format and build our project around a video call. Drawing inspiration from the game Keep Talking and Nobody Explodes, we created a 15-minute, 2-3 player escape room that takes place in a Zoom call and your browser.
Walkthrough
Email Invitation
The experience starts in your inbox: you and your teammates have just completed ASAN space cadet training. You all receive invites to a virtual commencement webcast, held over Zoom.
top: Zooming Away email invitation, bottom: Space cadet virtual background
Virtual Commencement
When the time comes, you join the Zoom call, and the ceremony kicks off with a special congratulatory message from Jim Fredstine, Director of ASAN.
A special message from Jim Fredstine, Director of ASAN.
A few minutes in, ASAN cosmonaut Rachel Ride dials in urgently and interrupts, "Mayday!" Her space capsule is tumbling out of control and set to hit an asteroid in 10 minutes. She needs your help down at ground control to stabilize it and get it back on course before it explodes.
Rachel Ride interrupts the webcast with an emergency call for help.
Diagnostic Guide and Control Console
Facing an unexpected first mission, you urgently dig up the ASAN diagnostic guide you received during training and remote dial into the command console from your browser.
An excerpt of the ASAN diagnostic guide. (full guide here)
The space cadet control console.
Through interacting with Rachel over video call and flipping through the guide to determine the correct console commands, you and your fellow cadets can stabilize Rachel's space capsule and help steer her back on course. But can you repair everything in under 10 minutes?
How It Works
The space cadets (players) communicate with the cosmonaut (live actor) and her space capsule through two channels: the Zoom video call and the control console.
Flowchart of Zooming Away components.
Audio and Video FX
To sell the illusion that the live actor is dialing in from a space capsule, we process her microphone and webcam feeds before sending them to Zoom. We pass the audio through Logic Pro X, where we apply EQ, distortion, and other plugins to achieve a lo-fi, crackly sound. We run the video through Snap Camera, where we add a head-tracking heads-up display, a shaking effect, a scanning effect, and color correction, using custom AR lenses made in Lens Studio.
Control Console, Actor's Panel, and Server
Hidden from the players is the actor's panel, which helps the live actor direct the game and locate relevant information.
The actor's panel.
Game Controls: The start, pause, and resume game buttons allow the live actor to control the clock.
Line Prompter: At the bottom, a line prompter gives the actor hints and starter lines for directing the players, though the majority of her dialogue is left to improvisation.
Capsule Information: The middle section of the actor's panel contains a range of information about the space capsule, which the actor can or cannot read depending on the capsule state. For example: in this screencap, we know that the number on the outside of the window is 21737, but in order to see what's inside the window, we'll need to close the window. If the players ask about the inside of the window, the actor could say, "The window's open. I can't see inside." The players should then figure out that they need to hit the "Close Window" command in their control console.
The actor's panel and space cadet control console are connected via web sockets (Socket.IO) and a web server (Node.js) that keeps track of the game state.
Contributions
I designed all graphic assets, video FX, and audio FX, and played the role of the cosmonaut during our final presentation. Dillon wrote the puzzles; developed the control console, actor's panel, and server; and played the role of the Director of ASAN during the presentation. The two of us worked together to build the narrative and integrate all the components into one cohesive experience.
Reflections
Zooming Away received glowing reviews from our guest participants and class instructor! They praised its level of immersion and its use of a Zoom call as a storytelling medium.
My biggest takeaway from this project is that building interactive live experiences for our laptop screens opens more doors than it closes. We lost the chance to experiment with physical space, but we gained the opportunity to explore how AR lenses and web sockets can play a role in producing engaging experiences in our own homes.
Where could this take us in the future?
| collaborators | Dillon Zhang |
| tags | Experiential Storytelling · Escape Room · Interactive Design |
| technologies | Photoshop · Illustrator · Lens Studio · Snap Camera · Logic Pro X · SocketIO · Node.js · Javascript |
| context | MIT 21M.737 Interactive Design and Projection for Live Performance |
| completed | May 2020 |