In the MIT theatre class Interactive Design and Projection for Live Performance (21M.737), our first assignment is to introduce ourselves through a short "self-portrait" performance.
I'm a huge fan of sci-fi and dystopias (catch me binge-watching Black Mirror). So I wondered, how could I add a dystopian twist to sharing fun facts about myself? I came up with the idea of an airport security scan gone too far.
Watch the full demo video. (1:52)
Going Interactive
In the first iteration of this project, I created a simple video, projected it onto a blank wall in the classroom space, and then stood in front of it during my performance. Because the video's content and timing were fixed, I needed to practice timing my actions — walking onstage and raising/lowering my arms — until I got it right.
Later in the class, I revisited the piece and made it interactive, with the help of a Kinect sensor, NI mate, and Isadora. Now, I no longer need to follow the video; the video follows me! The project tracks and uses the positions of my hands to progress through a sequence of scenes.
Behind the Scenes
Creating this piece led me through a variety of skills, from motion graphics to sensor integration, which made it all the more fun.
I started by hand-animating the motion graphics in After Effects.
Motion graphics in After Effects.
In Logic Pro X, I recorded the voiceover, added music and sound effects, and mixed the audio.
Sound design in Logic Pro X.
In NI mate, I configured the Kinect sensor and set up communication between NI mate and Isadora: NI mate transmits Kinect data over OSC and real-time video input via Syphon.
I set up OSC and Syphon listeners in Isadora, assembled the video and audio clips, and implemented transition logic to integrate everything into an interactive experience.
Interactive logic in Isadora.
What's Next
There's so much territory to explore here. While my scan video now responds to arm movement, its content is still fixed — it would ID everyone as me. But what if it didn't? What if it could dynamically display any user's biographical data, hobbies and interests, and psychological profile? Would that even be possible?
From a technological standpoint, it totally is. Access to our Facebook data alone (our profiles, reacts, check-ins, groups, and liked pages) could potentially unlock all of the above information. During the 2016 presidential election, Cambridge Analytica infamously psychologically profiled American voters based on their online activity. I can imagine this piece extending into an art installation that did the same. If I took the scan in that direction, the very existence of such a project would raise awareness and speak volumes about how little control we have over our data in today's society.
With a growing level of government surveillance, it's also not too hard to imagine a future in which even airport security procedures would tap into the wealth of data surrounding our digital identities.
This short, fun self-portrait project doubles as a piece that raises serious questions about where our digital privacy is headed in the future.
| tags | Experiential Storytelling · Motion Graphics · Interactive Design |
| technologies | Kinect · Isadora · NI mate · After Effects · Logic Pro X · OSC · Syphon |
| context | MIT 21M.737 Interactive Design and Projection for Live Performance |
| completed | May 2020 |