The Terminal - Reflection

Concept

The project speaks to The Nether and its Terminal by recreating such an alternate universe where participants are masked with different faces of different people, thus lending them fabricated identities in the virtual world. Participants in this alternate universe possess abilities that they wish for in the real world (in this particular case: playing music like Beethoven), which begs the question of whether they should stay indefinitely despite a lack of authenticity and intimacy from other people in the real world.

A demo of The Terminal

Main Technicality

  • The body tracking algorithm was based on machine learning computer vision (Tensorflow’s poseNET)
  • The mask image is placed according to the position of the ears. Junior and I calculate the distance between the ears and based on the x and y coordinates of such, overlay the image on the middle points between the two ears.
  • The tilt of the head is also taken into consideration by finding the angle made by the line going through the two ears with the horizontal plane. The music is played whenever the tracking method detects that both of the wrists positions are on the top third portion of the screen, and is paused otherwise.

    • Reflection

  • What parts were most effective?

    • I think the storyline was nuanced, but simple enough to engage the audience and make them associate this with The Nether and the Terminal in such a short period of time given. Also, the ability to control the music with hands movement added some playfulness to the piece that later proved enticing to the participants.

  • What were some challenges you had to work through?

    • At first, we intended the project to be an improv performance with random characters being assigned to the participants. However, after more careful consideration, we instead decided to craft a narrative with pre-chosen characters so that the performance could be more logically tight and packed with nuances.

    Moreover, from a technical standpoint, we tried to come up with a function that can detect hand movements and translate it into the rhythm of the music being played; however, as the tracking method proves to be too unstable for such precision, we simply opted for a much simpler workaround: play/pause a pre-recorded piece of music.

  • Did audience members react they way you expected?

    • I expected this theatrical installation to be lighthearted but can still speak for the Nether world and the Terminal. And for the most part, the audience reacted the way expected them to be. Also, for the stand-alone body tracking installation itself, it was interesting to see some participants try hard to figure out the way music control worked, which amazed both of us because it was a really simple mechanism.

  • How did you work together as a group?

    • As have only 2 people in our group, our collaboration was tight-knit and efficient as would bounce off ideas and reciprocally receive feedback from the other forming a continuous stream of conversation towards idea conceptualization and idea development. As both of us had experience working with p5.js before, sharing a glitch project where I could code together and see each other’s contribution proved to be a convenient way to divide the workload: taking turns to add features into the gitch page.