Storm

Kinect Interactive Instrument | April 2020

Storm is a dance performance that explores the body as a vehicle for expressions with the synthesis of sound, motion and spatial awareness.

This project is a prototype as part of my ongoing exploration of the crossover between music/sound, space, touch and awareness of the body. Enabled by Kinect and the way it’s programmed, the piece combines choreography and music composition in one single process and uses the naturally synchronized flow of events to create a narrative.

The performance uses Kinect as a spatially reactive instrument, which maps the body as a physical object to the virtual space, which contains the triggers for sound. When collision between the body and a trigger is detected in virtual space, a sample is played. As shown from the video, there are currently 16 pre-recorded samples laid out in a 4×4 grid, which are all placed roughly at the height of the top of my head.

The grid layout creates a spatial framework for arrangement of expressions. For sound composition, I put the samples in a few different categories, each used for different effects (See diagrams below). The more intense samples are triggered as larger events, and they’re meant to be played sequentially and not too frequently. These are put in each of the 4 corners so they won’t be triggered all at once. Then there are the smooth ambient sound effects such as running water and bells, which persist throughout the piece on and off and won’t create disturbance. The rest of the samples include other sound effects such as percussion and bass. In a way, the arrangement is the most important part of the composition, and the combination and layout are the result of many rounds of testing.

Once the setup is ready, the performance operates on building the pace and tension through body movement. With the spatial awareness of the sound locations at the backdrop, very dance move is created with an intention of setting up the next event, and with sound playing, the feedback of the sound in turn affects the expression of the dance move.