index

Making a Concert Hall Come Alive with Generative Art

There are many ways people combine technology with music. I’ve always been of the mindset that good music doesn’t need technology to be captivating. But technology, when used tastefully, is not a gimmick---it’s a tool that extends the expressive capability of a performer.

How might we create interactive visualizations that synergize with live performers? I draw from examples of living systems to design visualizations that make a concert hall live and breathe with the performers that inhabit it.

In 2018, Reena Esmail published a piece called “Darshan” for solo violin. Six years later, I became awe-struck by it. It draws on Hindustani classical music using the Charukeshi Raag. The emotional landscape builds from a single chord that is slowly embellished with every phrase. The key feature of the piece is that it feels improvisatory---the performer can be free with time---yet it is richly structured. In a way, it is like the G Minor Adagio from solo Bach: melodic lines weave in between harmonic pillars creating a balance between freedom and structure.

I set my performance of Reena Esmail’s Darshan to live visualizations in 2024. I used gyroscopic sensors and accelerometry from an iPhone to capture the movements of my bow arm. This data was streamed wirelessly (UDP protocol) in real time to an animation I coded in Processing. The visuals used algorithmic design to map acoustics and motion to little snippets of synthetic life. Small creatures moved across the screen pushed by the force of my bow arm, fractal structures mimicked trees, strands of DNA-looking things flitted on the screen.

More recently, I became interested in how immersive performance might encourage audience agency.