Mike McCracken

Last night was my friend Memmi’s percussion recital which I helped with. I don’t have any photos or video to put up here yet (I will add something if I encounter it) but I did want to describe some of the stuff I did. It was my first performance doing live video interaction (using Jitter) and it was very fun. There were three musical movements that I did some stuff for and I’ll describe those, but first some background on the subject matter.

The piece was about St. Hildegard von Bingen, a figure Memmi finds inspirational for several reasons. For one, Hildegard’s work was multifaceted - she worked in natural sciences, medicine, music and more. This inspires Memmi (who has been heads down working on her PhD in music for several years now) to be allowed to pursue other interests. I can totally respect that. There is a general attitude in art and science that unless you focus and specialize, you aren’t going to be able to contribute to anything meaningful. There is some underlying truth in a certain respect (certainly specialists can and have contributed significantly), but it forgets the value of of the renaissance man (or woman) who through the diversity of their interests can make connections and contributions that a specialist wouldn’t be able to. Buckminster Fuller even takes the attitude that to specialize is to thwart ones potential as a human.

Another inspirational aspect of Hildegard was her ability to speak out, make her voice heard as much as she did, and survive. Most people (women especially) in her position would be thought to be evil or possessed and be burnt at the stake. She had visions, spoke out against the church (dangerous thing to do in the 12th century) and was generally a bit of a trouble maker, but managed to play her cards just right and be deemed by the pope divinely inspired rather than a witch. Winding up with two monasteries is a far preferable outcome to asserting opinions against church practices than being burnt alive.

Hildegard was also thought to be a synesthete. This is also quite interesting for Memmi as she has colored hearing (sounds are involuntarily associated with colors - correlating by pitch).

Movement one was improvised marimba playing with recorded voice and imagery that progresses through elements (as they were in Hildegard’s time) and colors associated with them that Keely put together. I made a simple patch that made the video sound responsive so that the intensity of sound would bring out the color associated with that period in the piece. By the end (sky / air) we were looking at bright clouds pulsating whitish yellow with the voice and marimba.

The second movement dealt directly with colored hearing. We coded in (as close as we could) Memmi’s colors associated with different pitches. Memmi played the xylosynth sending MIDI commands into my patch, which created dots with color and size reflecting the pitch and intensity of the note. The dots would initially fade away, then overlap and combine to fill in the screen. initially, they revealed a camera feed showing Memmi playing, then after the main transition in the piece they added color on top of the camera feed, bringing the image up to white.

In the third piece, we worked with paintings the Hildegard had done based on her visions, and the recurrence of themes of illumination in Hildegard’s writing. Some think hers is the first documented case of migraines because she writes in detail about being overcome with white light. So I mixed a video of the paintings that Keely made with a camera feed that had Memmi’s mallets color keyed in to create white light that feeds back on itself, the effect being first kind of glowing, sparkling mallets, then as I ramp up the feedback the mallets start leaving cloudy tracers which eventually grow to consume the screen with their light. During this piece Memmi played a composition of Hildegard’s that she arranged for marimba.

The show was interspersed with lectures and chants and was generally very well received, and a lot of fun to do. If I find pictures, I’ll post them. Unfortunately, the camera we borrowed from the UW to record the whole show came with the wrong power adapter and no battery so we didn’t get that, bummer. But there was at least one person snapping photos, so there’s hope!