Embodiment 2: The Multisensory Experience of Music
Why am I emphasizing how Deaf composers create visual and spatial access to music? Why discuss the embodiment of sound in music videos? Because we need to recognize how Deaf composers make music visually accessible.
I see ASL music videos as parallel to M. Remi Yergeau’s (2014) activist rhetoric, in which she advocated for valuing embodied experience. In contrast to uncaptioned music videos that privilege the aural mode, ASL music videos incorporate Deaf musicians’ multimodal practices in a rhetorical message that could be interpreted as: we will make our own music because “we are the movers” (Yergeau, 2014, Toward Criptastic Hacking section, para. 5). Deaf composers embody the multisensory experience of music through the movement of sound in visual-spatial form—such as in Figure 3 below, a screenshot from Sean Forbes’ (2010) “I'm Deaf,” in which the word DEAF vibrates across the screen.
Figure 3: Screenshot from Sean Forbes’ (2010) video, “I'm Deaf”
To quote the activist message of Deaf Professional Arts Network (D-PAN, n.d.), “We have the power to level the playing field and bring music to everyone, and we’re not going to stop. It’s everybody’s music” (Our Story section, para. 12). Their videos bring music to the eyes and the body in ways that can be accessed by more senses and bodies. Their videos resonate with the ways that Jay Dolmage’s (2008b) and Yergeau’s (2014) have framed access as a verb that reflects “how bodies and spaces move and interact with each other” (Yergeau, 2014, Toward Criptastic Hacking section, para. 4). We continually interact with differences and can always recreate new ways to communicate to different senses.
Access is a process, “a way to move” (Yergeau, 2014, Toward Criptastic Hacking section, para. 4), and Deaf composers are not going to stop moving to bring music to more audiences. Just as importantly, we composition scholars and students should not stop moving in interaction with different modes and bodies as we continually explore different ways to communicate to our audiences.
The embodiment of sound in visual form reflects the ways that we can communicate across multiple senses to capture sensory meaning. Seeing music through visual text reinforces Gunther Kress and Theo van Leeuwen’s (2001) theory of “multimodality in which common semiotic principles operate in and across different modes, and in which it is therefore possible for music to encode action, or images to encode emotion” (p. 2). Visually striking compositions such as Sean Forbes’ (2011) video “Sean Forbes ‘Let’s Mambo’ Ft. Marlee Matlin” (see clip below) highlight the potential of non-aural elements in visually expressing the melody of music.
I see ASL music videos as an extension of the vibrant visual and kinesthetic experience embodied by ASL musicians in their performances. Steph Ceraso (2014) drew from deaf percussionist and composer Evelyn Glennie’s performances to accentuate how “the vibratory aspect of sound” is a multimodal event that involves the “convergence of sight, sound, and touch” (p. 104). Ceraso explored how the percussionist experiments with the relationship between sound and visible movement in her performances and wrote that Glennie’s approach to sonic experiences can be replicated by anyone who experiences sound in the body. I resoundingly agree; the dynamic visual text in ASL music videos show that deaf and non-deaf viewers (who can see) can sense the pulsations of sound in multisensory forms.
In a similar vein, Jeannette Di Bernardo Jones (2015) used her exploration of the visual performance of Deaf musicians to challenge “the hearing world to think about the alternative modes of hearing that a deaf musical experience offers, calling for critical reflection on how many of the current assumptions about music are connected to how people hear with their ears” (p. 55). She shows how Deaf musicians visually and kinesthetically express musical vibrations through signs in the air.
Certainly, Deaf individuals reinforce the visual and kinesthetic experience of music through signs that express the lyrics of songs; we move our hands and bodies in accentuated rhythm with the beat. We recreate music in and through our bodies—and the embodied experience of feeling music is recreated once again through visual text that moves through the screen of ASL music videos. To explore the multisensory experience of music, I turn to an ASL version of “Fireflies.”
NEXT: Example #1: "Fireflies"