Copy make (video gestual scoring system)

Copy make


Copy make is a demonstration video about the physicality of sound and proposes new methodologies of working relationships between composer and performers in an open and visually centred collaborative approach.  You can read further about this project here in a peer-reviewed paper in Leonardo Music Journal, Volume 30, 2020

The copy make installation came about in a workshop where the premise was to explore notions of perception and music, musical preconditions, ‘music without sound’, and ‘composition beyond music’. My response was to make visible the invisible world of music through a display of its gestural processes. This allowed for a transference of a diverse array of parameters involving; rotation, closing, expanding, roughening, smoothing, demarcation, recursion, inversion, punctuation, dynamic change, velocity, and duration. 

Sound and movement are recorded on a glass surface to be used as a multi-screen video score for musicians to use with an instrument or sounding object. Text, numbers and symbols are written over videos containing gestural movements – a technique called digital annotation that has been used in the field of dance and anthropology. The emphasis is on the line-semantics of the performers movements. A gestural matrix allows for multiple points of focus for the performer to reiterate, interpret and retrace actions within a given frame (outlined here using yellow tape). This project was intended to be sound-based, but there is scope for it to be explored within a choreographical context. The video theorises potential uses of this way of working that could have multiple performance outcomes. It is a platform for experimentation and for realising a fixed work, and is open to anyone who wants to discover its possible uses.

Copy-make explores the embodied process of correspondence as a compositional tool and framework for thinking about gesture in music. I use correspondence in an ongoing inquiry carried out through collaborative relationships and between: the body and the music instrument; a sounding musical gesture and the nonsounding; and visual-aural and haptic-gestural relations that are part of the scoring and rehearsal process. The documentation of movement and its correspondence renders my scores somewhat like somatic maps. I use the term “line-mapping” to explain this process of discovering, transcribing, notating and transforming gestural pathways.


Cross-modality in a music process

As a method of scoring for the body, I developed an installation, Copy-make, that focuses on the physicality of sound and proposes new methodologies of working relationships between composers and performers.

A camera and a microphone are positioned facing a glass window surface. Performers are invited to make movements and sounds on the glass using either a musical instrument or a sounding object, or by tapping and rubbing on the glass itself. The recording devices capture short looping videos that are projected onto a screen to the left. The performer uses these videos as a score to inform their gestural actions. The emphasis is on the line-semantics generated by the performer’s movements that are placed within a multigestural matrix. This allows the performer to see and interpret their lines and then remake, reiterate or retrace these actions within the given frame  The performer is instructed to make corresponding gestures by superimposing complementary (or contrary) lines.

Copy make
Image 1: The Copy make score where text, dynamics, zones (Z) and timings are overlaid on the videos. Lines are drawn with a black marker, and a transparent rectangle scrolls through each screen to indicate which section to perform. (© Daniel Portelli)
Copy Make
Image 2: Photograph of the interactive video installation Copy make in Graz, 2015, as part of Peter Ablinger’s “Composition Beyond Music” workshop, run by Georg Nussbaumer. (© Daniel Portelli)

In this work, gesture is conveyed through a cross-modal medium, where the body is used to transmit musical instruction. The duration, shape, rotation, posture, location and speed are all used to inform gestural choices. Sound is partly indeterminate but does have equal weight in the composers’ and performers’ decisions.

When I use the term cross-modal, I am referring to the sensory modalities: the aural, the visual, the motor, the tactile and the imagination, and how these physical or perceived processes can correspond within the work. In Listening, Jean- Luc Nancy states: “Nothing is said of the sonorous that must not also be true for the other registers” (Nancy, 2007). Copy-make then offers a platform for this dialogue between the senses and demonstrates that our sensory registers have an immense capacity for corresponding. It challenges musicians to think through a kinesthetic-based methodology of composition within a collaborative environment. I also reference Gritten, King and Welch, who suggest “that musical gestures are cross-modal and that gestures include non-sounding physical movements as well as those that produce sound” (Gritten et al. 2016).

The installation becomes a performance, with audiences watching as the performer constructs the work. The performer is not simply imitating the videos but goes through a rigorous process of transforming them, internalizing them and superimposing complementary lines of movement. The work offers a playground or sketching process for generating ideas. It allows the performer to build an archive of music gestures and then make choices to correspond with it: moving under their line, over it, with it, or against it, smoothly or dynamically in a wavy motion. Copy-make can be adapted and customized in myriad ways, depending on the instrument(s) being used and the composers’/performers’ relationships and how they envision the work.

This work brings to the foreground the cross-modality of music and the framing and mapping of the delicate nuances of touch. Think of musicians’ hands on their instruments or even how sound vibrations touch cochlear hair cells before becoming signals in the brain. In this work, the body transmits the score.

The work aims for an inversion of eye-ear relations, making visible the invisible world of music. The use of videos also makes visible an unfolding sonic-gestural process. What is seen, heard and interacted with in the score are processes including but not limited to recursion, inversion, rotation, closing, expanding, roughening, smoothing, demarcations, punctuations, dynamic changes, velocities and durations.


A. Gritten, E. King and G. Welch, New Perspectives on Music and Gesture (Abingdon, U.K.: Routledge, 2016) p. 6.

 Jean-Luc Nancy, Listening (New York: Fordham Univ. Press, 2007) p. 71.

Daniel Portelli, Music gesture and the correspondence of lines: Collaborative video mediation and methodology. (Leonardo Music Journal and MIT Press, 2020) Vol. 30.