Beyond the score: Music visualization and digital humanities


  • Margaret Lam

    Investigates Challenges Related to the Transmission, Preservation and Organization of Music Knowledge
    1. Works independently as a researcher, writer and designer as well as a collaborator on various projects. You can learn more about her work at
    Search for more papers by this author


Editor's Summary

Among the humanities, music is especially powerful for reflecting time and place and evoking personal experiences and memories. Yet music research in the digital humanities has attended little to socio-cultural context, tradition, history and performance knowledge, instead focusing on the musical score and technical elements. To have an impact on music scholarship, digital research in the music domain must be considered more broadly. An example is the work of Alexander Chan, whose cutting edge music research may be appreciated as interactive installations but gets little recognition within digital humanities. Music research has fallen short in its use of digital technology to understand theory, composition, musicology and the preservation and transmission of music knowledge. Despite music's evocative power, its conceptual scope in digital research is narrow.

Music – beyond the scores, recordings and signal analysis that occupies information scientists and music librarians – has the amazing ability to bring human history to life.

The ethos of a time and place is inevitably reflected by the humanities, and music is no exception. We know this effect through our own personal experiences: the way couples have their song, the way music takes you back to a time in a different place, the soundtrack of car radios, movies and television shows that accompany us through various life stages. For many music scholars the socio-cultural aspect of music is an important part of their research. However, current conceptualization of music research within digital humanities does not fully address the socio-cultural element in music research. Efforts within the digital humanities are focused primarily on music as either information or artifact. While the elements of music such as pitch and rhythm are common to all music researchers to various degrees, it is not fully representative of the diverse approaches that music research can take [1]. Conceptualization of what constitutes music within digital humanities is necessarily narrow as we experiment with different applications of technologies to support music research. Yet, as I will suggest in this short piece, a broader conceptualization of music is necessary for digital humanities research in the domain of music to have real impact on music scholarship.

Music as Information, Music as Knowledge

When we conceptualize music as information, we are referring to elements of music such as pitches, chords, tempo and dynamics on one level, and contextual information such as genres, performers, dates and instrumentation on another. The development of specialized markup languages for music (see and the largest hub for music information retrieval (MIR) research at the University of Illinois at Urbana-Champaign ( reflect a score-centric approach as the dominant research paradigm in music information research today [2]. In other words, music information research is striving to enrich existing musical scores with metadata for better indexing and search on the one hand, while also working to extract information from audio signals and translating it into decipherable musical notations on the other [3].

These problems are unique research challenges that only arise within in the domain of music, but there is little research on issues dealing with music as knowledge, such as how to perform music, how to discern or attribute human intention within a piece of music and how to situate a piece of music within a musical tradition. Such issues are important to music researchers and have much more overlap with the research interests in the humanities than the score-centric approaches.

Information visualization of music has also focused on music as information, by offering a visual way of highlighting the relationship between selected structural elements (see for example: In other examples of Alexander Chan's work (, the creation or composition of music is dependent on input from external environments (such as the movement of commuters in a subway station in Sonata for the Unaware) and virtual instruments with which we create music, such as a line drawn on a screen (as in his work Crayong).

The Scope of Music Scholarship

While the examples of Chan's work appear to belong more in the realm of digital arts rather than digital humanities, it is without question a product of music research, in the sense that his research output is an interactive installation, rather than a musical score or a research paper. Yet, despite the common reliance on elements of music, or music information, in music information visualization and in digital humanists' current approach to music, it is noteworthy that research output such as Chan's is an area that is not often addressed within digital humanities.

The difference between the traditional conceptualization of music research and cutting edge innovations is not an insurmountable one. It may be worthwhile here to outline the spectrum of interests and specializations that may be the context for a music researcher, based on personal experience. Broadly speaking, these areas are the following:

  • Theory and Composition: the creation of music; composition techniques, musical systems

  • Performance: performance practices, historical performance

  • Pedagogy: methodology for the transmission of different musical practices

  • Musicology: analysis of Western music, historical musicology

  • Ethnomusicology: anthropological and sociological approach to studying music

While music scholars may align themselves with one or more of these research areas, most are familiar with the type of research in all the areas. The fields that are most preoccupied by music information (that is, pitch, musical structure and so forth), are those engaged in theory and composition and musicology. For all the other areas of music research, while it is no doubt a necessary skill to be able to work with music information, it is not their primary research interest. Instead, it is music as a socio-cultural and essentially human phenomenon that they are interested in. There is recognition among music scholars of the way changing and emerging technologies are changing music practices around the world, and by proxy, the way they conduct and share their research [4]. Chan's work, for example, would be recognized as the work of a composer who is working with technology at an advanced level. The use of technology is seen as a natural extension of what composers have always done.

Such specializations within music – which most would regard as already a specialized area of study and research – create subtle divisions not only between the broad areas of research listed above, but also within each of those research approaches. Anyone who self-identifies as a music researcher will no doubt have something to say about the way I have divided and organized the list. I do not intend it to be an authoritative way of organizing, but rather as a way of highlighting the nuances behind the concept of music research.

Looking Forward

Music has the ability to bring not only history to life, but also to bring the lesser-known corners of our contemporary world to the fore. In conversation with ethnomusicologists, who are among the most actively boundary-crossing music researchers encountered, it is clear that collaboration between digital humanists and music researchers is not only a great possibility, but also extremely necessary. Ethnomusicologists and their peers deal with unique challenges related to the archiving of, and fair access to, their data from their field work. Traditional music archives do not allow them to balance the need to give the communities they work in ownership and access to their field recordings. There is currently no easy way for them to share their research in a meaningful way with a general public or to actively preserve musical cultures that are transient or dying. While there is no lack of ideas from music researchers as to what new technology can do for their work, there is a lack of time, resources and expertise available to them. In the context of the music-related research within the digital humanities, there is a tremendous opportunity to further the way we preserve and transmit music knowledge. This opportunity is only possible if we broaden current conceptualization of what constitutes music research within the digital humanities and if we strive to better understand and appreciate the diversity of research approaches within music as a field itself.