The Million Song Dataset (henceforth the MSD) is a gigantic collection of “audio features and metadata for a million contemporary popular music tracks” (tracks—not songs per se) that became available for download from Columbia University in February 2011. An explanation of the MSD is available in an eponymous article (available at the same website) written by the dataset's creators:Footnote 1 a Columbia electrical engineering Ph.D. student, Thierry Bertin-Mahieux, and his advisor, Daniel P. W. Ellis, both of whom represent Columbia's Laboratory for the Recognition and Organization of Speech and Audio (LabROSA); and Brian Whitman and Paul Lamere, employees of the commercial company the Echo Nest. The Echo Nest, which touts itself as “the world's leading music intelligence company,”Footnote 2 does not interact with the general public directly; rather, it services other companies such as Spotify, Clear Channel, and eMusic, which draw on the Echo Nest's data and analyses to further their individual commercial enterprises. The Echo Nest is more or less performing the same role for LabROSA, the main difference being that the purpose of the information's distribution is to encourage academic study; the MSD is funded in part by the National Science Foundation.
The “metadata” of the MSD includes track title, artist name, date, and the like. The “audio features” consist of rhythmic, timbral, loudness, and pitch information in numerical form; no actual audio files are included. Still, the dataset is so large—273 GB, the same size as some laptop hard drives—that it is broken into several smaller downloads comprising thousands of files each. (Because downloading the entire set will take a very long time, the creators have also compiled a subset of information on 10,000 recordings, which is less than 2 GB total.) Once the dataset is downloaded, special software is required to read its HDF5 format. The MSD is thus not something a scholar simply peruses, or even easily searches through; it is designed primarily for large-scale statistical analysis carried out by algorithms written in computer code.
Some significant statistical work using the MSD has already been published. The most notable is the Scientific Reports article “Measuring the Evolution of Contemporary Western Popular Music,”Footnote 3 which was picked up by the popular press because of its provocative claim that recent “mainstream” music, compared to its own past, “shows less variety in pitch transitions . . . timbral palette, and . . . volume dynamics.” With such a large corpus, experiments run on this dataset will undoubtedly yield numerous results that are statistically significant. In fact, the MSD is currently—and promises to remain for many years—the largest open-access collection of popular music audio information in the world; it thus represents one of the most exciting developments in the scientific study of popular music. But because of its complexity and sheer size, the dataset itself offers no obvious, practical uses for most scholars in the humanities, except perhaps for those working in forensic musicology, who might find it a great source for song precedents (although finding the precedents would require specialized computing knowledge). As a music theorist with no formal training in statistics and no recent programming experience, I am not qualified to evaluate the MSD's technical aspects. On the other hand, I do feel at liberty to comment on the musical underpinnings of the project as a whole—specifically the MSD's methods for quantifying musical parameters and its definition of “contemporary popular music.”
Among the musical parameters that are measured, pitch seems especially problematic. The procedure for analyzing pitch involves partitioning the audio stream into durational “segments,” each of which constitutes “a set of sound entities (typically under a second) . . . [that is] relatively uniform in timbre and harmony”; the pitch contents of these segments are converted into “chroma vectors” (pitch-class strengths) based on their “relative dominance” (doubtless based primarily on amplitude); these vectors are then somehow used to compute both a tonal center represented by 0–11, where C=O (this integer is misleadingly labeled a “key”), and also a “mode,” which is merely “minor,” “major,” or “no result”; additionally, there are “confidence” numbers that indicate the dataset's relative certainty regarding its own estimations for center and mode. Yet as music scholars well know, center and mode can be very difficult characteristics to ascribe to popular music. It would be helpful to know exactly how the key, mode, and confidence numbers are derived from the chroma vectors; in any case, the assumption that each and every popular song exhibits only one center and mode is highly suspect, as music theorists have demonstrated repeatedly.Footnote 4 Although the confidence numbers act as caveats to the center and mode readings, the initial goal of pursuing a single center and mode per track is itself misguided. I would think creating algorithms that allow for multiple outputs for center and mode would not have been insurmountably difficult to implement.
Perhaps the most problematic aspect of the MSD, at least from the perspective of a humanities scholar, is that its repertory is not quite what it claims to be. Granted, “contemporary popular music” is difficult, or impossible, to define with precision. The dataset is admirable for its inclusive nature, featuring songs in numerous languages—English, Spanish, German, Dutch, Japanese, Chinese, etc. (and thus the songs are not simply “Western” as stated in the Scientific Reports article, as well as in the dataset's eponymous article)—and songs in innumerable styles—from film scores to noise rock to Brazilian psychedelia to Siegfried and Roy's show music. But does polka king Frankie Yankovic really count as contemporary popular music? Does Art Tatum? What about Brian Ferneyhough, Charles Wuorinen, Pierre Boulez, and John Cage? Surely Stravinsky, Schoenberg, Debussy, Tchaikovsky, Brahms, Wagner, Schumann, Chopin, Mendelssohn, Schubert, Beethoven, Mozart, Haydn, Handel, Bach, Rameau, Vivaldi, and Palestrina cannot be called “contemporary popular music” without distorting that expression beyond recognition. Yet every one of these musicians appears in the dataset. By contrast, shouldn't inclusion of the Beatles be a no-brainer? We do find Beatles songs in the MSD: “I Want to Hold Your Hand,” for example, arrives to us in versions by the Beatles Revival Band, the Beatles Tribute Band, the Ultimate Beatles Cover Band, the B-Sides, the Lovejoys, the Monsters, Al Green, Ray Conniff, Jack Nitzsche, Glen Adams (this version appears twice), the Golden Boys (also twice, and sung in Portuguese), and conductor Arthur Fiedler (again twice, in orchestral arrangement). But the only Beatles tracks we encounter are the 1961 Hamburg performance of “Ain't She Sweet” from their stint as the backing band for Tony Sheridan (this performance appears three times in the dataset), and several interviews the band members gave during their career and after their breakup. The inclusion of these interviews indicates that not only do the terms “contemporary” and “popular” (and “Western”) fail to describe the entirety of the MSD, but so too does the term “music.” I cannot determine the extent to which these three terms are misrepresentative of the MSD as a whole, but until someone performs a statistical analysis on the dataset itself showing that these problems will not affect the overall numbers, I will remain a bit skeptical of any grander claims based on the dataset's content. I truly hope this task is taken up; it would be a real shame if the MSD were to fall short of its overwhelming potential.
