How we discovered the three revolutions of American pop

Our recent scientific analysis of the “fossil record” of the Billboard charts prompted widespread attention, particularly the findings about the three musical “revolutions” that shaped the musical landscape of the second half of the 20th century.

The journey that got me here started in 1992, when I was a sheltered 12-year-old boy, growing up in a small town in Germany. But I had just discovered a window to the world: every Sunday, Radio Luxembourg transmitted the original US Top 40, in English, and I was addicted. So much so that when we absolutely had to go and visit grandma, I set the timer to record the show on compact cassette. Listening back one night I heard the marvellous sound of a man singing “Mama, ooh”, in a song so unusual, so exuberantly creative I had to re-listen over and over – it became the musical love of my life.

It was, of course, Queen’s Bohemian Rhapsody, which resurfaced in the charts after the song’s appearance in the film Wayne’s World. I had stumbled across a gem from the 1970s in a sea of 1990s music. Little did I know then that I would grow up to analyse the difference between 70s and 90s pop, and much more.

Revolution #1

In 2011, nearly 20 years and a couple of amateur bands later, I found myself in London, with an engineering PhD in music information retrieval in my pocket, and in my head a new passion: Darwinian evolution. An eccentric mixture, granted, but London wouldn’t be London if one couldn’t find another eccentric with complementary passions. I found Armand Leroi, an evolutionary biologist with a love for music, and culture in general.

We’d already studied the evolution of musical loops produced by genetic algorithms, Bob MacCallum’s DarwinTunes, but it was clear that eventually we would have to transition to real human music, and what is more indicative of human musical taste than the charts? Conveniently, while waiting for my next academic job, I was working at the music recommendation service Last.fm with its large collection of audio recordings. Last.fm were happy for me do research on the music, and to carry away some of the data as well, so by the time I started my new job at Queen Mary’s we had a rich dataset of original recordings from the Billboard charts – more than 17,000.

Revolution #2

So rich was the dataset that it took us a long time to work out what to do with it. Long discussions ensued, evenings spent in intense arguments in front of our computers. In the end we decided to look at two properties: harmony and timbre, or tone colour. Our ambition was to show what would happen if we measured these on a great number of songs in ways that traditional, manual musicology couldn’t achieve.

And so we set out to data-mine 50 years of Billboard charts in ways that music enthusiasts and musicologists alike could relate to. What’s more, Armand was adamant to explore what evolutionary biologists would explore in the fossil records of organisms: diversity and bursts of fast evolution.

What did we find? Personally, I love to observe the evolution of chords. Tracing the decline of jazz and blues chords, however sad, was as fascinating as the surge in the use of more funky, disco-y chords in the 70s. Our timbral topics also turned out to show very clear trends in instrumentation: the double hump pattern of loud guitar sounds (around 1970 and the late 1980s), the beautiful wave of declining orchestral sounds, and the awesome rise of percussive sounds in the 1980s.

Revolution #3

What has captured public attention most are the revolutions, perhaps because the press exaggerated our findings to suggest that hip hop was better or more influential than the Beatles. What we did find is that the charts changed most rapidly around three distinct periods: 1964 when the charts turned to guitar rock; 1983, when technologies like synths and drum machines altered production, and hence timbre, forever; and 1991, when rap and hip hop finally made it into the charts big time.

The change in 1991 appeared most significant because – unlike all other genres – hip hop put so much emphasis on speech and beats that harmony was pushed into the background, or omitted completely. Also, we found that in terms of the features we measured, the US charts were most uniform in the mid-1980s, a result that resonates with my perception.

What this all shows is that we can relate human perception of music to patterns in recordings, which in turn allows us to understand the mystery of music, how it comes about, and how it changes. Many people I’ve talked to are fascinated with this new way of looking at music – some are horrified. Yes, it will take time to get used to, but ultimately I believe that our increased understanding will add to the enjoyment of and fascination with music, just like the stars in the sky are beautified by our growing knowledge of the universe.

Matthias Mauch receives funding from the Royal Academy of Engineering.