A.I.’s Present on Beethoven’s Birthday | Beethoven X

According to Zach Evans, founder of music A.I. lab, Harmonai, the challenge for his industry is to turn the nuclear bomb into nuclear energy. It’s an analogy for overcoming the anxiety that some in the arts community have expressed towards A.I. and instead, leveraging the technology for our benefit. 

Classical music may not seem like the obvious space for testing A.I. The term itself, ‘classical music’, seems almost in opposition to the futuristic notions evoked by A.I. However, in 2021, just as Zach was starting to experiment with how machine learning might open new frontiers for music, a group of computer scientists, musicologists, and musicians were using the technology to solve one of history’s greatest musical conundrums. 

Some consider Ludwig van Beethoven to have been “the world’s first superstar,” who famously died before he was ready, in the wake of releasing his 9th Symphony. When he died in 1827, he left behind only notes of an unfinished 10th Symphony that was commissioned by the London Philharmonic Society. 

Generations of composers and music experts have wondered what Beethoven’s 10th Symphony would have been. Some believe it would have even eclipsed his 9th Symphony, widely regarded as his magnum opus. In 1988, musicologist Barry Cooper even undertook the painstaking effort to extrapolate 250 bars of music that he said was a faithful composition of what the 10th Symphony’s first movement (symphonies generally have three sections, or movements) may have sounded like. That’s as far as Cooper could get, though, as Beethoven’s sketches became too sparse to reliably interpret them into the other two movements. 

In October 2021, however, emerging technology in deep machine learning let the world hear what Beethoven’s 10th Symphony could have sounded like, thanks to the Beethoven X/AI project. 

As early A.I. technology captured imaginations in 2019, a new effort to flesh out Beethoven’s genius began taking shape. One year prior, Ahmed Elgamaal, Director of Rutgers University’s Art & A.I. Lab, and his team released AICAN, billed as a nearly autonomous art program which could generate innovative images of its own. Word about AICAN spread to Dr. Matthias Röder, the director of the Karajan Institute, an organization in Salzburg, Austria, that promotes music technology. In an article for The Conversation, Elgamaal wrote about how Röder reached out to ask him if A.I. could help with Beethoven’s birthday project.  

With support from European telecom company, Telkom, the two put their plan together to recruit experts in computer science, musicology, and music history to build a type of A.I. model that didn’t yet exist. 

Collaborators on the musical side were tasked with examining Beethoven’s sketches and deciding where the various pieces would best fit within the structure of a classical symphony. Meanwhile, the computer scientists had to translate all of Beethoven’s works, including the sketches, into a computer-readable format, and figure out the more intangible task of training an A.I. model to compose music like Beethoven. 

“We needed to figure out how to take a short phrase, or even just a motif, and use it to develop a longer, more complicated musical structure, just as Beethoven would have done,” Elgamaal wrote. “For example, the machine had to learn how Beethoven constructed the Fifth Symphony out of a basic four-note motif.”

The team knew they were on the right track after passing the first litmus test, where a pianist performed a few minutes of their A.I. generated piece in front of an audience of experts at the Beethoven Museum (located in the home where Beethoven was born in Bonn, Germany). The audience was challenged to determine where Beethoven’s sketches ended and the notes generated by the A.I. began. Only those most familiar with Beethoven’s final sketches were able to hear the difference. 

The team unveiled their completed work 18 months later: the remaining two full movements of Beethoven’s 10th Symphony, spanning about 40 minutes, performed by the Beethoven Orchestra of Bonn. 

Even at the time, Elgamaal anticipated criticism from some for using A.I. to mimic the composer who is considered by many as the best of all-time. 

“We anticipate some pushback to this work – those who will say that the arts should be off-limits from A.I., and that A.I. has no business trying to replicate the human creative process. Yet when it comes to the arts, I see A.I. not as a replacement, but as a tool – one that opens doors for artists to express themselves in new ways.”

The full performance is available on Youtube, and a 52 minute film documenting the process behind-the-scenes, Beethoven X/AI, was released as a TV special in 2021. 

Check out our conversation with Zach Evans, Founder of Harmonai, that explores A.I.’s impact in the music production industry more in Ep12. Zach Evans: Stability AI’s Harmonia on Open Source Generative A.I. Music.