ALYSIA: an AI that writes melodies for your lyrics

Have you ever written your own lyrics but struggled to find the perfect melody to accompany it? Well, the great gods of technology and music might have heard your plea.

Margareta Ackerman, an assistant professor at Florida State University, together with David Loker from the technology advisory firm Orbitwerks, has developed an artificial intelligence that uses machine learning to write a melody for your lyrics.

“I was studying singing while I was doing my PhD in computer science,” Ackerman was quoted as saying by NewScientist. “Over time, I started to think of computers as creative partners instead of tools, which could maybe help me write my songs.”

The machine learning system is called ALYSIA (automated lyrical song-writing application). It processes each line of text and then associates each syllable with a specific musical note. It determines the combination of note and syllable by analysing various elements such as the syllable’s position in the given word and how it correlates with the five preceding musical notes.

By using this method, ALYSIA is able to write whole accompanying scores for sets of lyrics, or provide a range of melody options for the various segments of lyrics, effectively taking the place of a co-creator. Although Ackerman and Loker originally developed the AI to specifically write pop tunes, they claim that the system could be modified to write for other genres as well.

ALYSIA makes use of two models to write the melodies. One is focused on rhythm and the other on pitch. Each of them was trained on the melody line and lyrics of 24 pop songs.

The system was then used to create melodies to accompany two sets of words written by Ackerman, “Now that you’re gone/I just realised that I’m all alone”. The system was also tested on the lyrics of a vaudeville classic, namely I’m Always Chasing Rainbows by Harry Carroll, as Ackerman and Loker were interested in seeing how ALYSIA would reimagine the song as a pop song.

A rather interesting AI project, ALYSIA stands out thanks to its lyrics-based approach to composition

Although the attempt to automate musical composition is not necessarily a new endeavour, David Cope at the University of California says that what makes ALYSIA different is that it uses the lyrics as the starting point as opposed to the melody. But even though he admits that the system is impressive, he does feel that ALYSIA’s musical compositions showcase an “almost annoying” lack of harmony.

Rebecca Fiebrink, a researcher in machine learning and music at Goldsmiths, University in London, questions the approach of using lyrics as the starting point.

“Is this really solving the compositional process for people who want to make music? Creating a melody without additional accompaniment, like the system does, is the easiest thing to achieve,” Fiebrink was quoted as saying.

Ackerman admits that the songs, as of yet, aren’t exactly award-winning material but says that this is only the start. She originally planned to target ALYSIA at the electronic music community but is currently hard at work repurposing the system for professional songwriters with the help of classical composers.

Ackerman’s vision is to eventually create a system will be capable of composing all aspects of a song on its own.

“We want to design a program able to generate the music, the lyrics, and ideally even the production of the singing by itself.”