Robots are taking over our manual jobs, and now it seems they will be taking over creative ones, too. Shimon the robot made his first appearance as a musician in 2017 when he performed some “cover” tunes on the marimba. At that time, he was more focused on covers than composing his own music. Now, he’s learned to write lyrics, sing, dance, and compose his own melodies. He is even going on tour soon to promote his first album.
Shimon creates his music in a similar way to other creative robots or even people. He is fed 50,000 lyrics from existing human music, including hip-hop, jazz, and progressive rock. The data provides Shimon with a foundation in which he can begin composing his lyrics from the patterns and rules he’s learned.
A researcher on the team who helped build Shimon, Richard Savery, said:
“There are lots of systems that use deep learning, but lyrics are different. The way semantic meaning moves through lyrics is different. Also, rhyme and rhythm are obviously super important for lyrics, but that isn’t as present in other text generators. So, we use deep learning to generate lyrics, but it’s also combined with semantic knowledge.”
Professor Gil Weinberg of Georgia Institute of Technology is the proud creator of Shimon. He developed the robot to be able to write lyrics based on a theme. “You’ll get a word like ‘storm,’ and then it’ll generate a whole bunch of related words, like ‘rain.’ It creates a loop of generating lots of material, deciding what’s good, and then generating more based on that,” explained Savery.
Professor Weinberg speaks about Shimon in the video below:
The lyrics are made up of phrases and words that sound like a sentence at a glance, but when you pay attention, they don’t make any sense. For example, “home is where the rainbow comes,” and “there may be music or a star into your mind.” However, a significant amount of music people write these days doesn’t make sense, so Shimon is not that different from them.
Shimon has also been given a makeover, which enhanced his stage presence and performance. He once was an expressionless metal ball on a mechanical arm, but now he has eyebrows that wiggle, a mouth that sings in sync with the music, and head movements that portray his emotions.
During intense parts of the song, he bends his head down towards the marimba, or sometimes he looks to his human bandmates. Additionally, a team from Spain’s Pompeu Fabra University developed a unique voice for Shimon using machine learning algorithms trained on hundreds of songs.
When the robot jams with his bandmates, he can add in some melodies of his own making. However, most of them are composed by Professor Weinberg to fit Shimon’s songs. According to Weinberg, this robot-human collaboration is just the beginning. In the next few months, Shimon and the band will be releasing an album comprised of eight to ten songs. Then, they will go on tour.
Weinberg said:
“I think we have reached a level where I expect the audience to just enjoy the music for music’s sake. This is music that humans, by themselves, wouldn’t have written. I want the audience to think, ‘There’s something unique about this song, and I want to go back and listen to it, even if I don’t look at the robot.'”
Check out Shimon jamming out with his human bandmates in the video below:
