Renaissance Polyphony Meets Visuals Created By Artificial Intelligence

by Jarrett Hoffman
Published November 23, 2020

The vocal quartet New York Polyphony sings a lamentation by Francisco de Peñalosa in the audiovisual work ‘Aleph Earth.’

What started out as a cold-call email from one University of Oregon professor to another developed into an ambitious multimedia work that might best be summed up as, well, many different things.

A lament on climate change. A modern vehicle for sharing early music with new audiences. And perhaps a new blueprint for creative projects at a time when the coronavirus pandemic continues to challenge artistic collaboration.

More simply, Aleph Earth is a 12-minute audiovisual work of art.

The audio, from the vocal quartet New York Polyphony, is the world premiere recording of Spanish Renaissance composer Francisco de Peñalosa’s Lamentationes Jeremiae Feria V,  taken from the ensemble’s September 2019 album Lamentationes on BIS Records. (Read EMA’s CD review here.)

The visuals are the handiwork of the University of Oregon’s Artificial Intelligence Creative Practice (AICP) Research Group. Director Colin Ives and multimedia artists Zachary Boyt and Thomas Newlands developed a machine-learning algorithm that found abstract patterns within videos that were fed into it, including footage of melting icebergs and catastrophic bushfires. Then they trained that model of artificial intelligence to use those patterns to create its own visualizations.

Iceland drone footage used to generate images in ‘Aleph Earth’

In addition, the team found a way for the algorithm to “react” to the vocal interplay of the Peñalosa recording, thus generating its imagery in direct response to the performance by New York Polyphony countertenor Geoffrey Williams, tenor Steven Caldicott Wilson, baritone Christopher Dylan Herbert, and bass Craig Phillips.

The premiere came in late August at the Currents New Media Virtual Festival, and although a trailer is available, future showings of Aleph Earth are still to be determined; it’s available for booking as a virtual concert or educational event through Opus 3 Artists.

Maybe just as interesting as the final product is how and why it came together. To tell that story, let’s go back to the message that was sent out of the blue — and that was even a little apologetic, as bass Phillips said, laughing, during a recent conference call with AICP director Ives for EMA.

“I’m somewhat new to the University of Oregon,” Phillips said. “I’ve been here for three years, and developing relationships with colleagues across the campus is difficult to do as a new guy. So I literally started reading bios.”

What he and his New York Polyphony colleagues were looking for was a different way in. “We’ve been trying to find ways to share early music with people who aren’t already coming to it with some expertise, so having a different delivery mechanism has always been interesting to us,” Phillips said. “As champions of this repertoire, we’re always looking for ways to say, ‘Please take a listen to this. It will change you.’”

The Peñalosa seemed to be fitting for such an undertaking — something largely undiscovered on which to shed more light, and a work with enough length and emotional drama to carry its own weight.

An image in ‘Aleph Earth’ generated from the Iceland drone footage above.

The response from Ives was immediate enthusiasm. “The timing was perfect,” he said. “My AICP group was looking for a project that we could really dig deep into, and as soon as I listened to this music, I was in love with it. It’s just so beautiful, and it was obvious that we needed to do something with it.”

The text of the Peñalosa comes from the Book of Lamentations, which depicts the destruction of Jerusalem in 586 BCE and is generally considered to have been written by the prophet Jeremiah. Ives, it should be noted, has long been interested in religion, in particular the lament, which he sees as not just a passive action, but also as something powerful and personal, “bringing us to a place where we can enact change individually.”

Perhaps it was also clear from the start that the natural world would find a way in. “A good portion of my work as an artist-technologist is about ecological issues,” Ives said. “Craig read my bio, so it must have been in the back of his mind that that’s where I might want to go.”

It’s not hard to see how those two threads came together. “Jeremiah laments how it was the behaviors of his own people that invited the havoc,” Phillips said. “The text is simultaneously saying ‘Why have you forsaken us?’ and ‘We got what we deserved,’ so there’s a real symmetry with man’s role in climate change.” Oregon’s devastating wildfires this fall, just as Aleph Earth was being seen for the first time, proved to be a coincidental yet sadly fitting backdrop to this statement on the climate crisis.

That said, Phillips and Ives cautioned against boxing in the work with a single, clear-cut analysis. “We have our own interpretation, but the thing about successful art is that it doesn’t beat you over the head with some agenda,” Phillips said. “It allows you to bring to it whatever you’re thinking.”

Colin Ives, director of the University of Oregon’s Artificial Intelligence Creative Practice Research Group

He sees a connection between that creative philosophy and this style of music. “There’s something very objective about Renaissance polyphony — something very cool and distant to it. We’ve presented this music all over the place, and the response we get from people is that it presents a canvas onto which you can project.”

A point of contrast might be the operas of Puccini, who “very clearly wants you to feel a specific way,” Phillips said. “A lot of Renaissance music is moving in a human way, but it’s abstract, so in that respect, it gave Colin and his team a lot of playing space. And my role in this was: Here you go.”

Handing off the recording might sound simple, but there’s a story behind that as well. Thanks to the audio format used by BIS Records, the album’s producer “was able to separate out each channel,” creating a dedicated recording for each part, Phillips said. “Had we worked with a label less committed to hi-fi sound, that wouldn’t have been possible.”

That was critical in order for the algorithm to analyze the interplay of the four voices, the aspect of the Peñalosa “which for me is what’s so moving and powerful,” Ives said.

Still, since the singers had all recorded at the same time, there was overlap among the four individual recordings. Enter AICP’s Zachary Boyt. “We needed to isolate each voice, and Zach went through note by note to pull each one out for our data set,” Ives said. “It was beautiful to work with him — he’d gone through the piece so intimately that he knew every aspect of how each note was delivered.”

Months of work had already gone into Aleph Earth by spring of this year, but the outbreak of the novel coronavirus still made its impact felt. “When the world blew up in March, there was a pivot,” Phillips said. “Early on, we were thinking about making this some sort of installation in a non-traditional performance or acoustic space, where listeners could respond and react. Obviously, we had to move away from that.”

Francisco de Peñalosa (c. 1470-1528)

The pandemic also influenced the visuals of the introduction, which includes four texts floating by, each one appearing on-screen based on the entrances of the singers. “Two of them are ancient Hebrew texts, and two are from current news articles — one on climate issues, and one on COVID,” Ives said.

That’s a symbolic gesture, as the words are difficult to make out against a cloudy backdrop. But in an abstract way, those articles reflect the opening of the Peñalosa text: We watch the news of those two tragedies passing by, just as Jeremiah looks down over Jerusalem.

Perhaps most meaningfully, in today’s daunting landscape for the arts, Aleph Earth represents a new possibility when possibilities are slim: using technology to build something new around a performance that came from before.

“Globally, from a standpoint of the performing arts, we’re looking at a kind of mass extinction event,” Phillips said. “We’re forced to look at different ways of reaching an audience in a time when I think people are hungry for substance and for things that at least transport them from our current reality.”

Jarrett Hoffman is the managing editor at and served as a fellow at the 2014 Rubin Institute for Music Criticism. As a clarinetist, he performs in the Hudson Valley and New York City and runs a private studio.

Recent EMA Recording & Book Reviews

More News & Reviews

Scroll to Top