The data topic chosen for this project is text prompt which is widely seen in text-to-image generation models. Redefining the traditional creative process, information from a text prompt undergoes natural language processing models and artificial intelligence systems and is eventually rendered as a set of images. This complex technology allows users to create unique and convincing images with just their imagination and several words.
Findings and Insights
As design students, we were intrigued with the idea of a creative output heavily relying on technology and AI models when we came across the news of Jason Allen winning first place in the Colorado State Fair with his AI-generated Theatre D’opera Spatial image. Though this phenomenon sparked a controversy and raised questions regarding the nature and status of art, our team was more interested in exploring the capabilities and potential of this recent development in AI learning models.
Although most people are only familiar with the use of text prompts in AI image generators, our group experimented with text prompts in other areas — text-to-text and text-to-audio. We kept an open mind and had fun during the development of this project. Hence, with this project, we hope to convey that such AI models do exist, are accessible and are open to endless possibilities.
Design Decisions
From the beginning of our project, we had already decided that our outcome would be a hip-hop, rap album. With this in mind, all the parts of the arrangement reflected this decision, from the lyrics of the song, to the melodies, as well as the voice used to vocalise the song. In addition to the audio components, the visuals generated for the album’s cover also reflect the visual language of hip-hop culture. We generated an image that portrayed a large side-profile portrait, and paired the image together with large blackletter and graffiti style fonts; just like the hip hop artists we are referring to.
Challenges
The greatest hurdle we encountered is our lack of background knowledge and experience in music production.
Because of this, we resort to just experimenting and testing out all the methods we can, thus we don’t
have the capability to understand when to stop generating new sounds, and when we should re-do it again.
Furthermore, we are also limited in our capabilities of creating a very specific sound, for example, drum
beats. As most of the sound we create comes in a mp3 file format, and not MIDI, it is difficult to adjust
or take out individual parts of the generated sound. However, with this overall lack in experience, there
is a larger reliance on just using whatever the AI creates, minimising human interference.
Feedback
Week 9: Joanne asked us to ponder what we want our outcome to be like; do we want our listeners to
easily recognize our songs to be created by AI, or will we try to create something more seamless, and
make it difficult to tell?
Week 12: Gideon urged us to continue to push the boundaries of our songs even further and to create an
outcome that borders between noise and music. Our next steps were to try to combine as many
different sounds as possible to try to break these limits. We also wanted to create “levels” for varying difficulties in listening.
Week 14: Joanne and Vikas noted that our outcomes reminded them of Radiohead, both the visual and
audio aspects of our project. They also said that the songs we labelled as “hard listening” are
actually easier to listen to, because it doesn’t try to become anything other than experimental, noise
music that has no boundaries. On the other hand, the “easy listening” music is trying to mimic
something that it isn’t; thus it makes something sound off and strange.
Achievements
This was a very enjoyable project to work on, perhaps because we had no expectations for the outcomes, and mainly focused on the process of experimenting with all the different AI softwares. The unexpected results that emerged let our team to just go with the flow of the AI’s outcome; we weren’t the sole creators that controlled everything.