People are using AI music generators to create hateful songs

ActiveFence, an online trust and safety operations management service, has reported a rise in the misuse of AI music creation tools by malicious actors to create and spread offensive songs targeting minority groups. These songs, which are being shared in hate speech-related communities, aim to incite hatred towards ethnic, gender, racial, and religious groups. The concern is that with the availability of free music-generating tools, such content could be produced at scale, similar to the spread of misinformation and hate speech through image, voice, video, and text generators.

Generative AI music tools like Udio and Suno, which allow users to add custom lyrics to generated songs, are being exploited. Users have found ways to bypass the platforms' content filters by using phonetic spellings of offensive terms or altering spacings and spellings. ActiveFence has found links to AI-generated songs promoting hate speech, conspiracy theories, and violence. The company's CEO, Noam Schwartz, has called on music generation platforms to implement prevention tools and conduct more extensive safety evaluations to curb this trend.

Key takeaways

Malicious actors are misusing generative AI music tools to create and spread offensive songs targeting minority groups, according to ActiveFence.
These AI-generated songs aim to incite hatred and celebrate acts of violence, and their creation and distribution are increasing as more users learn how to generate and share them.
Users are finding ways to bypass content filters on platforms like Udio and Suno by using phonetic spellings of offensive terms and altering spacings and spellings when referring to acts of violence.
ActiveFence's CEO, Noam Schwartz, calls on music generation platforms to implement prevention tools and conduct more extensive safety evaluations to combat this issue, but acknowledges that solutions may be temporary as users find new ways to defeat moderation.

People are using AI music generators to create hateful songs | TechCrunch

Key takeaways

Discussion (0)