Generative AI music tools like Udio and Suno, which allow users to add custom lyrics to generated songs, are being exploited. Users have found ways to bypass the platforms' content filters by using phonetic spellings of offensive terms or altering spacings and spellings. ActiveFence has found links to AI-generated songs promoting hate speech, conspiracy theories, and violence. The company's CEO, Noam Schwartz, has called on music generation platforms to implement prevention tools and conduct more extensive safety evaluations to curb this trend.
Key takeaways:
- Malicious actors are misusing generative AI music tools to create and spread offensive songs targeting minority groups, according to ActiveFence.
- These AI-generated songs aim to incite hatred and celebrate acts of violence, and their creation and distribution are increasing as more users learn how to generate and share them.
- Users are finding ways to bypass content filters on platforms like Udio and Suno by using phonetic spellings of offensive terms and altering spacings and spellings when referring to acts of violence.
- ActiveFence's CEO, Noam Schwartz, calls on music generation platforms to implement prevention tools and conduct more extensive safety evaluations to combat this issue, but acknowledges that solutions may be temporary as users find new ways to defeat moderation.