Facebook’s new AI stickers can generate Elmo with a knife

Meta's AI-generated stickers feature in Facebook Messenger and Instagram Messenger is being misused to create potentially offensive images, including copyright-protected characters. The feature, which is available to some users on a limited basis, allows the creation of AI-generated sticker images from text-based descriptions. Despite filters to catch offensive content, many inappropriate combinations are slipping through. Some of the questionable images shared include violent or inappropriate depictions of popular characters and public figures.

This isn't the first time AI-generated imagery has caused issues on social media, with similar problems arising from uncensored open-source image models. The incident highlights the challenges facing moderation teams in managing AI-powered apps and online spaces. In response to the controversy, Meta has indicated that it will continue to improve the features and strengthen filters based on user feedback.

Key takeaways

Meta's new AI-generated stickers feature in Facebook Messenger and Instagram Messenger is being misused to create potentially offensive images of copyright-protected characters.
The AI stickers feature allows users to create simulated sticker images from text-based descriptions, which are then shared in chats.
Despite filters to catch offensive content, many inappropriate combinations are slipping through, including violent or inappropriate depictions of well-known characters and figures.
Meta has been criticized for releasing this feature without stricter safeguards, and it's unclear whether they will remove the feature or strengthen the filters in response to the misuse.

Facebook’s new AI stickers can generate Elmo with a knife

Key takeaways

Discussion (0)