JFrog's analysis suggests that some of these malicious uploads could be part of security research aimed at bypassing Hugging Face's security measures and collecting bug bounties. However, the public availability of these dangerous models presents a real and significant risk. The findings highlight the security risks posed by AI ML models, a problem that has not been adequately addressed by stakeholders and technology developers. JFrog calls for increased vigilance and proactive measures to protect the ecosystem from malicious actors.
Key takeaways:
- At least 100 instances of malicious AI ML models were found on the Hugging Face platform, some of which can execute code on the victim's machine, providing attackers with a persistent backdoor.
- JFrog's security team discovered these models despite Hugging Face's security measures, indicating a significant risk of data breaches and espionage attacks.
- One highlighted case involved a PyTorch model uploaded by a user named 'baller423', which contained a payload that could establish a reverse shell to a specified host, evading detection by embedding the malicious code within the trusted serialization process.
- JFrog suggests that some of the malicious uploads could be part of security research, but the public availability of these dangerous models poses a real risk that shouldn't be underestimated, calling for increased vigilance and proactive measures to protect the ecosystem.