Cybersecurity experts have discovered two malicious machine learning models on the Hugging Face platform, utilizing an unconventional masking technique using “broken” Pickle files. This method enabled the models to evade protective measures intended to detect malicious models.
According to Reverseglabs, the Pickle files extracted from Pytorch archives contained malicious Python code at the beginning. In both instances, the malicious payload was the Platform-Aware Reverse Shell, connected to a hardcoded IP address.
Identified as Nullifai, the method aims to circumvent protective mechanisms that analyze loaded models. The risky repositories on Hugging Face were pinpointed as glockkr1/Ballr7 and Who-R-U0000/00000000000000000000000000000000000.
Although these models are believed to be more of a proof of concept rather than part of an active supply chain attack, their presence raises concerns among cybersecurity professionals.
The Pickle serialization format, commonly used to distribute machine learning models, is known for its vulnerabilities. It permits the execution of arbitrary code upon loading and de-serialization, making it an appealing target for attacks.
The two malicious models were stored in Pytorch format, which is essentially a compressed Pickle file. However, instead of the standard ZIP archive employed by Pytorch, the attackers utilized the 7Z format. This tactic enabled the models to evade the Picklescan scanning system utilized by Hugging Face to detect suspicious files.
These models’ Pickle files disrupt the deserialization process immediately after executing the malicious code, resulting in an error during object analysis. Despite this, it was discovered that these “broken” files could still partially evade detection due to Picklescan’s characteristics.
The primary vulnerability lies in the sequential execution of Pickle’s instructions. If the malicious code is placed at the beginning of the file, it may be executed before the analysis tool detects a structural violation. In response to this discovery, Picklescan was promptly updated to address this issue.
This incident underscores the risks associated with utilizing the Pickle format to distribute machine learning models. Despite the security measures implemented by Hugging Face, threat actors persist in finding ways to bypass them. Enhancing the scrutiny and analysis of code is imperative for platforms that work with machine learning models.