Cloud security provider Wiz found two critical architecture flaws in generative AI models uploaded to Hugging Face, the leading hub for sharing AI models and applications.
In a blog post published on April 4, Wiz Research described the two flaws and the risk they could pose to AI-as-a-service providers.
These are:
- Shared Inference infrastructure takeover risk
- Shared Continuous Integration and Continuous Deployment (CI/CD) takeover risk
Shared Inference Infrastructure Takeover Risk
Upon analyzing several AI models uploaded on Hugging Face, Wiz researchers discovered that some were sharing inference infrastructure.
In the context of generative AI, inference refers to a model that makes predictions or decisions based on previously trained models and input data.
The inference infrastructure allows execution of an AI model — either “on edge” (e.g. Transformers.js), via an application programming interface (API) or following an Inference-as-a-Service model (e.g. Hugging Face’s Inference Endpoints).
“Our research found that inference infrastructure often runs untrusted, potentially malicious models that use the ‘pickle’ format,” Wiz researchers wrote.
An AI model in ‘pickle’ format is a serialized, compressed version of a trained model saved using the Python pickle module.
Because it’s a compressed version of the model, it is more compact and takes up less space than storing the raw training data.
However, Wiz noted that malicious pickle-serialized models could contain remote code execution payloads, potentially granting the attacker escalated privileges and cross-tenant access to other customers' models.
Shared CI/CD Takeover Risk
A continuous integration and continuous deployment (CI/CD) pipeline is an automated software development workflow that streamlines the process of building, testing and deploying applications.
It essentially automates the steps that would otherwise be done manually, leading to faster releases and fewer errors.
Wiz researchers found that attackers may attempt to take over the CI/CD pipeline itself and perform a supply chain attack.
No tags.
