Malicious Hugging Face AI Model Posing as OpenAI Delivered Infostealer Malware
In a stark reminder that the AI gold rush comes with poisonous nuggets, a Hugging Face repository masquerading as an official OpenAI release delivered c...
In a stark reminder that the AI gold rush comes with poisonous nuggets, a Hugging Face repository masquerading as an official OpenAI release delivered credential-stealing malware to Windows machines before being taken down — and it racked up an estimated 244,000 downloads along the way. That number may have been artificially inflated by the attackers, but the real danger is that developers and data scientists routinely clone such models directly into corporate environments, handing attackers a golden ticket to source code, cloud credentials, and internal systems. This isn’t just another phishing scam; it’s a supply-chain time bomb aimed squarely at the AI ecosystem.
What Is Hugging Face and Why Is It a Target?
Hugging Face is the world’s largest public repository for AI models, hosting thousands of open-source models, datasets, and inference scripts. Developers often pull these artifacts directly into Docker containers, CI/CD pipelines, or even production environments without rigorous security checks. The platform’s popularity — combined with the assumption that “open source is safe” — makes it a prime vector for supply-chain attacks.
- Model registries like Hugging Face are becoming the new npm or PyPI for AI teams — which means attackers are following the same playbook.
- The attack does not target the model file itself (e.g.,
.pth,.h5), but the setup scripts, loader files, and README instructions that accompany it. - Previous warnings about Pickle-serialized exploits already highlighted that malicious code can be hidden inside model files, but this attack used a simpler, script-based infection chain.
The Core News: Fake OpenAI Repository “Open-OSS/privacy-filter”
Security firm HiddenLayer discovered that the repository Open-OSS/privacy-filter was an almost perfect clone of OpenAI’s legitimate Privacy Filter release. The fake repo copied the original model card, branding, and even instructions — with one critical difference: it told users to run start.bat on Windows or python loader.py on Linux/macOS. That script was the infection vector.
- The repository reached the top of the trending list on Hugging Face within 18 hours, gathering 667 likes (likely manipulated by the attackers).
- HiddenLayer confirmed that the actual download count of ~244,000 may have been inflated via bots or automated scripts to increase perceived legitimacy.
- The malicious code was designed to evade detection:
loader.pybegan with decoy code that looked like a normal AI model loader before executing the hidden payload.
| Real Repository | Fake Repository |
|---|---|
| Official OpenAI Privacy Filter | Open-OSS/privacy-filter |
| Legitimate model card | Copied almost verbatim |
| Standard install instructions | Instructs to run loader.py or start.bat |
| Safe | Delivers Rust-based infostealer |
Why This Matters: The AI Supply Chain Is Wide Open
This attack is not a one-off. AI development workflows are uniquely vulnerable because they combine executable code, model binaries, dependency files, and notebooks — all in one easily shareable package. Traditional Software Composition Analysis (SCA) tools inspect libraries and container images, but they aren’t designed to sniff out malicious loader logic in a repository’s README or startup script.
- Developers trust Hugging Face as a peer-reviewed ecosystem, but the platform’s scanning primarily targets Pickle-based exploits, not scripting-layer attacks.
- The malware targeted Windows machines, extracting credentials from Chromium/Firefox browsers, Discord, cryptocurrency wallets, FileZilla, and system info.
- It also attempted to disable Windows Antimalware Scan Interface and Event Tracing, making it harder for defenders to detect the infection.
The stakes: if a developer clones a compromised model into a corporate environment, the attacker can pivot to cloud consoles, CI/CD secrets, and production databases. This is not a theoretical risk — IDC’s November 2025 FutureScape report predicts that by 2027, 60% of agentic AI systems will require a bill of materials to track artifacts, source, and versioning.
Technical Breakdown: How the Infection Chain Worked
The Loader Script
HiddenLayer’s analysis revealed that loader.py executed the following sequence:
- Decoy code that looked like a normal model loading routine.
- Disabled SSL verification to bypass certificate checks.
- Decoded a base64-encoded URL pointing to
jsonkeeper.com— a legitimate command-and-control (C2) service. - Fetched a remote instruction from that C2 and passed it to PowerShell on Windows.
- The PowerShell command downloaded a batch file from an attacker-controlled domain.
- The batch file established persistence by creating a scheduled task disguised as a Microsoft Edge update.
- The final payload was a Rust-based infostealer that exfiltrated browser data, crypto wallets, and system credentials.
Wider Campaign
HiddenLayer also identified six additional Hugging Face repositories using nearly identical loader logic and sharing the same infrastructure. This suggests an organized campaign aimed at compromising AI development environments at scale.
Mitigation Steps
- Anyone who cloned
Open-OSS/privacy-filterand ran the files on Windows should treat the system as compromised and re-image it. - Browser sessions should be invalidated — even if passwords are not stored locally, session cookies can bypass MFA in some cases.
- Hugging Face has removed the repository, but similar repos may still exist.
Competitive Landscape: How Other Players Are Responding
Other public model registries face similar threats:
| Platform | Known Vulnerabilities | Security Measures |
|---|---|---|
| Hugging Face | Pickle deserialization, malicious scripts in notebooks/model cards | Pickle scanning, but no deep script sandboxing |
| GitHub | Poisoned repositories, fake stars, compromised actions | Code scanning, Dependabot, but not model-specific |
| PyPI / npm | Dependency confusion, malicious packages | Manual review, automated scanning, 2FA |
| TensorFlow Hub | Model weight poisoning, backdoor triggers | Limited scanning, mostly trust-based |
The key gap: no platform currently sandsboxes setup scripts or validates that a repository’s instructions match its intended behavior. Attackers can simply copy a legitimate README and swap out the “install” command.
What This Means for AI-Tool and AI-News Publishers
For Indian developers, content creators, and AI newsletter writers, this story offers several actionable angles:
- Write a practical guide: “How to Verify a Hugging Face Model Is Safe Before You Clone It” — show steps like checking contributor history, verifying hashes, and running scripts in a sandbox.
- Create a checklist for AI teams: Include warnings against running
setup.pyorloader.pydirectly from untrusted repos; recommend using Docker containers with no internet access. - SEO opportunity: Keywords like “Hugging Face malware”, “AI supply chain attack 2025”, “OpenAI fake model” are trending. A detailed blog post with comparisons will perform well.
- Interview angle: Reach out to HiddenLayer or IDC’s Sakshi Grover for quotes on the need for AI bill of materials — this is a long-term trend.
- Tool comparison: Write a piece comparing SCA tools (Snyk, Black Duck, Trivy) with AI-specific scanners (HiddenLayer’s own tool, Lacework). Include a table of features.
Challenges Ahead / Risks / Limitations
- Attribution is hard: The attackers used jsonkeeper.com as a C2, making it easy to rotate payloads without changing the repo.
- Detection is reactive: Traditional security tools miss script-based attacks that don’t match known malware signatures.
- Trust-based culture: The AI community values openness and rapid iteration, but that directly conflicts with security best practices.
- Platform responsibility: Hugging Face’s scanning currently focuses on Pickle files, not shell scripts. Until that changes, similar attacks will succeed.
- User behavior: Developers often skip reading README thoroughly — they just run the command. The fake repo’s “python loader.py” instruction blended in perfectly.
Final Thoughts
The Open-OSS privacy-filter attack is a wake-up call that the AI ecosystem cannot rely on trust alone. As AI models become embedded in everything from chatbots to enterprise agents, the supply chain must adopt the same maturity as traditional software — including signed artifacts, reproducible builds, and mandatory scanning of executable scripts. The attackers are already moving; the rest of us need to catch up before the next 244,000-download incident hits closer to home.
FAQ
How did the fake Hugging Face repo manage to get so many downloads?
The attackers likely used bots or automated scripts to inflate the download count and likes, making the repository appear popular and trustworthy to real developers.
What operating systems were affected by this malware?
The infection chain was designed to target Windows machines specifically, using PowerShell to execute the final payload. Linux and macOS users who ran loader.py may have triggered decoy code but not the full infection, though HiddenLayer advises caution.
Can I get infected just by browsing a malicious Hugging Face model card?
No — the infection requires a user to download and run the included scripts (start.bat or python loader.py). Simply viewing the repository page or model card is safe.
Has Hugging Face taken down all similar malicious repositories?
Hugging Face removed the identified Open-OSS/privacy-filter repo, but HiddenLayer found six additional repos with identical logic. The platform is likely investigating further, but more may emerge.
What should I do if I already cloned and ran files from a suspicious Hugging Face repo?
Treat the machine as compromised immediately. Re-image the operating system, rotate all passwords, invalidate browser sessions, and check for unauthorized access to cloud services.
Will this attack change how AI model registries are secured in the future?
Yes — industry analysts like IDC predict that by 2027, 60% of agentic AI systems will require a bill of materials and provenance tracking. Platforms may also start sandboxing setup scripts and requiring code signing for uploaded repositories.
