Malicious Hugging Face AI Model Posing as OpenAI Delivered Infostealer Malware

In a stark reminder that the AI gold rush comes with poisonous nuggets, a Hugging Face repository masquerading as an official OpenAI release delivered credential-stealing malware to Windows machines before being taken down — and it racked up an estimated 244,000 downloads along the way. That number may have been artificially inflated by the attackers, but the real danger is that developers and data scientists routinely clone such models directly into corporate environments, handing attackers a golden ticket to source code, cloud credentials, and internal systems. This isn’t just another phishing scam; it’s a supply-chain time bomb aimed squarely at the AI ecosystem.

---

What Is Hugging Face and Why Is It a Target?

Hugging Face is the world’s largest public repository for AI models, hosting thousands of open-source models, datasets, and inference scripts. Developers often pull these artifacts directly into Docker containers, CI/CD pipelines, or even production environments without rigorous security checks. The platform’s popularity — combined with the assumption that “open source is safe” — makes it a prime vector for supply-chain attacks.

Image: Developers often clone models directly into corporate environments, exposing sensitive systems.

• Model registries like Hugging Face are becoming the new npm or PyPI for AI teams — which means attackers are following the same playbook.
• The attack does not target the model file itself (e.g., .pth, .h5), but the setup scripts, loader files, and README instructions that accompany it.
• Previous warnings about Pickle-serialized exploits already highlighted that malicious code can be hidden inside model files, but this attack used a simpler, script-based infection chain.

The Core News: Fake OpenAI Repository “Open-OSS/privacy-filter”

Security firm HiddenLayer discovered that the repository Open-OSS/privacy-filter was an almost perfect clone of OpenAI’s legitimate Privacy Filter release. The fake repo copied the original model card, branding, and even instructions — with one critical difference: it told users to run start.bat on Windows or python loader.py on Linux/macOS. That script was the infection vector.

• The repository reached the top of the trending list on Hugging Face within 18 hours, gathering 667 likes (likely manipulated by the attackers).
• HiddenLayer confirmed that the actual download count of ~244,000 may have been inflated via bots or automated scripts to increase perceived legitimacy.
• The malicious code was designed to evade detection: loader.py began with decoy code that looked like a normal AI model loader before executing the hidden payload.

Real Repository	Fake Repository
Official OpenAI Privacy Filter	Open-OSS/privacy-filter
Legitimate model card	Copied almost verbatim
Standard install instructions	Instructs to run loader.py or start.bat
Safe	Delivers Rust-based infostealer

---

Why This Matters: The AI Supply Chain Is Wide Open

This attack is not a one-off. AI development workflows are uniquely vulnerable because they combine executable code, model binaries, dependency files, and notebooks — all in one easily shareable package. Traditional Software Composition Analysis (SCA) tools inspect libraries and container images, but they aren’t designed to sniff out malicious loader logic in a repository’s README or startup script.

• Developers trust Hugging Face as a peer-reviewed ecosystem, but the platform’s scanning primarily targets Pickle-based exploits, not scripting-layer attacks.
• The malware targeted Windows machines, extracting credentials from Chromium/Firefox browsers, Discord, cryptocurrency wallets, FileZilla, and system info.
• It also attempted to disable Windows Antimalware Scan Interface and Event Tracing, making it harder for defenders to detect the infection.

The stakes: if a developer clones a compromised model into a corporate environment, the attacker can pivot to cloud consoles, CI/CD secrets, and production databases. This is not a theoretical risk — IDC’s November 2025 FutureScape report predicts that by 2027, 60% of agentic AI systems will require a bill of materials to track artifacts, source, and versioning.

Image: The AI supply chain is increasingly targeted by attackers exploiting trust in open-source repositories.

Technical Breakdown: How the Infection Chain Worked

The Loader Script

HiddenLayer’s analysis revealed that loader.py executed the following sequence:

1. Decoy code that looked like a normal model loading routine.
2. Disabled SSL verification to bypass certificate checks.
3. Decoded a base64-encoded URL pointing to jsonkeeper.com — a legitimate command-and-control (C2) service.
4. Fetched a remote instruction from that C2 and passed it to PowerShell on Windows.
5. The PowerShell command downloaded a batch file from an attacker-controlled domain.
6. The batch file established persistence by creating a scheduled task disguised as a Microsoft Edge update.
7. The final payload was a Rust-based infostealer that exfiltrated browser data, crypto wallets, and system credentials.

Wider Campaign

HiddenLayer also identified six additional Hugging Face repositories using nearly identical loader logic and sharing the same infrastructure. This suggests an organized campaign aimed at compromising AI development environments at scale.

Mitigation Steps

• Anyone who cloned Open-OSS/privacy-filter and ran the files on Windows should treat the system as compromised and re-image it.
• Browser sessions should be invalidated — even if passwords are not stored locally, session cookies can bypass MFA in some cases.
• Hugging Face has removed the repository, but similar repos may still exist.

Competitive Landscape: How Other Players Are Responding

Other public model registries face similar threats:

Platform	Known Vulnerabilities	Security Measures
Hugging Face	Pickle deserialization, malicious scripts in notebooks/model cards	Pickle scanning, but no deep script sandboxing
GitHub	Poisoned repositories, fake stars, compromised actions	Code scanning, Dependabot, but not model-specific
PyPI / npm	Dependency confusion, malicious packages	Manual review, automated scanning, 2FA
TensorFlow Hub	Model weight poisoning, backdoor triggers	Limited scanning, mostly trust-based

The key gap: no platform currently sandsboxes setup scripts or validates that a repository’s instructions match its intended behavior. Attackers can simply copy a legitimate README and swap out the “install” command.

---

What This Means for AI-Tool and AI-News Publishers

For Indian developers, content creators, and AI newsletter writers, this story offers several actionable angles:

1. Write a practical guide: “How to Verify a Hugging Face Model Is Safe Before You Clone It” — show steps like checking contributor history, verifying hashes, and running scripts in a sandbox.
2. Create a checklist for AI teams: Include warnings against running setup.py or loader.py directly from untrusted repos; recommend using Docker containers with no internet access.
3. SEO opportunity: Keywords like “Hugging Face malware”, “AI supply chain attack 2025”, “OpenAI fake model” are trending. A detailed blog post with comparisons will perform well.
4. Interview angle: Reach out to HiddenLayer or IDC’s Sakshi Grover for quotes on the need for AI bill of materials — this is a long-term trend.
5. Tool comparison: Write a piece comparing SCA tools (Snyk, Black Duck, Trivy) with AI-specific scanners (HiddenLayer’s own tool, Lacework). Include a table of features.

Challenges Ahead / Risks / Limitations

• Attribution is hard: The attackers used jsonkeeper.com as a C2, making it easy to rotate payloads without changing the repo.
• Detection is reactive: Traditional security tools miss script-based attacks that don’t match known malware signatures.
• Trust-based culture: The AI community values openness and rapid iteration, but that directly conflicts with security best practices.
• Platform responsibility: Hugging Face’s scanning currently focuses on Pickle files, not shell scripts. Until that changes, similar attacks will succeed.
• User behavior: Developers often skip reading README thoroughly — they just run the command. The fake repo’s “python loader.py” instruction blended in perfectly.

---

Final Thoughts

The Open-OSS privacy-filter attack is a wake-up call that the AI ecosystem cannot rely on trust alone. As AI models become embedded in everything from chatbots to enterprise agents, the supply chain must adopt the same maturity as traditional software — including signed artifacts, reproducible builds, and mandatory scanning of executable scripts. The attackers are already moving; the rest of us need to catch up before the next 244,000-download incident hits closer to home.

---

FAQ

How did the fake Hugging Face repo manage to get so many downloads?

The attackers likely used bots or automated scripts to inflate the download count and likes, making the repository appear popular and trustworthy to real developers.

What operating systems were affected by this malware?

The infection chain was designed to target Windows machines specifically, using PowerShell to execute the final payload. Linux and macOS users who ran loader.py may have triggered decoy code but not the full infection, though HiddenLayer advises caution.

Can I get infected just by browsing a malicious Hugging Face model card?

No — the infection requires a user to download and run the included scripts (start.bat or python loader.py). Simply viewing the repository page or model card is safe.

Has Hugging Face taken down all similar malicious repositories?

Hugging Face removed the identified Open-OSS/privacy-filter repo, but HiddenLayer found six additional repos with identical logic. The platform is likely investigating further, but more may emerge.

What should I do if I already cloned and ran files from a suspicious Hugging Face repo?

Treat the machine as compromised immediately. Re-image the operating system, rotate all passwords, invalidate browser sessions, and check for unauthorized access to cloud services.

Will this attack change how AI model registries are secured in the future?

Yes — industry analysts like IDC predict that by 2027, 60% of agentic AI systems will require a bill of materials and provenance tracking. Platforms may also start sandboxing setup scripts and requiring code signing for uploaded repositories.