Pillar Security Uncovers Novel Attack Vector That Embeds Malicious Backdoors in Model Files on Hugging Face

New supply chain vulnerability in popular GGUF model format allows attackers to embed malicious instructions that bypass existing AI security controls.

TEL EVIV, Israel, July 09, 2025 (GLOBE NEWSWIRE) -- Pillar Security, a leading company in AI security, discovered a novel supply chain attack vector that targets the AI inference pipeline. This novel technique, termed "Poisoned GGUF Templates," allows attackers to embed malicious instructions that are processed alongside legitimate inputs, compromising AI outputs.

The vulnerability affects the widely used GGUF (GPT-Generated Unified Format), a standard for AI deployment with over 1.5 million files distributed on public platforms like Hugging Face. By manipulating these templates, which define the conversational structure for an LLM, attackers can create a persistent compromise that affects every user interaction while remaining invisible to both users and security systems.

“We’re still in the early days of understanding the full range of AI supply chain security considerations,” said Ziv Karliner, CTO and Co-founder of Pillar Security. “Our research shows how the trust that powers platforms and open-source communities—while essential to AI progress—can also open the door to deeply embedded threats. As the AI ecosystem matures, we must rethink how AI assets are vetted, shared, and secured.”

How the "Poisoned GGUF Template" Attack Works

This attack vector exploits the trust placed in community-sourced AI models and the platforms that host them. The mechanism allows for a stealthy, persistent compromise:

• Attackers embed malicious, conditional instructions directly within a GGUF file’s chat template, a component that formats conversations for the AI model.
• The poisoned model is uploaded to a public repository. Attackers can exploit the platform’s UI to display a clean template online while the actual downloaded file contains the malicious version, bypassing standard reviews.
• The malicious instructions lie dormant until specific user prompts trigger them, at which point the model generates a compromised output.
  

"What makes this attack so effective is the disconnect between what's shown in the repository interface and what's actually running on users’ machines," added Pillar’s Ariel Fogel, who led the research. "It remains undetected by casual testing and most security tools."

The AI Inference Pipeline: A New Attack Surface

The “Poisoned GGUF Templates” attack targets a critical blind spot in current AI security architectures. Most security solutions focus on validating user inputs and filtering model outputs, but this attack occurs in the unmonitored space between them.

Because the malicious instructions are processed within the trusted inference environment, the attack evades existing defenses like system prompts and runtime monitoring. An attacker no longer needs to bypass the front door with a clever prompt; they can build a backdoor directly into the model file. This capability redefines the AI supply chain as a primary vector for compromise, where a single poisoned model can be integrated into thousands of downstream applications.

Responsible Disclosure

Pillar Security followed a responsible disclosure process, sharing its findings with vendors, including Hugging Face and LM Studio, in June 2025. The responses indicated that the platforms do not currently classify this as a direct platform vulnerability, placing the responsibility of vetting models on users. This stance highlights a significant accountability gap in the AI ecosystem.

Mitigation Strategies

The primary defense against this attack vector is the direct inspection of GGUF files to identify chat templates containing uncommon or non-standard instructions. Security teams should immediately:

• Audit GGUF Files: Deploy practical inspection techniques to examine GGUF files for suspicious template patterns. Look for unexpected conditional logic (if/else statements), hidden instructions, or other manipulations that deviate from standard chat formats.
• Move Beyond Prompt-Based Controls: This attack fundamentally challenges current AI security assumptions. Organizations must evolve beyond a reliance on system prompts and input/output filtering toward comprehensive template and processing pipeline security.
• Implement Provenance and Signing: A critical long-term strategy is to establish model provenance. This can include developing template allowlisting systems to ensure only verified templates are used in production.
  

The Pillar platform discovers and flags malicious GGUF files and other types of risks in the template layer.

Read the full report: https://www.pillar.security/blog/llm-backdoors-at-the-inference-level-the-threat-of-poisoned-templates

About Pillar Security

Pillar Security is a leading AI-security platform, providing companies full visibility and control to build and run secure AI systems. Founded by experts in offensive and defensive cybersecurity, Pillar secures the entire AI lifecycle - from development to deployment - through AI Discovery, AI Security Posture Management (AI-SPM), AI Red Teaming, and Adaptive Runtime Guardrails. Pillar empowers organizations to prevent data leakage, neutralize AI-specific threats, and comply with evolving regulations.

Contact person:

Hadar Yakir
info@pillar.security

A photo accompanying this announcement is available at https://www.globenewswire.com/NewsRoom/AttachmentNg/d767a026-13f9-419d-827f-7ade3b92a24d

Malicious GGUF Templates - Attack Surface

While most developers and AI security vendors focus on validating user inputs and filtering model outputs, this research reveals the critical blind spot between them: the chat template layer.

Legal Disclaimer:

EIN Presswire provides this news content "as is" without warranty of any kind. We do not accept any responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you have any complaints or copyright issues related to this article, kindly contact the author above.