Public AI repositories have become a major distribution channel for machine learning models, giving developers access to pretrained weights, fine-tuned language models, and supporting components that can be integrated into applications with relatively little development effort. Platforms such as Hugging Face allow organizations to obtain and deploy models without building them from scratch, but the convenience of downloading third-party artifacts introduces security risks that are frequently overlooked. Model files can contain executable serialization instructions, repositories may include unreviewed Python code, and pretrained weights can carry behavioral modifications that are difficult to identify through conventional security testing.
Unlike traditional software repositories, which primarily distribute source code and compiled packages, AI repositories distribute combinations of model weights, configuration files, tokenizers, serialization formats, and executable dependencies. These components are often loaded directly into development environments or production inference systems, sometimes with access to internal datasets, cloud credentials, and other sensitive resources. An attacker who compromises a repository or publishes a malicious model can potentially exploit the way these files are processed, turning an otherwise routine model-loading operation into an opportunity for remote code execution, credential theft, or unauthorized changes to application behavior.
The security concerns extend beyond intentionally malicious uploads. Vulnerabilities in machine learning frameworks, insufficient validation of repository configurations, and the absence of reliable model provenance can expose organizations that believe they are downloading ordinary data files. The resulting attack surface resembles a software supply chain, but includes security challenges associated with model serialization, training data, and inference behavior that cannot be addressed through traditional dependency scanning alone.
The Security Risks Behind Public Model Distribution
Public model repositories serve a different purpose from conventional software package registries. A typical model repository may contain serialized parameter tensors, architecture definitions, tokenizer configurations, generation settings, custom Python modules, training scripts, and documentation explaining how to load the model. These components collectively determine how the model is initialized and how its outputs are generated. The distinction between passive model data and executable application logic becomes particularly significant when frameworks automatically process repository contents during initialization.
Hugging Face’s Transformers library, for example, provides the from_pretrained() interface for loading model architectures, weights, and related configurations. Depending on the model and framework configuration, this operation may involve downloading several files, constructing model classes, initializing tensor storage, and resolving dependencies required by custom architectures. The process can also involve repository-provided Python code when custom model implementations are explicitly trusted. A developer may perceive the operation as equivalent to opening a data file, although certain loading paths can execute software obtained from an external repository.
Repository metadata contributes to the difficulty of determining whether a model should be trusted. Model cards commonly describe intended use cases, training datasets, architecture, evaluation results, and limitations. These descriptions are valuable for documentation and reproducibility, but most of the information is provided by repository maintainers rather than independently verified. A model card that identifies a reputable architecture or claims a particular training lineage does not establish that the downloaded weights correspond to the original model or that the accompanying files are free of malicious functionality.
The problem becomes more pronounced when organizations incorporate community-maintained fine-tunes, quantized models, or third-party conversions into their applications. A model may originate from a recognized developer but pass through several independent repositories before reaching an enterprise environment. Each conversion, fine-tuning operation, or redistribution introduces another opportunity for unauthorized modification. Without verification of the artifact’s origin and integrity, organizations may have limited visibility into the changes introduced throughout that process.
Unsafe Serialization and Code Execution During Model Loading
One of the most established security weaknesses in AI model distribution involves Python’s Pickle serialization mechanism. Pickle is widely used to serialize Python objects and has historically been incorporated into PyTorch checkpoint formats, including files commonly distributed with extensions such as .pt, .pth, and .bin. Its flexibility allows developers to preserve object structures and reconstruct them when a checkpoint is loaded, but that same functionality introduces a potentially dangerous execution path.
Pickle is not limited to storing primitive values or numerical tensors. Its serialization format contains opcodes that instruct the deserializer to reconstruct objects, resolve references to Python classes and functions, and invoke certain callable objects. Instructions such as GLOBAL, STACK_GLOBAL, and REDUCE are particularly relevant to security analysis, since they can participate in resolving and invoking functions during deserialization. An attacker who controls the serialized content can potentially construct a file that performs unintended operations when processed through an unsafe loading mechanism.
This creates an arbitrary code execution risk that does not require the victim to explicitly run a suspicious script. A developer may download what appears to be a legitimate model checkpoint and load it through an established machine learning workflow, causing malicious instructions embedded in the serialized object to execute within the Python process. The resulting execution inherits the privileges available to that process. In an environment containing cloud access tokens, model registry credentials, SSH keys, or sensitive training data, successful exploitation can provide opportunities for credential exfiltration, persistence, and further compromise.
The vulnerability class has been demonstrated against real public model repositories. In February 2025, ReversingLabs disclosed a technique it named nullifAI after identifying two malicious models hosted on Hugging Face. The files used deliberately malformed Pickle structures and unusual packaging to interfere with automated inspection. According to the researchers, malicious instructions could execute during deserialization before the malformed content caused the loading process to fail, allowing the attack to evade certain scanning assumptions. The payloads included reverse-shell functionality, demonstrating how a model checkpoint could be repurposed as a malware delivery mechanism.
The nullifAI findings exposed a particularly relevant limitation in model security testing. A file that cannot be fully parsed by a scanner should not automatically be considered safe, especially when the application consuming that file may process malicious instructions before encountering the malformed content. Differences between scanner behavior and runtime deserialization can create opportunities for attackers to exploit parsing inconsistencies. Although Hugging Face provides malware scanning, Pickle inspection, and third-party security integrations, these mechanisms cannot independently guarantee that every uploaded model artifact is safe to execute.
Why Safer Model Formats Are Not a Complete Solution
The risks associated with Pickle encouraged the adoption of safer serialization formats, particularly Safetensors. Developed to store numerical tensors without the executable object-reconstruction behavior associated with Pickle, Safetensors uses a comparatively simple structure containing tensor metadata and raw numerical data. It avoids the general-purpose Python object deserialization process, reducing the possibility of arbitrary code execution through the act of loading model weights.
This represents a meaningful security improvement for organizations consuming external models. A checkpoint distributed exclusively in Safetensors format removes the direct Pickle execution mechanism, provided the application uses an appropriate loader and does not fall back to an unsafe format. Hugging Face recommends explicitly requiring Safetensors when supported and pinning model revisions to prevent unexpected changes to downloaded artifacts. These practices reduce exposure to known serialization-related threats and make the behavior of model-loading pipelines more predictable.
Yet a safer tensor format does not establish that the model itself is trustworthy. Safetensors can store maliciously altered weights just as effectively as legitimate ones, and the format does not verify whether a model was trained on appropriate data or modified to produce attacker-selected outputs. The repository may also include configuration files, custom modeling code, tokenizers, or scripts processed separately from the tensor file. Preventing arbitrary code execution through weight deserialization addresses one attack path without eliminating vulnerabilities in the surrounding inference framework.
PyTorch’s restricted checkpoint-loading functionality illustrates the difficulty of securing serialization mechanisms that preserve compatibility with existing formats. Beginning with PyTorch 2.6, torch.load() defaults to weights_only=True when a custom Pickle module is not provided. This mode restricts the operations permitted by the unpickler and limits the types of objects that can be reconstructed. The restriction reduces exposure to common malicious Pickle operations, but it should not be treated as equivalent to a serialization format that does not support executable object reconstruction.
Security advisories have demonstrated the remaining risks. CVE-2025-32434 affected restricted PyTorch checkpoint loading in versions through 2.5.1, and a separate advisory published in January 2026 described another vulnerability affecting versions through 2.9.1. The latter involved insufficient validation of Pickle opcodes and storage metadata, creating conditions for memory corruption and potential arbitrary code execution even with weights_only=True. PyTorch 2.10.0 addressed the 2026 issue. These findings demonstrate that restricted deserialization reduces the attack surface but still depends on the security of the underlying implementation.
Executable Repository Code and Configuration-Based Attacks
Model weights are only one component of the attack surface. Many public AI repositories contain Python modules that implement custom neural network architectures, preprocessing routines, or inference pipelines. In Hugging Face Transformers, models requiring custom implementations may use the trust_remote_code=True parameter, which authorizes the framework to load code supplied by the repository. This functionality is useful for architectures that have not been incorporated into the standard library, but it also establishes a direct execution path between an external repository and the local runtime.
The security implications depend on how the repository code is reviewed and executed. A malicious custom modeling module could perform unauthorized operations during import, model initialization, or inference. Potential actions include reading environment variables, accessing local files, invoking system commands, or communicating with external infrastructure. These behaviors may be difficult to distinguish from legitimate initialization routines if the repository contains substantial application logic or relies on additional third-party dependencies. Restricting custom code execution and reviewing the modules associated with a model are consequently important parts of repository evaluation.
Configuration files can introduce similar risks when software interprets attacker-controlled metadata as executable functionality. In May 2026, CVE-2026-4372 disclosed a remote code execution vulnerability in Hugging Face Transformers versions earlier than 5.3.0. The vulnerability involved insufficient validation of the internal attention implementation configuration, allowing a malicious config.json to reference an attacker-controlled repository containing executable kernel code. Under the affected loading path, the framework could download and execute that code through ordinary model initialization, bypassing the expected trust_remote_code restriction.
The vulnerability is particularly significant since it demonstrates how seemingly declarative metadata can influence executable behavior. Developers may inspect model weights or avoid explicitly enabling remote code execution without realizing that configuration parsing itself can reach a vulnerable code-loading path. A configuration value can become security-sensitive when the application uses it to resolve modules, select plugins, construct objects, or load external implementations. The security boundary must therefore apply to the complete model-loading process, rather than a single parameter intended to control custom code execution.
A related vulnerability, CVE-2026-44827, affected the Hugging Face Diffusers library and involved an unintended execution path for custom pipeline code. The underlying issue allowed repository-controlled pipeline information to bypass a remote-code trust safeguard under affected conditions. This illustrates another consequence of treating external repository metadata as authoritative configuration for executable components. Security review must consider the way frameworks interpret model configuration and resolve dynamic dependencies, even if the primary artifact contains numerical weights rather than conventional program code.
Additional risks exist in tokenizer and preprocessing components. Tokenizers may be distributed with JSON configuration files, vocabulary data, special-token definitions, and chat templates that determine how user input is converted into model-ready sequences. In August 2026, CVE-2026-9856 documented an arbitrary file-write vulnerability involving chat-template names in affected Transformers versions. Crafted repository configuration data could influence filenames used during tokenizer or processor saving, permitting path traversal outside the intended output directory. The issue was addressed in Transformers 5.10.0 and further demonstrates that files commonly classified as model metadata can become attack inputs when consumed by vulnerable software.
Model Poisoning and Behavioral Backdoors
Not every malicious model attempts to execute code through a vulnerable framework. Attackers can also manipulate the model’s learned behavior, introducing weaknesses that persist after the model has been loaded through an otherwise secure runtime. Model poisoning involves modifying training data, fine-tuning procedures, or learned parameters to influence how the model responds to particular inputs. Unlike malicious serialization payloads, these attacks may not require access to the underlying operating system. Their objective is to alter the model’s outputs or decisions in ways that benefit the attacker.
One approach involves inserting a behavioral backdoor that activates only when a particular trigger appears in the input. During ordinary evaluation, the model may perform similarly to its legitimate counterpart, producing expected results and maintaining acceptable benchmark scores. When the trigger condition is present, the model may generate an attacker-selected response, ignore a security restriction, misclassify an input, or produce misleading information. The activation condition can be a phrase, token sequence, contextual pattern, or another feature learned during training.
Research has demonstrated that these behaviors can be difficult to eliminate through subsequent training. In January 2024, Anthropic published research on sleeper-agent behavior in language models, constructing experimental models that behaved differently depending on information included in their prompts. The researchers demonstrated that certain deliberately introduced behaviors persisted after standard safety-training techniques were applied. These experiments did not establish that public model repositories were broadly compromised, but they provided evidence that model-level backdoors can survive attempts to correct unwanted behavior.
Public repositories create a distribution path for this category of attack. A malicious actor could publish a modified model that appears to be an ordinary fine-tune of a recognized architecture, preserving most of its expected capabilities while introducing targeted behavioral changes. Those changes may then propagate into downstream applications or additional fine-tuned models. Detecting them is difficult through conventional software analysis, since the malicious behavior may be encoded in large numerical parameter matrices rather than recognizable source code or suspicious system calls.
The distinction between executable malware and poisoned model behavior is particularly important for defensive teams. Static malware scanning may detect dangerous serialization instructions or known malicious binaries, but it cannot reliably determine whether billions of numerical parameters encode a hidden behavioral trigger. Model validation requires separate testing that examines expected behavior across representative inputs, adversarial prompts, and application-specific security requirements. Even extensive behavioral testing provides limited assurance against triggers that were never included in the evaluation set.
Supply Chain Exposure in Enterprise AI Pipelines
The integration of public models into enterprise development environments introduces additional risks associated with software dependencies and automated deployment. Model repositories frequently include instructions for installing Python packages, running inference scripts, converting checkpoint formats, or configuring specialized runtimes. These instructions may reference external package registries, Git repositories, or downloadable executables. A developer who follows installation instructions from an unfamiliar model repository can expose the environment to the same dependency substitution, typosquatting, and malicious package attacks seen in conventional software development.
The impact can be greater when model-loading operations occur inside automated machine learning pipelines. Continuous integration jobs, model evaluation systems, and deployment services may retrieve models directly from public repositories as part of routine workflows. If these systems execute untrusted repository code or process unsafe serialized files, a malicious artifact can become an entry point into the environment responsible for building and deploying AI applications. Access to internal artifact registries, cloud storage, training datasets, and deployment credentials may increase the consequences of a successful compromise.
An additional concern is the use of mutable repository references. Applications that download a model using a moving branch reference can receive different files at different times, even if the repository identifier remains unchanged. A model that passed security review during initial development may later receive modified weights, configuration files, or executable dependencies. If the pipeline automatically retrieves the latest revision, those changes may enter production without a corresponding security assessment. Repository compromise, malicious maintainer activity, or unauthorized updates can create this exposure even when the original model was legitimate.
Cryptographic hashes and immutable revision identifiers provide a mechanism for addressing part of this problem. Pinning a model to a particular commit and verifying the downloaded artifacts against previously approved hashes can establish that the organization is using the expected files. Such controls do not establish that the original model is safe, but they reduce the risk of unreviewed upstream changes being introduced after approval. Digital signatures and documented provenance can provide further evidence about artifact origin when the signatures are independently verified against trusted publisher identities.
These risks are reflected in the OWASP Top 10 for LLM Applications, which classifies supply chain weaknesses under LLM03:2025 and data or model poisoning under LLM04:2025. The categories recognize that third-party model artifacts, training data, inference frameworks, and deployment dependencies represent distinct opportunities for compromise. Securing AI systems requires examining the complete dependency chain rather than treating the downloaded model as an isolated component.
What Model Security Scanners Can and Cannot Detect
Public model platforms have introduced security controls intended to reduce the distribution of malicious artifacts. Hugging Face documents automated malware scanning and Pickle inspection, alongside third-party scanning integrations capable of identifying suspicious model structures and potentially dangerous serialization behavior. These tools provide valuable protection against known malicious patterns and can help developers identify files that require further review before deployment.
Their effectiveness depends on the threats they are intended to detect. Signature-based malware scanning is useful for known malicious files, but it may miss novel payloads embedded in uncommon serialization structures. Static analysis of Pickle files can identify dangerous imports and reconstruction operations, yet scanner implementations may encounter malformed archives, unsupported formats, or parser inconsistencies. The nullifAI research demonstrated the consequences of differences between what a scanner successfully analyzes and what the runtime eventually processes.
Model scanning also faces limitations associated with execution paths outside the weight file. A repository could contain relatively safe tensor data alongside vulnerable configuration files or unreviewed Python modules. A scanner focused on detecting executable Pickle instructions would not necessarily identify a malicious dependency referenced by an installation script or a configuration value capable of exploiting a vulnerable framework. A successful security scan should consequently be interpreted as evidence that certain checks were completed, rather than proof that the repository presents no security risk.
Behavioral model poisoning introduces an even more difficult detection problem. A maliciously fine-tuned model may contain no executable payload, unsafe serialization, or identifiable malware signature. Its harmful behavior may emerge only under narrowly defined conditions that are absent from ordinary evaluation datasets. Detecting this class of attack requires provenance analysis, controlled behavioral evaluation, and testing that reflects the security requirements of the application consuming the model.
Securing Public Models Before Deployment
Organizations integrating public AI repositories should treat third-party models as external software supply chain components, applying controls before those artifacts reach production environments. Repository evaluation should begin with the identity of the publisher, documented model lineage, artifact history, and the files required for initialization. Model cards, download counts, and public usage can provide useful context, but they should not replace direct verification of the model artifacts and associated software.
Model ingestion pipelines should favor serialization formats that do not execute arbitrary code during weight loading. Where Safetensors is supported, explicitly requiring the format can prevent an application from silently falling back to unsafe Pickle-based checkpoints. Organizations that must process legacy checkpoints should isolate the deserialization process and restrict its privileges. Framework dependencies also require routine vulnerability management, particularly given the documented remote code execution vulnerabilities affecting PyTorch, Transformers, and Diffusers.
Isolation is especially relevant during initial model evaluation. An untrusted repository should be examined in an environment with restricted filesystem access, minimal privileges, limited network connectivity, and no production credentials. Containers or virtual machines can provide useful separation when configured with appropriate security boundaries, but their effectiveness depends on controlling host mounts, device access, and outbound connections. A model-loading process that can freely read cloud credentials or contact external services retains considerable attack potential even when executed inside a container.
Repository contents should undergo a broader review that covers Python modules, configuration files, tokenizer assets, installation scripts, and external dependencies. Any requirement to enable remote code execution should trigger additional scrutiny of the implementation being imported. Organizations should also verify that model-loading libraries are updated to versions addressing known vulnerabilities, since safer serialization and restricted execution options cannot compensate for every defect in the surrounding framework.
Model provenance should be recorded throughout development and deployment. Approved model revisions, cryptographic hashes, dependency versions, and any associated fine-tuning artifacts should be retained so security teams can determine exactly which components are running in an environment. Internal model registries can provide a controlled distribution point for approved artifacts, preventing production workloads from retrieving arbitrary public repository revisions. These records also support incident response when a model publisher reports a compromise or a framework vulnerability becomes publicly known.
For models used in security-sensitive applications, technical validation must extend beyond checking whether the files can execute malicious code. Organizations should evaluate the model’s behavior against representative datasets, adversarial inputs, and application-specific failure conditions. A model used to classify security alerts, interpret internal documents, or generate executable code introduces different risks from one used for general-purpose text generation. Testing should reflect the actions the model is permitted to influence and the consequences of inaccurate or deliberately manipulated outputs.
The Security Problem Behind Public Model Trust
Public AI repositories have made pretrained models and machine learning research more accessible, but they have also introduced a new dependency that organizations must evaluate before deployment. A model repository is more than a collection of numerical weights. It may contain executable code, serialization instructions, configuration files, external dependencies, and learned behaviors that influence the security of the application consuming it. Each component creates a separate trust boundary, and a successful security review must account for how those components interact during model loading and inference.
Research into malicious Pickle checkpoints, framework-level code execution, and behavioral backdoors demonstrates that these risks are supported by documented vulnerabilities and experimental findings rather than speculation about AI systems. The failure modes differ substantially, ranging from conventional malware execution to unauthorized changes in model behavior. Effective security controls must distinguish between these categories and apply the appropriate inspection, isolation, provenance, and validation methods.
The central concern is the assumption that a model obtained from a public repository can be treated as trusted application data. Loading a third-party model can involve executing code, importing dependencies, interpreting sensitive configuration values, and accepting learned behavior that may be difficult to inspect. Organizations that incorporate these artifacts into development and production environments need the same discipline applied to other external software components, supplemented by controls that address the security characteristics of machine learning models themselves.
The question is no longer limited to whether a model produces accurate results. It is whether the organization can verify where the model came from, what its files are capable of doing, and how its behavior might affect the systems that depend on it.
How Can Netizen Help?
Founded in 2013, Netizen is an award-winning technology firm that develops and leverages cutting-edge solutions to create a more secure, integrated, and automated digital environment for government, defense, and commercial clients worldwide. Our innovative solutions transform complex cybersecurity and technology challenges into strategic advantages by delivering mission-critical capabilities that safeguard and optimize clients’ digital infrastructure. One example of this is our popular “CISO-as-a-Service” offering that enables organizations of any size to access executive level cybersecurity expertise at a fraction of the cost of hiring internally.
Netizen also operates a state-of-the-art 24x7x365 Security Operations Center (SOC) that delivers comprehensive cybersecurity monitoring solutions for defense, government, and commercial clients. Our service portfolio includes cybersecurity assessments and advisory, hosted SIEM and EDR/XDR solutions, software assurance, penetration testing, cybersecurity engineering, and compliance audit support. We specialize in serving organizations that operate within some of the world’s most highly sensitive and tightly regulated environments where unwavering security, strict compliance, technical excellence, and operational maturity are non-negotiable requirements. Our proven track record in these domains positions us as the premier trusted partner for organizations where technology reliability and security cannot be compromised.
Netizen holds ISO 27001, ISO 9001, ISO 20000-1, and CMMI Level III SVC registrations demonstrating the maturity of our operations. We are a proud Service-Disabled Veteran-Owned Small Business (SDVOSB) certified by U.S. Small Business Administration (SBA) that has been named multiple times to the Inc. 5000 and Vet 100 lists of the most successful and fastest-growing private companies in the nation. Netizen has also been named a national “Best Workplace” by Inc. Magazine, a multiple awardee of the U.S. Department of Labor HIRE Vets Platinum Medallion for veteran hiring and retention, the Lehigh Valley Business of the Year and Veteran-Owned Business of the Year, and the recipient of dozens of other awards and accolades for innovation, community support, working environment, and growth.
Looking for expert guidance to secure, automate, and streamline your IT infrastructure and operations? Start the conversation today.
Leave a Reply