LiteLLM Supply Chain Attack 2026: How a Poisoned Security Scanner Backdoored 47,000 Downloads

On March 24, 2026, two malicious versions of the litellm Python package appeared on PyPI. Within 46 minutes, they were downloaded 46,996 times. The entry point was not a vulnerability in LiteLLM's own code. The attacker got in through Trivy, a trusted open source security scanner running inside LiteLLM's CI/CD pipeline.

This incident is not a simple case of a compromised developer account. It is a multi-stage, coordinated supply chain campaign by a threat actor known as TeamPCP, who had already hit Trivy and Checkmarx KICS before pivoting to LiteLLM. The full attack chain reveals exactly how trust in security tooling becomes the exploit itself.

TL;DR: What Happened at a Glance

LiteLLM Attack Timeline

How the LiteLLM Attack Was Discovered

The compromise came to light through accidental side effects rather than proactive monitoring. Callum McMahon at FutureSearch was testing a Cursor MCP plugin that pulled in litellm as a transitive dependency. Almost immediately after Python started, his machine became unresponsive due to complete RAM exhaustion.

Tracing the source, McMahon found litellm_init.pth, a 34,628-byte file sitting in site-packages/, double-encoded in base64. The RAM exhaustion was not an intentional feature of the malware. It was a bug: the .pth mechanism fires on every Python interpreter startup, and because the payload spawned a new subprocess, that child process re-triggered the .pth file, creating an unintentional fork bomb. The fork bomb made the attack visible. Without it, the credential exfiltration could have run silently in thousands of environments before anyone noticed.

McMahon published his findings on futuresearch.ai, and the disclosure spread to r/LocalLLaMA, r/Python, and Hacker News within the hour, reaching 324 points before PyPI quarantined the packages.

The LiteLLM Attack Chain: From Trivy to LiteLLM

The attack on LiteLLM began five days before the malicious packages appeared on PyPI. The entry point was Trivy, the open source container and vulnerability scanner built by Aqua Security and widely used in developer CI/CD pipelines.

March 19: TeamPCP rewrote Git tags in the trivy-action GitHub Action repository to point to a malicious release tagged v0.69.4. Because many pipelines pin to a tag name rather than a commit SHA, they silently pulled the malicious version. The compromised action carried the same credential-harvesting payload and exfiltration infrastructure used in all subsequent operations.

March 24: LiteLLM's CI/CD pipeline executed Trivy as part of its build process, pulling it from apt without a pinned version. The compromised Trivy action ran inside the GitHub Actions runner and exfiltrated the PYPI_PUBLISH token from the runner's environment. With that credential in hand, the attacker bypassed LiteLLM's official GitHub-based release workflow entirely and uploaded malicious packages directly to PyPI.

The Pwn Request: How CI/CD Trust Became the Exploit

The Trivy compromise began with a GitHub Actions misconfiguration known as a Pwn Request-using the pull_request_target trigger instead of the safer pull_request. This setting let code from forked pull requests run with full repository permissions and secrets. In February 2026, a user named MegaGame10418 exploited this in the trivy-action repo, accessing aqua-bot credentials and rewriting official Git tags. As a result, any pipeline referencing v0.69.4 instead of a commit SHA unknowingly pulled malicious code.

The fix is straightforward but requires deliberate action from every repository maintainer who accepts external contributions:

• Replace pull_request_target with pull_request for workflows that run untrusted code.
• Where pull_request_target is genuinely required for write-back operations, add a mandatory approval gate before any secrets-touching jobs run.
• Reference all external GitHub Actions by full commit SHA (uses: aquasecurity/trivy-action@a6e08f…) rather than by mutable tag name.
• Audit all workflows for the combination of pull_request_target AND secrets.* in the same job step, which is the precise condition that makes a Pwn Request exploitation viable.

GitHub's own security documentation has called out this pattern since 2021. The Trivy exploit demonstrates that the AI and security tooling ecosystem has not adopted the guidance at scale.

Two Delivery Mechanisms

The two malicious versions used distinct injection techniques, each escalating the attack surface:

• 1.82.7 (Source Injection): The payload was base64-encoded and embedded directly inside litellm/proxy/proxy_server.py. It executes when anything imports litellm.proxy, the standard import path for LiteLLM's proxy server mode. A developer using only the SDK without the proxy component would not trigger this payload.
• 1.82.8 (.pth File): This version added litellm_init.pth to site-packages/. The .pth mechanism fires on every Python interpreter startup with no import required, including when pip itself starts Python, when an IDE's language server initializes, or when python -c runs a single command. The act of running pip install was sufficient to execute the malware.

LiteLLM's Three-Stage Credential Harvester and Worm

Stage 1: Information Collection

The first stage operated as a structured credential harvester, systematically scanning the compromised system for:

• System data: Hostname, environment variables, running processes, network routing tables, and recent authentication logs from /var/log/auth.log.
• Credentials: SSH private keys, .env files, Git credentials, API keys from shell history, Slack and Discord webhook tokens, CI/CD configuration files (Jenkins, Travis CI, Terraform), and password hashes from /etc/shadow.
• Cloud credentials: AWS credentials and configuration files, GCP and Azure service account files, cloud metadata via direct IMDS queries with full IMDSv2 signing implemented in the script, AWS Secrets Manager, and SSM Parameter Store.
• Container and orchestration secrets: Docker config.json registry credentials, Kubernetes kubeconfig files, service account tokens, cluster secrets, and certificates.
• Cryptocurrency wallets: Bitcoin, Ethereum, Solana, Cardano, and Monero wallet files and seed phrases, including Ledger hardware wallet configurations.

Stage 2: Encryption and Exfiltration

The payload did not transmit data in plaintext, making network-level interception harder to interpret. The exfiltration process followed these steps:

1. A 32-byte AES-256 session key is generated via openssl rand.
2. Collected data is encrypted with AES-256-CBC using PBKDF2 key derivation.
3. The session key is encrypted with a hardcoded 4096-bit RSA public key (OAEP padding).
4. Everything is bundled into an archive named tpcp.tar.gz.
5. The bundle is POST'd via HTTPS to models.litellm.cloud, which was registered on March 23, one day before the attack.

The tpcp.tar.gz naming convention and the identical RSA public key prefix appear across all three TeamPCP operations (Trivy, KICS, and LiteLLM), and Wiz researchers confirmed the key match. This is the strongest technical attribution link connecting all three incidents to the same operator.

Stage 3: Persistence and Lateral Movement

Once data was exfiltrated, the payload established long-term access on the host:

Local persistence:

• Writes a backdoor Python script to ~/.config/sysmon/sysmon.py (or /root/.config/sysmon/sysmon.py for root processes).
• Creates a systemd user service at ~/.config/systemd/user/sysmon.service with the innocuous description "System Telemetry Service".
• Registers and enables the service, ensuring it survives reboots.

The persistence script polls https://checkmarx.zone/raw every 5 minutes for a URL, downloads the file to /tmp/pglog, and executes it. At the time of analysis, the endpoint returned YouTube URLs. The operator can switch to delivering live payloads at any point with no change required on compromised hosts.

Kubernetes lateral movement: If the script detects a Kubernetes service account token at the standard mount path, it reads all secrets across every namespace. It then attempts to deploy a privileged pod to every node in kube-system using alpine:latest. These pods mount the host filesystem and install the sysmon backdoor directly on the underlying node, completing a full cluster-wide compromise. Malicious pods appear as node-setup-{node_name} in the kube-system namespace.

TeamPCP Malware Evasion Techniques

TeamPCP's payload was not written carelessly. Several deliberate design choices were made to delay detection, blend with legitimate system behavior, and survive standard security tooling reviews.

Obfuscation at the file level:

The litellm_init.pth file stores its payload as a double-encoded base64 string. The first decoding layer produces a second base64 string. Only the second decoding produces executable Python. Most grep-based scans searching for import subprocess or exec( in site-packages would miss this without recursive base64 decoding.

Mimicking legitimate system services:

The persistence backdoor writes to ~/.config/sysmon/sysmon.py with the systemd service description "System Telemetry Service." The directory name sysmon deliberately echoes the legitimate Microsoft Sysinternals sysmon monitoring tool, common in security-aware environments. An analyst doing a quick review of running services would not immediately flag a service named sysmon with a telemetry description.

Encrypted, short-lived exfil artifacts:

The collected credential bundle is encrypted with AES-256-CBC before being written to disk as /tmp/tpcp.tar.gz. The file is deleted after the POST completes. The short on-disk window reduces the chance of filesystem scanners or EDR tools catching the unencrypted data. Even if the file is intercepted in transit, AES-256-CBC with PBKDF2 key derivation and RSA-4096 session key wrapping means the content is unreadable without the attacker's private key.

C2 traffic that blends with normal HTTPS:

The persistence script checks https://checkmarx.zone/raw every 5 minutes over HTTPS on port 443. Most network monitoring configurations do not deep-inspect outbound HTTPS traffic. The polling interval of 300 seconds sits well below the threshold that behavioral anomaly detection systems typically use to flag periodic beaconing.

Decentralized C2 infrastructure:

The CanisterWorm component uses the Internet Computer Protocol (ICP) for its command-and-control layer. Unlike domain-based C2, ICP canisters run on a decentralized blockchain and cannot be suspended by a registrar, hosting provider, or law enforcement takedown request. This is the evasion technique with the longest operational shelf life in the payload set.

Disclosure suppression via botnet:

When community members filed GitHub issue #24512 to report the compromise, TeamPCP deployed 73 previously compromised developer accounts to flood the issue with 88 comments in 102 seconds. The compromised maintainer account then closed the issue. This social-layer evasion technique was designed to delay the community response long enough for the C2 registration period, the malicious package window, and the credential exfiltration to complete without a coordinated incident response.

About TeamPCP: Phase 09 of an Ongoing Campaign

Disclosure Suppression: 88 Bot Comments in 102 Seconds

When community members began reporting the compromise in GitHub issue #24512, the response from TeamPCP was systematic and rapid. The attackers posted 88 bot comments from 73 unique accounts in a 102-second window between 12:44 and 12:46 UTC. These accounts were not purpose-created profiles; they were previously compromised developer accounts, lending them a surface-level appearance of legitimacy.

Rami McCarthy's analysis found 76% account overlap between this botnet and the one used to suppress the earlier Trivy vulnerability disclosure. Using the compromised krrishdholakia maintainer account, TeamPCP closed issue #24512 as "not planned" and pushed commits to unrelated repositories with the message "teampcp update." The community opened a parallel tracking issue (#24518) and continued discussion on Hacker News, eventually forcing the maintainer to rotate all credentials and confirm the compromise.

LiteLLM Attack Impact: 47K Downloads, 88% Dependency Exposure

The malicious versions were available on PyPI for approximately 3 hours. During the 46-minute window before PyPI quarantine, 46,996 downloads occurred. FutureSearch's analysis of the full PyPI dependency graph found 2,337 packages on PyPI depend on litellm, of which 2,054 (88%) had version specifications that allowed the compromised versions at the time of the attack.

Downstream projects that filed security PRs on March 24 to pin away from the compromised versions included DSPy, MLflow, OpenHands, CrewAI, langwatch, Arize Phoenix, and several others. Notably, Aider was confirmed safe because it already pinned to litellm==1.82.3.

Version Pinning and Exposure

Your version specifier in requirements.txt or pyproject.toml determined whether you were exposed:

Across 2,337 dependent packages: 12% had no constraint, 59% had a lower bound only, and 16% had an upper bound that still included 1.82.x. Only 3% were upper-bounded to exclude 1.82.x, and 9% were pinned to a safe exact version. Lock files (uv.lock, poetry.lock, pip-compile output) protect your builds but do not protect downstream consumers who install your library fresh.

Why pip Hash Verification Failed Against LiteLLM 1.82.7/1.82.8

Hash verification confirms a file matches what PyPI advertised. It does not tell you whether that content is malicious. In the 1.82.8 release, the litellm_init.pth file was correctly declared in the wheel's RECORD file with a valid hash, so pip install --require-hashes would have passed without any alert. The package cleared every standard integrity check because it was published using legitimate, stolen credentials: no hash mismatch, no typosquat, no name confusion.

The only install-time signal that could have flagged it is inspecting whether a .pth file contains patterns like subprocess, base64, or exec, a check no widely deployed pip plugin performs automatically.

The .pth File: Python's Overlooked Attack Surface

A .pth file in Python's site-packages directory runs executable code on every interpreter startup, before any application code or defensive checks. This technique, documented as MITRE ATT&CK T1546.018, has a known upstream CPython issue (#113659) with no patch applied. The critical difference between LiteLLM's two malicious releases is the trigger: the 1.82.7 payload only fired when litellm.proxy was explicitly imported, limiting exposure to proxy deployments. The 1.82.8 .pth payload ran at pip install itself, meaning every environment that resolved to that version was compromised at installation time, regardless of whether the application ever ran.

LiteLLM Backdoor Detection: 5 Commands to Check Your Systems

Run these commands to determine if your environment is compromised:

Step 1: Check your installed version

pip show litellm | grep Version

If the output shows 1.82.7 or 1.82.8, treat the system as compromised. Do not just upgrade; the payload may have already run.

Step 2: Check for the sysmon backdoor

ls -la ~/.config/sysmon/sysmon.py 2>/dev/null && echo "BACKDOOR FOUND"
ls -la ~/.config/systemd/user/sysmon.service 2>/dev/null && echo "PERSISTENCE FOUND"
systemctl --user status sysmon.service 2>/dev/null

Step 3: Check for malicious .pth files

find $(python3 -c "import site; print(' '.join(site.getsitepackages()))") -name "*.pth" -exec grep -l "base64\|subprocess\|exec" {} \;

Step 4: Verify file hashes

find / -path "*/litellm/proxy/proxy_server.py" 2>/dev/null -exec shasum -a 256 {} \;
# Malicious hash: a0d229be8efcb2f9135e2ad55ba275b76ddcfeb55fa4370e0a522a5bdee0120b
find / -name "litellm_init.pth" 2>/dev/null -exec shasum -a 256 {} \;
# Malicious hash: 71e35aef03099cd1f2d6446734273025a163597de93912df321ef118bf135238

Step 5: Check for network indicators

grep "models.litellm.cloud\|checkmarx.zone" /var/log/syslog 2>/dev/null
kubectl get pods -A | grep "node-setup-"

The Power of Automated Local Scanning

Manual command-line checks are appropriate for a single compromised machine, but most organizations managing Python environments at scale need automated scanning integrated into both their CI pipelines and their local developer workflows. This incident created a clear playbook for what automated tooling should be checking and when.

Scanning for known-malicious packages:

Tools such as pip-audit, Google's osv-scanner, and Anchore's grype query the Open Source Vulnerabilities (OSV) database and PyPI's advisory feed. Once PyPI quarantined versions 1.82.7 and 1.82.8, these advisories were published and available for automated querying. A pipeline running pip-audit or osv-scanner on every build would have flagged any environment that resolved to the malicious versions before deployment.

pip-audit
osv-scanner --lockfile requirements.txt

Detecting suspicious .pth files:

Standard vulnerability scanners do not inspect .pth files for behavioral patterns. A targeted scan for .pth files containing execution primitives can be scheduled as a cron job or CI step:

find $(python3 -c "import site; print(' '.join(site.getsitepackages()))") -name "*.pth" | xargs grep -l "exec\|base64\|subprocess\|__import__" 2>/dev/null

Any match should be treated as a high-severity finding. Legitimate .pth files almost never contain these patterns; they typically contain only filesystem path entries.

Secrets scanning for post-compromise credential exposure:

If a system was compromised, the most critical question is which credentials were accessible. GitGuardian's ggshield can scan your filesystem, shell history, and git repositories for secrets that were present at the time of infection:

ggshield secret scan path ~ --recursive
ggshield secret scan docker <image_name>

This is particularly important for cloud environments because the payload queried both ~/.aws/credentials and the IMDS endpoint. Keys that were not stored in files but were available via instance metadata may not appear in a filesystem scan, requiring separate audit of AWS CloudTrail logs for GetSecretValue and GetParameter API calls issued around the attack window.

EDR-based detection of Python startup hook abuse:

Endpoint Detection and Response platforms that monitor process creation events should be configured to alert when a Python interpreter spawns unexpected child processes at startup. The fork bomb behavior in the 1.82.8 payload, caused by the .pth file spawning a subprocess that re-triggered the .pth file, is precisely the kind of anomalous process tree that EDR behavioral heuristics can catch, even for payloads with no known hash signature at the time of initial infection.

Integrating pip-audit into your CI pre-deployment gates, scheduling weekly .pth audits across developer machines, and enabling EDR alerts for Python startup anomalies together create a detection layer that would have caught this attack at multiple points in the kill chain.

LiteLLM 1.82.7/1.82.8 Cleanup Steps

If you have NOT installed 1.82.7 or 1.82.8:

Pin to <=1.82.6 immediately in all environments until a verified clean release is available.

If you HAVE installed 1.82.7 or 1.82.8, follow these steps in order:

Remove persistence artifacts:

rm -f ~/.config/sysmon/sysmon.py
rm -f ~/.config/systemd/user/sysmon.service
systemctl --user disable sysmon.service 2>/dev/null
systemctl --user daemon-reload
rm -f /tmp/tpcp.tar.gz /tmp/session.key /tmp/payload.enc /tmp/.pg_state /tmp/pglog

Rotate all credentials on the affected system, including SSH private keys, AWS access keys, GCP service account keys, Azure service principals, API keys from .env files and CI/CD secrets, Docker registry credentials, Kubernetes configs and service account tokens, database passwords, Git credentials, and cryptocurrency wallet seed phrases.

Audit AWS Secrets Manager and SSM Parameter Store. The payload queries these directly if instance metadata is accessible via IMDS.

Audit your Kubernetes cluster for node-setup-* pods in kube-system. If a service account token was present on the host, all secrets across all namespaces may have been read.

Reinstall from a clean environment rather than upgrading in place: pip install "litellm<=1.82.6"

Indicators of Compromise

The Broader Pattern: Security Tools as Attack Vectors

TeamPCP did not target application code first. They went after tools with trusted, elevated access inside automated pipelines: a container scanner (Trivy), an infrastructure scanner (KICS), and an AI model router (LiteLLM). Each of these tools holds broad access to credentials and environment variables by design, making them high-value targets. LiteLLM's role as a centralized LLM gateway amplifies this further, since a single compromised host can expose API keys for multiple providers at once. The pattern echoes the Ultralytics supply chain attack of 2024 and points to a broader gap: the AI developer ecosystem has not yet adopted the security practices standard in more mature software supply chains.

Why Trusted Publishers and Sigstore Would Have Prevented This

PyPI's Trusted Publishers ties package uploads to verified GitHub Actions workflows, meaning a stolen API token alone would not have been enough to publish a malicious release. Sigstore attestations go further, creating a cryptographic link from a specific source commit to the published artifact. Since the malicious LiteLLM releases had no matching commits in the official repository, a Sigstore-enforced policy would have blocked them outright. Both controls are free to adopt, and their absence shows that credential-based publishing remains a single point of failure in an otherwise sound supply chain.

Conclusion

The LiteLLM supply chain attack demonstrates how trust cascades through modern software infrastructure. A security scanner that developers trusted implicitly became the instrument of compromise. The stolen credential from that scanner opened a door to one of the most widely downloaded AI Python packages on PyPI. And a Python startup hook mechanism, acknowledged as a security risk by CPython maintainers but still unpatched, guaranteed execution before any defensive measure could intervene.

The 46,996 downloads in 46 minutes represent 46,996 opportunities for credential theft, persistent backdoor installation, and Kubernetes cluster compromise. Teams whose systems were exposed during the window should assume credentials were stolen and rotate accordingly. Teams whose systems were not exposed should treat this incident as a concrete case study for why version pinning, Trusted Publishers, Sigstore attestations, and pinned GitHub Action SHAs are non-negotiable controls in any modern CI/CD pipeline.

The pattern TeamPCP has established across nine documented campaign phases is clear: attack the tools that protect software, and you gain access to the software itself.

Frequently Asked Questions

Was the LiteLLM Docker image affected?
No. Customers running the official LiteLLM Proxy Docker image were not impacted. That deployment path pins dependencies in requirements.txt and does not rely on the compromised PyPI packages.

If I never ran my application, am I still at risk?
It depends on which version you installed. If your resolver picked 1.82.8, yes - the .pth file executes during pip install itself, before any application code runs. If you resolved to 1.82.7, you are only at risk if you explicitly imported litellm.proxy.

Would using uv instead of pip have protected me?
Not automatically. However, FutureSearch's analysis of download logs showed that safe version installs skewed toward uv users with lock files pinning to a known version, while the malicious versions skewed toward pip resolver behavior that picks the latest version. A uv.lock file generated before the attack window would have kept you on a safe version.

Did pip hash verification offer any protection here?
No. The litellm_init.pth file is correctly declared in the wheel's RECORD file with a matching hash. Running pip install --require-hashes would have passed. The malicious content was published using legitimate stolen credentials, so there was no integrity violation to detect.

How do I confirm my Kubernetes cluster is clean?
Run kubectl get pods -A | grep node-setup- to check for malicious pods. If any are found, examine all secrets across all namespaces as they should be treated as compromised. Remove the pods with kubectl delete pod -n kube-system <pod-name> and rotate all cluster secrets.

What is the significance of the ICP-based C2 infrastructure?
TeamPCP's use of the Internet Computer Protocol (ICP) for their CanisterWorm component creates C2 infrastructure that cannot be taken down through standard domain seizure or hosting suspension. Traditional incident response relies on getting malicious domains taken offline. ICP canisters operate on a decentralized blockchain, making that response option unavailable.

How did the community stop the disclosure from being suppressed?
When the attacker used the compromised maintainer account to close GitHub issue #24512 and flooded it with 88 bot comments in 102 seconds, community members immediately opened a parallel tracking issue (#24518) and continued discussion on Hacker News. The public visibility of the Hacker News thread, which reached 324 points, created enough external pressure that the legitimate maintainer was able to regain control and confirm the compromise within hours.

What is the single most important change teams should make after this incident?
Adopt PyPI Trusted Publishers and pin all GitHub Actions references to commit SHAs rather than mutable tags. The entire attack chain from Trivy to LiteLLM was made possible by two unpinned references: a floating tag in the trivy-action GitHub Action and an unpinned apt install of Trivy in the CI pipeline.