A Poisoned PyPI Package Quietly Hit 36 Percent of Cloud Environments Through LiteLLM

A developer runs pip install litellm. Nothing looks wrong. No warnings, no errors. The package installs cleanly, the application starts, and within seconds, a multi-stage payload is quietly sweeping through the environment — collecting AWS credentials, Kubernetes configs, CI/CD secrets, and SSH keys before encrypting everything and shipping it to attacker-controlled servers.

The operation was designed to be invisible. The only reason anyone noticed was an accidental fork bomb that caused Python to exhaust RAM, which made a developer trace the problem back to a freshly installed package and find litellm_init.pth: a 34,628-byte file buried in site-packages/, double base64-encoded, designed to fire on every Python interpreter startup.

Without that accident, the operation might have run silently for weeks across the 3.4 million environments that install LiteLLM daily.

This is the story of how a poisoned security scanner became the entry point to one of the most targeted supply chain attacks on AI infrastructure to date — why your AI pipeline is now a primary attack surface, and what a concrete investigation and response workflow looks like using Strobes AI.

What LiteLLM Is and Why Attackers Picked It Specifically

LiteLLM is a Python middleware library that acts as a unified proxy for 100+ LLM APIs. Developers drop it into their stack and call OpenAI, Anthropic, AWS Bedrock, Google Vertex AI, and Azure OpenAI through a single interface instead of writing separate integration code for each provider.

That architectural convenience is exactly what made it valuable to attackers.

Most compromised packages yield one credential: maybe an AWS key from one application, maybe a GitHub token from one repo. A compromised LiteLLM environment yields everything, because LiteLLM holds credentials for every AI provider the application uses simultaneously. AWS Bedrock credentials, Anthropic API keys, OpenAI tokens, GCP service accounts, Azure service principals — all from a single environment with a credential density no typical application comes close to.

This was not accidental targeting. This was precision.

This Was Not a One-Off Compromise

Before getting into what the malware did, it is important to understand what this incident actually was: part of a coordinated, multi-stage supply chain campaign targeting developer tooling.

The attackers have been active since at least December 2025, operating with the consistency of a professional threat operation. They have been linked to LAPSUS$ — the same actor behind breaches of Microsoft, Okta, NVIDIA, and Samsung.

Their strategy is not spray-and-pray. They target developer tooling specifically because these tools are trusted, widely installed, and run with elevated permissions in exactly the environments attackers want access to. Their payloads share consistent infrastructure across operations: the same RSA key pair, the same exfiltration bundle naming, and the same C2 polling mechanism — making it clear this is a deliberate, ongoing campaign and not isolated incidents.

Here is the full campaign timeline with real timestamps:

Date	Event
Late Feb 2026	Attackers exploit a pull_request_target workflow in Trivy's CI to exfiltrate the aqua-bot publishing credentials
Mar 19, 17:43 UTC	Trivy v0.69.4 GitHub Action tags rewritten to point to a malicious release
Mar 23, 12:58 UTC	Checkmarx KICS GitHub Action compromised; exfiltration domain models.litellm.cloud registered one day before the LiteLLM attack
Mar 24, 10:39 UTC	Malicious LiteLLM 1.82.7 published to PyPI using stolen credentials
Mar 24, 10:52 UTC	LiteLLM 1.82.8 published 13 minutes later with an escalated delivery mechanism
Mar 24, 11:48 UTC	Public disclosure issue opened on GitHub
Mar 24, 12:44 UTC	88 bot comments from 73 accounts flood the disclosure issue in 102 seconds; compromised maintainer account closes it as "not planned"
Mar 24, 13:38 UTC	PyPI quarantines the malicious versions

Three hours. That is the window between publication and quarantine. For a package with 3.4 million daily downloads, three hours is a long time.

One detail worth calling out: the attackers used an AI agent for automated attack targeting as part of this operation. The same technology being adopted to defend environments is being weaponized to compromise them.

Two Delivery Mechanisms, One Goal

The two malicious versions used different injection techniques, and understanding both matters for detection.

Version 1.82.7 used source injection. The payload was base64-encoded and embedded directly inside litellm/proxy/proxy_server.py. It executes when anything imports litellm.proxy — the standard import path for LiteLLM's proxy server mode.

Version 1.82.8 used a .pth file. This is the more dangerous mechanism. The file litellm_init.pth was placed in site-packages/ and fires on every Python interpreter startup with no import required. The payload executed when pip ran, when an IDE's language server started Python, and during CI/CD build steps — not just when your application code ran.

There is also a reason standard integrity checks did not catch this. The .pth file is correctly declared in the wheel's RECORD file with a matching hash. Running pip install --require-hashes would have passed. The package passes all standard verification because the malicious content was published using legitimate stolen credentials. There is no hash mismatch, no suspicious domain in the package metadata, no misspelled package name. The attack bypassed hash verification entirely — not by breaking it, but by going through it legitimately.

What the Payload Actually Collected

The three-stage payload was professionally built, obfuscated with base64-encoded Python, and used AES-256-CBC encryption with a 4096-bit RSA key to secure exfiltration. The same RSA key appeared in the Trivy and KICS payloads, making it the strongest single attribution link across all three operations.

Stage 1 collected everything an attacker could want from a developer environment:

• Environment variables, including API keys and tokens
• SSH private keys
• .env files and Git credentials
• API keys from shell history
• Slack and Discord webhook tokens
• CI/CD configs including Jenkins, Travis CI, and Terraform
• Password hashes from /etc/shadow
• AWS credentials and config files
• GCP and Azure service account files
• Cloud metadata via direct IMDS queries with full IMDSv2 signing
• AWS Secrets Manager and SSM Parameter Store contents
• Docker config.json registry credentials
• Kubernetes kubeconfig files, service account tokens, secrets, and certificates
• Database credentials from config files
• Cryptocurrency wallet files and seed phrases

Stage 2 encrypted and exfiltrated everything to models.litellm.cloud — a domain registered the day before the attack, deliberately designed to look like a legitimate LiteLLM domain.

Stage 3 established persistence and lateral movement. A backdoor Python script was written to ~/.config/sysmon/sysmon.py, disguised as a systemd service called "System Telemetry Service." The script polls checkmarx.zone/raw every five minutes for live payloads. The operator can switch from returning YouTube URLs to delivering live attack code at will. If a Kubernetes service account token was present, the payload read all secrets across every namespace and attempted to deploy a privileged pod to every node in kube-system. Those pods mounted the host filesystem.

Why AI Pipelines Are the New Soft Target

This incident did not happen in isolation. It is part of a structural shift in how attackers approach developer environments.

According to recent industry research, supply chain attacks are now averaging 26 incidents per month — twice the rate from early 2024. AI-enabled supply chain attacks specifically jumped 156% last year. The average time to detect a compromise is one year. OWASP has recognized supply chain vulnerabilities in its GenAI Top 10 as LLM03:2025, alongside the number three position on the traditional OWASP Top 10 overall.

The economics of AI development explain why this will keep getting worse. Building LLMs from scratch is prohibitively expensive. Every team building AI products depends on third-party frameworks, pre-trained models, fine-tuning tools, and deployment platforms. LiteLLM, LangChain, LlamaIndex: these are not optional dependencies that developers can avoid. They are how AI applications get built. The attack surface is inseparable from the development model itself.

What makes AI pipelines particularly vulnerable compared to traditional application stacks is how credentials accumulate. A conventional web application might have a database connection string and an API key or two. An AI application running LiteLLM holds credentials for every cloud provider it touches, plus the IAM permissions needed to access training data, vector databases, model artifacts in cloud storage, and inference infrastructure. The credential density is an order of magnitude higher.

You Cannot Remediate What You Cannot See

The core problem is simple to state and hard to solve: you cannot remediate what you do not know is compromised. Most security programs lack the visibility to answer the three questions this incident immediately demands:

• Q1: Which of our repositories use LiteLLM, and are any of them running 1.82.7 or 1.82.8?
• Q2: Which environments ran those versions, and what credentials did those environments have access to?
• Q3: Do any of our other dependencies pull in Trivy, KICS, or Checkmarx as a transitive dependency — meaning the same credential theft risk exists even without a direct LiteLLM dependency?

Most teams answer these questions manually: check Slack, ask engineers, grep through repos, cross-reference against a spreadsheet. That process takes hours to days. Attackers use that time.

The structural answer is continuous SBOM generation for AI-specific dependencies, SCA analysis that covers the full transitive dependency tree, and agentic investigation that connects package exposure to actual credential blast radius automatically.

How Strobes AI Hunts Threat Exposure Across Your Entire Codebase

Strobes SCA scans your full dependency tree and cross-references every flagged package against Strobes Threat Intel, enriching each finding with EPSS scores, CISA KEV status, and active exploitation data — so the output is a ranked priority queue, not a flat list. For deep validation, credential blast radius mapping, and persistence checks, Strobes deploys its agentic AI ecosystem, running that work autonomously across every affected environment in parallel while every proposed action waits behind an explicit approval gate before anything executes.

SBOM Generation

Strobes AI agents clone every repository in your GitHub organization and parse every dependency manifest they find. The result is a timestamped table showing exactly which repositories are running affected LiteLLM versions, in which environments, with color-coded risk status — not a best guess from memory.

Repository	LiteLLM Version	Environment	Risk Status
ai-inference-service	1.82.8	Production	CRITICAL
ml-pipeline-worker	1.82.7	Production	CRITICAL
dev-chatbot-prototype	1.82.6	Development	Safe
internal-llm-proxy	1.82.8	Staging	HIGH

SCA Analysis with Threat Intel Enrichment

Finding LiteLLM directly is the easy part. LiteLLM was compromised through Trivy — a transitive dependency — which means the real question is whether anything deeper in your stack pulls in the affected tools. Strobes SCA traverses the full dependency tree, five levels deep if needed, and cross-references every finding against Strobes Threat Intel: EPSS exploit probability, CISA KEV status, active in-the-wild exploitation, and patch availability. The output is not a raw list of flagged packages. It is a ranked priority queue that tells you which ones need action today and which can wait.

Persistence Check

Upgrading the package is not enough. The payload installed a backdoor before anyone could react. Strobes AI agents run these detection commands directly against affected environments through sandbox execution — no manual SSH required:

# Sysmon backdoor
ls -la ~/.config/sysmon/sysmon.py

# Systemd persistence service
systemctl --user status sysmon.service

# Exfiltration artifacts
ls /tmp/tpcp.tar.gz /tmp/session.key /tmp/payload.enc

# Suspicious .pth files
find $(python3 -c "import site; print(' '.join(site.getsitepackages()))")   -name "*.pth" -exec grep -l "base64|subprocess|exec" {} \;

# Kubernetes lateral movement pods
kubectl get pods -A | grep "node-setup-"

Prioritize by Business Risk

Strobes AI ranks remediation using Threat Intel data alongside asset context: EPSS score, KEV listing, active exploitation status, credential type, and environment sensitivity. Production environments with static cloud credentials and confirmed active exploitation go first. Staging environments with real API keys follow. Dev sandboxes with isolated test credentials are last. This ordering prevents the chaos of rotating every credential simultaneously while keeping the highest-risk gaps closed first.

This is the same risk-driven prioritization model that separates effective exposure management from checkbox compliance.

Mobilize with Full Audit Trail

AI agents create specific GitHub issues per affected repository, each containing the exact upgrade version, which credentials to rotate, whether to audit CI/CD logs for the March 24 window, and Kubernetes remediation steps where needed. Every proposed action requires explicit approval before it executes, with the full decision trail logged in the workspace.

Incident Report

The final output is a complete report generated from live data: an executive summary covering which repositories were affected, which credentials were rotated, what was found and cleared, and what still needs action. No manual assembly. Ready to distribute the moment generation completes.

Indicators of Compromise

Keep this handy. Immediately check your environments against these IOCs.

Affected package versions: litellm 1.82.7 and 1.82.8

File	SHA-256
litellm_init.pth (1.82.8)	71e35aef03099cd1f2d6446734273025a163597de93912df321ef118bf135238
proxy_server.py (1.82.7)	a0d229be8efcb2f9135e2ad55ba275b76ddcfeb55fa4370e0a522a5bdee0120b
sysmon.py (backdoor)	6cf223aea68b0e8031ff68251e30b6017a0513fe152e235c26f248ba1e15c92a

Network indicators: Exfiltration to https://models.litellm.cloud/ (POST) and C2 polling at https://checkmarx.zone/raw (GET every 5 minutes).

Filesystem artifacts: ~/.config/sysmon/sysmon.py, ~/.config/systemd/user/sysmon.service (description: "System Telemetry Service"), /tmp/tpcp.tar.gz, /tmp/session.key, /tmp/payload.enc.

Kubernetes: Pods named node-setup-{node_name} in kube-system. Container name: setup, image: alpine:latest.

The Pattern Worth Paying Attention To

The attackers behind this campaign are not done. The same infrastructure, the same RSA key, and the same exfiltration bundle naming convention has appeared across Trivy, KICS, Checkmarx, and now LiteLLM. Each operation follows the same playbook — targeting tools that are trusted by developers, run early in the build process, and have broad read access to the systems they operate on. Whatever comes next will fit that same profile.

The security programs that will respond fastest are not the ones with the best manual processes. They are the ones that already know what is running in their AI stack, already have SCA running across their full dependency trees as part of a continuous exposure management program, and already have an agentic investigation workflow ready to map credential exposure the moment a new indicator of compromise appears.

That capability is not built during an incident. The next one is already in motion.

See the Workflow With Your Own Repositories

Strobes AI gives you continuous SBOM generation across your codebase, SCA analysis that covers the full transitive dependency tree, and agentic investigation that connects package exposure to real credential blast radius before the next PyPI incident hits.

• Request a Demo: strobes.co/demo — walk through the LiteLLM detection and response workflow with your own repositories
• Read More: strobes.co/blog
• Talk to Us: hello@strobes.co