AI Software
This technique has been observed in real-world attacks on AI systems.
Adversaries may target software packages that are commonly used in AI-enabled systems or are part of the AI DevOps lifecycle. This can include deep learning frameworks used to build AI models (e.g. PyTorch, TensorFlow, Jax), generative AI integration frameworks (e.g. LangChain, LangFlow), inference engines, and AI DevOps tools. They may also target the dependency chains of any of these software packages [\[1\]][1]. Additionally, adversaries may target specific components used by AI software such as configuration files [\[2\]][2] or example usage of AI packages, which may be distributed in Jupyter notebooks [\[3\]][3].
Adversaries may compromise legitimate packages [\[4\]][4] or publish malicious software to a namesquatted location [\[1\]][1]. They may target package names that are hallucinated by large language models [\[5\]][5] (see: Publish Hallucinated Entities). They may also perform a [AI Supply Chain Rug Pull](/techniques/AML.T0109) in which they first publish a legitimate package and then publish a malicious version once they reach a critical mass of users.
[1]: https://pytorch.org/blog/compromised-nightly-dependency/ "Compromised PyTorch-nightly dependency chain between December 25th and December 30th, 2022." [2]: https://www.pillar.security/blog/new-vulnerability-in-github-copilot-and-cursor-how-hackers-can-weaponize-code-agents "New Vulnerability in GitHub Copilot and Cursor: How Hackers Can Weaponize Code Agents" [3]: https://medium.com/mlearning-ai/careful-who-you-colab-with-fa8001f933e7 "Careful Who You Colab With: abusing google colaboratory" [4]: https://aws.amazon.com/security/security-bulletins/AWS-2025-015/ "Security Update for Amazon Q Developer Extension for Visual Studio Code (Version #1.84)" [5]: https://www.trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/slopsquatting-when-ai-agents-hallucinate-malicious-packages "Slopsquatting: When AI Agents Hallucinate Malicious Packages"