Malware Families PROMPTFLUX and PROMPTSTEAL Exploit Large Language Models at Runtime
Google Threat Intelligence Group identified two distinct malware families—PROMPTFLUX and PROMPTSTEAL—that dynamically query large language model (LLM) APIs during execution. These families represent a significant evolution in malware tactics, as they integrate AI capabilities directly into their operational workflows. PROMPTFLUX uses LLM APIs to rewrite its own code, likely to evade detection and analysis by security tools. PROMPTSTEAL, attributed to the Russia-linked APT28 group, employs LLMs to generate Windows commands specifically designed to exfiltrate sensitive documents from compromised systems.
The discovery underscores a new trend in cyber threats: AI-in-the-loop malware. Unlike traditional malware that relies on static payloads or predefined logic, these families dynamically adapt their behavior by interacting with external AI systems. This shift introduces complexities for defenders, as traditional signature-based detection methods may struggle to identify malware that alters its structure or functionality in real time.
PROMPTSTEAL’s ability to generate contextually relevant Windows commands—such as those used for document theft—demonstrates how AI can be weaponized to automate sophisticated attack chains. For instance, the malware may query an LLM to produce commands that mimic legitimate system activities, reducing the likelihood of triggering alerts. This approach aligns with APT28’s historical focus on targeted operations, particularly in geopolitically sensitive regions like Ukraine.
Google responded to the threat by disabling compromised API keys and implementing additional safeguards for its Gemini model. The company’s actions reflect the broader challenge of securing AI systems against misuse, particularly when APIs are exposed to untrusted environments. The incident also highlights the need for organizations to monitor and audit API usage, as malicious actors may exploit legitimate access points to achieve their objectives.
Technical Mechanisms and Operational Impact
PROMPTFLUX and PROMPTSTEAL operate by embedding API calls within their execution logic, allowing them to interact with LLMs without requiring preconfigured payloads. By offloading parts of their functionality to an LLM, these malware families can bypass static analysis techniques that depend on known patterns or hashes.
The integration of LLMs into malware operations introduces several technical challenges for cybersecurity practitioners. Additionally, the use of natural language processing (NLP) to craft commands could allow attackers to evade keyword-based detection mechanisms. This adaptability makes AI-driven malware particularly resilient to conventional defenses.
APT28’s use of PROMPTSTEAL in Ukraine further illustrates the tactical advantages of AI-in-the-loop malware. By generating commands on-the-fly, PROMPTSTEAL can tailor its actions to the specific context of the compromised system, increasing the likelihood of successful data exfiltration while minimizing the risk of detection.
The malware’s reliance on external APIs also introduces dependencies that could be exploited for attribution or disruption. For instance, if an LLM API is temporarily unavailable or returns unexpected responses, the malware’s behavior may become erratic or fail entirely. This dependency could provide defenders with opportunities to disrupt operations, though it also raises questions about the reliability of AI-driven attack chains in real-world scenarios.
Broader Implications for AI Security and Defense
The emergence of PROMPTFLUX and PROMPTSTEAL signals a broader trend in which adversaries increasingly incorporate AI into their toolsets. This shift is not limited to state-sponsored actors; criminal groups may also adopt similar techniques to enhance their capabilities. The use of LLMs for malware development and execution lowers the barrier to entry for sophisticated cyber operations, as attackers can generate custom payloads or commands without requiring deep technical expertise.
The incident also underscores the importance of securing API endpoints and enforcing strict access controls. Google’s decision to disable compromised API keys and harden its Gemini model serves as a model for other organizations facing similar risks.
Finally, the use of AI in malware operations highlights the need for greater transparency and accountability in the development and deployment of AI systems. As LLMs become more accessible, the potential for misuse grows, necessitating robust safeguards and ethical guidelines. Industry stakeholders must work together to establish best practices for securing AI models and APIs, ensuring that advancements in AI do not inadvertently create new avenues for cyber threats.