Faulty reasoning signals

• Confirmation Bias - 2.3% (27 hits)
• Anchoring Bias - 0%
• Availability Heuristic - 10.3% (122 hits)
• Representativeness Heuristic - 6.2% (73 hits)
• Hindsight Bias - 0%
• Overconfidence Bias - 13.4% (158 hits)
• Framing Effect - 1.4% (16 hits)
• Loss Aversion - 0%
• Status Quo Bias - 1.8% (21 hits)
• Sunk Cost Effect - 0%
• Optimism Bias - 4.7% (55 hits)
• Pessimism Bias - 9.9% (117 hits)

Article text

Hackers can use 9 of the most popular AI tools to assemble massive botnets

In the brief history of AI security, the prompt injection has quickly become the top threat.

Large language models are inherently unable to distinguish between legitimate instructions provided by users and malicious ones sneaked into emails, source code, and other third-party content the models are processing.

This makes it trivial to surreptitiously inject malicious commands that the LLM readily follows.

With no way to enforce this crucial boundary between trusted and untrusted sources, AI engine developers are left to erect elaborate guardrails designed to mitigate the damage rather than solve the root cause.

To date, most prompt injections have fallen into a class known as push, in which each potential victim is targeted.

For example, the adversary injects malicious instructions into an individual email or calendar invitation.

Because the injection must then be sent (or pushed) to each specific target, the scale of the attack is limited, hampering mass exploits that hit the Internet at large.

Meanwhile, pull-based attacks, in which an LLM actively seeks out the adversarial prompts planted on websites, remain limited.

With no way to lure large numbers of LLMs to a malicious site, these sorts of attacks don’t scale either.

Enter HalluSquatting

Now, researchers have devised a pull-based attack that changes all that.

A new attack the researchers have named HalluSquatting has the potential to assemble massive botnets, perform large-scale DDoSes, and infect devices at scale, a first for prompt-injection attacks.

The attack works against AI coding assistants and agents, including Cursor, Cursor CLI, Gemini CLI, Windsurf, GitHub Copilot, Cline, OpenClaw, ZeroClaw, and NanoClaw, which are all susceptible.

In the normal course of performing day-to-day activities, these assistants and agents routinely pull code and other resources from repositories and registries.

The HalluSquatting threat model.

Credit: Spira et al.

Short for adversarial hallucination squatting, HalluSquatting is built on an LLM’s inherent tendency to hallucinate the resource identifiers hosted in repositories and registries.

It works against coding agents and assistants, which commonly access high-privilege command lines to run code from third-party resources.

By predicting the identifiers LLMs are most likely to hallucinate and then registering and seeding them with instructions to install reverse shells or other malicious wares, the attack can indiscriminately infect massive numbers of devices without having to target each one.

“The scalable property of the attack enables the attacker to compromise a large number of users with minimal effort by targeting popular resources, thereby maximizing the likelihood that the squatted resource will be retrieved,” the researchers wrote in a paper published Wednesday.

“By exploiting integrated shells and terminals of agentic applications to run scripts and code, attackers can effectively ‘infect’ many independent agentic applications by embedding instructions to install reverse shells in the resources the attackers register.

Gaining access to distributed computational resources under attacker control opens the door to several high impact outcomes allowing attackers to achieve various goals.

For example, having the ability to compromise LLM applications with terminals allows the attacker to scale the number of ransomware attacks on different networks to maximize financial gain.

Alternatively, attackers can aggregate compromised machines into a botnet and use it for tasks that rely on substantial computing power, including (1) large-scale cryptocurrency mining (e.g., Smominru, WannaMine) or (2) performing distributed denial of service (DDoS) attacks against victims (e.g., Mirai).”

HalluSquatting is already receiving interest from fellow AI security researchers not involved in the study.

“This is very cool research, and the threat is very real,” Michael Bargury, CTO of security firm Zenity, wrote in an email.

“Like typosquatting, it’s a problem that’s not going away.

At the end of the day, it’s about the level of agency we allow our agents.

They *are* going to get fooled one way or the other.

That should be our assumption, and we should be resilient to that.”

Independent researcher Johann Rehberger wrote:

What’s interesting is that it shows that LLM resource resolution can become an attack path and an attacker can first probe models to find high-probability hallucinated candidates (like repo names, skill identifiers,etc) to squat and wait for agents to resolve and use them.

But the main point is that they found a cool technique to find resource names that are more likely by models to be used/confused with.

And that could mean many agents falling for such attacks in the wild.

AI tool makers frequently exaggerate the convenience and efficiency of their platforms.

Marketers claim the platforms lighten workflows by automating and streamlining tedious tasks.

They are much more reticent about the inherent flaws that can torpedo an entire project.

Attacks like HalluSquatting provide a potent reminder that some of the efficiencies are exaggerated since, at the end of the day, users must double-check details such as the location for each resource incorporated into a project.

It also provides a cautionary lesson on the unintended and potentially dire outcomes that can result when people rely too heavily on AI assistants.

LLMs don’t know how to say “I don’t know.”

The starting point for HalluSquatting is the inability of LLMs to accurately identify the location of a resource specified by the user.

When a developer, for instance, instructs a coding agent to clone a popular new repository, the LLM hallucinates its correct location up to 85 percent of the time.

When cloning a trending “skill,” a form of instruction, script, or resource that gives agents specialized capabilities and domain expertise, hallucinations can occur 100 percent of the time.

HalluSquatting focuses on trending resources because they aren’t included in the LLM training.

They also receive large numbers of downloads over a short period of time.

Table depicting the most frequently hallucinated owner/repo candidate per (target repository, foundational LLM) combination over 100 queries.

Owner shading: yellow = real GitHub owner, blue = registrable squat (owner does not exist on GitHub), red = misdirection (real but unintended owner), purple = placeholder string that cannot be registered as a GitHub username.

A ⋆ marks self-referential hallucinations (owner == repository name).

Credit: Spira et al.

The researchers say the inability of LLMs to provide the correct location is an inherent flaw that arises from training biases or from misinterpretations of instructions within the current context.

That means when a user prompts the coding assistant to clone a repository or skill—in the form of, say, “clone repo name” or “install skill name”—the bot frequently navigates to the wrong location to retrieve it.

Not only are these hallucinations inevitable, but they also occur at the foundational level of all six of the major LLMs, including Gemini-2.5-flash, Gemini-2.5-pro, GPT-5.1, GPT-5.2, Sonnet-4.5, and Opus-4.5.

Additionally, the most commonly provided incorrect locations that these LLMs hallucinate are easy to predict in advance.

All six LLMs follow common patterns when resolving the repository or skill name in a prompt with its official name in a repository or skill repository.

LLMs follow various hallucination patterns.

The one HalluSquatting exploits is described as being self-referential.

All six models produce repo-name/repo-name slugs that treat a repository name as the owner.

Exploiting the pattern requires no model probing.