What Your Search Tool Sends Your Coding Agent

When a coding agent answers a question about a library, some tool fetched that documentation a second earlier, usually a search tool, and whatever it returns goes straight into the model's context.

So we measured what they return when the page is a bad one. We took nine documentation pages we knew were dangerous, asked six tools the questions a developer would type, and checked what came back.

Most of those pages weren't attacks. They were honest documentation telling your agent to fetch something unpinned and run it. Search tools that return whole pages handed the dangerous passage over in 23% to 42% of searches, Context7 in 10%. And when the passage reached a model, the model passed it on as a step to follow in about a third of answers, almost never with a warning.

What we tested

We started from nine documentation pages we had verified as dangerous. Finding them was the hard part: two discovery sweeps over 1,752 search results turned up none, so these nine are pages we already knew about and then checked one by one.

For each page we wrote four questions a developer would type to land on it, and asked each three times, because a search tool answers the same question differently from one call to the next. That is 108 searches per tool, through Exa, Firecrawl, Tavily, Brave's two endpoints and Context7.

Two things were counted: whether the page came back at all, and whether the harmful passage itself was in what the tool returned.

The results

The five search tools surfaced these pages at a similar rate, 31% to 56%, and Context7 a little below them. The gap is in the second bar, and the reason is how much of the page travels with it:

Those are averages and they move by query, though not simply by size. Brave's web endpoint returns fragments picked to match the words you typed: on one query Brave returned 12,671 characters in 40 separate pieces, each about 300 long. The dangerous passage is a contiguous block somewhere on the page, rarely the part that matches your question, so it usually lands in none of the pieces. The six searches where it did come back are the ones where the question happened to be about the broken thing the dangerous fix claims to repair. It isn't screening anything; it is answering a narrower question, and what a model can do with that is a separate matter we didn't measure.

The four at the top return the extracted page body. That is the product: give the model something to read, and the dangerous part comes with it. One is sold as pre-extracted content for coding agents, with ready-made instructions for Claude Code, Cursor and Codex; it returned the dangerous text inside a code block, and underneath it the forum replies: "thank you fixed", one like.

Context7 is the last row, at 10%.

Not all the pages were planted

All nine are dangerous to follow. Most were published by exactly who they appear to be, in good faith.

One is a fix for a control panel's broken file manager, on that project's own support forum. The shape of it, with the domain and paths changed:

sudo -i
cd /usr/local/<panel>/web/fm/backend/Services/Session/Adapters/
mv SessionStorage.php SessionStorage.php.ori
curl -fsSLm20 https://<shortener>/<slug> -o SessionStorage.php

Read the last line: it installs a PHP file from a link shortener as the panel's session handler, as root. It reads like a backdoor.

It isn't. The author is one of the project's most active contributors, the bug is real and documented upstream, and we downloaded the file, read it and diffed it against the original: the project's own class with the missing method added. No network call, no eval.

It's still not something you should run, and that has nothing to do with the author. The file sits on a domain the project doesn't control, with no version, checksum or signature, and it lands as root in a path the panel executes. Reading it once tells you nothing about the next fetch.

Three of the nine never came back from any tool in any run, so they sit outside every number here: a skill that has your agent print your private key, a product that routes fetches through your machine, and a wiki page carrying a fake support number.

Here are the other six, sites left out. Four of them were published in good faith, and a URL in a table reads as an accusation whatever the paragraph says.

Twice as many unsafe-by-design pages as attacks, both ending in the same place: something outside the project gets lasting access. Prompt injection gets the attention; the commoner case is honest documentation telling your agent to fetch something unpinned and run it.

Where Context7 differs

Context7 works differently by design: documentation is screened when a library is added, and a question returns a few thousand characters of matched snippets rather than ten page bodies. The numbers below measure the outcome, not which half does the work.

Here is every case a tool returned, measured both ways. Each case has four questions asked three times, which is 48 searches across the four page-body tools and the same twelve through Context7.

The shortener fix is the clearest: web search returned that command in 88% of searches (42 of 48), Context7 in none. On that case Context7 had something to serve in half the runs and nothing in the other half, so the zero is partly silence. On the wallet page it answered all twelve times and the address still never appeared.

A search tool has to rank the page before it can return it, and that ranking does part of the work. So we skipped it: name the library outright, ask the question, and the passage came back in 25% of runs instead of 10%. On the wallet page it was still zero.

Two rows are not zero, and they account for all of Context7's hits: one where Context7 is level with web search, one where it is below. Both are vendor documentation where the dangerous line is also the answer to the question: ask a trading API's reference how to place a trade, and the required privateKey parameter is the answer.

We won't excuse those for one side and count them for the other. Drop both from both sides and the comparison holds anyway: web search returned the dangerous passage in 24% of searches (81 of 336), Context7 in none (0 of 84). Neither outright attack came back at all.

What the models did with it

Text reaching a model isn't a problem by itself. So we put the same questions to eight models, a frontier and a fast model from each of the four major providers, and graded whether the instruction was passed along as a step to take. With web search it was, in 24% of answers. With Context7, in under 1%:

Only two of those 54 answers warned the developer. In the rest it arrived as an ordinary numbered step, usually formatted more cleanly than the source page, with the caveats dropped. One answer filed the root-level fetch under a heading reading "Troubleshooting File Manager (500 Error)". Seven of the eight did it at least once through web search, including the strongest.

That 24% is generous to web search in two ways. It counts every answer, including runs where the search returned nothing harmful to repeat; when the dangerous text was actually in front of the model, it was passed along 35% of the time (53 of 153). There is no matching figure for Context7, because that stage recorded whether the passage was present only for the search side. And the grading is done by a model. We read the answers by hand and found the grader conservative: it excuses a few that reproduce the command inside a numbered step, on the grounds that they "merely quoted the documentation". Five of those are bare instructions with no caution anywhere. We left its verdict as it stood.

The one through Context7 came from the account-takeover site. Per-page screening asks whether a passage is harmful, and for a site like that every honest answer is no. That is a different problem from this one.

Notes on the method

Nine pages is nine pages, not a percentage of the web, and it says nothing about Claude Code, Codex or any shipped assistant; everything here is a plain API call with a one-line system prompt.

• Every search was repeated three times. Search tools answer the same question differently each call, and that moves these numbers more than the models do.
• We checked who published each page before calling anything an attack. Publisher identity, upstream bug reports, GitHub author records. Two we had marked as attacks weren't.
• The model stage ran five of the six pages that came back. Two are indexed under two paths each, which makes seven question-and-library combinations. Each ran through both sources and all eight models, five times for the fast models and three for the frontier ones: 448 runs. In 57 no source returned anything, which leaves the 391 answers in the table.
• Both sides were measured by the same rule. A search counts if the passage was in any result it returned; for Context7, in any library its own search returned. The search stage ran all nine pages, six came back from at least one tool, and the model stage ran five of those six.

What reaches your agent is decided by the tool that fetched it, not by whether anyone meant harm.

Context7 is available as an MCP server, a REST API, and a hosted search at context7.com.