BetterWebSearch MCP — smarter web search, better data extraction, flexible and extensible, open source
Web search for AI agents sounds simple: Search the web Open the results Extract useful information Feed it back to the model
Some websites work with a simple HTTP request. Others render almost everything through JavaScript. Search APIs often require accounts, API keys or paid plans, and dumping entire webpages into an LLM context wastes a huge amount of tokens.
👉 https://www.npmjs.com/package/better-web-search-mcp 👉 https://github.com/PhantomPixelDev/BetterWebSearch-MCP
It's a local, open-source Model Context Protocol server for web search, extraction and deep research.
Optional providers are also supported: DuckDuckGo — keyless Brave Search — optional Tavily — optional
Getting ten URLs isn't particularly useful if your agent can't actually read them.
If the page is JavaScript-heavy, BetterWebSearch can progressively fall back to structured application data and finally Playwright.
| Tool | Purpose | |---|---| | websearch | Search across providers, deduplicate and rerank results | | webresearch | Search, extract and return cited research passages | | deepsearch | Alias for webresearch | | webextract | Extract clean content from a URL | | webfind | Search inside a specific website | | webnews | Search recent news with diversity filtering |
it can perform the search, extraction and passage selection inside the MCP server and return a much smaller research payload.
The agent: performs a search opens the top five pages receives their extracted contents
The agent makes one webresearch call and receives selected passages with citations.
| | Traditional | webresearch | |---|---:|---:| | Payload | 820,229 chars | 110,973 chars | | Estimated tokens | ~205,000 | ~28,000 | | Wall clock | 137.3s | 49.6s |
The benchmark source is included in the repository, so you can reproduce it yourself:
Instead of searching only the exact sentence given by the model, BetterWebSearch can rewrite it into several related searches.
Those queries can run in parallel before the results are deduplicated and reranked.
This helps especially with searches where terminology differs between websites or languages.
If the same wire story is republished by five websites, that isn't really five independent sources.