
The crawler of DuckDuckGo, the US search engine that doesn’t track its users. Much of what it shows comes from Bing, but its own crawler, DuckDuckBot, improves its results, and a second one, DuckAssistBot, reads pages live for its AI answers (now called Search Assist). DuckDuckGo publishes the addresses of both.
Worth knowing: DuckDuckGo says it answers 3 billion searches a month (DuckDuckGo), and that the pages DuckAssistBot reads are “not used in any way to train AI models” (DuckDuckGo).
How it makes money: Mostly private adverts on its search, chosen by the words typed rather than a profile, with most ad clicks handled by Microsoft’s ad network, plus a subscription with a VPN and personal information removal. It publishes no revenue figures. (duckduckgo.com)
On this site: It mostly looks at the homepage, from DuckDuckGo’s published addresses: much of what DuckDuckGo shows comes from Bing anyway, so a glance may be all it needs.
To keep an up to date copy of the web, so they can show it in their results and sell advertising beside them. More and more, the same copy also feeds their AI answers. What they sell →