Mojeek
MojeekBot
The crawler of Mojeek, a small independent search engine from Brighton, England, that has been building its own index of the web since 2004 rather than borrowing Google's or Bing's.
The crawler of Mojeek, a small independent search engine from Brighton, England, that has been building its own index of the web since 2004 rather than borrowing Google's or Bing's.
Why? To keep an up to date copy of the web, so they can show it in their results and sell advertising beside them. More and more, the same copy also feeds their AI answers.