
RiverBot
The crawler of River AI, a company selling a platform for training and running AI models (“Your own frontier AI lab. In one API.”). Its page says RiverBot fetches public pages to train and test River’s models, one page at a time, without following links, and reads robots.txt at least once a day.
Worth knowing: It is one of the more open bots: it signs its requests, publishes its exact addresses, all 100 of them, and treats a robots.txt line saying ai-train=no as a full refusal (River).
How it makes money: It sells an API for training and running open AI models, billed by the million tokens, though its rates aren’t published. In August 2026, two months after launch, it raised $1.1 billion. (techcrunch.com)
On this site: It reads the rules every day and takes one page at a time, signed and from the addresses River publishes: an unusually polite collector, gathering pages to train AI models.
Writing and pictures are the raw material for training AI models, and AI search and assistants fetch pages to answer people’s questions. The answer is given inside the chat, so the person asking often never comes to the site itself. What they sell →