
DomainMetaDataBot
The crawler of DomainMetaData, a German company (arndt.ai GmbH) that sells lists of internet domain names. The bot visits a site’s public pages to note what technology it uses, its language and its subject, for DomainMetaData’s database of domains.
Worth knowing: It counts more than 850 million domains, fewer than half of them active, and finds hundreds of thousands of new ones every day (DomainMetaData).
How it makes money: It sells CSV lists of domain names for about 1,706 endings, unmasked on the Basic plan at US$55 a month or US$149 a year: “domain names only”, with “no owners, no emails, no WHOIS data”. (domainmetadata.com)
On this site: The rules, the homepage and Contact: enough to file this site in a database of domains by its language, subject and the software it seemed to run.
They map which sites link to which and how pages rank, and sell that map to marketers and agencies trying to climb the search results. What they sell →