
GiantCrawler
Its name points to its own website, giantcrawler.com, which says only “Crawler is offline for redevelopment.” Nobody says who runs it or what it collects.
On this site: A single look at the homepage, from a crawler whose own site says it is offline for redevelopment: still out working while it is being rebuilt.
Tools anyone can run from a laptop or a rented server: curl, Python scripts, small homemade scrapers, research crawlers. They could be a student’s project, someone building a dataset to sell, or the start of a scan; on their own they don’t say which. What they sell →