Programs without a company
The painted mark for GiantCrawler
Placeholder: mark still to be painted

GiantCrawler

Its name points to its own website, giantcrawler.com, which says only “Crawler is offline for redevelopment.” Nobody says who runs it or what it collects.

On this site: A single look at the homepage, from a crawler whose own site says it is offline for redevelopment: still out working while it is being rebuilt.

Why? Tools anyone can run from a laptop or a rented server: curl, Python scripts, small homemade scrapers, research crawlers. They could be a student’s project, someone building a dataset to sell, or the start of a scan; on their own they don’t say which. What they sell →