
The crawler that builds the index of Fundig, a business research service run by Stehrway in Canada. Fundig sends a second program, FundigResearch, to read public pages when its customers look into a company.
Worth knowing: Fundig’s page says FundigBot follows robots.txt, but FundigResearch doesn’t, and can only be stopped with an opt-out form (Fundig).
On this site: Just the rules and the homepage: enough to file this site as a business in Fundig’s index, perhaps under art, perhaps under shops.
They map which sites link to which and how pages rank, and sell that map to marketers and agencies trying to climb the search results. What they sell →