How can I set a Funnelback collection to ignore the website robots.txt file?
Assuming you can alter a site's robots.txt file, the recommended approach would be to add an entry like:
robots.txt
User-Agent: Funnelback Allow: *
A full update would then be required.
See also: https://docs.funnelback.com/crawler_robotagent_collection_cfg.html
