Ignore robots.txt file

How can I set a Funnelback collection to ignore the website robots.txt file?

Assuming you can alter a site’s robots.txt file, the recommended approach would be to add an entry like:

User-Agent: Funnelback
Allow: *

A full update would then be required.

See also: https://docs.funnelback.com/crawler_robotagent_collection_cfg.html