The crawl time mostly depends on the speed of the server being crawled and how quickly it responds.
There are various crawl settings that can affect the crawl time, e.g. values like the crawler request delay.
You also can use things like site profile configuration to crawl more agressively if the server can handle more requests.
The collection update history (on the analyse tab) shows you a history graph showing where time is spent during updates.
It is also worth checking your crawl logs to see if you're getting into a crawler trap (like crawling a calendar indefinitely).
