Asynchronous web page indexing

Hey,

I’m afraid nothing has changed on this front.

However it’s not completely impossible - we’ve done some experiments and had some success crawling a JS generated site using a proxy that is basically a headless browser which processes the Javascript and returns the resulting html.

A big drawback of this approach though is the speed of crawling and the fact that you’ll need to set up a bunch of other infrastructure to be able to achieve it.

regards