sometimes, spider is taking waaay to long to parse a site or some links. And by wayy to long i mean several minutes for each link. If the site has tens similar pages ....
Is this a bug? can a option be introduced like a time cap / site or /link?
thank you
Is this a bug? can a option be introduced like a time cap / site or /link?
thank you