Quantcast
Channel: GSA Email Spider — GSA SEO Forum
Viewing all articles
Browse latest Browse all 237

Add new URLs to Random is too slow?

$
0
0

Hello,

 

I am running GSA Email Spider on a very fast server with a 1 Gbps connection. If I add new URLs to the bottom, it can easily do 100 threads or even 300 threads, but the sites I scrape get HAMMERED and I get DoS complaints, so I need to add new URLs to Random.

When I do 'add new URLs to Random' it works nicely and doesn't hammer any sites, but it runs VERY SLOW, 5-10 threads, no matter what I put in the number of threads.

1. Is there any way to keep the URLs random and make it run at 100 threads? It's only using 20% CPU so I'm not sure why it's slow.

2. Is there any way to run more than one instance of GSA at the same time?

Overall the software is very solid, excellent code, I highly recommend it to everyone.


Viewing all articles
Browse latest Browse all 237

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>