e.crawler.1.5.8.source-code.crawler.properties Maven / Gradle / Ivy

Go to download

Show more of this group Show more artifacts with this name
Show all versions of crawler Show documentation

Simple java (1.6) crawler to crawl web pages on one and same domain.

There is a newer version: 1.5.11

## Override these properties by set a system property
com.soulgalore.crawler.nrofhttpthreads=5
com.soulgalore.crawler.threadsinworkingpool=5
com.soulgalore.crawler.http.socket.timeout=5000
com.soulgalore.crawler.http.connection.timeout=5000

# Auth like:
# soulislove.com:80:username:password,...
com.soulgalore.crawler.auth=

# Proxy properties, if you are behind a proxy.
## The host like: http:proxy.soulgalore.com:80
com.soulgalore.crawler.proxy=