Download SiteCrawler JAR file with all dependencies
SiteCrawler from group io.github.jasperroel (version 1.0.0)
This project provides a simple WebCrawler with retry-capabilities, functionality to distinguish between http/https sites.
It biggest feature is that it allows for plugins (or CrawlerActions), which allows you to hook your scripts into the crawling process.
It also allow for setting "blocked" URLs. Those URLs or patterns will not be crawled.
Artifact SiteCrawler
Group io.github.jasperroel
Version 1.0.0
Last update 30. July 2018
Organization Salesforce.com
URL https://github.com/forcedotcom/SiteCrawler
License The BSD 2-Clause License
Dependencies amount 3
Dependencies jcl-over-slf4j, htmlunit, commons-lang,
There are maybe transitive dependencies!
Group io.github.jasperroel
Version 1.0.0
Last update 30. July 2018
Organization Salesforce.com
URL https://github.com/forcedotcom/SiteCrawler
License The BSD 2-Clause License
Dependencies amount 3
Dependencies jcl-over-slf4j, htmlunit, commons-lang,
There are maybe transitive dependencies!
Page 1 from 1 (items total 1)