• Home
• com.virjar
• vscrawler-core
• 0.2.2
• source code

×

Project download

Many resources are needed to download a project. Please understand that we have to compensate our server costs. Thank you in advance.
Project price only 1 $

You can buy this project and download/modify it how often you want.

crawler-core.0.2.2.source-code.default_vsCrawler.properties Maven / Gradle / Ivy

#爬虫的默认配置,他会被vsCrawler.properties里面的配置项merge,作为真正生效的配置数据传递到各个组件
#工作目录,将会存在一些爬虫中间数据
vsCrawler.Working.directory=~/.vscrawler/work/

#预计 这个可能会被删除,考虑取消bloomFilter
seedManager.expectedSeedNumber=1000000


###########################默认爬虫配置###########################
#爬虫线程数目,默认10个线程
vsCrawler.vsCrawler.threadNumber=10

#初始种子文件
vsCrawler.vsCrawler.initSeedFile=


###########################多站点爬虫配置###########################