org.archive.modules.fetcher.FetchDNS_en.utf8 Maven / Gradle / Ivy

Go to download

Show more of this group Show more artifacts with this name
Show all versions of heritrix-modules Show documentation

This project contains some of the configurable modules used within the Heritrix application to crawl the web. The modules in this project can be used in applications other than Heritrix, however.

There is a newer version: 3.5.0

Show newest version

description:
DNS Fetcher. Handles DNS lookups.


accept-non-dns-resolves-description:
If a DNS lookup fails, whether or not to fallback to InetAddress 
resolution, which may use local 'hosts' files or other mechanisms. 


server-cache-description:
Used to do DNS lookups. 


digest-content-description:
Whether or not to perform an on-the-fly digest hash of retrieved
content-bodies.

 
digest-algorithm-description:
Which algorithm (for example MD5 or SHA-1) to use to perform an
on-the-fly digest hash of retrieved content-bodies.