org.archive.modules.canonicalize.FixupQueryString_en.utf8 Maven / Gradle / Ivy
Go to download
Show more of this group Show more artifacts with this name
Show all versions of heritrix-modules Show documentation
Show all versions of heritrix-modules Show documentation
This project contains some of the configurable modules used within the
Heritrix application to crawl the web. The modules in this project can
be used in applications other than Heritrix, however.
description:
Fixup the question mark that leads off the query string.
This rule returns 'http://www.archive.org/index.html' if passed
'http://www.archive.org/index.html?'. It will also strip '?&'
if '?&' is all that comprises the query string. Also strips
extraneous leading '&': Returns 'http://archive.org/index.html?x=y
if passed 'http://archive.org/index.html?&x=y.
Will also strip '&' if last thing in query string.
Operates on all schemes. This is a good rule to run toward the
end of canonicalization processing.