• Home
• org.apache.droids
• droids-norobots
• 0.2-incubating
• source code
• package.html

×

Project download

Many resources are needed to download a project. Please understand that we have to compensate our server costs. Thank you in advance.
Project price only 1 $

You can buy this project and download/modify it how often you want.

org.apache.droids.norobots.package.html Maven / Gradle / Ivy

  
Using norobots-rfc
   

   
Import the class
   
       import org.apache.http.norobots.NoRobotClient;
   
   
   
Create an instance for your user-agent
   
       NoRobotClient nrc = NoRobotClient("googlebot");
   
   
   
Parse a robots.txt at a site
   
       nrc.parse( new URL( "http://www.apache.org/" ) );
   
   
   
Ask if a url is allowed
   
       boolean test = nrc.isUrlAllowed( new URL( "http://www.apache.org/index.html" ) );