edu.berkeley.nlp.io.LexedTokenFactory Maven / Gradle / Ivy

Go to download

Show more of this group Show more artifacts with this name
Show all versions of berkeleyparser Show documentation

The Berkeley parser analyzes the grammatical structure of natural language using probabilistic context-free grammars (PCFGs).

The newest version!

package edu.berkeley.nlp.io;

/**
 * Constructs a token (of arbitrary type) from a String and its position
 * in the underlying text.
 */
public interface LexedTokenFactory {
  /**
   * Constructs a token (of arbitrary type) from a String and its position
   * in the underlying text.
   * @param str The String extracted by the lexer.
   * @param begin The offset in the document of the first character
   *  in this string.
   * @param length The number of characters the string takes up in
   *  the document.
   */
  public T makeToken(String str, int begin, int length);
}