• Home
• io.github.amayaframework
• amaya-tokenize
• 1.0.0
• source code
• Tokenizer.java

×

Project download

Many resources are needed to download a project. Please understand that we have to compensate our server costs. Thank you in advance.
Project price only 1 $

You can buy this project and download/modify it how often you want.

io.github.amayaframework.tokenize.Tokenizer Maven / Gradle / Ivy

package io.github.amayaframework.tokenize;

/**
 * An interface describing an abstract tokenizer.
 * Splits a string by delimiters and returns the result as an abstract {@link Iterable} instance.
 * 

 * A good way to implement it would be not to create any collection storing tokens,
 * but to implement an {@link java.util.Iterator} that performs splitting on the fly.
 */
public interface Tokenizer {

    /**
     * Splits string by given delimiter.
     *
     * @param target the specified string to be split
     * @param delim  the specified string containing delimiters
     * @return the {@link Iterable} instance, containing result of splitting
     */
    Iterable tokenize(String target, String delim);
}