All Downloads are FREE. Search and download functionalities are using the official Maven repository.

org.apache.lucene.analysis.cjk.package.html Maven / Gradle / Ivy

There is a newer version: 3.6.2
Show newest version







Analyzer for Chinese, Japanese, and Korean, which indexes bigrams (overlapping groups of two adjacent Han characters).

Three analyzers are provided for Chinese, each of which treats Chinese text in a different way.

  • ChineseAnalyzer (in the analyzers/cn package): Index unigrams (individual Chinese characters) as a token.
  • CJKAnalyzer (in this package): Index bigrams (overlapping groups of two adjacent Chinese characters) as tokens.
  • SmartChineseAnalyzer (in the analyzers/smartcn package): Index words (attempt to segment Chinese text into words) as tokens.
Example phrase: "我是中国人"
  1. ChineseAnalyzer: 我-是-中-国-人
  2. CJKAnalyzer: 我是-是中-中国-国人
  3. SmartChineseAnalyzer: 我-是-中国-人





© 2015 - 2024 Weber Informatics LLC | Privacy Policy