All Downloads are FREE. Search and download functionalities are using the official Maven repository.

iki.ok-ml-pipelines_2.11.0.5.1.source-code.README.md Maven / Gradle / Ivy

## About the "languages" folder and files

Most of these files are from the original software from Nakatani Shuyo.
Unfortunately, the data sources from which they were generated are not available.
It looks like the text comes from Wikipedia pages.

To generate your own language profile, see the main readme at https://github.com/optimaize/language-detector

km Khmer:
sources available, see https://github.com/optimaize/language-detector/issues/19

## About the "languages.shorttext" folder and files

These files are from the original software from Nakatani Shuyo.

Either they are for detecting language on short messages, or they are built from short message text, or
both, I don't know.


## About the "messages.properties" file

They are used in the CharNormalizer.





© 2015 - 2024 Weber Informatics LLC | Privacy Policy