ipe-tok.1.8.6.source-code.en-nonbreaker.txt Maven / Gradle / Ivy

Go to download
Show more of this group Show more artifacts with this name
Show all versions of ixa-pipe-tok Show documentation
The ixa pipes tokenizer (ixa2.si.ehu.es/ixa-pipes)
The newest version!
# Anything in this file, followed by a period (and an upper-case word), does NOT indicate an end-of-sentence marker.

#any single upper case letter  followed by a period is not a sentence ender (excluding I occasionally, but we leave it in)
#usually upper case letters are initials in a name
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z

#These are often followed by upper-case names, but do not indicate sentence breaks
Adj
Adm
Adv
Asst
Assn
Assoc
Bart
Bldg
Blvd
Brig
Bros
Capt
Cmdr
Col
Comdr
Con
Corp
Cpl
Dep
Dept
DR
Dr
Drs
Ens
Esq
ext
Fac
Gen
Gov
Hon
Hr
Hosp
Inc
Insp
Intl
Jr
Lt
Lieut
Ltd
MM
MR
MRS
MS
Maj
Messrs
Mlle
Mme
Mr
Mrs
Ms
Msgr
Op
Ord
Pfc
Plc
Ph
Prof
Pvt
Rd
Rep
Reps
Res
Rev
Rt
Sen
Sens
Sfc
Sgt
Sys
Sr
St
sq
Supt
Surg
tel
Univ

# freeling

a.a
a.a.a
a.a.u
a.b
a.b.a
abbr
abr
a.c
acad
aclu
a.d
a.e.c
a.f.l
afl-cio
afrik
a.i.a
a.k.c
a.l
a.l.a
alt
alta
a.m
a.m.a
a.m.p
a.m.u
antilog
a.p
arab
ariz
ark
a.s
ascap
at.no
at.wt
a.u
aug
a.v
avdp
ave
b.a
b.b.c
b.c
b.d
b.lit
b.mus
b.p
brig.gen
b.s
b.t.u
bul
bulg
cal
calif
cant
capt
c.c
c.d
cent
cento
c.e.o
c.g.s
chem
chin
chron
c.i.a
c.i.d
c.i.o
c.m
co
col
coll
colo
comdr
comp
com.pop
conn
cor
corp
cos
cot
coul
c.p.a
c.p.l
c.p.o
c.s.c
c.u
dan
dar
d.c
d.c.l
d.d
d.d.s
d.d.t
dec
del
dept
deut
dist
div
dr
d.sc
du
e.c
e.c.a
eccles
ecclus
ed
e.d.c
e.e
e.e.a
e.e.c
e.e.o.c
e.f.t.a
e.g
e.m.f
e.m.u
eng
enl
eph
e.r.a
e.r.p
e.s.c
esp
est
e.u
ev
ex
ezek
f.a.a
fac
f.a.o
f.b.i
f.c.c
f.d.a
feb
f.e.p.c
finn
fl
fla
floz
f.m
fr
ft
f.t.c
ga
gal
gall
gatt
g.d.p
gen
ger
g.m.t
g.n.p
g.o.p
gov
gr
grad
hab
hag
heb
h.m.s
hon
hr
hung
hz
i.a.u
i.b.m
i.b.t
i.c.a.o
i.c.b.m
i.c.c
icel
i.e
i.g.y
ilgwu
ill
i.l.o
i.m.f
inc
ind
inst
introd
i.q
i.r.a
i.r.b.m
i.r.s
isa
ital
i.t.u
i.u.p.a.c
i.w.w
jan
jap
j.d
jer
j.g
jr
kc
kg
kgb
kgm
k.k.k
kl
km
kw
kwh
ky
la
lam
lat
lb
lev
l.h.d
lib
lith
litt.b
litt.d
ll.b
ll.d
l.s.d
lt
lt.col
ltd
lt.gen
lt.gov
m.a
mac
maj.gen
mal
mass
mass.no
m.d
md
m.e
mev
mex
mg
m.h.g
mi
mich
min
minn
miss
mks
ml
mlle
mm
mme
mo
mont
m.p
mph
m.p.h
mr
mrs
m.s
ms
msgr
ms
mss
mt
mts
mus
mus.b
mus.d
n.a.a.c.p
n.a.f.t.a
n.a.s.a
n.a.s.d.a.q
n.a.t.o
n.b
n.b.a
n.c
n.c.a.a
n.c.o
n.dak
n.e
n.e.a
nebr
neh
nev
n.f
n.f.l
n.h
n.h.l
n.j
nl
n.l.r.b
n.mex
nnw
no
nor
nov
n.r.a
n.r.c
n.s
n.s.f
num
n.y
n.y.a
n.y.s.e
o.a.s
obad
oct
o.e
o.e.c.d
o.e.o
o.e.s
o.fr
o.h.g
okla
o.n
ont
op
o.p.a
o.s
o.s.c.e
o.s.s
o.z
pa
p.a.u
pd.d
p.e.i
pers
p.f.c
p.g.a
ph.b
ph.d
philip
pl
plc
p.m
po
pol
pop
port
prov
prov(s)
ps
pseud
pss
pt
pub
pvt
p.w.a
q.t
que
r.a
r.a.f
repr
rev
r.i
r.n
r.n.a
rom
r.o.t.c
r.p.m
rpm
r.r
r.s.f.s.r
r.s.v
rt.rev
rus
r.v
sam
sask
s.c
sc.d
s.dak
s.e
s.e.a.t.o
sec
sept
ser
sgt
s.j
skt
s.o.s
span
s.p.c.a
s.p.c.c
sp.gr
s.q
sr
s.s
s.s.r
st
s.t.d
s.t.e
s.t.p
s.w
swed
t.a.n
t.a.s.s
tenn
thess
tim
t.n.t
tr
turk
t.v.a
u.a.w
u.h.f
ukr
u.m.w
u.n
uninc
univ
u.n.r.r.a
u.p.i
u.s
u.s.a
u.s.a.f
u.s.c.g
u.s.m.c
u.s.n
u.s.o
u.s.s
u.s.s.r
u.t
va
var
ved
v.f.w
v.h.f
vol
vs
vt
w.a.c
w.c.t.u
w.e.u
w.f.t.u
wis
wmo
wpa
wt
wto
w.va
wyo
yd
y.m.c.a
y.m.h.a
y.w.c.a
y.w.h.a
zech
zeph

#misc - odd period-ending items that NEVER indicate breaks (p.m. does NOT fall into this category - it sometimes ends a sentence)
v
vs
i.e
rev
e.g

# sometimes occur before number, but break anyway
Nos
Nr

# dates
Jan
Feb
Mar
Apr
Jun
Jul
Aug
Sept
Oct
Nov
Dec
Mon
Tue
Wed
Thur
Fri
Sat
#states
Ala
Ariz
[A]rk
Calif
Colo
Conn
Dak
Del
Fla
Ga
[I]ll
Ind
Kans?
Ky
La
[M]ass
Md
Mich
Minn
[M]iss
Mo
Mont
Neb
Nev
Okla
[O]re
Pa
Penn
Tenn
Tex
Va
Vt
[W]ash
Wisc
Wyo