slmprune(1) [debian man page]

SLMPRUNE(1)						User Contributed Perl Documentation					       SLMPRUNE(1)

NAME

       slmprune - prune the back-off language model to a reasonable size

SYNOPSIS

       slmprune input_slm result_slm cut_option num...

DESCRIPTION

       This program uses entropy-based method to prune the size of back-off language model 'input_slm' to a specific size and write to
       'result_slm'.

       the third parameter cut_option can be [R|C]. It means the following numbers is the number for (R)eserve or (C)ut. If (C)ut, the num[k]
       means how many items in level K would be cut. If (R)eserve, num[k] means how many item would be reserved in level k.

       Note that we do not ensure that during pruning process,	exactly the the given number of items are cut or reserved, because some items may
       contains high level children, so could not be cut.

       Also it's your responsiblity to give right number of arguments based on 'input_slm'.

NOTE

       To get information of the back-off language model, try 'slminfo'.

AUTHOR

       Originally written by Phill.Zhang <phill.zhang@sun.com>.  Currently maintained by Kov.Chai <tchaikov@gmail.com>.

SEE ALSO

       slminfo(1), slmthread(1).

perl v5.14.2							    2012-06-09							       SLMPRUNE(1)

SLMSEG(1)						User Contributed Perl Documentation						 SLMSEG(1)

NAME

       slmseg - maximum matching segment Chinese text.

SYNOPSIS

       slmseg -d dict_file [option]... [corpus_file]...

DESCRIPTION

       slmseg is a tool for segmenting Chinese text into words using maximum matching algorithm. slmseg segments corpus_file, or standard input if
       no filename is specified, and write the segmented result to standard output.

OPTIONS

       -d dict_file
	   Use dict_file as lexicon. A default lexicon can be found at /usr/share/sunpinyin-slm/dict.utf8.

       -f,--format (text|bin)
	   Output Format, can be 'text' or 'bin'. default 'bin'.  Normally, in text mode, word text are output, while in binary mode, binary short
	   integer of the word-ids are written to stdout.

       -s, --stok STOK_ID
	   Sentence token id. Default 10.  It will be written to output in binary mode after every sentence.

       -i, --show-id
	   Show Id info. Under text output format mode, attach id after known words.  If under binary mode, print id(s) in text.

       -m, --model language-model-file Speficy the language model file. This file is always generated by slmthread.

NOTES

       Under binary mode, consecutive id of 0 are merged into one 0.  Under text mode, no space are inserted between unknown-words.

AUTHOR

       Originally written by Phill.Zhang <phill.zhang@sun.com>.  Currently maintained by Kov.Chai <tchaikov@gmail.com>.

SEE ALSO

       mmseg(1), ids2ngram (1).

perl v5.14.2							    2012-06-09								 SLMSEG(1)