Factored language model

From Wikipedia, the free encyclopedia

There are very few or no other articles that link to this one.
Please help introduce links in articles on related topics. After links have been created, remove this message.
This article has been tagged since August 2006.

The factored language model (FLM) is an extension of conventional Language model. In an FLM, each word is viewed as a vector of k factors: $w_i = \{f_i^1, ..., f_i^k\}$ . An FLM provides the probabilistic model $P (f | f i,..., f N)$ where the prediction of factor $f$ is based on $N$ parents ${f 1,..., f N}$ . For an example, if $w$ represents word token and $t$ represents Part of speech tag for English, the model $P (w i | w i - 2, w i - 1, t i - 1)$ gives a model for predicting current work token based on traditional Ngram model as well as Part of speech tag of the previous word.

A main advantage of factored language models is they allow users to put in linguistic knowledge such as explicitly model the relationship between word tokens and Part of speech in English, or morphological information (stems, root, etc.) in Arabic.

Like N-gram models, smoothing techniques are necessary in parameter estimation. In particular, generalized backing-off is used in training an FLM.