Probabilistic latent semantic analysis

From Wikipedia, the free encyclopedia

Probabilistic latent semantic analysis (PLSA), also know as Probabilistic latent semantic indexing (PLSI, especially in Information retrieval circles) is a statistical technique for the analysis of two-mode and co-occurrence data. PLSA evolved from Latent semantic analysis, adding a sounder probabilistic model. PLSA has applications in information retrieval and filtering, natural language processing, machine learning from text, and related areas. It was introduced in 1999 by Thomas Hofmann ^[1][2], and it is related to non-negative matrix factorization.

Compared to standard latent semantic analysis which stems from linear algebra and downsizes the occurrence tables (usually via a singular value decomposition), probabilistic latent semantic analysis is based on a mixture decomposition derived from a latent class model. This results in a more principled approach which has a solid foundation in statistics.

It is reported that the aspect model used in the probabilistic latent semantic analysis has severe overfitting problems. The number of parameters grows linearly with the number of documents and it's not a generative model, as it doesn't provide a generative procedure for documents^[3].

[edit] Evolutions of PLSA

• Hierarchical extensions:
  • Asymmetric: MASHA ("Multinomial ASymmetric Hierarchical Analysis") ^[4]
  • Symmetric: HPLSA ("Hierarchical Probabilistic Latent Semantic Analysis") ^[5]

• Discriminative models: The following models have been developed to address an often-criticized shortcoming of pLSA, namely that it is not a proper generative model for new documents.
  • Latent Dirichlet Allocation.

[edit] References and notes

1. ^ Thomas Hofmann, Probabilistic Latent Semantic Indexing, Proceedings of the Twenty-Second Annual International SIGIR Conference on Research and Development in Information Retrieval (SIGIR-99), 1999
2. ^ Thomas Hofmann, Learning the Similarity of Documents : an information-geometric approach to document retrieval and categorization, Advances in Neural Information Processing Systems 12, pp-914-920, MIT Press, 2000
3. ^ Bridges from pLSI to generative models however exist, via Fisher kernels, which make the link between pLSI and probalistic models, which are generative.
4. ^ Alexei Vinokourov and Mark Girolami, A Probabilistic Framework for the Hierarchic Organisation and Classification of Document Collections, in Information Processing and Management, 2002
5. ^ Eric Gaussier, Cyril Goutte, Kris Popat and Francine Chen, A Hierarchical Model for Clustering and Categorising Documents, in "Advances in Information Retrieval -- Proceedings of the 24th BCS-IRSG European Colloquium on IR Research (ECIR-02)", 2002

[edit] External link

• Probabilistic Latent Semantic Analysis