Named entity recognition

From Wikipedia, the free encyclopedia

Named entity recognition (NER) (also known as entity identification (EI) and entity extraction) is a subtask of information extraction that seeks to locate and classify atomic elements in text into predefined categories such as the names of persons, organizations, locations, expressions of times, quantities, monetary values, percentages, etc.

For example, a NER system producing MUC-style output might tag the sentence,

Jim bought 300 shares of Acme Corp. in 2006.
<ENAMEX TYPE="PERSON">Jim</ENAMEX> bought <NUMEX TYPE="QUANTITY">300</NUMEX> shares of <ENAMEX TYPE="ORGANIZATION">Acme Corp.</ENAMEX> in <TIMEX TYPE="DATE">2006</TIMEX>.

NER systems have been created that use linguistic grammar-based techniques as well as statistical models. Hand-crafted grammar-based systems typically obtain better results, but at the cost of months of work by experienced linguists. Statistical NER systems typically require a large amount of manually annotated training data.

Since about 1998, there has been a great deal of interest in entity identification in the molecular biology, bioinformatics, and medical natural language processing communities. The most common entity of interest in that domain has been names of genes and gene products.

Contents

[edit] Named entity types

In the expression named entity, the word named restricts the task to those entities for which one or many rigid designators, as defined by Kripke, stands for the referent. For instance, the automotive company created by Henry Ford in 1903 is referred to as Ford or Ford Motor Company. Rigid designators include proper names as well as certain natural kind terms like biological species and substances.

There is a general agreement to include temporal expressions and some numerical expressions such as money and measures in named entities. While some instances of these types are good examples of rigid designators (e.g., the year 2001) there are also many invalid ones (e.g., I take my vacations in “June”). In the first case, the year 2001 refers to the 2001st year of the Gregorian calendar. In the second case, the month June may refer to the month of an undefined year (past June, next June, June 2020, etc.). It is arguable that the named entity definition is loosened in such cases for practical reasons.

At least two hierarchies of named entity types have been proposed in the literature. BBN categories [1], proposed in 2002, is used for Question Answering and consists of 29 types and 64 subtypes. Sekine's extended hierarchy [2], proposed in 2002, is made of 200 subtypes.

[edit] Evaluation

Benchmarking and evaluations have been performed in the Message Understanding Conferences (MUC) organized by DARPA, International Conference on Language Resources and Evaluation (LREC), Computational Natural Language Learning (CoNLL) workshops, Automatic Content Extraction (ACE) organized by NIST, the Multilingual Entity Task Conference (MET), Information Retrieval and Extraction Exercise (IREX) and in HAREM (Portuguese language only).

State-of-the-art systems produce near-human performance. For instance, the best system entering MUC-7 scored 93.39% of f-measure while human annotators scored 97.60% and 96.95%.

[edit] See also

[edit] External links

[edit] Evaluation forums

[edit] Datasets and hierarchies

[edit] Open source or free

  • AbGene Biomedical named entity recognizer.
  • ABNER Biomedical named entity recognizer.
  • AIIAGMT Biomedical named entity recognizer.
  • ANNIE Information extraction package (a GATE component) with NER capabilities.
  • Balie Baseline implementation of named entity recognition.
  • ESpotter A domain and user adaptation approach for named entity recognition on the Web.
  • FreeLing An open source language analysis tool suite. See the online demo.
  • KeX A simple Knowledge EXtraction tool.
  • MinorThird Collection of Java classes for storing text, annotating text, and learning to extract entities and categorize text.
  • MutationFinder An information extraction system for extracting descriptions of point mutations from free text.
  • Named Entity Tagger Yet another demo system for named entity tagging. Allows users to enter their own text.
  • POSBIOTM/W NER client tool that enables users to automatically annotate biomedical-related entities.
  • Carabao MorphoLogic Mixed dictionary-based and heuristics-based named entity recognition for single words only.
  • Stanford NER NER client tool based on Java. Uses CRF algorithm.

[edit] Dual license (free and commercial version)

  • LingPipe Java Natural Language Processing software that includes a trainable named-entity extraction framework with first-best, n-best and confidence-ranked-by-entity output. Models available for various languages and genres. See the online demos.
  • Calais Named entity, fact and event extraction web service provided by Reuters

[edit] Commercial

  • AeroText An extensible, commercial natural language processing toolkit for entity, relationship, and event extraction.
  • Alethes Commercial Text Analytics Solution, entity extraction, information extraction, categorization, clustering, sentiment analysis for 8 different language.
  • Basis Technology's Rosette Entity Extractor (REX)
  • ClearForest Commercial natural language processing toolkit that includes NER.
  • Cortex Intelligence Commercial competitive intelligence web software that use entity extraction technology.
  • Expert System Commercial natural language processing, entity extraction, categorization rules and domain construction tool sets.
  • Inxight: Natural language processing, entity extraction and fact extraction in 32 languages.
  • ISYS An enterprise search product which includes automatic entity recognition
  • LCC CiceroLite Commercial and state-of-the-art extraction suite which includes entity extraction for English, Chinese and Arabic.
  • PolyAnalyst Commercial natural language processing suite with entity extraction tools
  • SRA NetOwl Commercial and state-of-the-art recognizer in its class (rule and statistical based) covering many scripts and including highly inflected languages such as Arabic.
  • Teragram multilingual entity extraction
  • Trifeed Ltd. Trifeed is a research and development software company operating in the field of text analysis and information extraction.
  • OpenEyes Commercial NLP suite with entity and information extraction engine and resource
Languages