> For the complete documentation index, see [llms.txt](https://www.mlcompendium.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.mlcompendium.com/natural-language-processing-nlp/named-entity-recognition-ner.md).
# Named Entity Recognition (NER)
Named entity recognition pulls people, places, and organizations out of free text, and the first question is which model family to trust. The page starts with milestone NER papers and the BiLSTM line of work, then compares the off-the-shelf tools (NLTK, polyglot, spaCy, and Stanford NER), then goes deeper into Stanford NER (SNER), and ends with spaCy, active learning, and the BiLSTM-CRF tutorials.
The same notes are in [ANONYMIZATION](/responsible-ai/fairness-accountability-and-transparency.md#anonymization), [BERT](/natural-language-processing-nlp/pretrained-language-models.md#bert), [CONDITIONAL RANDOM FIELDS (CRF)](/predictive-ml/probabilistic-models.md#conditional-random-fields-crf), [FLAIR](/deep-learning/representations.md#flair), and [SPACY](/natural-language-processing-nlp/nlp.md#spacy).
The map of the field comes first. [State of the art LSTM architectures using NN](https://web.archive.org/web/20180123225955/http://blog.paralleldots.com/data-science/named-entity-recognition-milestone-models-papers-and-technologies/) is the ParallelDots list of milestone papers, models, and technologies for recognizing named entities in unstructured text. A deep model needs a lot of labeled data, so the Medium series starts there: [Ner free datasets](https://medium.com/data-science/deep-learning-for-ner-1-public-datasets-and-annotation-methods-8b1ad5e98caf) covers the annotated, publicly available and mostly free datasets and the annotation methods behind them, and [bilstm implementation](https://medium.com/data-science/deep-learning-for-named-entity-recognition-2-implementing-the-state-of-the-art-bidirectional-lstm-4603491087f1) is part two on Medium, implementing the state-of-the-art bidirectional LSTM from Chiu and using glove embeddings. It is easy to implement in keras, and it is based on the following [paper](https://arxiv.org/abs/1511.08308).
A trained model is only one option; the other is to combine the tools that already exist. The [Medium](https://medium.com/district-data-labs/named-entity-recognition-and-classification-for-entity-extraction-6f23342aa7c5) post from District Data Labs runs NLTK entities, polyglot entities, sner entities, and finally an ensemble method wins all! The figure below is that comparison.
NLTK, Polyglot, SNER, and ensemble NER comparison.
The head-to-head that matters most in practice is spaCy against SNER. [Comparison between spacy and SNER](https://medium.com/@dudsdu/named-entity-recognition-for-unstructured-documents-c325d47c7e3a) for terms is Eduardo Andrade's NER on unstructured documents, where contracts and other exchanged documents carry personal information that may have to be anonymised before some employees see it. When there are no labels at all, \*\*\* [Unsupervised NER using Bert](https://medium.com/data-science/unsupervised-ner-using-bert-2d7af5f90b8a) is Ajit Rajasekharan tagging sentences with bert-large-cased and no fine tuning: 500 tagged sentences yielded about 1000 unique entity types, of which a select few were mapped to synthetic labels. When the entity types are your own, [Custom NER using spacy](https://medium.com/data-science/custom-named-entity-recognition-using-spacy-7140ebbb3718) is Kaustumbh Jaiswal's custom spaCy model, and [Spacy Ner with custom data](https://medium.com/@manivannan_data/how-to-train-ner-with-custom-training-data-using-spacy-188e0e508c6) is Manivannan Murugavel's walk through training spaCy on your own data with Python and getting the entities back out. The figure below is from the custom spaCy note.
Custom NER using spaCy.
Outside spaCy, a CRF can be built by hand. [How to create a NER from scratch using kaggle data, using crf, and analysing crf weights using external package](https://medium.com/data-science/named-entity-recognition-and-classification-with-scikit-learn-f05372f07ba2) is Susan Li's NER and classification with Scikit-Learn. [Another comparison between spacy and SNER - both are the same, for many classes.](https://medium.com/data-science/a-review-of-named-entity-recognition-ner-using-automatic-summarization-of-resumes-5248a75de175) is Mohan Gupta's review, which explains what NER is, how industry uses it, the libraries for it, and a code walk through of NER for resume summarization. The figure below is that resume comparison.
spaCy and SNER comparison on resume summarization.
The same comparison can be widened to the whole pipeline. [Vidhaya on spacy vs ner](https://www.analyticsvidhya.com/blog/2017/04/natural-language-processing-made-easy-using-spacy-%E2%80%8Bin-python/) — tutorial + code on how to use spacy for pos, dep, ner, compared to nltk/corenlp (sner etc). The results reflect a global score not specific to LOC for example, which the figure below shows.
Vidhaya spaCy versus NER tutorial results.
**Stanford NER (SNER)**
Since SNER keeps showing up in those comparisons, it is worth knowing how it works. The [SNER presentation - combines HMM and MaxEnt features, distributional features, NER has](https://nlp.stanford.edu/software/jenny-ner-2007.pdf) [many applications.](https://nlp.stanford.edu/software/jenny-ner-2007.pdf) When it labels a word, future observations are taken into account, forward and backward. Before training it, [How to train SNER, a FAQ with many other answers (read first before doing anything with SNER)](https://nlp.stanford.edu/software/crf-faq.shtml#a) is the Stanford Natural Language Processing Group's own FAQ. The SNER demo used to be linked here, with the note that capital letters matter, a minimum of one; it is kept at the end of the page.
To see where SNER stands, [State of the art NER benchmark](https://github.com/magizbox/underthesea/wiki/TASK-CONLL-2003) is the CoNLL 2003 task page in the Underthesea wiki. Two review papers compare the tools directly. [Review paper, SNER, spacy, stanford wins](http://www.aclweb.org/anthology/W16-2703) is Evaluating and Combining Name Entity Recognition Systems, by Ridong Jiang, Rafael E. Banchs, and Haizhou Li at the Sixth Named Entity Workshop, 2016. [Review paper SNER, others on biographical text, stanford wins](https://arxiv.org/abs/1308.0661) is A Comparison of Named Entity Recognition Tools Applied to Biographical Texts, written because the tools differ in processing method, entity types, the text they can handle, and input and output formats, which makes choosing one hard. [Another NER DL paper, 90%+](https://openreview.net/forum?id=ry018WZAZ) is a deep learning entry on OpenReview.
**Spacy & Others**
After the Stanford tool, the rest of the notes return to spaCy and to deep sequence taggers. [Spacy - using prodigy and spacy to train a NER classifier using active learning](https://www.youtube.com/watch?v=l4scwf8KeIA) is Explosion's video on training a new entity type with Prodigy, annotation powered by active learning. [Ner using DL BLSTM, using glove embeddings, using CRF layer against another CRF](https://web.archive.org/web/20190320104331/http://www.nlp.town/blog/ner-and-the-road-to-deep-learning/) is the NLP Town post on named entity recognition and the road to deep learning.
That BiLSTM-CRF design has a reference implementation. [Another medium paper on the BLSTM CRF with guillarue’s code](https://medium.com/intro-to-artificial-intelligence/entity-extraction-using-deep-learning-8014acac6bb8) is entity extraction with deep learning based on Guillaume Genthial's NER work, on the point that deep learning produced large accuracy gains over traditional methods. [Guillaume blog post, detailed explanation](https://guillaumegenthial.github.io/sequence-tagging-with-tensorflow.html) is the source: GloVe + character embeddings + bi-LSTM + CRF for sequence tagging (NER, POS), a bidirectional RNN and CRF in Tensorflow. A source for Italian used to sit here; it is kept at the end of the page.
The same family reaches past 90. [Another 90+ proposed solution](https://arxiv.org/pdf/1603.01360.pdf) is the paper Neural Architectures for Named Entity Recognition. [A promising python implementation based on one or two of the previous papers](https://github.com/deepmipt/ner) is the deeppavlov/ner repository for Named Entity Recognition. [Quora advise, the first is cool, the second is questionable](https://www.quora.com/How-can-I-perform-named-entity-recognition-using-deep-learning-RNN-LSTM-Word2vec-etc) is the Quora thread on doing NER with RNN, LSTM, and Word2vec.
The off the shelf solutions benchmark that used to close this list is kept at the end of the page. What stays is the ParallelDots piece from the start, read this time for a different reason: [Parallel api talk about bilstm and their 2mil tagged ner model (washington passes)](https://web.archive.org/web/20180123225955/http://blog.paralleldots.com/data-science/named-entity-recognition-milestone-models-papers-and-technologies/).
The Towards Data Science copy of the custom NER using spacy note is .
## Deprecated links
{% hint style="warning" %}
These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.
{% endhint %}
* State of the art LSTM architectures using NN This address no longer opens:
* Medium: Ner free datasets This address no longer opens:
* bilstm implementation This address no longer opens:
* Unsupervised NER using Bert This address no longer opens:
* How to create a NER from scratch using kaggle data, using crf, and analysing crf weights using external package This address no longer opens:
* Another comparison between spacy and SNER - both are the same, for many classes. This address no longer opens:
* SNER demo - capital letters matter, a minimum of one. This address no longer opens:
* Review paper SNER, others on biographical text, stanford wins This address no longer opens:
* Ner using DL BLSTM, using glove embeddings, using CRF layer against another CRF\*\*.\*\* This address no longer opens:
* For Italian This address no longer opens:
* Off the shelf solutions benchmark This address no longer opens:
* Parallel api talk about bilstm and their 2mil tagged ner model (washington passes) This address no longer opens:
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://www.mlcompendium.com/natural-language-processing-nlp/named-entity-recognition-ner.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
