Named Entity Recognition (NER)
Named entity recognition pulls people, places, and organizations out of free text, and the first question is which model family to trust. The page starts with milestone NER papers and the BiLSTM line of work, then compares the off-the-shelf tools (NLTK, polyglot, spaCy, and Stanford NER), then goes deeper into Stanford NER (SNER), and ends with spaCy, active learning, and the BiLSTM-CRF tutorials.
The same notes are in ANONYMIZATION, BERT, CONDITIONAL RANDOM FIELDS (CRF), FLAIR, and SPACY.
The map of the field comes first. State of the art LSTM architectures using NN is the ParallelDots list of milestone papers, models, and technologies for recognizing named entities in unstructured text. A deep model needs a lot of labeled data, so the Medium series starts there: Ner free datasets covers the annotated, publicly available and mostly free datasets and the annotation methods behind them, and bilstm implementation is part two on Medium, implementing the state-of-the-art bidirectional LSTM from Chiu and using glove embeddings. It is easy to implement in keras, and it is based on the following paper.
A trained model is only one option; the other is to combine the tools that already exist. The Medium post from District Data Labs runs NLTK entities, polyglot entities, sner entities, and finally an ensemble method wins all! The figure below is that comparison.
The head-to-head that matters most in practice is spaCy against SNER. Comparison between spacy and SNER for terms is Eduardo Andrade's NER on unstructured documents, where contracts and other exchanged documents carry personal information that may have to be anonymised before some employees see it. When there are no labels at all, *** Unsupervised NER using Bert is Ajit Rajasekharan tagging sentences with bert-large-cased and no fine tuning: 500 tagged sentences yielded about 1000 unique entity types, of which a select few were mapped to synthetic labels. When the entity types are your own, Custom NER using spacy is Kaustumbh Jaiswal's custom spaCy model, and Spacy Ner with custom data is Manivannan Murugavel's walk through training spaCy on your own data with Python and getting the entities back out. The figure below is from the custom spaCy note.
Outside spaCy, a CRF can be built by hand. How to create a NER from scratch using kaggle data, using crf, and analysing crf weights using external package is Susan Li's NER and classification with Scikit-Learn. Another comparison between spacy and SNER - both are the same, for many classes. is Mohan Gupta's review, which explains what NER is, how industry uses it, the libraries for it, and a code walk through of NER for resume summarization. The figure below is that resume comparison.
The same comparison can be widened to the whole pipeline. Vidhaya on spacy vs ner — tutorial + code on how to use spacy for pos, dep, ner, compared to nltk/corenlp (sner etc). The results reflect a global score not specific to LOC for example, which the figure below shows.
Stanford NER (SNER)
Since SNER keeps showing up in those comparisons, it is worth knowing how it works. The SNER presentation - combines HMM and MaxEnt features, distributional features, NER has many applications. When it labels a word, future observations are taken into account, forward and backward. Before training it, How to train SNER, a FAQ with many other answers (read first before doing anything with SNER) is the Stanford Natural Language Processing Group's own FAQ. The SNER demo used to be linked here, with the note that capital letters matter, a minimum of one; it is kept at the end of the page.
To see where SNER stands, State of the art NER benchmark is the CoNLL 2003 task page in the Underthesea wiki. Two review papers compare the tools directly. Review paper, SNER, spacy, stanford wins is Evaluating and Combining Name Entity Recognition Systems, by Ridong Jiang, Rafael E. Banchs, and Haizhou Li at the Sixth Named Entity Workshop, 2016. Review paper SNER, others on biographical text, stanford wins is A Comparison of Named Entity Recognition Tools Applied to Biographical Texts, written because the tools differ in processing method, entity types, the text they can handle, and input and output formats, which makes choosing one hard. Another NER DL paper, 90%+ is a deep learning entry on OpenReview.
Spacy & Others
After the Stanford tool, the rest of the notes return to spaCy and to deep sequence taggers. Spacy - using prodigy and spacy to train a NER classifier using active learning is Explosion's video on training a new entity type with Prodigy, annotation powered by active learning. Ner using DL BLSTM, using glove embeddings, using CRF layer against another CRF is the NLP Town post on named entity recognition and the road to deep learning.
That BiLSTM-CRF design has a reference implementation. Another medium paper on the BLSTM CRF with guillarue’s code is entity extraction with deep learning based on Guillaume Genthial's NER work, on the point that deep learning produced large accuracy gains over traditional methods. Guillaume blog post, detailed explanation is the source: GloVe + character embeddings + bi-LSTM + CRF for sequence tagging (NER, POS), a bidirectional RNN and CRF in Tensorflow. A source for Italian used to sit here; it is kept at the end of the page.
The same family reaches past 90. Another 90+ proposed solution is the paper Neural Architectures for Named Entity Recognition. A promising python implementation based on one or two of the previous papers is the deeppavlov/ner repository for Named Entity Recognition. Quora advise, the first is cool, the second is questionable is the Quora thread on doing NER with RNN, LSTM, and Word2vec.
The off the shelf solutions benchmark that used to close this list is kept at the end of the page. What stays is the ParallelDots piece from the start, read this time for a different reason: Parallel api talk about bilstm and their 2mil tagged ner model (washington passes).
The Towards Data Science copy of the custom NER using spacy note is https://towardsdatascience.com/custom-named-entity-recognition-using-spacy-7140ebbb3718.
Deprecated links
These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.
State of the art LSTM architectures using NN This address no longer opens: https://blog.paralleldots.com/data-science/named-entity-recognition-milestone-models-papers-and-technologies/
Medium: Ner free datasets This address no longer opens: https://towardsdatascience.com/deep-learning-for-ner-1-public-datasets-and-annotation-methods-8b1ad5e98caf
bilstm implementation This address no longer opens: https://towardsdatascience.com/deep-learning-for-named-entity-recognition-2-implementing-the-state-of-the-art-bidirectional-lstm-4603491087f1
Unsupervised NER using Bert This address no longer opens: https://towardsdatascience.com/unsupervised-ner-using-bert-2d7af5f90b8a
How to create a NER from scratch using kaggle data, using crf, and analysing crf weights using external package This address no longer opens: https://towardsdatascience.com/named-entity-recognition-and-classification-with-scikit-learn-f05372f07ba2
Another comparison between spacy and SNER - both are the same, for many classes. This address no longer opens: https://towardsdatascience.com/a-review-of-named-entity-recognition-ner-using-automatic-summarization-of-resumes-5248a75de175
SNER demo - capital letters matter, a minimum of one. This address no longer opens: http://nlp.stanford.edu:8080/ner/process
Review paper SNER, others on biographical text, stanford wins This address no longer opens: https://arxiv.org/ftp/arxiv/papers/1308/1308.0661.pdf
Ner using DL BLSTM, using glove embeddings, using CRF layer against another CRF**.** This address no longer opens: http://nlp.town/blog/ner-and-the-road-to-deep-learning/
For Italian This address no longer opens: https://www.qcri.org/app/media/4916
Off the shelf solutions benchmark This address no longer opens: https://www.programmableweb.com/news/performance-comparison-10-linguistic-apis-entity-recognition/elsewhere-web/2016/11/03
Parallel api talk about bilstm and their 2mil tagged ner model (washington passes) This address no longer opens: https://blog.paralleldots.com/data-science/named-entity-recognition-milestone-models-papers-and-technologies/
Last updated
Was this helpful?