{"id":259,"date":"2018-07-16T22:05:05","date_gmt":"2018-07-16T22:05:05","guid":{"rendered":"http:\/\/sarahjpurcell.sites.grinnell.edu\/sandbox_clone\/?page_id=259"},"modified":"2019-11-04T15:43:24","modified_gmt":"2019-11-04T15:43:24","slug":"natural-language-processing","status":"publish","type":"page","link":"https:\/\/his100.sarahjpurcell.sites.grinnell.edu\/spring-2021\/tutorials\/textual-analysis\/natural-language-processing\/","title":{"rendered":"Natural Language Processing (Stanford’s Named Entity Recognizer)"},"content":{"rendered":"

This tutorial was written by Katherine Walden<\/a>, Digital Liberal Arts Specialist at Grinnell College. Tutorial instructions were co-authored by Sarah Purcell (L.F. Parker Professor of History, Grinnell College) and Papa Ampim-Darko, a student research assistant at Grinnell College.<\/p>\n

This tutorial was reviewed by Gina Donovan<\/a> (Instructional Technologist, Grinnell College).<\/p>\n

This tutorial is adapted from Michelle Moravec\u2019s History in the City Stanford NER tutorial<\/a> and Rachel Burma\u2019s The Rise of the Novel Robinson Crusoe NER assignment<\/a>.<\/p>\n

\"Creative<\/a>
\nNatural Language Processing (Stanford\u2019s Named Entity Recognizer) is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License<\/a>.<\/p>\n

\n

Developed in 2006 by a team based out of Stanford University\u2019s National Language Processing Group, the Stanford Named Entity Recognizer<\/a> (NER) is a Java-based tool for recognizing and extracting named entities in an unstructured textual dataset. In this tutorial, we\u2019ll be using Stanford\u2019s NER to identify named entities in The Interesting Narrative of the Life of Olaudah Equiano, Or Gustavus Vassa, The African<\/em>, an autobiographical slave narrative published in 1789.<\/p>\n

\n
Data<\/h5>\n

1- Navigate to\u00a0http:\/\/sarahjpurcell.sites.grinnell.edu\/digital_methods\/files\/Equiano_Text.txt<\/a> in a web browser and save the \u201cEquiano_Text\u201d text file to your Desktop.. Open Equiano.txt<\/strong> to see the plain text utf-8 file<\/a> downloaded from Project Gutenberg.<\/p>\n

\"\"<\/a><\/p>\n

2-Copy the file to your Desktop<\/strong>, and right click on the file<\/strong> to open it in Notepad<\/strong>, the native Windows text editing program.<\/p>\n

\"\"<\/a><\/p>\n

3-Because the text file was downloaded from Project Gutenberg, it contains information at the beginning and end of the file that is not part of the original Equiano<\/em> text. Delete these lines, re-save the file to your Desktop, and close Notepad.<\/p>\n

\n
Opening Stanford\u2019s NER<\/h5>\n

4-The NER has already been downloaded and installed on the Library computers. To install on your own computer, visit the program\u2019s Download page<\/a>.<\/p>\n

\"\"<\/a><\/p>\n

5-To open the program, navigate to C:\\Program Files (x86)\\Stanford NER and double click on the Windows Batch File named ner-gui<\/strong>.<\/p>\n

\"\"<\/a>\"\"<\/a><\/p>\n

6-The program will launch two windows\u2014a Command Prompt shell that will run the program via the CLI, and a GUI interface. Both windows need to remain open for the program to run.<\/p>\n

\n
Loading data into the NER<\/h5>\n

7-Erase the sample text<\/strong> in the GUI interface window.<\/p>\n

8-Click File->Open File<\/strong>, and select the Equiano.txt<\/strong> file saved to your Desktop.
\n\"\"<\/a><\/p>\n

9-Text will appear in the GUI window once the file has loaded.<\/p>\n

\n
Identifying Classifiers in the NER<\/h5>\n

10-Next, we need to load a list of sample terms for the NER to use when analyzing our text.
\n\"\"<\/a><\/p>\n

11-Click Classifier->Load CRF from File<\/strong>. Navigate to\u00a0C:\\Program Files (x86)\\Stanford NER\\classifiers and open the classifiers folder.<\/p>\n

\"\"<\/a><\/p>\n

12-Stanford\u2019s NER includes classifier resource files with 3, 4, and 7 entity categories it can search for in your text.<\/p>\n

\"\"<\/a><\/p>\n

13-Select the english.all.3class.distim.crf.ser.gz<\/strong> file and click the Open<\/strong> icon. Three entity categories (organization, location, person), with color labels, will now display on the right-hand side of the GUI window.<\/p>\n

14-Click on the Run NER icon<\/strong> on the bottom of the GUI window to run the program.<\/p>\n

\"\"<\/a><\/p>\n

15-You can see the results being generated from the NER program in the CLI window.<\/p>\n

\"\"<\/a><\/p>\n

16-The GUI window now shows the color-coded results of the NER program\u2019s analysis.<\/p>\n

\"\"<\/a><\/p>\n

17-The GUI interface gives you options to export your tagged file. Click File->Save Tagged File As<\/strong>. Navigate to your Desktop and save the file as Equiano_Tagged.txt.<\/strong><\/p>\n

18-You could also copy the text generated in the CLI and paste into a text file to see a list with just the tagged entities.<\/p>\n

\"\"<\/a><\/p>\n

19-Right click on the icon for the Equiano_Tagged.txt<\/strong> file and open in Notepad<\/strong>.<\/p>\n

20-You\u2019ll see that the NER has added <LOCATION>, <PERSON>, or <ORGANIZATION> tags around the entities it recognized based on the terms in the classification file. We could use a combination of XML and XPath to isolate terms with particular tag categories.<\/p>\n

\n
Reflection questions<\/h5>\n