Linguistics/CSE 256: Statistical Natural Language Processing

Course information

Instructor information

Course Description

The goal of this course is to train you, the students, to do research in natural language processing — work that can potentially be published in the leading conferences and journals of the field. In addition to helping you succeed academically in this field (and related fields including AI, machine learning, and psycholinguistics), this is also great training if you are interested in doing NLP work in industry, either in a research lab (Google, Microsoft, Powerset, Yahoo, etc.) or in a startup.

Intended Audience

Graduate students in linguistics, computer science, engineering, cognitive science, psychology, and any other discipline who are interested in how to process natural language by computer. Highly motivated undergraduates are also welcome, but please talk to the instructor before enrolling.

Reading material

We're going to be using the two premier textbooks in the field for this course:

1. Jurafsky and Martin 2008: Speech and Language Processing (2nd Edition). Prentice Hall. We should be getting beta copies of the second edition!
2. Manning and Schütze 1999: Foundations of Statistical Natural Language Processing. MIT Press. Here's the online version.

There will also be at least one reading from the following new textbook:

1. Manning, Raghavan, and Schütze 2008: Introduction to Information Retrieval. Avaliable here.

We may occasionally read recent papers from the literature as well.

Skills required & suggested background

Working in natural language processing requires putting several different types of skills together:

• Math: basic probability theory, linear algebra, a bit of statistics
• Linguistics: basic syntax
• Computer science: data structures, algorithms
• Programming skills

You may not have all of these skills yet, but hopefully you have a substantial subset of them. It may require a bit of extra work for you to strengthen your background in any area where you're deficient — the focus within the class will be on how to put them together.

Mailing List

The mailing list for the class is ligncse256@ling.ucsd.edu. Sign up for the mailing list here.

Syllabus (subject to change!)

Requirements & grading

Your grade will be based on the following criteria:

1. Several written homework assignments, to be distributed at various times during the class;
2. Several programming assignments, to be distributed at various times during the class;
3. A final project

Collaboration is encouraged for homework assignments and final projects, but you must be explicit about who you collaborated with and what the division of labor was.

Leading Conferences and Journals in the field

Computational linguistics/NLP is a very conference-oriented field; many of the classic articles in the literature never wind up getting published in journals. The top conferences include:

• the Annual Meeting of the Association for Computational Linguistics (ACL)
• the conference on Empirical Methods in Natural Language Processing (EMNLP)
• the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)
• the Conference of the European Chapter of the Association for Computational Linguistics (EACL)

There are also some excellent workshops and conferences run regularly by "special interest groups", including

• the Conference on Computational Natural Language Learning
• the International Conference on Parsing Technology

and others. Finally, excellent work in computational linguistics/NLP also appears in machine learning, artificial intelligence, and other conferences, notably the Conference on Neural Information Processing Systems (NIPS).

The flagship and leading journal of the field is Computational Linguistics. Other excellent journals in the field include:

• Natural Language Engineering
• Computer Speech & Language