Making decisions in natural language processing problems often involves assigning values to sets of interdependent variables where the expressive dependency structure can influence, or even dictate, what assignments are possible. Structured learning problems such as semantic role labeling provide one such example, but the setting is broader and includes a range of problems such as name entity and relation recognition and co-reference resolution. The setting is also appropriate for cases that may require a solution to make use of multiple models (possible pre-designed or pre-learned components) as in summarization, textual entailment and question answering.
This semester, we will devote the course to the study of structured learning problems in natural language processing. We will start by recalling the ``standard” learning formulations as used in NLP, move to formulations of multiclass classification and from then on focus on models of structure predictions and how they are being used in NLP.
Through lectures and paper presentations the course will introduce some of the central learning frameworks and techniques that have emerged in this area over the last few years, along with their application to multiple problems in NLP and Information Extraction. The course will cover:
Models: We will present both discriminative models such as structured Perceptron and Structured SVM, Probabilistic models, and Constrained Conditional Models.
Training Paradigms: Joint Learning models; Decoupling learning from Inference; Constrained Driven Learning; Semi-Supervised Learning of Structure; Indirect Supervision
Inference: Constrained Optimization Models, Integer Linear Programming, Approximate Inference, Dual Decomposition.
Machine Learning class; CIS 419/519/520 or equivalent. Knowledge of NLP is recommended but not mandatory.
There will be
Course Projects (40%) - The project will be done in teams of sizes 2 or 3; teams will proposed projects and consult us. We will have milestones along define a few intermediate stages and results will be reported and presented at the end of each stage.
Reading assignments - Mandatory readings and additional recommended readings will be assigned every week.
Critical Surveys ( 6 + 6 + 6 + 12 = 30% ) - Four (4) times a semester you will write a short critical essay on one of the additional readings.
Presentations ( 20% ) - Once or twice you will present a paper from the additional readings (30 minutes, focusing on the mathematical/technical details of the paper). The presentations will be prepared in groups, whenever possible, and a group of presentations will form a coherent tutorial, whenever possible (more on that later).
Class Participation ( 10% )
There is no final exam.
This is an advanced course. I view my role as guiding you through the material and helping you in your first steps as an researcher. I expect that your participation in class, reading assignments and presentations will reflect independence, mathematical rigor and critical thinking.