Linguistics 274: Computational Psycholinguistics
Spring 2007

This is the course website for the reading seminar Computational Psycholinguistics, taught Spring quarter 2007. This course is a reading seminar covering a variety of computational modeling approaches to human language comprehension, production, acquisition, and representation. There is a strong emphasis on probabilistic approaches: at its core, the processing of natural language involves dealing with uncertainty all the time, and in psycholinguistic research probability theory is playing a larger and larger role in modeling how people deal with this uncertainty.

Prerequisites

There are no formal prerequisites for this seminar, but we will be reading some fairly advanced examples of computational modeling papers, and it can't hurt you to have a good background in this area. In particular, we'll be relying on ideas from probability theory and machine learning, so some background in this area is useful. Familiarity with parsing algorithms for natural language sentences is also useful; if you've never taken a computational linguistics class, you can get a head start by looking at the draft Chapter 12 of the upcoming second edition of Jurafsky and Martin.

Requirements

The requirements for participation in this seminar are that you show up, participate in discussion, lead discussion of a paper at some point during the quarter, and (if you are taking the course for credit) write a final paper (research or review) on some topic covered in the course.

Class schedule

This schedule is tentative and rest assured that it will be changed at least somewhat. You are encouraged to suggest additional readings on the topics listed below, or on topics that don't appear but you're interested in.

Date	Topic & Reading	Discussion leader
Monday 2 April 2007	Introduction and organization No reading
Thursday 5 April 2007	Historical material Yngve, V. (1960). A model and an hypothesis for language structure. In Proceedings of the American Philosophical Society, pages 444-466. [ .pdf ]	Roger
Monday 9 April 2007	Working memory in sentence comprehension (1) Just, M. A. and Carpenter, P. A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99(1):122-149. [ .pdf ] MacDonald, M. C. and Christiansen, M. H. (2002). Reassessing working memory: Comment on Just and Carpenter (1992) and Waters and Caplan (1996). Psychological Review, 109(1):35-54. [ .pdf ]	Lisa: Just & Carpeter Tanya: MacDonald & Christiansen
Wednesday 11 April 2007	Working memory in sentence comprehension (2) Lewis, R. L. and Vasishth, S. (2005). An activation-based model of sentence processing as skilled memory retrieval. Cognitive Science, 29:1-45. [ .pdf ] Lewis, R. L., Vasishth, S., and Dyke, J. V. (2006). Computational principles of working memory in sentence comprehension. Trends in Cognitive Science, 10(10). [ .pdf ]	Rebecca: Lewis & Vasishth Erin: Lewis et al.
Monday & Wednesday 16-18 April 2007	Roger out of town, no class
Monday 23 April 2007	Probability, expectations & information theory in sentence comprehension (1) Jurafsky, D. (1996). A probabilistic model of lexical and syntactic access and disambiguation. Cognitive Science, 20(2):137-194. [ http ] Hale, J. (2001). A probabilistic Earley parser as a psycholinguistic model. In Proceedings of NAACL, volume 2, pages 159-166. [ .pdf ] Levy, R. (2007). Expectation-based syntactic comprehension. Cognition. In press. [ .pdf ]	Alex: Jurafsky Hannah: Hale, Levy
Wednesday 25 April 2007	Probability & information theory in sentence comprehension (2) Hale 2001 & Levy 2007, continued Hale, J. (2006). Uncertainty about the rest of the sentence. Cognitive Science, 30(4):609-642. [ .pdf ]	Gabe: Hale
Monday 30 April 2007	Competition and local coherences in sentence comprehension (1) Tabor, W., Juliano, C., and Tanenhaus, M. K. (1997). Parsing in a dynamical system: An attractor-based account of the interaction of lexical and structural constraints in sentence processing. Language and Cognitive Processes, 12(2/3):211-271. [ http ] Tabor, W. and Hutchins, S. (2004). Evidence for self-organized sentence processing: Digging in effects. Journal of Experimental Psychology: Learning, Memory, and Cognition,, 30(2):431-450. [ .pdf ]	Roger
Wednesday 2 May 2007	Competition and local coherences in sentence comprehension (2) Gibson, E. (2006). The interaction of top-down and bottom-up statistics in the resolution of syntactic category ambiguity. Journal of Memory and Language, 54:363-388. [ .pdf ] Konieczny, L. (2005). The psychological reality of local coherences in sentence processing. In Proceedings of the 27th Annual Conference of the Cognitive Science Society. [ .pdf ]	Klinton
Monday 7 May May 2007	Computational approaches to semantic acquisition (1) Landauer, T. K. and Dumais, S. T. (1997). A solution to plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2):211-240. [ .pdf ]	Dan
Wednesday 9 May 2007	Computational approaches to semantic acquisition (2) Griffiths, T. L., Steyvers, M., and Tenenbaum, J. B. (in press). Topics in semantic representation. Psychological Review. [ .pdf ]	Adam
Monday 14 May 2007	Computational approaches to morphological processing Moscoso del Prado Martín, F., Kostic, A., and Baayen, R. H. (2004). Putting the bits together: an information theoretical perspective on morphological processing. Cognition, 94:1-18. [ http ]
Wednesday 16 May 2007	Computational approaches to lexical access & word reading (1) Kello, C. T. and Plaut, D. C. (2003). Strategic control over rate of processing in word reading: A computational investigation. Journal of Memory and Language, 48:207-232. [ .pdf ]	Danke
Monday 21 May 2007	Computational approaches to lexical access & word reading (2) Norris, D. (2006). The Bayesian reader: Explaining word recognition as an optimal Bayesian decision process. Psychological Review, 113(2):327-357. [ http ]	Albert
Wednesday 23 May 2007	Computational approaches to language learning (1) Swingley, D. (2005). Statistical clustering and the contents of the infant vocabulary. Cognitive Psychology, 50:86-132. [ .pdf ]	Dayne
Monday 28 May 2007	Memorial Day: no class
Wednesday 30 May 2007	Computational approaches to language learning (2) Goldwater, S., Griffiths, T. L., and Johnson, M. (2007). Distributional cues to word segmentation: Context is important. In Proceedings of the 31st Boston University Conference on Language Development. [ .pdf ] Frank, M. C., Goldwater, S., Mansinghka, V., Griffiths, T., and Tenenbaum, J. (2007). Modeling human performance in statistical word segmentation. In Proceedings of CogSci. [ .pdf ]	Roger
Monday 4 June 2007	Computational approaches to syntactic acquisition Klein, D. and Manning, C. D. (2004). orpus-based induction of syntactic structure: Models of dependency and constituency. In Proceedings of ACL. [ .pdf ] Solan, Z., Horn, D., Ruppin, E., and Edelman, S. (2005). Unsupervised learning of natural languages. Proceedings of the National Academy of Sciences, 102(33):11629-11634. [ http ]	Roger
Wednesday 6 June 2007	Uniform Information Density/Constant Entropy/Probabilistic approaches to langauge production Aylett, M. and Turk, A. (2004). The Smooth Signal Redundancy Hypothesis: A functional explanation for relationships between redundancy, prosodic prominence, and duration in spontaneous speech. Language and Speech, 47(1):31-56. [ http ] Keller, F. (2004). The entropy rate principle as a predictor of processing effort: An evaluation against eye-tracking data. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 317-324, Barcelona. [ .pdf ] Levy, R. and Jaeger, T. F. (2006). Speakers optimize information density through syntactic reduction. In Advances in Neural Information Processing Systems. [ .pdf ]	Andy

Readings and other references

The class schedule lists required readings and also related/background readings for each topic. Each lecture will focus on the required readings, and it will be assumed that you have done these readings before class and are ready to discuss them. The related readings are provided in case you are interested in further pursuing one or more of the topics covered in the class.

The readings above, listed in bibliographic order, can be found here.

You can also take a look at the research going on at the Computational Psycholinguistics Lab here at UCSD.

Software

Here is some related software that could be useful for investigating some of the models we'll cover in the class:

A prefix probability parser, related to the section on information-theoretic models. To use this parser you will need to install Java (version 1.4 or later) on your computer.

The topic modeling toolbox that Griffiths, Steyvers, and Tenenbaum (in press) used.