Real-Time Speech Comprehension

How does the mind make sense of what people say so accurately and fast? 

Watching closed captions—the transcription of the audio portion of a program—on a television in a noisy airport provides for an effective demonstration of the complexity associated with comprehending speech in real time. Speech is a highly complex, fleeting stimulus, with two to three words produced per second. Captioning systems often cannot keep up with this pace, and the visual portion of the speech and its written transcription quickly fall out of sync. Moreover, the transcription of the initial portion of an utterance is often tentative and needs to be modified in the light of what comes later, with time-consuming revisions. This state of affairs contrasts with our experience of listening to the audio part of the same program, where understanding what people mean as they talk seems fluid and effortless.

In order to investigate how the brain uses sensory information in real time to infer what is being said, we have placed people in situations in which what they see is relevant to what they hear. For instance, they follow spoken instructions to click on one pictured object, among others, using the computer mouse while we monitor their eye movements to the objects on the screen. People spontaneously orient their gaze toward the object that they believe is the one they must select. This technique can reveal how quickly people begin to make such ‘guesses’ as well as how much these guesses are influenced by biases or other prior expectations.

Research on this topic from the lab has resulted in many empirical reports as well as reviews for broader audience (see references below).

Selected publications:

Reviews

  • Dahan, D., & Ferreira, F. (2019). Language comprehension: Insights from research on spoken language. In P. Hagoort (Ed.), Human Language: from Genes and Brains to Behavior. MIT Press.
  • Dahan, D. (2013). Auditory word recognition. In H. Pashler (Ed.), The Sage Encyclopedia of the Mind, Volume 2 (pp. 783—784). Thousand Oaks, CA: Sage Publications.
  • Dahan, D. (2010). The time course of interpretation in speech comprehension. Current Directions in Psychological Science, 19, 121-126.
  • Dahan, D. (2009). Word recognition. In B. Goldstein (Ed.), The Sage Encyclopedia of Perception (volume 2, pp-1141-1145).Thousand Oaks, CA: Sage Publications, Inc.
  • Dahan, D., & Magnuson, J. S. (2006). Spoken-word recognition. In M. J. Traxler & M. A. Gernsbacher (Eds.), Handbook of Psycholinguistics (pp. 249-283). Amsterdam, The Netherlands: Elsevier.

Empirical reports