Ph.D - Pupillometry

Video describing Pupillometry-Roberto Gutierrez

Summary:

Many people are using eyetrackers to monitor language processing in real-time, by observing where the participant is looking on a screen, what I did was slightly different. I monitored the diameter of the pupil while people were listening to sentences. Put simply, pupil diameter changes real-time with changes in processing demands, the more processing, the larger the pupil diameter. I was able to gain insights into complex cognitive processes by measuring and time locking pupil diameter with sentences. The beauty of this, it that anyone, at any age can participate by looking att eh screen and listening to sentences.

Descrpition:

We conduct pupillometry studies, which means we monitor pupil diameter during the presentation of stimuli. Pupil diameter is known to be associated with mental effort; as cognitive load increases, pupil diameter increases. Psychophysiological research has shown pupillary size increases in response to pictures with emotional content and increases in working memory load, and even sentences with greater structural complexity were shown to elicit larger pupil diameters. Most of the studies that use pupillometry as an index of processing loads do so by presenting stimuli and monitoring pupil diameters for the 2-5 seconds following the stimuli. This gives a gross measure of mental effort. These studies provide very interesting data, but they do not provide any time course information. In our lab, we have designed multiple studies that use the continuous pupil diameter measures from the Tobii T-60. By time locking the stimulus presentation to pupil diameter we are able to gather a real-time measure of sentence processing loads. The idea here is that by measuring processing load across the full range of a sentence, we can tease apart accounts of sentence processing by examining how different types of information are processed in real time.

We record sentences using Adobe Audition 3.0. Then we use Windows Movie Maker to synchronize the sound file with a picture of a fixation cross. Tobii Studio does not allow us to present a picture and a sound file simultaneously, so we are forced to use other software programs to create movies that accomplish this feat. We present the videos in a Tobii Studio Project and then use the export function in Tobii Studio (Replay). This exports the raw data and stimulus presentation information from Tobii Studio. The following is an example of pupil diameter measurements from one sentence.

Each video begins by presenting the fixation cross followed by 750ms of silence to let the pupils accommodate to the current condition. To reduce noise in the data set, we average across conditions, but each sentence is a different length so we focus on one region of interest. In this case, it is the 500ms prior to the tense maker (i.e. /-ed/ in the word needed). To do this, we transform the pupil dilations into ‘percent change from baseline’ measures and the average across sentence types. We average the 500ms prior to the tense marker to create a local pupillary baseline, then subtract the baseline from the diameter and divide by the baseline.

Normalized data =

(Pupil Diameter – Baseline)/Baseline

We then average across subjects within each condition and thus, are able to look at the pupillary change from baseline in different sentence processing contexts.

For example, we can compare the effect a simple, but obvious, ungrammaticality has on sentence processing with a real-time continuous measure of mental effort by examining the ‘waveforms’ that result from processing these sentences.

1a. Tom told jenny that the sneaky man needed a tool from his messy tool box in the garage.

1b. *Tom told jenny that the sneaky man need a tool from his messy tool box in the garage.

Sentence 1a is grammatical and sentence 1b is ungrammatical (a violation of tense at the verb “need”).

I have only given one example above, but this graph represents the average of 20 different sentence pairs similar to the one above, from 13 different participants.

After visual inspection, we compare regions that appear to be different by running a simple T-test.

T-test for ranges

p-value

-500 t0 zero = 1.000

zero to 483 = 0.216

483 to 716 = 0.424

716 to1000 = *0.006

This indicates that pupil dilations become significantly larger while listening to sentences with a violation to subject-verb agreement approximately 716ms after the ungrammaticality. This type of information should prove very important in the field of psycholinguistics. Here is a copy of a poster I presented at a recent conference.

Here are the slide from the video Pupillometry-Roberto Gutierrez