Limerick1 workspace

Technical elements of a cognitive psychology thesis work submitted to the faculty of the Departments of Psychology and Linguistics at Swarthmore College in May 2019 in partial fulfillment for the degree of Bachelor of Arts.

Table of Contents

• Abstract
• In this repository
• Requirements
• Usage
  • Step 1: Gather data
  • Step 2: Generate files necessary for analysis
  • Step 3: Set up parameters for data analysis
  • Step 4: Acquire data analysis measures
  • Step 5: Edit data analysis measures format
  • Step 6: Statistical analysis
• Support and Contributing
  • Troubleshooting FAQ
  • Additional support
  • Future work
• Acknowledgements
• License

Abstract

Occupation of Inner Speech in the Working Memory During Silent Reading:

Effects of Articulatory Suppression on Anticipated Lexical Stress

As we silently read words on a page, we use an “inner speech” that includes rhythm and inflections as if we are reading aloud. We also use an inner speech when performing non-reading tasks such as remembering verbal materials, problem solving, mental calculation, and decision making. Present research has not yet determined whether these two uses of inner speech occupy the same cognitive resources. This paper presents findings from an eye-tracking study designed to explore whether the inner speech of silent reading occupies the same resources in working memory as other mental tasks. In the first part of the experiment, participants read stress-alternating homographs (e.g., PREsent, preSENT) embedded in limericks, which compelled them to initially expect the incorrect prosody of the homograph and thus encounter a reading cost. As they read, participants also performed articulatory suppression by repeating the word this aloud. The goal of this was to use the outer voice to occupy the relevant resources in working memory, thus rendering them unavailable to be used during silent reading—this allows us to see if those resources pertain to the inner voice of silent reading. Participants also completed the 18-item Varieties of Inner Speech Questionnaire (VISQ; McCarthy-Jones & Fernyhough, 2011), a self-assessment survey that assesses one’s relationship with inner speech. The goal of this was to see if participants who reported experiencing higher levels of inner speech in their everyday lives would be more susceptible to the reading costs prompted by articulatory suppression during rhythm-mismatching limericks. If the differences in reading costs between reading rhythm-matching and -mismatching limericks vanish while repeating this, and if participants who experience higher levels of inner speech are more affected by articulatory suppression, then this would show that these two inner voices occupy the same working memory resources. The study found some preliminary evidence that articulatory suppression can diminish the effect of stress clash on silent reading, as well as some preliminary indication that inner speech occupies working memory and that this machinery overlaps with the inner voice of the rehearsal component of the phonological loop.

Key words: inner speech, silent reading, working memory, phonological loop, articulatory suppression, eye movements

Back to top

In this repository

• awk_scripts/ - awk scripts for data wrangling
• data/ - collected data
• src/ - source code for data processing
• test/ - testing stuff; everything in here is an archive of things that can be ignored
• .gitignore - specifiles which files and folders to ignore when pushing code
• InnerSpeechWorkingMemory.pdf - copy of paper
• LICENSE - copyright terms
• README.md - (this page) contains information about the contents and usage of this repository
• a mountain of technical difficulties

See project wiki for more details about contents.

Back to top

Requirements

Compatibility

This package works with Mac and Linux terminals.

It works with Windows, too, but this is difficult for a variety of reasons: Command Prompt uses different commands than the ones described in the instructions below, when you do something wrong whilst running eyedry it crashes without showing you the error message, and retrieving files from Bash is difficult. If necessary, however, a workaround can be arranged by using the VisualStudio Code IDE and GitHub Desktop bash shell.

TL;DR: Best to use macOS or Linux.

Required programs

• python3.4+
• perl
• R
• GCC compiler for C (or substitute with another compiler)

Back to top

Usage

Step 1: Gather data

1. Create the experiment in ExperimentBuilder.
   • Ideally, we would have created the experiment entirely using UMass eyetracking software in order to avoid technical struggles of preparing data for analysis. However, this particular experiment involved a secondary task component that was not yet supported by the UMass setup at the time of deploy.
2. Run participants.
   • Experimenter script, debriefing, participant log, pre-registration, etc. are located in the EyeTrackingLabAdmin/Limerick1/ shared Drive folder.
   • Information about how stuff was run and where relevant files are located on the computers can be found in the Experimenter Script.
3. Convert data files from EDF to ASC
   • Each participant’s data will be located on the Host computer in e.g. Desktop/Exeriments/LimerickProsodyDeployed/LimerickProsodyA/results/subject0/
   • In that folder, there should be an .edf file titled e.g. subject0.edf. Double-click on that .edf file to generate an .asc file.
     • There’s some EDF to ASC converter software installed on the Host computer that allows it to do that. Double-clicking an EDF file on your personal computer won’t do anything if you don’t have that software installed.
4. Gather original ASC files and AOI folders
   • Once the .asc file has been generated on the Host computer, copy it onto a thumbdrive and place it into this package’s data/original_asc/ folder.
   • Also copy each of the eight Areas of Interest (AOI) folders into this package’s data/original_asc/ folder, so that it contains A_aoi/, B_aoi/, C_aoi/, D_aoi/, RevA_aoi/, RevB_aoi/, RevC_aoi/, RevD_aoi/.
5. Match participants with their list
   • This experiment used a Latin Square design. Participants saw one of eight possible lists. Analysis will require knowing which list each participant saw.
   • In src/reformat_asc/lists.py is a dictionary mapping lists to participants who saw that list. The eight lists are A, B, C, D, RevA, RevB, RevC, RevD.

Back to top

Step 2: Generate files necessary for analysis

From within the /src directory, run:

python3 runall.py

All input variables are already written into the respective scripts (but see the wiki for where to modify things during future adaptations). This single script executes the following consecutively:

• Deletes all data/ subfolders and their contents, except for original_asc/ and eyedry/, and then recreates them as empty folders for a fresh start
• reformat_asc.py
• copy_question_acc.py
• remove_participants_question_acc.py
• copy_Scripter2.pl
• copy_make_cnt.py
• copy_fix_align_v0p92.R
• copy_Robodoc.py

NOTE: It is crucial to only run this once, unless you go back and gather the original_asc files again. This is because remove_participants_question_acc.py excludes participants based on their relative performance on the ‘is this limerick dirty’ questions.

Back to top

Step 3: Set up parameters for data analysis

1. For each attempt at analysis, create a subfolder within data/eyedry_output/. In the end, we created five folders:

• data/eyedry_output/graphs1/
• data/eyedry_output/graphs2/
• data/eyedry_output/graphs3/
• data/eyedry_output/graphs4/
• data/eyedry_output/graphs5/

1. From within src/eyedry/, compile eyedry: (ignore any warnings)

   gcc eyedry.c -o eyedry

   NOTE: compiling only needs to be done once, ever, unless for some reason you make changes to eyedry.c (which you probably shouldn’t do)

2. From within src/, run the executable that we just created: (ignore any warnings)

   ./eyedry/eyedry

3. Eyedry will ask a series of questions. For details of what each question is asking, see the eyelab manual—a copy of this manual is located in EyeTrackingLabAdmin/UMassEyetrack/eyelabmanual.pdf. We answered as follows:

After answering these questions, the next that should appear is a “Which analysis?” list of possible analyses that can be performed. Continue to the next step.

Back to top

Step 4: Acquire data analysis measures

4.1. Overview of analysis (just read, nothing to do here)

• Prior to analysis: (following Breen and Clifton, 2011)

  • Eliminated participants who scored below 2.5 standard deviations below the median on the “is this limerick dirty” comprehension questions (standard practice).
    • This was done when we ran remove_participants_question_acc.py as part of runall.py
  • Eye movement data was cleaned of track losses, blinks, and long fixations over 800ms using EyeDoctor.
    • We cleaned track losses and blinks when we ran Robodoc.
    • We cleaned long fixations when we passed in parameters to EyeDoctor in the previous step.
  • Short fixations (<80ms) were incorporated into the nearest neighboring fixation within three characters, otherwise deleted.
    • We cleaned short fixations when we passed in parameters to EyeDoctor in the previous step.
  • Long fixations (>800ms) were deleted
    • We cleaned long fixations when we passed in parameters to EyeDoctor in the previous step.
  • Trials with blinks or track losses on the critical word were eliminated
    • We eliminated trials with blinks or track losses when we ran Robodoc.

• Analysis of standard eyetracking measures: (following Breen and Clifton, 2011; see also Rayner, 1998; Rayner et al.,1989)

  • First-pass time - sum of all fixations made from first entering to first leaving a region, eliminating trials on which no such fixations occurred.
  • Go-past time - sum of all fixation durations made from first entering a region to first leaving it to the right.
  • Probability of fixating in a region - no description provided.
  • Probability of regressing out of a region given that it was fixated during the first pass - no description provided.
  • First fixation duration for critical Region 3 - no description provided
    • Due to non-significance, Breen and Clifton (2011) did not report these data.

Back to top

4.2. Generate data analysis files (instructions here)

Each of the following measures will be obtained within the terminal starting from the “Which analysis?” prompt. Completion of each measure will return you to this prompt. Answer the questions as follows.

WARNING: Sometimes the program segfaults and crashes for various unknown reasons.

Shortcuts:

• 4.2.A. First-pass time
• 4.2.B. Go-past time
• 4.2.C. Probability of fixating a region
• 4.2.D. Probability of regressing out of a region
• 4.2.E. First fixation duration
• 4.2.F. Fin.

Back to top

4.2.A. First-pass time

Back to top

4.2.B. Go-past time

Back to top

4.2.C. Probability of fixating a region

Back to top

4.2.D. Probability of regressing out of a region

Back to top

4.2.E. First fixation duration

Back to top

4.2.F. Fin.

Press 0 to Quit.

Notice that all of these generated files got saved into src/. Create a new folder located at ../data/eyedry_output/graphs_1/original_eyedry_output_1 and move them all into there.

NOTE: the trace file (e.g. ../data/eyedry_output/graphs1/trace_graphs11.txt) contains a record of everything that was performed with EyeDry just now.

Back to top

4.3. Check that the analyses measures are actully adequate

Sample a few trials from the analyses files to compare “by hand” with the video viewer in ExperimentBuilder.

• Play the participants’ eye movements for those trials at 25% of the speed
• See if they seem to correspond with the purported firstPass duration, goPast duration, probabilityFixation, probabilityRegression, etc.

To match the participants’ measures in the analyses files back together with their eye-movement videos in ExperimenterBuilder, consider the following:

• The subject numbers (second column) in each of the IXS measures files (e.g. firstPassIXS) respectively correspond to participants in the data/robodoc_output/files_processed.lst file.

• The sequence number (first column) in each of the IXS measures files (e.g. firstPassIXS) correspond with the DataViewer TrialID according to this relationship:

    dataviewer_trialid = (sequence_number + 1) / 2 + 1

  • I had to do math to discover this relationship.
  • NOTE: The TRIALID from the original .asc file and the TRIALID from the DataViewer for the same limerick trial appear to be off-by-one. The .asc file starts counting at 0, while Dataviewer starts counting at 1.

• Double-check: The item number (third column) in each of the IXS measures files (e.g. firstPassIXS should correspond exactly with the TRIALID number in the original .asc file. Check the limerick from the .asc for that trial against the one for that sequence in src/scripter/input_to_scripter; they should be the same limerick.

• Triple-check: The condition number should match against the original .asc.
• Can also take a look at RPlots.pdf

Back to top

Step 5: Edit data analysis measures format

From within /src, run:

python3 edit_cond.py [INPUT_FOLDER] [OUTPUT_FOLDER]

The INPUT_FOLDER is the data/eyedry_output/graphs1/original_eyedry_output_1 folder, and the OUTPUT_FOLDER is the data/eyedry_output/graphs1/edited_eyedry_output_1 folder.

This script replaces missing values with 0 and replaces condition numbers with their word descriptions (e.g. 1 => match_tap) for each of five IXS measures files.

It also repairs the unexplainable error where some random lines in the measures files contained a zero condition value but a non-zero sequence value; for these, it replaces the sequence value wtih 0.

The edited files become automagically stored into data/eyedry_output/graphs1/edited_eyedry_output_1/ per the script (or whichever graphs folder was designated).

Step 6: Statistical Analysis

This section provides a brief overview of the analysis that we performed. For more information regarding the various attempts at analysis that we made, please reference the manuscript or the aptly-named data/eyedry_output/grafts.txt

6.1 Crunch numbers using Stat

6.1.a. NOT EXCLUDING ZERO FIXATIONS: (Graphs 1 & 5)

From within data/eyedry_output/graphs1/edited_eyedry_output_1/, run:

FOR REGION 2

awk -F"," '$4~/_/{print $2,$4,$6}' firstPassIXS_edited | tr "_" " "| ../../anova subj clashtype secondtask time

FOR REGION 3

awk -F"," '$4~/_/{print $2,$4,$7}' firstPassIXS_edited | tr "_" " "| ../../anova subj clashtype secondtask time

for each of the measures:

• firstPassIXS_edited
• firstFixIXS_edited
• goPastIXS_edited
• probRegressionIXS_edited
• probFixationIXS_edited

6.1.b. EXCLUDING ZERO FIXATIONS: (Graphs 2, 3, & 4)

FOR REGION 2

awk -F"," '$4~/_/ && $7!=0{print $2,$4,$6}' firstPassIXS_edited | tr "_" " "| ../../anova subj clashtype secondtask time

FOR REGION 3

awk -F"," '$4~/_/ && $7!=0{print $2,$4,$7}' firstPassIXS_edited | tr "_" " "| ../../anova subj clashtype secondtask time

6.1.c. UNBALANCED GRID DUE TO EXCLUDING ZERO FIXATIONS: (Graph 3)

For the firstPast and goPast measures, we removed the participant(s) who was making the R3 grid unbalanced.

6.2 Graph

• Copy the measures outputted from the Terminal into Excel (perhaps with the aid of a text-to-TSV converter such as https://www.browserling.com/tools/text-to-tsv)

• Make graphs

Support and Contributing

Troubleshooting FAQ

• Help I lost a data file.
  • Copies of every .edf should be stored on the Source computer.
• Help my participant is looking at the blue dot but it’s not triggering.
  • Recalibration can be performed between blocks at each of the two break point, but unfortunately we are not able to perform mid-block recalibration :( In the meantime just guide the participant’s gaze to find the trigger point using your finger, and remind them again to refrain from moving. Some damage repair can hopefully be absolved when we run fixAlign.R on the ascii files.
• Why are the subject numerbers assigned the way that they are?
  • I recognize my terrible and inconsistent naming convention.

Additional support

• If anything is buggy or unclear, please open an issue or pull request, or contact itang1@swarthmore.edu itang1@alum.swarthmore.edu.
• This repo is no longer actively being maintained as of May 2019.

Back to top

Future work

• As written in the Discussion section of the paper, Limerick2 will explore amendments to the eyetracker setup.

Acknowledgements

Thanks to Professor Dan Grodner for enormous guidance and so much invested time over the years; to Professors Dan Grodner and Nathan Sanders for giving birth to the experiment; to Drs. Mara Breen and Charles Clifton for providing the stimuli and research on which this study was based; to Drs. Charles Clifton, Adrian Staub, Andrew Cohen, Jesse Harris, and colleagues for creating and maintaining the eyetracking analysis software that we relied on; to the alumni and current students practicing research in the Swarthmore Psycholinguistics Lab; to the Swarthmore Department of Psychology for sponsoring the experiment; and to the undergraduate students at Swarthmore College who participated in the study.

Back to top

License

MIT License

(c) Copyright 2019 Swarthmore Psycholinguistics Lab etc.

This project was created for non-monetary educational purposes only (please don’t sue me)

Back to top