Auditory Stroop Using Spatial Stimuli

Article information

Abstract

INTRODUCTION

In his seminal article, John Ridley Stroop conducted three experiments that were the first to demonstrate stimulus interference [1]. In two classic experiments, Stroop first compared reading a list of words printed in black with reading the same list of words printed in incongruent colors. He found that there was little difference in reading time for the two lists. He then compared the naming of colors for a list of solid color squares with the naming of colors for a list of words printed in incongruent colors. Stroop noted that participants took much longer to complete the color reading in the second task than they had taken to name the colors of the squares in Experiment 2. This delay had not appeared in the first experiment. Such interference was explained by the automation of reading, where the mind automatically determines the semantic meaning of the word (it reads the word “red” and thinks of the color “red”), and then must intentionally check itself and identify the color of the word (the ink is a color other than red), a process that is not automatized [14]. This phenomenon is known as the Stroop effect.

The Stroop effect has been studied extensively in both visual and auditory modalities. More recently, auditory Stroop tasks have been investigated using speakers’ gender [2], pitch [3], loudness, and time [4]. Greg et al. [2] replicated studies of Green and Barber [5, 6] as they investigated the reaction times of participants when asked to make a judgment of the sex of the speaker by pressing a key labeled “Female” or “Male.” It was found that faster reaction times were obtained in the congruent condition. Spape et al. [3] investigated reaction times when asking participants to vocally respond “high” or “low” to sinusoidal tones of 550 or 1,050 Hz. Slower reaction times were observed when the stimuli were incongruent, as in previous research. Morgan et al. [4] found similar results in increased reaction times when responding to pitch, loudness, and duration of a presented word whose meaning is in conflict with the response label (i.e. high/low, loud/soft, and fast/slow). While the Stroop effect has been studied extensively, there is limited research available on incongruent auditory-spatial stimuli, in both the horizontal and vertical plane, and its effect on participants’ accuracy and reaction time when responding to the stimuli.

The impact of auditory cues on reaction time has implications for Augmentative and Alternative Communication (AAC) devices. AAC refers to methods and tools that are used as forms of communication by individuals whose natural speech does not meet their daily communication needs. AAC does not require a technology solution; however, speech-generating devices are often used. Recent studies have suggested that around 0.5% of the population could benefit from AAC [7]. With advances in technology and reduced cost to access of modern technologies, barriers to obtaining AAC devices have been reduced considerably. Moreover, these devices can be customized to meet the specific communications needs of each individual.

Individuals with adequate motor abilities are able to utilize an AAC system with a graphic interface, in which they select their desired speech output by quickly and accurately selecting a series of graphic images. However, many individuals that often utilize AAC also have physical disabilities that require a more involved interface for them to be useful. Those with more severe motor impairments may benefit from a system that includes a form of eye tracking or a selection method where items are presented sequentially until the desired item is highlighted, to which the individual will have one motion (e.g., large button press) to select the item of their choice. There is a sub-population of AAC users with concomitant visual impairments. Since this renders the user unable to view the screen, alternative solutions are required.

As many as 1 in 1,000 children in the United States have low vision or are legally blind. Seventy percent of children with cerebral palsy also have visual impairments [8]. In circumstances where an individual has both motor and visual impairments, a visual graphic interface would not suffice. One solution is to present items sequentially in the auditory domain.

Research to improve access to graphic interfaces on AAC devices has suggested a benefit to users by giving cues as to the relative location of items in an array to aid in navigation [9]. Current auditory scanning techniques rely on labels to represent classes of items for organization. However, there is a lack of information to help organize these items [8]. Two methods for making graphic interfaces more accessible for individuals with motor and visual impairments include the use of auditory icons and spatial auditory cues.

There are implications for how spatial sound cues may or may not facilitate orientation to graphic user interfaces, especially for those with limited visual abilities. The challenge is in providing the needed vocabulary and phrases in an easily re form. Research has proposed multiple ways visual cues and an understanding of visual processing could be of assistance in accessing AAC displays [10,11]. Lacking in this research, however, are the possibilities as to how auditory cues could also play a role.

The term sound localization refers to the ability to detect where a sound originates in space. When a sound source is located at a given angle from the center of the head, the auditory system utilizes slight differences in the timing and intensity of the sound as it reaches each ear (e.g., a sound coming from the left side will reach the left ear slightly sooner and with more energy than it does the right ear). These cues are known as interaural timing differences (ITDs) and interaural level differences (ILDs), respectively, and play important roles in sound localization. Other anatomical factors such as the pinna and the shape of the head also contribute to sound localization, particularly in the vertical plane. As an incoming sound enters the ear canal, portions of the sound are reflected by the folds of the pinna and the head. The auditory system then compares these reflected sound inputs to the direct sound input; some frequencies will be cancelled out while others are amplified. This is known as the head-related transfer function (HRTF). Sound sources originating from different locations in space will have unique HRTFs which the auditory system recognizes and perceives as directionality. The smallest detectable difference in the location of a sound source is known as the minimum audible angle (MAA) and represents the spatial resolution of the auditory system.

There are a number of appealing interference tasks that might address questions posed by this study. A visual-spatial Stroop task performed by Wühr [12] examined how the spatial orientation of rectangles containing congruent or incongruent semantic orientation words affected participants’ ability to process the visual stimuli. Wühr found that the Stroop effect was present in this novel task which extended Stroop interference effects to the combined visual-spatial modality. A study by Van der Burg, Brederoo, Nieuwenstein, Theeuwes, and Olivers [13] observed the effects of audiovisual semantic interference. Participants were presented with letters presented visually and/or auditorily and were instructed to ignore one or the other. Results showed that semantic information from both modalities produced interference when the information from either modality was relevant to the task. Given that a Stroop effect is present is spatial tasks and semantic auditory information interferes with relevant task information, an auditory-spatial Stroop task is a uniquely qualified task to assess the various elements of semantic auditory-spatial interference. The purpose of this study is to evaluate the influence of spatial auditory stimuli when the semantic meaning of the spoken word is incongruent with the location of the sound. To do this, single word utterances were presented in a simulated-spatial fashion in both the vertical plane and the horizontal (or azimuthal) plane as separate conditions. If the word spoken (e.g., “left”) matched the location of the sound source (e.g., from the left), then this presentation was deemed congruent. However, if the word used was different than the sound location, then it was an incongruent presentation. Based on previous auditory Stroop research, we predict that participants will respond to incongruent stimuli, in both planes, with decreased accuracy and prolonged reaction times. Similarly, we predict that since the stimuli in both planes are well beyond the MAA, performance between planes will be similar. By investigating the accuracy and reaction times of congruent and incongruent spatial stimuli, we can better understand an individual’s response to spatial stimuli and its significance in the auditory scanning approach of AAC devices for persons with visual and motor impairments.

METHODS

RESULTS

Each participant’s recording was analyzed for accuracy and reaction time using Adobe Audition. Accuracy was measured as whether the participant’s response correctly matched the location of the presented stimulus. A two-way 2 (plane: vertical, horizontal) ×2 (congruency: congruent, incongruent) within-subjects repeated measures ANOVA was performed to analyze accuracy. Accuracy results (proportion correct) revealed a significant interaction between plane and congruency, F(1, 19) =50.35, p <0.001, n_p² =0.726. In the vertical plane, incongruent accuracy was significantly lower than congruent accuracy, F(1,19)=62.68, p <0.001, n_p² =0.767. In the horizontal plane, incongruent accuracy was not significantly different than congruent accuracy, F(1,19) =3.22, p =0.09, n_p² =0.145. For congruent stimuli, accuracy in the horizontal plane was significantly higher than accuracy in the vertical plane, F(1,19)=28.30, p <0.001, n_p² =0.598. For incongruent stimuli, accuracy in the horizontal plane was also significantly higher than accuracy in the vertical plane, F(1,19)=369.44, p<0.001, n_p² =0.951. Results for accuracy in the two planes are shown in Figure 1.

Figure 1.

Overall accuracy performance is shown for each plane. In the vertical plane, congruent presentations significantly reduced performance. In the horizontal plane, performance was near perfect regardless of congruency.

Reaction times (RTs) were measured as the amount of time (milliseconds) between the end of the stimulus presentation to the beginning of the participant’s response. A two-way 2 (plane: vertical, horizontal)×2 (congruency: congruent, incongruent) within-subjects repeated measures ANOVA was run to analyze RTs. RT analysis revealed a significant interaction between plane and congruency, F(1, 19)=5.60, p<0.001, n_p² =0.228. In the vertical plane, congruent RTs were significantly faster than incongruent RTs, F(1,19)=4.55, p =0.05, n_p² =0.193. In the horizontal plane, congruent RTs were not significantly different than incongruent RTs, F(1,19)=0.00, p=0.95, n_p² =0.00. For congruent stimuli, RTs in the horizontal plane were significantly faster than RTs in the vertical plane, F(1,19)=45.66, p<0.001, n_p² =0.706. For incongruent stimuli, RTs in the horizontal plane were also significantly faster than RTs in the vertical plane, F(1,19)=93.62, p<0.001, n_p² =0.831. RT results in the two planes are shown in Figure 2. Descriptive statistics for accuracy and RT for the horizontal and vertical planes are shown in Table 1.

Figure 2.

Overall RT performance is shown for each plane. In the vertical plane, RTs were significantly longer when presentations were incongruent. RTs were low regardless of congruency.

Table 1.

Descriptive statistics for accuracy and RT

Chi-square goodness of fit tests were performed to analyze the errors made by participants. The first test was to assess the proportions of total errors made by participants per plane (“center” trails excluded). A Chi-square goodness of fit test indicated that there was a significant difference in the proportion of errors committed on stimuli presented in the vertical plane (0.95) compared to the expected equal distribution value of 0.50, X² (1, n=936)=768.27, p <0.001. Of the errors made on trials where the stimulus was presented in the vertical plane, a Chi-square goodness of fit test indicated there was no significant difference in the proportion of errors committed per semantic word presented (up =0.24, down =0.24 right=0.27, left=0.25) and the expected equal distribution value of 0.25, X² (3, n=892)=2.53, p=0.47. An analysis of errors made on “center” trials was also performed. A Chi-square goodness of fit test showed that there was a significant difference in the proportions of incorrect responses to “center” trials (up=0.79, down=0.15, right=0.06, left=0.00) and the expected equal distribution values of 0.25, X² (2, n =176) = 167.47, p<0.001.

DISCUSSION AND CONCLUSION

This study was designed to ascertain the feasibility of providing a two-dimensional auditory grid of cues. The Stroop design was used; listeners were instructed to report where a spoken word was located in space notwithstanding which word was spoken. Naturally, performance was more accurate and faster in congruent presentations when the word spoken (e.g., “right”) was paired with the sound coming from the right. Performance, however, was rather different in the vertical versus horizontal planes.

The data from this study show interference of incongruent performance for both accuracy and RT in the vertical plane. Accuracy was higher and RTs were lower in vertical-congruent presentations. However, participants were completely insulated from incongruency in the horizontal plane. Accuracy performance in the horizontal plane was near perfect regardless of congruency, and RTs were also quite low in this plane for both congruent and incongruent stimuli. We predicted a significant, negative impact of incongruent presentations, which held true for the vertical plane only. Given the difference in performance between the two planes, our second hypothesis proved incorrect.

It is interesting to note that there were a substantially greater number of female (18) than male (2) participants. However, examining differences in performance relative to gender was not an objective of this experiment. MacLeod [15] performed a review of the past 50 years of Stroop research. He noted that as of 1991 there were over a dozen studies that explicitly examined any differences between sexes in Stroop tasks; he concluded that the research shows there is no discernable difference between genders in Stroop tasks. He noted that while females may sometimes respond faster than males (especially in color-naming tasks), it is the result of women having a faster general response speed and was not influenced by any measure of interference, and concluded that there was no difference in Stroop interference between sexes at any age. The visual-spatial Stroop study performed by Wühr [12] used a participant demographic nearly identical to the current study (19 females, 3 males, mean age=21 years). Like our study, performance differences according to gender was not an objective of his study, and he did not report any findings or discussion points related to gender differences.

With regards to auditory spatial listening, research has shown that males have superior auditory spatial acuity. Studies by Lewald and Hausmann [16] and Zündorf, Karnath, and Lewald [17] have demonstrated that males are better at localizing target sounds in a complex multiple-source listening environment. In these experiments, participants were required to point to the spatial location of a target sound in two conditions: one with the target sound presented by itself, and the other with the target sound and distracter sounds presented simultaneously from different locations. The findings from both studies showed that in the complex environment with multiple distracters, males were more accurate at determining the location of the target sound source amidst multiple distracters, and the authors contribute this finding to males having superior sensory and attentional mechanisms when extracting spatial information from a complex scene with multiple sound inputs. While these results are intriguing, there are two key factors to consider with regards to our study. First, the studies by Lewald and Hausmann [16] and Zündorf et al., [17] found that in the condition with the target sound presented by itself, there was no difference in performance between males and females. In our study, we only used one single auditory input per trial; there were never multiple competing sounds for which the participant needed to attend selectively. Based on this paradigm, there should be no differences between males’ and females’ spatial localization performance on the stimuli used in the current study. Further, the experiments mentioned previously only tested spatial stimuli in the horizontal plane, at which performance proved to be near ceiling level in our experiment.

A study by Giguère, Lavallée, Plourde, and Vaillancourt [18] found that males had more accurate sound localization abilities in the vertical plane. The authors contribute this improved performance as a physical phenomenon from females having generally smaller sized pinna compared to males, and therefore females encode vertical HRTF information at higher frequencies than males. In our study, sound stimuli were recorded with KEMAR, which is modeled after the median human size head and torso. Accordingly, all sound stimuli were recorded with the same average HRTF and delivered via insert headphones which bypass the reflections provided by the pinna. Therefore, there should be no differences in the acoustic signal that males or females received in our study. Giguère et al. note that differences in cognitive abilities related to auditory spatial attention could be contributing to these findings, citing the Zündorf et al., [17] article. However, recall that the findings from that study only held in complex listening environments with multiple simultaneous sound sources, which our study did not have. Based on these factors, we do not believe that performance would be altered significantly by having more male participants.

Roeffler and Butler [19] argued that for a participant to locate stimuli in the vertical plane accurately, three criteria must be met: the stimulus must be complex, the stimulus must include frequencies above 7,000 Hz, and the pinna must be present. All three of these suggested requirements were met when recording our stimuli as our stimuli consisted of speech. We also used KEMAR to record our spatial stimuli in order to ensure the natural resonance of the pinna and ear canal was captured in our recordings.

The minimum audible angle (MAA) is the smallest angle between two stimuli that a listener can reliably identify a difference in location. The on-center MAA for the horizontal plane is 2–3° while vertical plane typically falls between 10–15°. This might indicate stronger salience for stimuli presented in the horizontal plane. However, the stimuli developed for this study were well beyond the MAA; we chose 45° for both planes. As such, the salience of the stimuli in each plane were likely maximized. The impact of congruency on accuracy in only the vertical plane suggests that the reduced salience of the localization cue renders it more vulnerable to errors when incongruency is present. The difficulty listeners experienced with vertical stimuli, therefore, is inherent to the human limitations of processing vertical cues.

The degree to which the horizontal performance was so high as well as being unhindered by words spoken warrants discussion. Participants produced near-perfect scores that were unaffected by the content, noting that the word contradicted the location of the sound. In the current study, there were three locations in this plane: left, center, and right. Given the degree of robustness in the data, a much larger number of elements may be easily discernible with little-to-no interference from the word spoken. This suggests that complex streams of auditory elements can be constructed while maintaining a high-level of performance. If this holds, this concept may then be used to facilitate sequential presentations in AAC devices. First, however, the maximum number of elements and angular distances achievable before performance drops or interference increases needs to be understood. Second, pilot implementations need to be tested with the target population who use AAC devices. Nevertheless, the current study shows clearly that the human performance in the vertical plane, while an attractive option as it allows multi-dimensional, spatial representation of sound, is simply not developed enough to be used as a facilitating measure in AAC devices. Rather, the horizontal plane shows much greater promise, and to a much greater degree than anticipated. There is a great deal of interest in reducing the memory demands of AAC user interfaces particularly with regard to scanning. Since scanning requires users to wait for different items to be presented before making their selection, information that assists users in recalling the order of potential messages is critical. For those with a visual impairment, spatial auditory stimuli would help engage the users in ways to track the locations of messages in a location based array rather than solely having to remember the order of messages serially. In a conversational context, separating speakers by their spatial location on the horizontal plane would help portray a natural group setting for the user.

By using the Stroop test to study human auditory performance in the vertical and horizontal domains, there are three conclusions. Since vertical performance, even in congruent presentations, is not impressive as well as vulnerable to the content of the spoken element, we do not recommend pursuing vertical auditory cues as a means to facilitate AAC representations in auditory space. Given the near-perfect performance as well as its resistance to contradictory spoken-word content, the use of horizontal, auditory-spatial representations for implementation in AAC devices warrants further investigation to improve the user experience.

Article information Continued