Comparison of Structured and Unstructured Discourse Tasks in Persons with Aphasia

Article information

Clin Arch Commun Disord. 2017;2(1):23-29
Publication date (electronic) : 2017 April 30
doi : https://doi.org/10.21849/cacd.2016.00038
1Department of Communication Sciences and Disorders, Appalachian State University, University of Central Arkansas, USA
2Department of Communication Sciences and Disorders, University of South Alabama, USA
3University of South Alabama, Speech Pathology and Audiology, USA
Correspondence: Kim C. McCullough, Appalachian State University, ASU Box, 32165, Boone, USA, Tel: +8282626063, Fax: +8282623153, E-mail: mcculloughkc@appstate.edu
Received 2016 September 30; Revised 2016 December 17; Accepted 2016 December 19.

Abstract

The informativeness of the discourse of persons with aphasia (fluent vs. non-fluent) under structured and conversational tasks was investigated. Ten individuals, five with fluent aphasia and five with non-fluent aphasia, who had suffered a single left hemisphere stroke, participated. Structured and conversational discourse samples were analyzed using the Correct Information Units (CIUs) analysis system. Significant differences between structured and conversational tasks were observed when comparing total number of words and total number of CIUs. Participants with fluent aphasia performed better than those with non-fluent aphasia during the conversational task. No differences were found between the groups for total number of words, total number of CIUs, and percentage of CIUs during structured tasks.

INTRODUCTION

Individuals with aphasia are at a particularly high risk of limited participation in everyday life activities involving language, such as conversation. The World Health Organization model defines participation as “involvement in a life situation” [1]. Participation can be characterized as the result of a complex interaction between an individual’s health condition and his/her own personal factors, as well as the environmental circumstances in which her/she lives [2]. Participatwion in conversational interactions is important not only for exchanging information but also for maintaining social relationships, which are critical for wellness.

Traditionally, standardized aphasia assessments and conversational analysis systems have focused on language rules, deficits, and patterns rather than how well language communicates information to listeners. Language impairments that negatively impact communication can restrict a person’s ability to fully participate. Therefore, improving their ability to participate in conversation should be a primary objective of therapy [3]. To examine the impact language impairments can have on participation, many have examined the informativeness and efficiency of the discourse production of persons with aphasia [49]. The Correct Information Units analysis system (CIUs) was developed to evaluate the informativeness of discourse production in response to different stimulus materials, ranging from conversational tasks to single and sequenced picture description tasks [5]. This rule-based system was created for distinguishing words that would be informative to a listener from those that would not. A CIU is a word that is intelligible in context, accurate in relation to the stimulus, and relevant to and informative about the stimulus. Words do not have to be grammatically correct to be included in the CIU count. Brookshire and Nicholas [5] suggest that the following two measures could be used to examine the performance of tasks: percentage of words that are CIUs and number of CIUs per minute.

Discourse Production and Use of CIUs

Several studies have employed CIU analysis as a tool to determine the informativeness of the discourse of participants with aphasia [69]. In 1993, Brookshire and Nicholas [6] attempted to determine the reliability, stability and sensitivity of CIU analysis. Participants included 20 non-impaired participants and 20 participants with aphasia. Six exhibited non-fluent aphasia and 14 exhibited fluent aphasia. Discourse production was elicited by 10 stimuli: four single pictures, two picture sequences, two requests for personal information, and two requests for procedural information. For scoring purposes, number of words and number of CIUs were identified, then each sample was timed. Time, words, and CIUs were used to calculate three measures: words per minute, CIUs per minute, and percentage of CIUs.

Results indicated good session-to-session stability for all measures. Changes in scores between sessions were smaller for the three measures (words per minute, CIUs per minute, and percentage of CIUs) than for the two counts (number of words and number of CIUs). There was a significant difference between the participants without aphasia and participants with aphasia on each of the five measures. Although the three calculated measures discriminated the participants without aphasia from the participants with aphasia better than the two count measures, the CIUs per minute was the most dependable measure for distinguishing between the groups. This measure, however, was not as informative about performance as calculating words per minute and percentage of CIUs. These two measures evaluated the contribution of rate and informativeness to performance. Overall, these results suggest the informativeness of discourse of persons with aphasia can be reliably scored using the CIU procedure. Boyle [14] also found the measures of number of correct CIUs and words per minute were stable enough to use for both group research studies and clinically.

CIUs also have been employed to measure the effects of speech sample size on test-retest stability [7]. The authors did not report which conversational task elicited more informative speech samples. However, they did indicate that the test-retest stability of words per minute and the percentage of CIUs obtained from speech samples are affected by the size of the speech sample. There are no specified requirements regarding the duration of a speech sample necessary to obtain adequate test-retest stability, but as the size of the sample increased, the standard error of measurement (SEM) decreased. Although the authors did not address the issue of how well a single speech sample might represent a participant’s communicative competence, they indicated that discourse elicited with several types of stimuli is likely to be most representative of the participant’s speech competence.

Finally, the use of CIU analysis to score performance deviation in the discourse production of persons with aphasia also has been investigated [8]. Performance deviations are considered to be productions that do not qualify as words or as CIUs. Results from a study by Brookshire and Nicholas [8] indicated speech samples from participants with aphasia (fluent and non-fluent) contained greater percentages of inaccurate words, false starts, and part-words than non-impaired participants. Comparing the participants with fluent aphasia and non-fluent aphasia, the group with fluent aphasia produced greater percentages of unnecessary exact repetition, whereas the group with non-fluent aphasia produced greater percentages of the word ‘and,’ as well as non-word fillers. These results suggested that the type of aphasia does influence communicative performance.

Discourse Production: Type of Task Matters

It has been reported [9,10] that CIU analysis can vary across discourse type. Roberts and Wertz [10] compared structured and unstructured language tasks in participants with aphasia. Twenty individuals with aphasia participated. Spontaneously produced language samples were videotaped. Two language samples, a conversation, and the PICA subtest I (a verbal test where the subject describes what an object is used for), were analyzed at 1, 3, 6, 9, and 12 months post-onset for each participant.

Results indicated differences do exist between conversational and elicited language samples. Utterance length was longer in conversation, but syntactic accuracy was better in the elicited tasks. The authors emphasized that standardized testing, as well as analyzing conversational skills, is important for evaluating, planning treatment, and measuring progress.

Doyle, Goda, and Spencer [9] also investigated the communicative informativeness of discourse by persons with aphasia during a structured task and a seven-minute segment of a conversational task [9]. The CIU analysis procedure was used to obtain the following for each sample: a) total words, b) number of CIUs, c) percentage of CIUs, and d) percentage of accurate main concepts.

Results revealed no significant differences across the two tasks when the number of words and the number of CIUs were compared. However, the participants did produce a greater percentage of CIUs and informative units in conversational discourse than in the structured discourse task. Despite the lack of significant differences, these results support that although the number of informative words produced across the two tasks did not differ, the efficiency with which information was communicated was greater during conversational discourse.

Shadden et al. [11] reported that an understanding of the ways in which task demands influence language use is needed to determine the most effective activities for aphasia management. Doyle et al. [9] indicated that measuring informativeness under conversational discourse conditions is the most valid method for determining the communicative ability of participants with aphasia, whereas, procedures that require the participants to describe pictured events or information are more effective means of measuring specific language abilities. Structured procedures allow for better control, but they differ from conversational samples with respect to their functional purpose, contextual elements, and cognitive demands.

There is limited research concerning the association between type of aphasia and performance of discourse production under structured and conversational tasks. Roberts and Wertz [10] found differences between structured and unstructured tasks, whereas Doyle et al. [9] found no significant differences between discourse tasks. The purpose of this study is to examine the informativeness of the discourse production of persons with aphasia under structured and conversational tasks. An attempt was made to answer the following questions: 1) Do discourse tasks (structured vs. conversational) effect the informativeness of discourse production?; and 2) Does type of aphasia (fluent vs. non-fluent) influence the informativeness of discourse production?

METHODS

Ten individuals with aphasia (i.e., five with fluent aphasia and five with non-fluent aphasia) participated. Type of aphasia was classified according to the classification categories provided by the Western Aphasia Battery (WAB) [12]. Only participants with an aphasia quotient (AQ) of 55 or above were included in the sample as the focus of this investigation was primarily on verbal communication, and the conversation required auditory comprehension abilities. Institutional Review Board approval was obtained and informed consent was given by all participants. All participants met the following inclusion criteria: 1) English speaker; 2) at least two months post-onset to insure medical stability and to allow for adjustment to the aphasia; 3) no evidence of additional neurologic disorders (e.g., dementia, Parkinson’s disease); 4) no evidence of psychiatric disorder (as reported by caregiver); and 5) completion of an audiometric screening. Descriptive information for each participant is provided in Table 1. The mean age of participants was 63.7 years with a range of 38–83 years of age. All participants presented with at least a high school education. The mean time post onset was 61 months with a range of 2.5–202 months. WAB aphasia quotients ranged from 45–78 with a mean of 65.

Descriptive Information of Participants

Video recordings of each participant were collected during an experimental session lasting approximately 60 to 90 minutes. Each session consisted of a spontaneous communication sample and a formal assessment utilizing the WAB.

Conversational discourse samples were obtained from each of the 10 participants during a spontaneous communication task. Samples were segmented into speaker turns consisting of seven minutes of conversation as other researchers have found this to be an adequate range of time [9]. The investigator, a certified speech-language pathologist, and the participant engaged in a conversation with target topics including work, history, and illness. Alternate topics included activities, interests, life changes, education, and family. An attempt was made to keep the topics and the form of the conversations similar for all participants; however, the exact content of the conversation depended largely on the participant.

Structured discourse samples were collected using the picture description task of the “picnic scene” from the WAB. Samples were analyzed using the entire discourse sample from the WAB picture card. The picture was printed on a 5X7-inch card and placed in front of each participant. Participants were instructed to describe what they saw happening in the picture, using complete sentences.

The structured and conversational discourse samples for each participant were orthographically transcribed. The transcripts were analyzed according to the procedures and rules for Correct Information Units. For each sample, the following measures were obtained: total number of words, total number of CIUs, and percentage of CIUs. To be included in the word count, words had to be intelligible in context, but not necessarily accurate, relevant, or informative about the stimulus. In order for the words to be included in the CIU count, they had to be accurate, relevant, and informative to the stimulus. Each CIU consisted of a single word, each of which was included in the word count.

All analyses were conducted by individuals who were trained to apply the CIU analysis in the same way. Intra- and inter-judge agreement were assessed for both types of discourse samples. To accomplish transcript reliability, three of the 10 transcripts were randomly chosen for review by the co-investigator, a certified SLP, and the investigator. The co-investigator was provided with three video samples and the transcripts generated by the investigator. She was instructed to watch the videos and indicate any discrepancies in the original transcripts. Inter-judge percentage agreement was 96%. Intra-judge percentage agreement was 97%.

Intra- and inter-judge reliability were also assessed for the measures of total number of words and number of CIUs. Intra-judge reliability was determined by having the investigator re-assess all measures for three of the 10 subjects selected randomly. The investigator completed a second count of total words and number of CIUs analysis without access to the original ratings. Intra-judge percent agreement was 96% for word count and 93% for CIU count.

Inter-judge reliability measurements were completed for all measures. The co-investigator trained to criterion for all measures, and then completed the analysis procedure for three randomly chosen participants without access to the original analysis. Comparisons were then made between these ratings and the original ratings made by the investigator. Inter-judge percentage agreement was 98% for word count and 88% for CIU count. Results are presented in Table 2.

Percent Agreement of Intra-judge and Inter-judge Reliability for CIU Analysis Procedures

RESULTS

Table 3 presents each individual participant’s structured task performance for total number of words, total number of CIUs, and percentage of CIUs. Table 4 presents each individual participant’s conversational task performance for total number of words, total number of CIUs, and percentage of CIUs.

Individual Participant Performance on Structured Discourse Task

Individual Participant Performance on Conversational Discourse Task

To answer the primary research question, an independent-samples t-test was conducted to compare informativeness during structured and conversational tasks using total number of words, total number of CIUs, and percentage of CIUs. Results are presented in Table 5. All tests indicated a significant difference (alpha level .05) between total number of words and total number of CIUs for both tasks. However, no significant differences were found for percentage of CIUs for either task.

Comparison of total words, # of CIUs and % of CIUs between discourse tasks

To determine whether type of aphasia (fluent vs. non-fluent) influences informativeness of discourse during structured and conversational tasks, the Mann-Whitney, a non-parametric version of the t-test, was computed for total number of words, total number of CIUs, and percentage of CIUs for both tasks. Results are presented in Table 6.

Results of Nonparametric Statistics for Structured and Conversational Tasks

Significant differences were found between the fluency groups for conversational tasks when measuring for total number of words (.028) and total number of CIUs (.028). Results indicate that participants with fluent aphasia performed better than participants with non-fluent aphasia during the conversational discourse task. However, no significant differences were found when comparing fluency groups for total number of words (.465), total number of CIUs (.753), and percentage of CIUs (.834) during structured tasks.

DISCUSSION AND CONCLUSIONS

Difference between structured and conversational tasks when comparing number of total words and total number of CIUs was observed (<.001). However, no differences were found for percentage of CIUs for either task. Several explanations may account for these findings. Just as Doyle, Goda, and Spencer [9] discussed, the functional purpose, contextual elements, and cognitive demands of the structured and conversational tasks in this experiment were substantively different.

First, during the structured task, participants were asked to describe a picture (WAB picnic scene) in full detail using complete sentences, which had no functional purpose to the participant. During the conversational task, participants were able to discuss topics pertaining to their lives, which allowed the conversational partner to learn more about the participant.

Additionally, the cognitive demands for each task varied in the type of information required to complete the task and the natural conversation between the two partners. The structured task required specific answers with no cues to help complete the task. In contrast, the conversational task was open-ended and assistance was given by the conversational partner. Therefore, each of these factors may have affected participant performance.

Previously, studies have reported that the discourse of persons with aphasia varies both on measures of control of verbal output [11], and completeness and accuracy of main concepts [5]. Brookshire and Nicholas [7] suggested that discourse elicited with several types of stimuli is likely to represent individual speech competency. As a case in point, the results of the present study revealed that the participants with fluent aphasia were more informative in their discourse with the conversational task than the participants with non-fluent aphasia. Linguistic abilities (i.e., sentence formulation, topic initiation, and topic maintenance) that were not addressed in the structured task were more completely revealed during the conversational task. For example, Participants #5 and #8 presented with similar age, fluency type, and aphasia quotients (78.7 vs. 72.9). However, Participant #5 performed significantly better on production of discourse during structured and conversational tasks. Participant #5’s words were 67% accurate and informative during the picture description activity, whereas Participant #8’s words were 46% accurate and informative. Furthermore, Participant #5 uttered 1043 words and Participant #8 uttered 623 words during conversation with 78% and 60% of them respectively being relative to the topic of discussion. Several potential factors could account for Participant #5’s superior communication ability. First, each subject discussed topics of interest and was eager to talk. Whereas Participant #5 had longer utterances that were informative about the topic, Participant #8 presented with perseveration and increased word-finding difficulties. Finally, Participant #5 seemed to have better control of verbal output, which allowed him to convey information to the listener with less difficulty.

In agreement with the results obtained in this study, Roberts and Wertz [10] indicated that differences do exist between structured and conversational tasks. Although Roberts and Wertz [10] did not utilize Brookshire and Nicholas’ CIU analysis, t-units were used to determine overall utterance, clausal lengths, and semantic accuracy. Their results indicated that participants performed better in conversational tasks than structural tasks.

Influence of Aphasia

Results indicated that participants with fluent aphasia performed better than participants with non-fluent aphasia during the conversational discourse task. However, no significant differences were found when comparing fluency groups for total number of words, total number of CIUs, and percentage of CIUs during structured tasks. Brookshire and Nicholas [8] reported that differences do exist between participants with fluent and non-fluent aphasia when comparing discourse production. Their results indicate that participants with fluent aphasia present with a higher percentage of unnecessary repetitions, whereas the participants with non-fluent aphasia exhibit more fillers during conversation.

Although the results of this investigation indicated no significant differences between participants with fluent and non-fluent aphasia for structured tasks, differences were identified between the two groups regarding their performance on the conversational task. Specifically, the participants with fluent aphasia performed better than the participants with non-fluent aphasia during the conversational task. For example, Participants #3 and #4 both had similar WAB aphasia quotients (68.5 & 67.7, respectively), age, and education. Nonetheless, the participant with fluent aphasia (#3), expressed 529 words with 72% being accurate and relevant to topic of discussion. The participant with non-fluent aphasia (#4), only produced 212 words with 57% being informative about the topic.

The majority of past research has focused on comparisons between non-impaired participants and participants with aphasia. More information is needed to provide a better understanding of how type of aphasia influences the informativeness of discourse production. Additionally, continued investigation is needed to determine the effectiveness of the application of the CIU analysis procedure. Research should include a comparison of the reliability and stability of the discourse performance of participants with fluent and non-fluent aphasia. Furthermore, it is also important to extend the present study to a larger group of participants with aphasia. In conclusion, more clinical application of the use of CIU analysis in assessment and treatment of participants with aphasia is needed to support the findings in past and present investigations.

References

1. World Health Organization. How to use the ICF: A practical manual for using the International Classification of Functioning, Disability and Health (ICF). Exposure draft for comment Geneva: WHO; 2013. p. 5.
2. Eadie TL. The ICIDH-2: Theoretical and clinical implications for speech-language pathology. Journal of Speech-Language Pathology and Audiology 2001;25(4):181–200.
3. Wilkinson R. Interaction focused intervention: a conversation analytic approach to aphasia therapy. Journal of Interactional Research in Communication Disorders 2010;1:45–68.
4. Boles L. Conversational discourse analysis as a method for evaluating progress in aphasia: A case report. Journal of Communication Disorders 1998;31:261–274.
5. Brookshire RH, Nicholas LE. A system for scoring main concepts in the discourse of non-brain-damaged and aphasic speakers. In : Lemme ML, ed. Clinical Aphasiology 21Austin, TX: Pro-Ed; 1992. p. 87–99.
6. Brookshire RH, Nicholas LE. A system for quantifying the informativeness and efficiency of the connected speech of adults with aphasia. Journal of Speech and Hearing Research 1993;36:338–350.
7. Brookshire RH, Nicholas LE. Speech sample size and test-retest stability of connected speech measures for adults with aphasia. Journal of Speech and Hearing Research 1994;37:399–407.
8. Brookshire RH, Nicholas LE. Performance deviations in the connected speech of adults with no brain damage and adults with aphasia. American Journal of Speech-Language Pathology 1995;4:118–123.
9. Doyle PJ, Goda AJ, Spencer KA. The communicative informativeness and efficiency of connected discourse by adults with aphasia under structured and conversational sampling conditions. American Journal of Speech-Language Pathology 1995;4:130–134.
10. Roberts JA, Wertz RT. Comparison of spontaneous and elicited oral-expressive language in aphasia. In : Lemme ML, ed. Clinical Aphasiology 18Austin, TX: Pro-Ed; 1989. p. 479–488.
11. Shadden BB, Burnette RH, Eikenberry BR, DiBrezzo R. All discourse tasks are not created equal. In : Prescott TE, ed. Clinical Aphasiology 20Austin, TX: Pro-Ed; 1991. p. 327–342.
12. Kertesz A. The Western Aphasia Battery New York: Grune & Stratton; 1982.
13. Oelschlaeger ML. Participation of a conversational partner in the word searches of a person with aphasia. American Journal of Speech-Language Pathology 1999;8:62–71.
14. Boyle M. Test–Retest Stability of Word Retrieval in Aphasic Discourse. Journal Speech, Language, and Hearing Research 2014;57:966–997.

Article information Continued

Table 1

Descriptive Information of Participants

Participant Gender Age Education MPO AQ WAB type Fluency
1 F 65 16 202.0 73.4 Broca NF
2 F 78 12 2.5 64.6 TS F
3 M 46 12 59.0 68.5 Anomia F
4 F 38 12 120.0 67.7 Broca NF
5 M 76 12 6.0 78.7 Anomia F
6 F 70 15 91.0 57.6 Broca NF
7 F 83 18 24.0 58.9 Wernicke F
8 M 65 12 23.0 72.9 Anomia F
9 F 56 16 26.0 61.4 Broca NF
10 M 48 12 4.0 69.0 Broca NF

Age and Education are reported in years.

MPO = months post onset; AQ = Western Aphasia Battery Aphasia Quotient; NF = Non-fluent; F = Fluent; TS = Transcortical Sensory.

Table 2

Percent Agreement of Intra-judge and Inter-judge Reliability for CIU Analysis Procedures

Word count CIU count
Intra-judge reliability
 Structured 99% 87%
 Conversational 93% 98%
 Total sample 96% 93%

Inter-judge reliability
 Structured 99% 82%
 Conversational 97% 93%
 Total sample 98% 88%

Table 3

Individual Participant Performance on Structured Discourse Task

Participant Total words # of CIUs % of CIUs
1 85 46 54.00
2 58 14 24.10
3 92 43 46.70
4 20 15 75.00
5 95 64 67.30
6 109 32 29.30
7 135 73 54.00
8 26 12 46.10
9 51 24 47.00
10 71 24 33.80
Mean 74.2 34.7 44.73

Table 4

Individual Participant Performance on Conversational Discourse Task

Participant Total words # of CIUs % of CIUs
1 380 224 58.90
2 334 215 64.30
3 529 387 73.10
4 212 122 57.50
5 1,043 817 78.30
6 232 135 58.10
7 762 431 56.50
8 623 372 59.70
9 303 202 66.60
10 497 283 56.90
Mean 491.5 318.8 62.99

Significance level set at .05 (2-tailed).

Table 5

Comparison of total words, # of CIUs and % of CIUs between discourse tasks

Mean Std. Dev. Sig.
Structured total words 74.20 36.28 < 0.001
Conversational total words 491.40 261.33 < 0.001
Structured # of CIUs 34.70 21.32 0.001
Conversational # of CIUs 318.80 204.25 0.001
Structured % of CIUs 47.82 18.14 0.987
Conversational % of CIUs 59.60 3.983 0.167

Table 6

Results of Nonparametric Statistics for Structured and Conversational Tasks

Structured Total words # of CIUs % CIUs
N 10 10 10
Mann-Whitney 9.00 11.00 11.50
2-Sig. Tailed 0.465 0.753 0.834
Conversational
N 10 10 10
Mann-Whitney 2.000 2.000 7.000
2-Sig. Tailed 0.028 0.028 0.251

Significance level was set at .05 (2-tailed).