Development and preliminary psychometric analysis of a quick screening form for referrals of individuals with neurogenic communication disorders to speech-language pathologists by laypersons

Article information

Clin Arch Commun Disord. 2020;5(1):18-30
Publication date (electronic) : 2020 April 30
doi : https://doi.org/10.21849/cacd.2019.00164
1Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong SAR
2School of Communication Sciences and Disorders, University of Central Florida, Orlando FL, USA
Correspondence: Anthony Pak-Hin Kong, School of Communication Sciences and Disorders, University of Central Florida, Orlando FL, USA, Tel: +1-407-823-4791, Fax: +1-407-823-4816, E-mail: antkong@ucf.edu. Diana Wai-Lam Ho, Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, Tel: +852-3400-3636, Fax: +852-2334-0185, E-mail: stu.swallow@polyu.edu.hk
Received 2019 December 21; Revised 2020 April 27; Accepted 2020 April 27.

Abstract

Purpose

The study aimed at developing and determining the preliminary reliability and validity of a new screening form for use by laypersons to refer individuals with neurogenic communication disorders to speech-language pathologists.

Methods

Thirty-two native Cantonese speakers with neurogenic communication disorders were tested with the proposed screening form by fourteen volunteer raters and well-established speech and language tests for aphasia, dysarthria and apraxia of speech (AOS) by ten speech-language pathology students.

Results

Statistical analyses showed that the internal consistency and inter-rater reliability were high in all the tasks for aphasia and most of the tasks for dysarthria and AOS of the proposed screening form. It was further identified that some of the tasks for aphasia, dysarthria, and AOS would be valid for laypersons to identify individuals with communications disorders.

Conclusions

The results indicated that the tasks for screening aphasia, dysarthria, and AOS should be carefully selected so that untrained laypersons would be able to make correct judgements. Further investigations involving a larger sample size are needed to determine areas to enhance the sensitivity and discriminating power of the current screener to be used by laypersons without a clinical background.

INTRODUCTION

Neurogenic communication disorders, including aphasia (receptive and/or expressive language disorders), dysarthria and apraxia of speech (AOS) (motor speech disorders), are consequences of damages to the nervous system [1]. Kong [2] estimated that about 80,000 individuals were diagnosed with aphasia after stroke at a given time in Hong Kong. There was no information on the incidence of motor speech disorders. As cerebrovascular diseases have close associations with motor speech disorders [3,4], the prevalence of motor speech disorders should not be underestimated given the number of new cases with stroke every year in Hong Kong, i.e., 25,000 [5].

It is important to provide speech and language pathology services for individuals with neurogenic communication disorders as early as possible so as to improve prognosis, preferably while they are still staying in the hospitals or rehabilitation centers [3]. The major caseloads of speech-language pathologists in adult clinical settings are swallowing disorders, followed by dementia and aphasia, and then other cognitive communication disorders and dysarthria [6]. It is common to have co-occurrence of dysphagia, aphasia, and dysarthria [7]. In Hong Kong, speech-language pathologists in community settings may receive referrals from any source to provide assessment and treatment to patients though speech-language pathologists in hospitals may only receive referrals from doctors. According to the results of a recent clinician survey study, the intensity and frequency of current speech and language pathology services in Hong Kong for individuals with acquired neurogenic communication disorders fell short of expectations and most “best practice” recommendations set in developed countries [8].

There are limited standardized assessment tools for assessing neurogenic communication disorders by speech-language pathologists in Hong Kong; among them include the Cantonese Aphasia Battery (CAB) [9] and Main Concept Analysis (MCA) [10] for assessment of language disorders as well as the Hong Kong versions of Birmingham Cognitive Screen (HK-BCoS) [11] and Oxford Cognitive Screen (HK-OCS) [12] for screening of cognitive disorders. Currently, there is a lack of standardized and published tools for assessing motor speech disorders in Hong Kong Cantonese. The Frenchay Dysarthria Assessment-2 (FDA-2) [13] and Apraxia Battery for Adults-2 (ABA-2) [14] were translated into Cantonese and used by speech-language pathologists in Hong Kong but these adaptations have not been formally standardized with the local population. Currently, there is no layperson-oriented screener of communication disorders for the Cantonese-speaking population. If an individual can receive a reliable and valid screening by a layperson after damage to the nervous system, the number of individuals with neurogenic communication disorders being referred to speech-language pathologists should increase. An individual can then receive an assessment and follow-up treatment as early as possible.

In order to ensure that a proposed screening measures what it is purported to measure and is stable across application, different types of reliability and validity should be considered [15]. For reliability, internal consistency and inter-rater reliability are two common measures [15]. Internal consistency measures the consistency among different sections of the same test and is usually measured with Cronbach’s alpha, a statistic calculated from the pairwise correlations between items of the test [16]. Cronbach’s alpha higher than 0.8 indicates good reliability, between 0.7 and 0.8 indicates acceptable reliability, below 0.7 indicates questionable reliability. Inter-rater reliability measures the degree of agreement between different raters on the items, after the raters have done the ratings individually [17]. For validity, content, construct, and concurrent validity are common measures. Content validity is used to reflect the degree to which a measure is representative in the domain of interest [15]. It is often done by having a group of experts in the relevant field verify that the test measures what it is supposed to measure. Construct validity is used to reflect whether a measure relates to the underlying constructs [18] and may be established by principal component analysis. Principal component analysis may be performed by calculating the corrected item-total correlation coefficients on each item to determine their sensitivity to discriminate the performance of the participants in the measure [19]. Items with low corrected item-total correlation coefficients are considered as “low discriminating”, which indicates a poor sensitivity to differentiate the performance between participants [20]. A minimum of corrected item-total correlation coefficient of 0.3 is set as the criterion of data deletion based on the suggestion of [20]. Principal component analysis presumes that the constructs are not inter-correlated to each other and allows discrimination of the testing items into the proposed constructs. It is used for analysis of the dispersion of observations, highlighting possible groupings and detecting the variables that are responsible for the dispersion [21]. The Kaiser-Meyer-Olkin Measure of Sampling Adequacy is one of the procedures that can indicate the proportion of variance in the variables which might be caused by underlying factors. Value which is close to 1.0 generally indicates that a factor analysis may be useful while a value which is less than 0.5 indicates that a factor analysis will not be very useful with the data [22]. Bartlett’s test of sphericity is used to indicate if the variables are unrelated and therefore unsuitable for structure detection. Value which is less than 0.05 of the significance level indicates that a factor analysis may be useful with the data [23]. Concurrent validity is used to determine whether the testing items in a proposed measure relate to existing similar measures. It is done by doing correlation of a new test with a measure that has previously been validated [16].

In this study, we aimed to develop a reliable (in terms of internal consistency and inter-rater reliability) and valid (in terms of content, construct, and concurrent validity) screening form for use by laypersons to refer individuals with neurogenic communication disorders to speech-language pathologists. Such a screening form, when fully validated, will be a valuable tool for many healthcare settings in Hong Kong to triage clients with different types and degrees of acquired neurogenic communication disorders, and can potentially be adapted and modified for use in other countries.

METHOD

Participants

Individuals with neurogenic communication disorders

Thirty-two native Cantonese speakers, mean age=51.16 years, SD=13.80, range=24–71 years, were recruited from the speech and language pathology clinic of a University in Hong Kong and community rehabilitation centres. Seven of them were at the age of 20–39 (21.88%, 7/32), twenty at the age of 40–59 (62.50%, 20/32) while the remaining five (15.63%, 5/32) were at the age of 60 or above. They were 19 males (59.38%, 19/32) and 13 females (40.63%, 13/32) with at least 6 months post onset. The majority of them (28/32, 87.50%) had suffered from cerebrovascular accident and four of them (12.50%, 4/32) had suffered from traumatic brain injury with no concomitant psychological or degenerative diseases. All the individuals were informed of the purpose and procedures of the study and provided written informed consent to participate in the study in compliance with the Committee for Human Subjects Ethics of the University.

Volunteer raters

Fourteen volunteer raters (aged from 20–29) who had not received any speech and language pathology training were recruited using convenience sampling [15]. The volunteer raters were responsible for administering the proposed screening form to the individuals with neurogenic communication disorders. The profile of the volunteer raters is outlined in Table 1.

Profile of the volunteer raters

Student clinicians

Ten master-level speech and language pathology students were also recruited by random sampling. They were responsible for implementing the Cantonese Aphasia Battery (CAB) [9], Cantonese versions of Frenchay Dysarthria Assessment-2 (FDA-2) [13] and Apraxia Battery for Adults-2 (ABA-2) [14] to the individuals with neurogenic communication disorders after training by the first author.

Development of screening form

Testing items

The last author had initially proposed 95 items under four areas (including expressive language, receptive language, dysarthria and AOS) based on the well-established assessment tools (i.e., CAB, FDA-2, and ABA-2 adapted in Cantonese) and reports in the literature. The items were sent to three clinical staff of the university with more than 10 years of experience in working with communication disorders of neurogenic population (first three authors) to review. Then the expert panel sent individual feedback on the wordings, content and scoring methods in the form to the last author. After the last author had revised the form, the expert panel reviewed the 95 items again in a face-to-face meeting to discuss together any unresolved suggestions on item instructions, wordings and scoring schemes. The last author then prepared a finalized version of the screening form (Table 2).

The construct of the proposed screening form

For easy administration of the screening in a variety of setting, common daily items, including cellphone, water bottle, key, pencil, tissue, plastic bag were used for comprehension of single words and commands and newspaper, glasses, watch, wallet, and notebook were used for object naming.

Scoring system

Section 1–10 and 12 use a binary scoring system with “1” and “0” to represent “presence/yes” and “absence/no”. Section 11 requires the rater to write down the names of fruit items orthographically. Section 13 requires the rater to write down the words in picture description orthographically and rated the fluency and content using a Likert scale with four levels (fluency: usually fluent, sometimes fluent, single words only, no response; content: 5 items or above, 3–4 items, 1–2 items, no response). Section 14 to 17 use a Likert scale with three levels (normal, mildly impaired, severely impaired). Section 18 and 19 require the rater to judge if the speech rate is normal, too fast, too slow or no response and the number of syllables with misarticulation. Section 20 and 21 require the rater to record the number of syllables in production (same number of syllables as the target, more than the target number of syllables, less than the target number of syllables, no response) and if the articulation is correct.

Procedure

Individuals with neurogenic communication disorders were assigned to volunteer raters and student clinicians randomly. Half of them were tested with standardized assessments before the proposed screening form and half were tested with the proposed screening form before the standardized assessments. The sessions were audio-recorded. All the individuals with neurogenic communication disorders needed less than 30 minutes to finish the proposed screening form and less than two hours to finish the whole assessment. The student clinicians recorded all the results of standardized tests and the volunteer raters recorded the results of the proposed screening form during the testing procedures.

DATA AND STATISTICAL ANALYSIS

In order to ensure that the proposed screening form is reliable and valid, the internal consistency, inter-rater reliability and content, construct and concurrent validities were investigated.

Cronbach’s alpha coefficient was done on the test items to see if the items were under the same construct so as to address the internal consistency.

For inter-rater reliability, the audio recording of five selected individuals with neurogenic communication disorders (i.e., two with severe aphasia and motor speech disorder, one with moderate aphasia and motor speech disorder, one with moderate aphasia but no motor speech disorder, and one with no aphasia and mild motor speech disorder) were independently reviewed and scored by second raters. The five selected individuals covered the range of neurogenic communication disorders in the sample. The score of each section in the proposed screening form obtained by the first and second raters were compared using Pearson’s r coefficients correlation. Since the second raters could only judge the performance of individuals with neurogenic communication disorders based on audio recording, the inter-rater reliability of section 3 noun comprehension, 4 verb comprehension, 5 simple command, and 6 sequential command in which the individuals with neurogenic communication disorders responded by pointing or manipulating the objects were not obtained.

Content validity was established by an expert panel composed of three qualified speech-language pathologists who had at least 10 years of experience in daily management of individuals with neurological impairments.

Construct validity was investigated by principal component analysis to evaluate if the items can be discriminated into the four areas (i.e., expressive language, receptive language, dysarthria and AOS) as proposed. Factor analysis was done further to identify the underlying relationships between the measured variables.

For concurrent validity, scores from all different sections except section 1 orientation questions in the proposed screening form were compared with similar tasks in CAB [9], FDA-2 [13], and ABA-2 [14] adapted in Cantonese using Pearson correlation coefficient. The expected correlated items are listed in Table 3. The orientation questions in the proposed screening form were not compared with the questions for spontaneous speech in CAB as the questions in CAB were used to assess the information and fluency of an individual in conveying message instead of orientation.

Expected correlation between the proposed screening form, CAB, FDA-2, and ABA-2 (Cantonese)

RESULT

Internal consistency

Cronbach’s alpha of the test items in the four proposed areas was examined to establish the internal consistency (Table 4). It should be noted that section 18 and 19 were included in both dysarthria and AOS areas in the analysis.

Reliability of the proposed screening form in terms of internal consistency

Inter-rater reliability

The scores of the five randomly selected individuals with neurogenic communication disorders obtained by two different untrained volunteer raters was compared using Pearson’s r coefficients correlation. The inter-rater reliability of the proposed screening form was shown in Table 5.

Inter-rater reliability of the proposed screening form

Statistically significant results were found in most sections except section 15 respiration in speech, 18 AMR (rate) and (articulation), 19 SMR (rate) and (articulation), 21 repeated trials (syllable) and (articulation), indicating that insignificant inter-rater reliability in these sections.

Content validity

Following the methodology in Huck [15] and Scott and Xie [18], the content validity was established by an expert panel with three clinical staff of the university with more than 10 years of experience in working with communication disorders of neurogenic population by careful selection of the testing items independently and in a meeting.

Construct validity

To address the construct validity, item analysis was performed to identify items with low corrected item-total correlation coefficients, which indicated that the items should not be included in the set [19]. Before calculating the corrected item-total correlation coefficient, items 3.3 noun comprehension, 20.1, and 20.2 increased word length were removed since they have no variance among all the individuals with neurogenic communication disorders and thus should not correlate with the overall score of the measurement. Table 6 shows the corrected item-total correlation coefficients of all the items in the proposed screening form.

Item analysis of the proposed screening form

Based on the results, a total of 12 items (i.e., 1.1 orientation question, 3.1, 3.3, and 3.5 noun comprehension, 8.2 noun naming, 10.1a, 10.2a, and 10.3a repetition, 12.2 and 12.3 sentence completion, 20.1a and 20.2a increased with length) were removed from the screening form in the subsequent analyses because of low corrected item-total correlation coefficients. Then, principal component analysis was carried out to evaluate if the items could be discriminated into the four areas as proposed. The four areas (factors) accounted for 46.70%, 10.21%, 8.69%, and 6.30% of the variance respectively. The value for Kaiser-Meyer-Olkin test was 0.53, which indicated a moderate level of suitability for performing factor analysis of the data [22]. The Bartlett’s test of sphericity for x2 was 855.43 (d.f. 325, p<0.001), suggesting that the data could provide significant common variances which could be summarized into factors [23]. Based on the four areas (factors), an anchor variable was chosen for each proposed area (factor) by identifying the variable with the highest loading. The loading of various variables is shown in Table 7.

Factor analysis of the proposed screening form

Based on the results, factor 1 was loaded by all receptive and expressive language tasks as well as most of the motor speech tasks which were designed to detect AOS except item 18a AMR (rate) and 19a SMR (rate). Factor 2 constituted most of the motor speech tasks designed to detect dysarthria, except item 14 intelligibility. Factor 3 was only loaded by item 18a AMR (rate) and 19a SMR (rate). No anchor variable was found in factor 4.

Concurrent validity

For concurrent validity, the Pearson’s r coefficient between the comparable aspects of the proposed screening form, CAB [9], FDA-2 [13], and ABA-2 [14] adapted in Cantonese were obtained (Table 8).

Correlation analyses between the proposed screening form and CAB, FDA-2 or ABA-2 (Cantonese)

The results suggested that all testing items in the language areas of the proposed screening form were positively and significantly correlated with the corresponding areas in CAB. In addition, three out of four comparable items (i.e., section 14 intelligibility, 16 voice quality, 17 volume) relating to dysarthria were also statistically significant. For the area of AOS, only item 20a increased word length (syllable) and 20b increased word length (articulation) showed significant relationships with the corresponding scores in ABA-2. Insignificant results were found in item 15 respiration in speech, 18 AMR (rate) and (articulation), 19 SMR (rate) and (articulation), and 21 repeated trials (syllable) and (articulation).

DISCUSSION

In this study, we aimed to develop a reliable and valid screening form for use by laypersons to refer individuals with neurogenic communication disorders to speech-language pathologists. This was done by analyzing the internal consistency, inter-rater reliability and content, construct and concurrent validities of a proposed screening form.

Internal consistency

For internal consistency, good reliabilities were found in the receptive language (α=0.90) and expressive language (α= 0.83) areas and an acceptable reliability was found in the dysarthria area (α=0.80) (Table 4). However, the reliability of the AOS area was considered as questionable (α=0.62) because α was<0.70 [17]. Considering that section 18 and 19 were included in both dysarthria and AOS areas in the analysis, the questionable reliability in the AOS area was contributed by section 20 (increased word length) and 21 (repeated trials). As section 20 and 21 required the rater to record the number of syllables produced by the individuals with neurogenic communication disorders (same number of syllables as the target, more than the target number of syllables, less than the target number of syllables, no response) and if the articulation was correct, the results showed that these items or requirements are particularly challenging to laypersons. It is necessary to find out if the challenge is created by the requirement of recording both the number of syllables and judging articulation errors so we may include only one requirement or the requirement of recording the number of syllables or judging articulation errors is difficult for laypersons so we should remove either requirement.

Inter-rater reliability

For inter-rater reliability, most sections demonstrated strong inter-rater agreements (r>0.80) [24], except section 15 Respiration in speech, 18 AMR (rate) and (articulation), 19 SMR (rate) and (articulation), 21 repeated trials (syllable) and (articulation), which were found to have no statistically significant result. The result might indicate that the tasks with lower inter-rater agreement (respiration in speech, AMR and SMR (rate and articulation), repeated trials (syllable and articulation)) are rather subjective or the tasks were difficult for the raters to judge the performance of individuals with neurogenic communication disorders and so the raters did not have much agreement when compared to other items with higher degree of correlation. For sections 18, 19, and 21, it is recommended to find out if the challenge is created by the requirement of recording both the rate/number of syllables and judging articulation errors so we may consider only one requirement or the requirement of recording the rate/number of syllables or judging articulation errors is difficult for laypersons so we should remove either requirement. It is recommended to remove Section 15 respiration in speech as some descriptive terms such as “normal/grossly normal/abnormal most of the time” and “out of breath”, “insufficient breath support” were already used to aid judgement of normal/abnormal respiration.

Content validity

The content validity of the proposed screening form was developed by careful selection of the test items. The 95 items covered all the relevant areas on well-established assessment tools in Cantonese (i.e., CAB, FDA-2 and ABA-2) as decided by an expert panel with three clinical staff of the university with more than 10 years of experience in working with communication disorders of neurogenic population. A panel of experts to review the test specifications and the selection of items should improve the content validity [25]. Even with a small sample size of 32 individuals with neurogenic communication disorders in this study, the results generally showed strong content validity in relation to the well-established assessment tools.

Construct validity

For construct validity, item analysis was performed and a ceiling effect [26] was found in 12 items with low-discriminating power (i.e., item 1.1 orientation question, 3.1, 3.3, and 3.5 noun comprehension, 8.2 noun naming, 10.1a, 10.2a, and 10.3a repetition, 12.2 and 12.3 sentence completion, 20.1a and 20.2a increased with length) (Table 6). Since over 90% of individuals with neurogenic communication disorders responded correctly to these items, these items were too easy for them and so failed to discriminate their performance and were removed in the subsequent analyses. In order to improve the overall sensitivity of the proposed screening form, these items should be removed or replaced by more difficult items. For example, noun comprehension or naming may be replaced by low frequency nouns or verbs. However, repetition, sentence completion and increased with length may not have relevant more “difficult” items and may be removed.

Principal component analysis was performed but indicated that the proposed structure could not be confirmed since the results did not provide a good fit for the proposed four areas (factors) (i.e., expressive language, receptive language, dysarthria, and AOS) [27]. An alternative factor structure was indicated. The results showed that factor 1 was loaded by all receptive and expressive language tasks (section 1 to 13) as well as most of the motor speech tasks which were designed to detect AOS (section 18b, 19b, 20, 21) (Table 7). Only item 18a AMR (rate) and 19a SMR (rate) for AOS were not included. Since receptive and expressive language were collapsed into the same factor, it indicated that receptive and expressive language tasks would have some shared features. Some of the comprehension tasks require verbal responses from the individuals with neurogenic communication disorders, e.g., section 1 orientation question and 2 yes/no questions. On the other hand, in order to complete the expressive language tasks, the individuals with neurogenic communication disorders need to comprehend the verbal instructions. Individuals with aphasia would have impairment in both receptive and expressive language, though comprehension may be less impaired than production [28]. Besides, section 14 intelligibility, which was proposed to be under dysarthria, was also grouped into factor 1 with predominately language tasks (Table 7). Previous studies found that the patients’ reduced speech intelligibilities due to different etiologies would also exert significantly negative influence on their communication effectiveness [29,30]. Therefore, the reduced intelligibility of the individuals with neurogenic communication disorders could likely pose a challenge to untrained raters for their judgement of patients’ expressive language performance. In addition, section 18b AMR (articulation), 19b SMR (articulation), 20 increased word length, and 21 repeated trials, which were designed to detect AOS were also loaded into factor 1 (Table 7). AMR, SMR, increased word length and repeated trials required the individuals to repeat the testing items. Repetition is a commonly used assessment task for both aphasia as in CAB [9] and AOS as in ABA-2 [14]. In CAB, the raters are required to record the correct/incorrect productions of the individuals in terms of number of syllables and units. In ABA-2, the raters are also required to record and count the correct/incorrect productions of the individuals. Apparently, repetition tasks appear to be more like a language task instead of a motor speech task. Besides, there is a high comorbidity between AOS and aphasia. The result in this study may reflect that AOS rarely presents in its pure form which might make it almost indistinguishable from aphasia [31]. Broca’s aphasia commonly co-exist with AOS [31], and thereby increase the difficulty for the untrained raters to distinguish the performance of the individuals in AOS tasks from language tasks.

Factor 2 constituted most of the motor speech tasks relating to dysarthria including section 15 respiration in speech, 16 voice quality, 17 volume. Respiration, phonation and prosody are parameters on motor speech for dysarthric patients [32].

Factor 3 was only loaded by item 18a AMR (rate) and 19a SMR (rate) which were considered as different from other motor speech tasks in the proposed screening form. It should be noted that item 18b AMR (articulation) and 19b SMR (articulation), which required judgement of correct/incorrect articulation, were collapsed into factor 1. On the other hand, item 18a AMR (rate) and 19a SMR (rate) required the raters to decide whether the speech rates of individuals with neurogenic communication disorders were normal or too fast/slow. This could be highly subjective for untrained raters who have no experience working with individuals with motor speech disorders. Gadesmann and Miller [33] also reported poor inter-rater reliability (i.e., r=0.28) between ten untrained raters and a significant divergence between ratings of untrained raters and speech-language pathologists (t=6.07, p<0.001) in AMR and SMR.

Concurrent validity

For concurrent validity, the Pearson’s r coefficient between the comparable aspects of the proposed screening form and standardized tests CAB [9], FDA-2 [13] and ABA-2 [14] adapted in Cantonese were performed (Table 8). Most of the sections in the proposed screening form showed statistically significant relations with the relevant sections in CAB, ABA-2 and FDA-2 (Table 8), except for section 15 respiration in speech, 18 AMR, 19 SMR, and 21 repeated trials. The result of concurrent validity in these sections echoed the insignificant results in inter-rater reliability (section 15 respiration in speech, 18a AMR (rate), 18b AMR (articulation), 19a SMR (rate), 19b SMR (articulation), 21a repeated trials (syllable), and 21b repeated trials (articulation) as stated in the section of inter-rater reliability above). Judgement of respiration in speech, rate and articulation of AMR and SMR tasks are especially difficult for untrained raters. On the contrary, section 20 increased word length was found to have significant concurrent validity and inter-rater reliability. Section 20 increased word length only required individuals with neurogenic communication disorders to repeat a phrase with fewer than 5 syllables. As mentioned previously, the results showed that it was difficult for untrained raters to judge if the respiration of an individual was normal in speech, the speech rate in AMR and SMR was normal, slow or fast, count the number of articulation errors in AMR and SMR and count the number of syllables and articulation errors in repeated trials. The individuals with neurogenic communication disorders would repeat the target syllables for multiple times, i.e., ten times rapidly for SMR and AMR (10 to 30 syllables), three times consecutively for repeated trials (9 to 15 syllables) and it would be challenging for the untrained raters to judge the performance. Untrained raters were only able to judge if the number of syllables was fewer than five.

Design of recording format

In addition to the above-mentioned psychometric properties of the screener, the design of the recording format might also have an effect on the difficulty of the rating tasks. Different rating scale formats with varying numbers of response categories and varying label formats would have effects on the responses of the questionnaires [34]. In section 20 increased word length of this study, boxes were provided under each syllable of the stimuli for recording, which enabled the raters to document the errors by putting a tick/cross in the box on the spot. However, this kind of visual aid was not provided in section 18 AMR, 19 SMR and 21 repeated trials. Raters were required to mark down the number of errors. Some raters verbally reported that the boxes were useful in facilitating their judgement. For further modification, boxes shall be given for recording each production in motor speech tasks in order to facilitate the judgement of the untrained raters.

Summary of analyses in reliability and validity

Table 9 is a summary of the analyses in internal consistency, inter-rater reliability, construct, and content validities. The highlighted sections, including yes/no questions, comprehension and production of verbs, comprehension of simple and sequential commands, responsive naming, repetition (articulation), divergent naming, fluency, information, intelligibility, voice quality, and volume, may be considered as passing all four criteria and can be used as reliable and valid items in the screening form. Voice quality and volume should be considered under the area of dysarthria while the remaining should be considered under aphasia and AOS. The form and untrained raters cannot distinguish between aphasia and AOS.

Summary of reliability and validity of different sections in the proposed screening form

LIMITATION OF STUDY AND DIRECTION OF FUTURE RESEARCH

Small number of individuals with neurogenic communication disorders

The small number of individuals with neurogenic communication disorders is one of the limitations. Suggested minimums for sample size were 3 to 20 times the number of variables [35]. Because there are 21 variables in the proposed screening form, the sample size should be at the range of 63 to 420. Based on the estimate of 80,000 individuals with aphasia after stroke at a given time in Hong Kong [2], a sample size of 383 participants is recommended for a confidence level of 95% and a confidence interval of 5. There were only 32 individuals with neurogenic communication disorders in this study. Albeit the small sample size, the pilot construction of the screener proved it to be sensitive and valid. The next step is to recruit a significantly larger sample size to allow proper investigation of the psychometric properties of the screener.

Sample bias on individuals with the types of aphasia

Secondly, a large proportion of the individuals with neurogenic communication disorders were anomic (43.75%,14/32) or Broca’s aphasia (28.13%, 9/32). Anomic aphasia is the mildest form of acquired language impairments [36,37]. Individuals with anomic aphasia could generally obtain high scores in the receptive language parts of the screening form which might take account for the ceiling effect in some of the tasks. Broca’s aphasia commonly co-exist with AOS [31]. This might also explain why most tasks for detecting AOS were collapsed with receptive and expressive language tasks in the proposed screening form. More individuals with different types of aphasia, as well as pure aphasia or motor speech disorders should be recruited to further validate the underlying construct of the proposed screening form.

Sample bias on demographic information of volunteer raters

Ninety-three percent (13/14) of the volunteer raters had undergraduate degrees or above and all of them were young (aged from 20–29 years). To ensure that the screening form can be used by the general public, volunteer raters with more diverse demographic background (e.g., wider range of education background and age groups) should be recruited.

Moreover, the number of motor speech items (i.e., 32) was far fewer than the language items (i.e., 63), which could confound the underlying construct of the screening form. Therefore, more items for motor speech disorders such as volitional/automatic speech and maximum phonation time may be considered in the screening form [32].

CONCLUSIONS

A reliable and valid screening form was proposed for use by laypersons to refer individuals with neurogenic communication disorders to speech-language pathologists. The screening form may include yes/no questions, comprehension of verb, simple and sequential commands, responsive naming, verb naming, divergent naming, fluency, information, and intelligibility in the areas of aphasia and AOS and voice quality and volume in the area of dysarthria. The screening form and untrained raters were not able to discriminate between aphasia and AOS.

ACKNOWLEDGEMENTS

The authors would like to thank all the individuals with neurogenic communication disorders, volunteer raters and student clinicians who participated in the study.

References

1. Brookshire RH, McNeil MR. Introduction to neurogenic communication disorders 8th ed.th ed. St. Louis, MO: Elsevier; 2015.
2. Kong A. Family members’ report on speech-language pathology and community services for persons with aphasia in Hong Kong. Disabil Rehabil 2011;33:2633–2645.
3. Vidović M, Sinanović O, Sabaskić L, Haticić A, Brkić E. Incidence and types of speech disorders in stroke patients. Acta Clin Croat 2011;50:491–494.
4. Strand EA, Duffy JR, Clark HM, Josephs K. The apraxia of speech rating scale: a tool for diagnosis and description of apraxia of speech. Journal of Communication Disorders 2014;51:43–50.
5. Hong Kong Stroke Fund [internet] Hong Kong: 2018. [cited 2018 April 4]. Available from: http://www.strokefund.org/aboutus_part1.php.
6. American Speech-Language-Hearing Association. SLP health care survey report: caseload characteristics and trends 2005–2017 Rockville, MD: Gail Brook, Surveys and Analysis; 2017.
7. Flowers HL, Silver FL, Fang J, Rochon E, Martino R. The incidence, co-occurrence, and predictors of dysphagia, dysarthria, and aphasia after first-ever acute ischemic stroke. J Commun Disord 2013;:46238–46248.
8. Kong APH, Tse CWK. Clinician survey on speech pathology services for people with aphasia in Hong Kong. Clin Arch Commun Disord 2018;3:201–212.
9. Yiu EML. Linguistic assessment of Chinese speaking aphasics: development of a Cantonese aphasia battery. J Neurolinguistics 1992;7:379–424.
10. Kong APH. The use of main concept analysis to measure discourse production in Cantonese-speaking persons with aphasia: a preliminary report. J Commun Disord 2009;42:442–464.
11. Kong APH, Chan J, Lau JKL, Bickerton WL, Weekes B, Humphreys G. Developing a Cantonese version of Birmingham Cognitive Screen for stroke survivors in Hong Kong. Commun Disord Q 2018;39:387–401.
12. Kong APH, Lam PHP, Ho DWL, Lau JK, Humphreys G, Riddoch J, Weekes B. The Hong Kong version of the Oxford Cognitive Screen (HK-OCS): validation study for Cantonese-speaking chronic stroke survivors. Aging Neuropsychol Cogn 2016;23:530–548.
13. Enderby P, Palmer R. Frenchay dysarthria assessment 2nd ed.th ed. Austin, TX: Pro-Ed; 2008.
14. Dabul B. Apraxia battery for adults 2nd ed.th ed. Austin, TX: Pro-Ed; 2000.
15. Huck SW, Cormier WH, Bounds WG. Reading statistics and research New York, NY: Harper & Row; 1974.
16. Hunt EF, Colander DC. Social science: an introduction to the study of society New York, NY: Routledge, Taylor & Francis Group; 2017.
17. George D, Mallery P. SPSS for windows step by step: a simple guide and reference 110 update 4th ed.th ed. Boston, MA: Allyn & Bacon; 2003.
18. Scott JL, Xie Y. Quantitative social science London: SAGE; 2005.
19. Nunnally JC, Bernstein IH. Psychometric theory 3rd ed.th ed. New York, NY: McGraw Hill; 1994.
20. Ferketich S. Focus on psychometrics: aspects of item analysis. Res Nurs Health 1991;14:165–168.
21. Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis New Jersey, NJ: Prentice Hall; 2009.
22. Cerny CA, Kaiser HF. A study of a measure of sampling adequacy for factor-analytic correlation matrices. Multivariate Behav Res 1977;12:43–47.
23. Snedecor GW, Cochran WG. Statistical methods 8th ed.th ed. Iowa, IA: Iowa State University Press; 1989.
24. Gwet K. Handbook of inter-rater reliability Maryland, MD: Advanced Analytics Press; 2014.
25. Foxcroft C, Paterson H, Le Roux N, Herbst D. Psychological assessment in South Africa: a needs analysis Pretoria, South Africa: Human Science Research Council; 2004.
26. Dean K, Walker Z, Jenkinson C. Data quality, floor and ceiling effects, and test–retest reliability of the mild cognitive impairment questionnaire. Patient Relat Outcome Meas 2018;9:43–47.
27. Bollen K. Structural equations with latent variables New York, NY: John Wiley & Sons; 1998.
28. Cho-Reyes S, Thompson CK. Verb and sentence production and comprehension in aphasia: Northwestern assessment of verbs and sentences (NAVS). Aphasiology 2012;26:1250–1277.
29. McAuliffe MJ, Fletcher AR, Kerr SE, O’Beirne GA, Anderson T. Effect of dysarthria type, speaking condition, and listener age on speech intelligibility. Am J Speech Lang Pathol 2017;26:113–123.
30. Joubert K, Bornman J, Alant E. Speech intelligibility and marital communication in amyotrophic lateral sclerosis: an exploratory study. Commun Disord Q 2011;33:34–41.
31. Patidar Y, Gupta M, Khwaja G, Chowdhury D, Batra A, Dasgupta A. A case of crossed aphasia with apraxia of speech. Ann Indian Acad Neurol 2013;16:428–431.
32. Duffy J. Motor speech disorders: substrates, differential diagnosis, and management 3rd ed.th ed. St. Louis, MO: Elsevier Mosby; 2013.
33. Gadesmann M, Miller N. Reliability of speech diadochokinetic test measurement. Int J Lang Commun Disord 2008;43:41–54.
34. Weijters B, Cabooter E, Schillewaert N. The effect of rating scale format on response styles: the number of response categories and response category labels. Int J Res Mark 2010;27:236–247.
35. Mundfrom DJ, Shaw DG, Ke TL. Minimum sample size recommendations for conducting factor analyses. Int J Test 2005;5:159–168.
36. Dronkers NF, Baldo JV. Language: aphasia. In : Stein JP, ed. Reference module in neuroscience and biobehavioral psychology Cambridge: Elsevier; 2009. p. 343–348.
37. Kong APH. Analysis of neurogenic disordered discourse production: from theory to practice 1st ed.th ed. New York, NY: Psychology Press; 2016.

Article information Continued

Table 1

Profile of the volunteer raters

Variables Elements Frequency Percent
Gender Male 7 50.00% (7/14)
Female 7 50.00% (7/14)

Education level Associate degree 1 7.14% (1/14)
Undergraduate 6 42.88% (6/14)
Postgraduate 7 50.00% (7/14)

Major Occupational therapy 6 42.89% (6/14)
Language 4 28.57% (4/14)
Psychology 2 14.30% (2/14)
Social work 1 7.14% (1/14)
Computer science 1 7.14% (1/14)

Table 2

The construct of the proposed screening form

Area Section Number of testing items
Receptive language 1 Orientation questions 5
2 Yes/no questions 5
3 Noun 5
4 Verb 5
5 Simple commands 5
6 Sequential commands 5

Expressive language 7 Responsive naming 5
8 Noun 5
9 Verb 5
10a Repetition (syllable) 5
10b Repetition (articulation) 5
11 Divergent naming 1
12 Sentence completion 5
13a Fluency 1
13b Information 1

Dysarthria 14 Intelligibility 1
15 Respiration in speech 1
16 Voice quality 1
17 Volume 1

Dysarthria and AOS* 18a Alternate Motion Rate (AMR) (rate) 3
18b Alternate Motion Rate (articulation) 3
19a Sequential Motion Rate (SMR) (rate) 1
19b Sequential Motion Rate (articulation) 1

AOS 20a Increased word length (syllable) 5
20b Increased word length (articulation) 5
21a Repeated trials (syllable) 5
21b Repeated trials (articulation) 5
*

Section 18 AMR and 19 SMR were considered as both dysarthria and AOS areas, as they were highly sensitive indices of motor speech impairments relating to both speech motor planning/programming and execution (Duffy, 2013).

Table 3

Expected correlation between the proposed screening form, CAB, FDA-2, and ABA-2 (Cantonese)

Area Section of the screening form Validation task
Receptive language 2 Yes/no questions Yes/no questions (CAB)
3 Noun comprehension Auditory word recognition (CAB)
4 Verb comprehension Auditory word recognition (CAB)
5 Simple commands Sequential commands (CAB)
6 Sequential commands Sequential commands (CAB)
Total score of the receptive language Total score of the auditory verbal comprehension (CAB)

Expressive language 7 Responsive naming Responsive speech (CAB)
8 Noun naming Object naming (CAB)
9 Verb naming Object naming (CAB)
10a Repetition (syllable) Repetition (CAB)
10b Repetition (articulation) Repetition (CAB)
11 Divergent naming Word fluency (CAB)
12 Sentence completion Sentence completion (CAB)
13.1 Fluency Spontaneous speech – fluency (CAB)
13.2 Information Spontaneous speech – information (CAB)
Total score of the expressive language Total score of naming, repetition and spontaneous speech (CAB)

Sum of receptive and expressive language Aphasia quotient (CAB)

Dysarthria 14 Intelligibility Intelligibility: conversation (FDA-2)
15 Respiration in speech Respiration in speech (FDA-2)
16 Voice quality Laryngeal in speech (FDA-2)
17 Volume Laryngeal volume (FDA-2)
Total score of Dysarthria Severity of dysarthria (FDA-2)

Dysarthria and AOS 18a AMR (rate)* Diadochokinetic rate (ABA-2)
18b AMR (articulation)* Diadochokinetic rate (ABA-2)
19a SMR (rate)* Diadochokinetic rate (ABA-2)
19b SMR (articulation)* Diadochokinetic rate (ABA-2)

AOS 20a Increased word length (syllable) Increased word length (ABA-2)
20b Increased word length (articulation) Increased word length (ABA-2)
21a Repeated trials (syllable) Repeated trials (ABA-2)
21b Repeated trials (articulation) Repeated trials (ABA-2)
Total score of AOS Severity of AOS (ABA-2)
*

Although section 18 AMR and 19 SMR were considered as both dysarthria and AOS areas (Duffy, 2013), correlations were only done with ABA-2 as the tasks in the proposed screening form were designed based on ABA-2. The description in the tasks of FDA-2 was considered as too difficult for naïve listener to judge the performance.

Table 4

Reliability of the proposed screening form in terms of internal consistency

Construct of the screening form A
Receptive language 0.90
Expressive language 0.83
Dysarthria 0.80
AOS 0.62

Table 5

Inter-rater reliability of the proposed screening form

Section of the screening form R
1 Orientation questions 1.00**
2 Yes/no questions 1.00**
7 Responsive naming 0.97**
8 Noun naming 1.00**
9 Verb naming 1.00**
10a Repetition (syllable) 1.00**
10b Repetition (articulation) 0.97**
11 Divergent naming 1.00**
12 Sentence completion 1.00**
13.1 Fluency 1.00**
13.2 Information 0.94*
14 Intelligibility 1.00**
15 Respiration in speech 0.54
16 Voice quality 0.92*
17 Volume 0.89*
18a AMR (rate) 0.91
18b AMR (articulation) 0.76
19a SMR (rate) 0.58
19b SMR (articulation) 0.65
20a Increased word length (syllable) 0.99**
20b Increased word length (articulation) 0.90*
21a Repeated trials (syllable) 0.87
21b Repeated trials (articulation) 0.76
*

Correlation significant at the 0.05 level;

**

Correlation significant at the 0.01 level.

Table 6

Item analysis of the proposed screening form

Section of the screening form Item no. Corrected item-total correlation coefficients Cronbach’s alpha if item deleted
1 Orientation questions 1.1 −0.11* 0.92
1.2 0.79 0.91
1.3 0.41 0.91
1.4 0.65 0.91
1.5 0.25 0.92

2 Yes/no questions 2.1 0.49 0.91
2.2 0.39 0.92
2.3 0.53 0.91
2.4 0.32 0.92
2.5 0.63 0.91

3 Noun comprehension 3.1 −0.18* 0.92
3.2 0.57 0.91
3.4 0.44 0.91
3.5 0.28* 0.92

4 Verb comprehension 4.1 0.67 0.91
4.2 0.65 0.91
4.3 0.71 0.91
4.4 0.44 0.91
4.5 0.62 0.91

5 Simple commands 5.1 0.71 0.91
5.2 0.68 0.91
5.3 0.71 0.91
5.4 0.61 0.91
5.5 0.55 0.91

6 Sequential commands 6.1 0.53 0.91
6.2 0.58 0.91
6.3 0.52 0.91
6.4 0.65 0.91
6.5 0.65 0.91

7 Responsive naming 7.1 0.68 0.80
7.2 0.85 0.80
7.3 0.64 0.80
7.4 0.78 0.80
7.5 0.73 0.80

8 Noun naming 8.1 0.65 0.80
8.2 0.25* 0.81
8.3 0.55 0.80
8.4 0.58 0.80
8.5 0.84 0.80

9 Verb naming 9.1 0.49 0.80
9.2 0.32 0.80
9.3 0.46 0.80
9.4 0.59 0.80
9.5 0.61 0.80

10 Repetition 10.1a (syllable) 0.07* 0.81
10.1b (articulation) 0.53 0.88
10.2a (syllable) 0.07* 0.81
10.2b (articulation) 0.37 0.88
10.3a (syllable) 0.26* 0.81
10.3b (articulation) 0.69 0.87
10.4a (syllable) 0.48 0.80
10.4b (articulation) 0.57 0.87
10.5a (syllable) 0.75 0.80
10.5b (articulation) 0.66 0.88

11 Divergent naming 11 0.83 0.91

12 Sentence completion 12.1 0.39 0.80
12.2 0.07* 0.81
12.3 0.22* 0.81
12.4 0.46 0.80
12.5 0.30 0.81

13.1 Fluency 13.1 0.60 0.79
13.2 Information 13.2 0.70 0.78

14 Intelligibility 14 0.46 0.77

15 Respiration in Speech 15 0.50 0.74

16 Voice quality 16 0.61 0.72

17 Volume 17 0.51 0.74

18 AMR 18.1a (rate) 0.35 0.76
18.1b (articulation) 0.66 0.88
18.2a (rate) 0.48 0.75
18.2b (articulation) 0.75 0.86
18.3a (rate) 0.71 0.71
18.3b (articulation) 0.63 0.87

19 SMR 19.1 (rate) 0.37 0.76
19.2 (articulation) 0.74 0.89

20 Increased word length 20.3a (syllable) 0.63 0.84
20.3b (articulation) 0.50 0.88
20.4a (syllable) 0.76 0.82
20.4b (articulation) 0.44 0.88
20.5a (syllable) 0.57 0.85
20.5b (articulation) 0.61 0.87

21 Repeated trials 21.1a (syllable) 0.55 0.85
21.1b (articulation) 0.80 0.86
21.2a (syllable) 0.64 0.85
21.2b (articulation) 0.46 0.88
21.3a (syllable) 0.59 0.84
21.3b (articulation) 0.57 0.87
21.4a (syllable) 0.69 0.83
21.4b (articulation) 0.80 0.86
21.5a (syllable) 0.63 0.84
21.5b (articulation) 0.80 0.86
*

Items with low discriminating power (corrected item-total correlation coefficients <0.30) are highlighted.

Table 7

Factor analysis of the proposed screening form

Area Variables of the screening form Factor 1 Factor 2 Factor 3 Factor 4
Receptive language 1 Orientation questions 0.88* −0.14 −0.02 0.22
2 Yes/no questions 0.77* −0.34 −0.21 −0.07
3 Noun comprehension 0.51* 0.27 −0.40 −0.52
4 Verb comprehension 0.85* −0.02 −0.31 −0.18
5 Simple commands 0.83* −0.26 −0.18 −0.23
6 Sequential commands 0.82* −0.27 0.15 0.18

Expressive language 7 Responsive naming 0.90* −0.11 −0.10 0.25
8 Noun naming 0.84* −0.12 −0.14 0.18
9 Verb naming 0.86* −0.08 0.02 0.12
10a _Repetition (syllable) 0.71* −0.17 0.05 0.34
10b _Repetition (articulation) 0.75* 0.19 0.34 −0.08
11 _Divergent naming 0.83* −0.02 −0.12 0.14
12 _Sentence completion 0.47* −0.34 0.02 0.25
13.1 Fluency 0.61* 0.34 −0.08 0.43
13.2 Information 0.66* 0.30 −0.45 0.07

Dysarthria 14 Intelligibility 0.70* 0.25 0.17 −0.05
15 _Respiration in speech 0.23 0.63* −0.24 0.26
16 Voice quality 0.37 0.62* −0.16 0.02
17 Volume 0.13 0.72* −0.24 0.23

Dysarthria and AOS 18a AMR (rate) 0.17 0.33 0.71* 0.22
18b _AMR (articulation) 0.56* 0.42 0.41 −0.45
19a SMR (rate) 0.17 0.25 0.64* 0.31
19b _SMR (articulation) 0.58* 0.34 0.39 −0.42

AOS 20a _Increased word length (syllable) 0.70* −0.35 0.26 −0.16
20b _Increased word length (articulation) 0.64* −0.07 0.40 −0.26
21a _Repeated trials (syllable) 0.82* −0.09 −0.07 −0.17
21b _Repeated trial (articulation) 0.80* −0.29 0.17 0.01
*

Items with the highest loading are highlighted in the column of the corresponding factor.

Table 8

Correlation analyses between the proposed screening form and CAB, FDA-2 or ABA-2 (Cantonese)

Section of the screening form Validation task r
2 Yes/no questions Yes/no questions (CAB) 0.79**
3 Noun comprehension Auditory word recognition (CAB) 0.46**
4 Verb comprehension Auditory word recognition (CAB) 0.82**
5 Simple commands Sequential commands (CAB) 0.75**
6 Sequential commands Sequential commands (CAB) 0.78**
Total score of the receptive language Total score of the auditory verbal comprehension (CAB) 0.92**
7 Responsive naming Responsive speech (CAB) 0.75**
8 Noun naming Object naming (CAB) 0.80**
9 Verb naming Object naming (CAB) 0.73**
10a Repetition (syllable) Repetition (CAB) 0.46**
10b Repetition (articulation) Repetition (CAB) 0.62**
11 divergent naming Word fluency (CAB) 0.85**
12 Sentence completion Sentence completion (CAB) 0.67**
13.1 Fluency Spontaneous speech – fluency (CAB) 0.70**
13.2 Information Spontaneous speech – information (CAB) 0.77**
Total score of the expressive language Total score of naming, repetition and spontaneous speech (CAB) 0.88**
Sum of receptive and expressive language Aphasia quotient (CAB) 0.91**
14 Intelligibility Intelligibility: conversation (FDA-2) 0.38*
15 Respiration in speech Respiration in speech (FDA-2) 0.20
16 Voice quality Laryngeal in speech (FDA-2) 0.38*
17 Volume Laryngeal volume (FDA-2) 0.39*
Total score of Dysarthria Severity of dysarthria (FDA-2) 0.39*
18a AMR (rate) Diadochokinetic rate (ABA-2) 0.35
18b AMR (articulation) Diadochokinetic rate (ABA-2) −0.15
19a SMR (rate) Diadochokinetic rate (ABA-2) 0.24
19b SMR (articulation) Diadochokinetic rate (ABA-2) −0.15
20a _Increased word length (syllable) Increased word length (ABA-2) 0.70**
20b _Increased word length (articulation) Increased word length (ABA-2) 0.70**
21a Repeated trials (syllable) Repeated trials (ABA-2) 0.26
21b Repeated (articulation) Repeated trials (ABA-2) 0.34
Total score of AOS Severity of AOS (ABA-2) 0.43*
*

Correlation significant at the 0.05 level;

**

Correlation significant at the 0.01 level.

Table 9

Summary of reliability and validity of different sections in the proposed screening form

Area Section Internal consistency Inter-rater reliability Construct validity Content validity
Receptive language 1 Orientation questions Good Significant Low CITI Significant

2 Yes/no questions* Good* Significant* Factor 1* Significant*

3 Noun Good Significant Low CITI Significant

4 Verb* Good* Significant* Factor 1* Significant*
5 Simple commands* Good* Significant* Factor 1* Significant*
6 Sequential commands* Good* Significant* Factor 1* Significant*

Expressive language 7 Responsive naming* Good* Significant* Factor 1* Significant*

8 Noun Good Significant Low CITI Significant

9 Verb* Good* Significant* Factor 1* Significant*

10a Repetition (syllable) Good Significant Low CITI Significant

10b Repetition (articulation)* Good* Significant* Factor 1* Significant*
11 Divergent naming* Good* Significant* Factor 1* Significant*

12 Sentence completion Good Significant Low CITI Significant

13a Fluency* Good* Significant* Factor 1* Significant*
13b Information* Good* Significant* Factor 1* Significant*

Dysarthria 14 Intelligibility* Acceptable* Significant* Factor 1* Significant*

15 Respiration in speech Acceptable Non-significant Factor 2 Non-significant

16 Voice quality* Acceptable* Significant* Factor 2* Significant*
17 Volume* Acceptable* Significant* Factor 2* Significant*

Dysarthria and AOS 18a AMR (rate) Acceptable Non-significant Factor 3 Non-significant
18b AMR (articulation) Acceptable Non-significant Factor 1 Non-significant
19a SMR (rate) Acceptable Non-significant Factor 3 Non-significant
19b SMR (articulation) Acceptable Non-significant Factor 1 Non-significant

AOS 20a Increased word length (syllable) Questionable Significant Low CITI Significant
20b Increased word length (articulation) Questionable Significant Factor 1 Significant
21a Repeated trials (syllable) Questionable Non-significant Factor 1 Non-significant
21b Repeated trials (articulation) Questionable Non-significant Factor 1 Non-significant

Low CITI: sections with low corrected item-total correlation coefficients.

*

Sections that are passing all criteria for reliability and validity.