| Home | E-Submission | Sitemap | Contact us |  
top_img
Clinical Archives of Communication Disorders > Volume 2(2); 2017 > Article
Kim and Lombardino: Exploring the Effects of Narration and Pictures on Learning for Sudents with Reading Deficits

Abstract

Although narration is frequently recommended for students with reading deficits, benefits of narration have not been empirically investigated, particularly during the multimedia learning instruction. This study was designed to examine the effect of multimedia instruction on the comprehension of students with and without reading deficits. In experiment 1, the comprehension abilities of grade school and college students with typical reading ability and college students with reading deficits were compared on comprehension tasks using a simultaneous narration+picture presentation. In experiment 2, the multimedia conditions were varied: narration+picture, narration+picture+text, narration-only. The performance of college students with reading deficits was affected differentially by the multimedia presentations. They performed as well as their reading able peers in the narration+picture and narration+ picture+text conditions while they exhibited significantly lower comprehension scores than their peers in the narration-only condition, supporting the premise that multimedia presentations facilitate learning in students who face academic challenges.

INTRODUCTION

Varying forms of media representation have been used to study their efficacy in students’ learning outcomes. To date, data show that multimedia forms are more effective in facilitating learning than a single form representation [1]. Multimedia format can increase the likelihood of information being remembered [2], foster meaningful learning [3], and arouse students’ interest [4]. By and large, literature on learning with multimodal representations has focused on studying populations of typical learners, that is students without learning difficulties [5]. Dallas and Upton [6] suggest that for students with learning difficulties, alternative media and technology has even greater potential. It helps to reduce the effects of students’ deficits on learning, increase student access to information inside and outside the classroom, and decrease their emotional and behavioral problems. In order to maximize students’ academic success and motivation, it is important to know how students with learning difficulties perform in different learning conditions. The aim of the current study was to investigate the effect of multimedia presentation on the comprehension abilities of students with and without reading deficits. Peebles and Cheng’s [7] three factor model for representational comprehension which includes (a) display factor (presentation conditions); (b) person factor (reading abilities, age); and (c) task factor (retention and transfer tasks) was used to guide the methodology of this study.

Display factor: presentation condition

Cognitive Load Theory [8,9], one of the most influential theories in educational research and design, aims to understand how cognitive load influences human learning. It typically addresses the construct of cognitive load with three types of factor that provide the foundation for learning: intrinsic, extrinsic, and germane load. Intrinsic load is internal to the subject, thus, cannot be changed by instructional alternation. It is evoked by the complexity of the subject matter being learned. An example of intrinsic load is the principles that entail a subject matter such as how rain is formed. In contrast, extrinsic load represents outside influences on learning such as modifications in instructional design. An example of extrinsic load would be the use of multimedia interventions, such as text and pictures or narration and picture, in a comprehension task. Finally, germane load represents influences brought to the task by the learner such as the learner’s prior knowledge, efforts, or motivation. Extrinsic factors represent the most malleable ones in the context of teaching and learning. Moreno [10] noted that multimedia instruction provides an excellent example of a methodology that can be used to create alternative learning environments by manipulating extrinsic load.

Person factor: reading abilities and age

Learning disabilities is the largest classification of disabilities [11] and more than 80% of students with learning disabilities experience problems in reading [12,13]. Most of these students are educated in mainstream classrooms without specific instructional accommodations, leaving many at high risk for failure to succeed [14]. Alternative methods of instruction in academic areas that provide these students with more than a single modality (e.g., reading) hold potential for facilitating learning. The most frequently recommended compensatory strategy involves audio recordings of academic materials typically learned through reading [15]. Students with reading deficits can learn better from audio material than reading material because audio material can reduce the extrinsic cognitive load, which frees capacity for intrinsic cognitive load (processing content) and in turn promotes learning. By compensating for the effect of reading deficits on learning, students can access information beyond their reading skill and obtain required curricular content.
With the advance of multimedia technology, text can be accompanied by simultaneous narration along with supplementary visual aids (e.g., pictures, animations, maps). However, there have been a few studies on the effects of multimedia on learning for students with reading deficits and most studies to date have examined the effect of adding visual representations to text on reading comprehension [16]. Berget et al. [5] investigated whether the addition of graphics to text enhanced performance in search task of students with reading deficits. The students performed poorer than typical readers when search targets contained text, but the two groups were not different in the graphic-only condition. Similarly, Kim and Wiseheart [17] found that performance of students with reading deficits was as good as their reading able peers’ performance in the graph reading tasks when the graphs did not contain words, while their performance was lower when the graphs contained words. Holmqvist, Brante, & Nyström [18], however, did not find benefits of pictures for students with reading deficits. They compared learning of students with and without reading deficits in two presentation conditions: text-only and text+picture. Students with reading deficits performed better in the text-only condition than in the text+ picture condition. Even though their learning conditions did not focus on aural representation and the results are mixed, these previous studies clearly demonstrated that performance of students with reading deficits is affected by presentation conditions.
Data on multimedia learning suggests that younger students tend to be less purposeful and strategic than adults in their comprehension of multimodal messages [19]. Children are more likely to view pictures or text as discrete items, instead of integrating the textual and graphical information in context, and they are less equipped with learning strategies necessary for successful learning with multimedia presentation. McTigue [20] explored the extent to which six grade students comprehend science text with and without pictures. The existence of pictured was helpful for children when the text described a familiar concept, but not helpful when the text addressed less familiar concepts. While McTigue used text that was matched to the children’s reading skill level, this linguistic modification did not guarantee successful comprehension of multimodal text. In fact, the addition of pictures may have resulted in added cognitive demands, possibly impeding their learning.

Task factor: retention and transfer

Learning outcomes can be assessed with different types of tasks or questions. With a few exceptions, research on multimedia learning includes at least two different tasks: retention and transfer tasks. Retention task is to measure students’ ability to recall and retrieve relevant information from what they have learned while transfer task is to measure students’ ability to apply what they have learned to a new situation or a new problem. Retention task seems to be easy, however, there are variables that raise the difficulty level. For example, procedural knowledge task, that requires students to describe an activity in a stepwise manner, often involves temporal order and spatial relationships [21]. For example, students can be asked to describe how rain is formed in various steps, instead of recalling a discrete piece of information. Examination of knowledge transfer across problems, settings, or domains is one of the most sensitive measures of learning [22]. Learning occurs when learners actively engage in cognitive processes including paying attention to incoming information, organizing them into coherent representations, and integrating the representations with knowledge [23]. This active learning process is underneath problem-solving transfer. There are often variations in students’ performance between retention and transfer tasks [24]; therefore, we gain a broader insight into multimedia learning by having both tasks to study the efficacy of multimedia instruction for students with reading deficits.

The present study

While the use of narration has been shown to facilitate learning in students with reading deficits, the impact of additional combinations of multimedia interventions have not been studied in this population. Hence, two experiments were designed to explore the effects of multimedia learning instruction for students with reading deficits. The first experiment was designed to examine the extent to which reading skill and age contributed to learning as tested through comprehension in a multimedia condition using simultaneous narration+ picture representations. The second experiment was designed to compare the extent to which reading skill contributed to learning as tested through comprehension across three multimedia formats: narration+picture, narration+picture+text, narration-only conditions.
If narration+picture presentation helps learning of students with reading deficits, their performance should be comparable to their reading able peers’ performance. In order to know whether addition of pictures to narration would improve or rather impede learning for students with reading deficits, in the second experiment, we examined how students with and without reading deficits performed differently across three multimedia instruction conditions: narration+picture, narration+picture+text, and narration-only. If pictures are helpful, students’ performance in the narration+picture and/or in the narration+picture+text condition would be superior to students’ performance in the narration-only condition. Alternatively, if pictures do not affect learning, students’ performance in the three conditions would be similar to each other.

Methods

Participants

All potential participants were individually assessed to ensure that they met the inclusionary criteria for this study. To be included in the reading deficits (RD) group, participants must have (a) reported a history of reading difficulties beginning in childhood and (b) scored at or below one standard deviation of the mean on a standardized test of word level reading. To be included in the typical reading skills (TR) groups (college students or children), participants must have reported an absence of a developmental history of reading difficulties at any time while in school. Their normal reading status was confirmed by their average or above average performance on the tests administered to the students in the RD group.
Following the screening process, 77 students were selected for participation in the study. The final cohorts of participants was composed of 30 college students with typical reading skills (TR adult group; mean age=22.07), 17 college students identified with a reading disability (RD group; mean age= 22.82), and 30 children with typical developing reading skills (TR child group; mean age=10.19). Children in 4-6 grades were chosen for the study because at this stage of development, they are capable of listening to text and processing its meaning [25]. We found no difference in the independent variables (i.e., prior knowledge and standardized scores in verbal and visual abilities) and the dependent variables between grades (4th, 5th, and 6th grades) in the TR child group (ps>.05). All participants were native speakers of English and reported negative histories for pervasive cognitive deficits, behavioral disturbance, neurological illness, psychiatric illness, hearing impairment, or uncorrected visual impairments (Table 1).
All participants were tested with the following standardized measures: Test of Word Reading Efficiency (TOWRE; 26), verbal ability subtest from Woodcock Johnson III Test of Cognitive Abilities (WJ-III-COG; 27), and visual sequential memory from Test of Memory and Learning, second edition (TOMAL-2; 28). An assessment measure of prior knowledge related to the experimental task, developed by Moreno and Mayer [29], was given to all participants to determine their prior meteorological knowledge. This measure required participants to rate each of seven weather-related statements (e.g., “I know what a cold front is”) by checking their answer on a five-point scale (1=very low, 5=very high).
Subject groups’ scores for each of the measures are shown in Table 1. The three groups’ average scores on verbal ability and visual sequential memory fell within one standard deviation of the mean score for their respective ages and no difference was found among the groups’ scores for prior meteorological knowledge. As expected, however, the participants in the RD group had significantly lower mean reading scores than both the typically developing adults and children (p < 0.0001). Furthermore, even though the RD group performed within one standard deviation of the mean for verbal ability, their mean score was significantly lower than the mean score of both the typically developing adults and children (p < 0.001). This profile of lower verbal ability of RD subjects has been reported in literature [3033]. Thus, we report analysis of dependent variable after controlling for participants’ verbal ability.

Experimental materials

Learning materials and comprehension questions used in the present experiment were adapted from Mayer and Johnson [34]. These materials included a verbal narration along with pictures for teaching students a lesson on the formation of lightning. When using the material with adult participants, the experiment included a single lesson, comprising 13 power point slides containing 15 idea units. Each slide included a picture with a concurrent audio clip. The audio clips were recorded by a female using Windows® Movie Maker 2001 [35].
When using the material with the child participants, the same number [15] of idea units, the same number [13] of slides, and the same pictures were used, however, the verbal explanations were shortened so as to be appropriate for the 4th–6th grade children. Readability scores were computed using the Flesch-Kincaid grade-level formula [36]. For the children’s material, the readability score was 89.6 (very easy to read), which indicates that the text was appropriate for students from fourth grade and beyond. For the college students’ material, the readability score of the text was 78.5 (fairly easy to read), which indicates that the text was appropriate for students from 7th grade and beyond. For children, the total lesson took 85 seconds with an averaging viewing of 7 seconds per slide and for the adults, the total lesson lasted 90 seconds with an average viewing of 8 seconds per slide. The slides advanced automatically after the narration was completed.
Prior to the experiment, a pilot study was conducted to test the experimental stimuli and procedures. A total of twelve students (6 college students and 2 fourth graders, 2 fifth graders, and 2 six graders) participated in the pilot study. They rated a collection of pictures and statements for clarity and comprehensibility from 1 (very low) to 5 (very high). Their average scores were all above 4.0 (average score of 4.6). One child reported that she did not know the meaning of ‘droplets’ in the sentence, ‘the cloud is composed of ice crystals and water droplets.’ Therefore, we changed the water droplets to water drops.
Following the slides, the participants were asked five open-ended comprehension questions. The first question was a retention question (“Based on the lesson you just listen to, please describe how lightning is formed as specifically as possible”). This was followed by four transfer questions. The questions were “What could you do to decrease the intensity of lightning?” “Suppose you see clouds in the sky, but not lightning, why not?” “What does air temperature have to do with lightning?” “What causes lightning?” In Mayer’s original task, students read the questions and wrote down their answers. However, in the current study, given the reading difficulties of the participants in the RD group and the age range of younger students in the TR child group, the experimenter read the questions to all participants, recorded the answers that they provided verbally, and transcribed all answers after the experiment was completed.
The maximum score achievable was 15 for the retention task and 8 for the transfer task. For the retention task, the lightning script was broken into 15 idea units, each unit expressing one main event (e.g., the air movements created electrical charges). For the transfer tasks, the author of the original experiment, Mayer, provided a list of acceptable answers and then the first author (Kim) and Mayer discussed the possible additional answer lists. One point was awarded for each correct answer and the points for each question were added to compute the total transfer score (maximum 8 points). Two independent raters who were not aware of the reading group membership of each participant determined the retention and transfer scores. Interrater agreements in scoring was above .90. For discrepancies, a third rater arbitrated and generated a consensus rating.

Procedure

Participants individually completed tasks in the following order: (a) general background and meteorology prior knowledge questionnaire, (b) verbal ability and reading assessment, (c) experimental task, and (d) visual sequential memory assessment. The entire procedure took approximately 45–60 minutes. For the experimental trial, the experimenter presented oral instructions, stating that the student would be presented with an explanation of how the process of lightning works followed by questions based on the material. Participants were seated in front of a computer showing pages repeating the instructions. Then, they were told to wear head-phones. The audio configuration was checked before each experiment and participants were able to adjust the volume control. When they pressed the space bar, the presentation began and screens were automatically advanced. When the presentation was completed, the experimenter asked five questions about the content. No time limit was given to answer the questions.

Results

The goal of this experiment was to assess students’ ability to retain and transfer knowledge from a multimedia lesson (narration+picture) on the formation of lightning by examining their responses to a series of questions following the lesson. Data from 77 students (30 college students with TR; 17 college students with RD; and 30 children with TR) were analyzed to examine multimedia comprehension using Analyses of Covariate (ANCOVAs) with group membership as the independent variable. Retention and transfer responses to the comprehension questions served as the dependent variables and WJ-III-COG verbal ability score was used as a covariate. The verbal score was included to ensure that findings were not attributable to differences in language ability (see Appendix for detailed information on students with reading deficits).
For the retention task, mean scores (standard deviation) were 7.06 (2.96) for college students with RD, 6.47 (2.20) for college students with TR, and 6.50 (2.93) for children with TR (see Figure 1). Group membership (RD group, TR adult group, TR child group) was not significantly related to the retention score, F(2, 73)=1.11, p>0.05. Verbal ability, the covariate, was not significant, F(1, 73)=2.71, p >0.05. For the transfer task, mean scores (standard deviation) were 3.18 (1.42) for college students with RD, 3.47 (1.43) for college students with TR, and 2.37 (1.58) for children with TR. Group membership was significantly related to the transfer score, F(2, 73)=6.02, p=0.003, ηp2 =0.14. Bonferroni post hoc tests revealed that typically developing children had significantly lower scores than the college students with RD, t(73)=−2.48, p=0.04, r=0.35, and college students with TR, t(73)= −3.38, p =0.003, r =0.44. Again, verbal ability was not significant, F(1, 73)=3.25, p>0.05 (Figure 1).
In summary, college students with RD performed as well as their peers with TR when information was presented with both pictures and narration. Interestingly, grade school children with TR performed as well as the college students with and without RD on the retention task, however, their performance was lower in the transfer task than the college students. Findings from experiment 1 lead to more specific questions regarding the advantages of narration for students with RD. Experiment 2 was conducted to compare the performance of college students with and without reading difficulties in two additional multimedia conditions.

EXPERIMENT 2

The purpose of Experiment 2 was to determine whether the different types of multimedia instruction (narration+picture, narration+picture+text, narration-only) would affect learning of students with and without reading deficits.

Method

Participants

For two learning conditions (narration+picture+text and narration-only), a total of 104 college students (62 with TR and 42 with RD) were recruited. Six students (three from the TR group and three from the RD group) were excluded from analyses. Among the remaining 98 students, 47 students (30 students in TR group and 17 in RD group), participated in the narration+picture+text condition and 51 students (29 students in TR group and 22 students in RD group) participated in the narration-only condition. Data from experiment 2 were compared to data from experiment 1 in which the narration+ picture condition was studied (Table 2).
We administered the same test materials that were used in Experiment 1 to the Experiment 2 participants and again, the two groups (RD and TR groups) did not differ in their age, prior knowledge, and visual sequential memory (ps>0.05) as shown in Table 2. As expected, the reading score for the RD group was significantly lower than the reading score for the typical reading group (p <0.0001). Comparing the reading scores of RD groups in the three learning conditions (narration+ picture, narration+picture+text, narration-only) yielded no differences among the three groups (ps>0.05). Similar to experiment 1, while verbal ability of the subjects in the RD group was within the average range, their mean score was significantly lower than the score of the TR group. Again, verbal ability was used as a covariate.

Materials and apparatus

The learning materials and comprehension questions for the lightning formation were identical to those used in the experiment 1. In the narration-only condition, participants listened to the narration in the absence of pictures. In the narration+ picture+text condition, text and narration contained the identical words. The presentation time of three learning materials (narration+picture, narration+picture+text, narration-only) was same for each screen, as screens were automatically advanced by the computer system.

Procedure

As in the experiment 1, participants individually completed tasks in the following order: (a) general background and meteorology prior knowledge questionnaire, (b) verbal ability and reading assessment, (c) experimental task, and (d) visual sequential memory assessment. The entire procedure took approximately 45–60 minutes.

Results

Data from 145 students (47 in narration+picture, 47 in narration+picture+text, 51 in narration-only) were analyzed to examine multimedia comprehension using ANCOVAs with group membership (TR group and RD group) and learning condition (narration+picture, narration+picture+text, narration-only) as the independent variables, retention and transfer responses to the comprehension questions as the dependent variables, and WJ-III-COG verbal ability score as a covariate (Figure 2).
For the retention task, mean scores (standard deviation) for students with RD and students with TR were 7.06 (2.96) and 6.47 (2.20) in the narration+picture condition, 6.00 (2.75) and 6.63 (2.79) in the narration+picture+text condition, and 3.91 (2.94) and 6.66 (2.93) in the narration-only condition, as shown in Figure 2. Group membership (RD or TR group) was not significantly related to the retention score, F(1, 138)=0.43, p>0.05; however, the learning condition was significantly related to the retention score, F(2, 138)=4.62, p=0.01, ηp2 =0.05.
A mean score in the narration-only condition was significantly lower than a mean score in the narration+picture condition t(138)= −1.58, p =0.01. A significant interaction was found for group membership and learning condition, F(2, 138)=4.05, p =0.02, ηp2 =0.06. Bonferroni post hoc tests revealed that scores for the RD group were significantly lower than scores for the TR group in the narration-only condition, t(138)= −2.61, p=0.01, r=0.22. In contrast, the two groups did not differ in their performance in the narration+picture or narration+picture+text conditions, ps>0.05. Furthermore, the RD group’s score in the narration-only condition was significantly lower than the RD group’s score in the narration+ picture condition, t(138)= −3.63, p=0.001, r=0.30, and in the narration+picture+text condition, t(138) =−2.66, p =0.02, r=0.22. Verbal ability, the covariate, was significantly related to the retention task performance, F(1, 138)=7.88, p=0.006, ηp2 =0.05.
For the transfer task, mean scores (standard deviation) for students with RD and TR were 3.18 (1.42) and 3.47 (1.42) in the narration+picture condition, 3.00 (1.36) and 3.53 (2.16) in the narration+picture+text condition, and 1.95 (1.32) and 3.48 (1.73) in the narration-only condition. Neither the group membership nor the learning condition significantly related to the transfer task performance, F(1, 138)=1.31, p>0.05 for group membership and F(2, 138)=2.85, p>0.05 for learning condition. Verbal ability was significantly related to the transfer task performance, F(1, 138)=12.05, p=0.001, ηp2 =0.08.
In summary, students with RD performed as well as their peers with typical reading skills when information was presented in the narration+picture and narration+picture+text formats. By contrast, students with RD performed poorer than their peers when information was presented in the narration-only format. The group difference was largest in the retention task.

DISCUSSION

The purpose of this study was to investigate how multimedia affect learning of students with and without reading deficits. Specifically, in the first experiment, we compared comprehension performance of college students with reading deficits to that of college students and younger children with typical reading skills in studying the effects of age and reading deficits on learning information presented in an narration+picture condition. In the second experiment in which only college students were studied, the learning conditions involved three presentation formats: narration+picture, narration+picture+ text, and narration-only. To measure students’ comprehension, the students’ abilities to both recall and transfer knowledge from the task were examined.
The experiments led to the three key results. Firstly, college students with reading deficits performed as well as their peers when instruction included a narration+picture presentation. This finding is consistent with the previous studies showing that alternative learning methods can improve performance of students who have academic difficulties [3840]. Secondly, children with typical reading skills recalled the same amount of information as college students, however, they were less skillful in applying or transforming their knowledge to solve new problems. Thirdly, comprehension of college students with reading deficits was affected by learning conditions. As stated in the first finding, their performance was comparable to their reading able peers’ in the narration+picture setting. Findings from the second experiment revealed that students with reading deficits showed significantly lower performance than their peers when information was presented in the narration-only condition. In contrast, their performance was as good as typical readers’ performance in the narration+picture and narration+picture+text learning conditions.

Theoretical implications

Multimedia effect of learning refers to enhanced learning when we use more than one concurrent modality for information presentation [41]. In this study, we investigated the impact of multimedia learning by comparing students’ performance in the narration+picture and narration-only conditions. While the previous studies mainly identified the facilitative effect of multimedia instruction in students without learning difficulties [42,43], this study expanded this line of research to investigate the same question with students who have reading deficits. Data from this study showing that students’ learning increased when pictures were added to narration is consistent with Mayer’s [44] cognitive theory of multimedia learning in which he posits that learning through simultaneous visual and verbal channels strengthens cognitive associations for acquiring knowledge. Similarly, Morett, Clegg, Blalock, and Mong [45] studied multimedia effects on map learning and navigation. College students recalled more driving routes and navigated turns more accurately to reach the destination when oral narrative directions and visual maps were presented concurrently than when one modality was presented alone.
In addressing the advantages of cross-modal input, it appears that pictures compensate for the transiency of auditory information. Auditory information is transient in nature and disappears quickly, leaving little time for information processing. The simultaneous presentation with narration and pictures may serve to quickly direct students to the visual presentations while the concepts, that are associated with the visuals, are being described in narration. Meaningful and interactive pictures aid in visualizing the physical appearance of entities. This is particularly helpful for instruction in science knowledge and medical concepts that are often unfamiliar to people and contain abstract and complex concepts and words. In fact, Houts, Doak, Doak, and Loscalzo [46] reviewed the role of pictures in health education and found that pictures facilitate the comprehension of medical information presented verbally. Furthermore, students tend to report audio-visual presentations as more enjoyable and interesting [47]. They prefer stories with pictures to ones without pictures. When asking students with reading disabilities about their preferred learning style, Beacham and Alty [48] found that the majority (97%) of students preferred a visual-verbal form (diagram+verbal presentation) to a verbal-only form. However, as noted by Houtes et al. [46] and Massa and Mayer [49], picture preference is affected by cultural factors and picture characteristics as well as personal characteristics and does not necessarily lead to better performance. Therefore, further investigation is needed to connect the cognitive learning styles to performance improvement for students with deficits.
In contrast to the students with reading deficits, the superior performance on the narration+picture condition over the narration-only condition was not observed in the students with typical reading skills. Their performance was similar across the three conditions, unlike the students with reading deficits. This data suggest that the auditory information in the current study may not be challenging to the students with typical reading skills. The transiency of auditory information is particularly problematic when verbal information is difficult [50]. Carney and Levin [47] found that the more complex the text, the more likely the students benefit from concurrent pictures. In addition to the difficulty level of the text, function of picture can be the reason. In the current study, pictures did not contain novel information but instead depicted key concepts described in text. Thus, it was not necessary to understand pictures to answer the comprehension questions. Leahy, Chandler, and Sweller [51] compared students’ comprehension in two conditions, when audio and visual material provided different information and when audio and visual material provided redundant information. When the information from the two media was different, multimedia format (audio+video) was beneficial. In contrast, when the information was redundant, multimedia format did not help students’ comprehension.

Practical implications

One of the most frequently recommended accommodations for students with reading deficits consists of using audiobooks [52]. This is a common recommendation in this age of technology when audio versions of text is readily available (e.g., text-to-speech software and reading pens). Indeed, the use of audio materials can be powerful since students with reading deficits have the opportunity not only to build required content knowledge, vocabulary, and comprehension but also to learn plot structures, themes, and textual cohesion, by removing the restraints of decoding skills [53]. Based on findings from this study, however, the provision of audio material alone does not necessarily facilitate novel knowledge in students with reading deficits. This implies that without teaching appropriate learning strategies, use of audio materials may not result in the expected outcome.
An interesting study by Kombartzky, Ploetzner, Schlag, and Metz [54] gives us insight into how to teach learning strategies in multimedia environments. They compared three groups’ learning from multimedia materials. The first group learned from material without strategy, the second group learned from material with strategies (using a worksheet to summarize content during and after listening to narration) but use of the strategies was not monitored. The third group learned from material with strategies and their strategy use was monitored. Both the second and third group learned significantly more than the first group, but learning was most successful when the use of strategies was monitored. The researchers explained that at first, using instructional strategies can be demanding and might even temporarily impede learning. Therefore, students may not voluntarily use the strategies. Until the learned strategies become internalized, students need to be closely monitored and encouraged. This explanation might account for the reason that Boyle et al. [15] did not find differences in the performance of a group learning from audio textbooks alone and a group learning from audio textbook combined with a complementary strategy (using worksheet to take notes about important information). The authors speculated that the strategies demanded a great deal of mental effort because students did not have any previous training in using learning strategies and correspondingly, they had difficulty synthesizing the relevant information of paragraphs into the worksheet. Further studies are needed to explore how students are interacting with the learning strategies for audio material, particularly for students in secondary and post-secondary educational settings given the dearth of research with older students [53].

Limitations and future direction

Although this study found that learning of postsecondary students with reading deficits is affected by multimedia instruction that included narration, there are a few limitations that are worth noting and more work is needed in conceptualizing and measuring the effects of multimedia learning for students with reading deficits. First, this study adopted system-paced multimedia presentations. Students neither controlled the speed of presentation nor repeated the presentation. An important next step is to test how students with reading deficits will perform on multimedia instruction in learner-paced multimedia presentations. Van den Broek, Segers, and Verhoeven [55] pointed out the possibility of differences in learners’ performance on the system-paced versus learner-paced learning instruction. Next, even though our participants were not officially diagnosed with attention deficits, their attention or memory span might still affect their performance, especially in the narration-only condition. Further research should employ direct measures of specific cognitive abilities including attention and memory. Finally, as noted above, pictures of this study have a limited function, describing the key elements of statements verbally provided. It will be interesting to study how students with reading deficits perform when visual and verbal multimedia material are representing different information. In this regard, a systematic classification of relationships between images and words, as noted by Kong [56], could serve as guide for developing subtypes of pictorial images for use in multimedia materials.

CONCLUSION

It is a misunderstanding that technology can unconditionally improve learning of students. An excessive dependence on new technology raises the risk of malfunction in teaching, learning, and student success. To prevent this result, the choice of multimedia and instructional methods should be grounded in theoretically-developing and empirically-testing models [57,58]. The work presented here demonstrated that students with reading deficits can benefit from multiple forms of information representation, particularly when pictures combined with verbal information. Across two experiments, our results consistently demonstrate that students with reading deficits learn through multiple external representations as well as their peers without reading deficits. In addition, our study indicated that just providing audio material does not necessarily improve performance of students with reading deficits. Spoken verbal information is an important tool in instruction for both traditional and high-tech, computer-based learning environment. The present study shows that multimedia approach assisting spoken information has a great potential for academic success of students with reading deficits.

Notes

Due to the unequal sample size, we followed Rosnow, Rosenthal, and Rubin (37) for the effect size calculation.

Figure 1.
Mean scores for college students and children on the retention (top) and transfer (bottom) tasks in Experiment 1. Error bars represent standard error of the mean. RD, reading deficits; TR, typical reading skills. *p<0.05.
cacd-2-2-116f1.gif
Figure 2.
Mean scores for college students with and without reading deficits on the retention (top) and transfer (bottom) tasks in Experiment 2. Error bars represent standard error of the mean. RD, reading deficits; TR, typical reading skills. *p<0.05.
cacd-2-2-116f2.gif
Table 1.
Characteristics of Participants with and without Reading Deficits in Experiment 1
College students with RD College students with TR Children with TR F statistics (2, 75) p value
Age (year) 22.82 (6.50) 22.07 (5.47) 10.19 (0.94) 65.51 <0.0001
Prior knowledge questionnaire (maximum score=12) 5.82 (1.87) 5.03 (1.92) 4.42 (2.14) 2.72 0.07
TOWRE Total Word Reading Efficiency (SS) (average=100) 80.88 (2.84) 100.90 (8.31) 101.51 (14.15) 25.33 <0.0001
WJ-III-COG verbal ability (SS) (average=100) 89.88 (9.16) 96.33 (7.71) 107.13 (14.78) 14.93 <0.001
TOMAL-2 Visual Sequential Memory (SS) (average=10) 10.35 (2.69) 11.17 (1.78) 11.16 (2.47) 0.83 0.43

Note. RD, reading deficits; TR, typical reading skills; TOWRE, Test of Word Reading Efficiency; WJ-III-COG, Woodcock Johnson III Test of Cognitive Abilities; TOMAL-2, Test of Memory and Learning, second edition; SS, Standard Score.

Table 2.
Characteristics of Participants with and without Reading Deficits in Experiment 2
College students with RD (N=56) College students with TR (N=89) F statistics (1, 143) p value
Age (year) 21.14 (4.42) 21.67 (5.08) 0.33 0.59
Prior knowledge questionnaire (maximum score=12) 5.30 (1.77) 5.04 (1.93) 0.33 0.56
TOWRE Total Word Reading Efficiency (SS) (average=100) 78.16 (5.26) 98.78 (7.62) 365.09 <0.0001
WJ-III-COG verbal ability (SS) (average=100) 88.20 (11.03) 97.61 (8.98) 28.20 <0.001
TOMAL-2 Visual Sequential Memory (SS) (average=10) 10.00 (2.62) 10.86 (2.40) 3.53 0.06

Note. RD, reading deficits; TR, typical reading skills; TOWRE, Test of Word Reading Efficiency; WJ-III-COG, Woodcock Johnson III Test of Cognitive Abilities; TOMAL-2, Test of Memory and Learning, second edition; SS, Standard Score.

REFERENCES

1. Ginns P. Meta-analysis of the modality effect. Learn Instr. 2005;Aug;15(4):313–331.
crossref
2. Goolkasian P, Foos PW. Presentation format and its effect on working memory. Mem Cognit. 2002;30(7):1096–1105.
crossref pmid
3. Mayer RE, Chandler P. When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? J Educ Psychol. 2001;93(2):390–397.
crossref
4. Mayer RE. Incorporating motivation into multimedia learning. Learn Instr. 2014;29:171–173.
crossref
5. Berget G, Mulvey F, Sandnes FE. Is visual content in textual search interfaces beneficial to dyslexic users? Int J Hum-Comput Stud. 2016;92:17–29.
crossref
6. Dallas BK, Upton TD. Maximizing access to postsecondary educational print materials for students with print-related disabilities. J Appl Rehabil Couns. 2011;42(1):35–42.
crossref
7. Peebles D, Cheng PCH. Modeling the effect of task and graphical representation on response latency in a graph reading task. Hum Factors. 2003;45(1):28–46.
crossref pmid
8. Sweller J. Cognitive load during problem solving: Effects on learning. Cogn Sci. 1988;12(2):257–285.
crossref
9. Sweller J, Ayres P, Kalyuga S. Cognitive load theory. 1. New York: Springer, 2011.

10. Moreno R. Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instr Sci. 2004;32(1/2):99–113.
crossref
11. Cortiella C, Horowitz DH. The state of learning disabilities: Facts, trends and emerging issues. New York: National Center for Learning Disabilities, 2014.

12. Kavale KA, Reese JH. The character of learning disabilities: An Iowa profile. Learn Disabil Q. 1992;15(2):74–94.
crossref
13. Stanovich KE. Progress in understanding reading: Scientific foundations and new frontiers. New York: Guilford Press, 2000. p. xxiiip. 536.

14. Maccini P, Gagnon JC, Hughes CA. Technology-based practices for secondary students with learning disabilities. Learn Disabil Q. 2002;25(4):247–261.
crossref
15. Boyle EA, Rosenberg MS, Connelly VJ, Washburn SG, Brinckerhoff LC, Banerjee M. Effects of audio texts on the acquisition of secondary-level content by students with mild disabilities. Learn Disabil Q. 2003;26(3):203–214.
crossref
16. Rau MA, Aleven V, Rummel N. Successful learning with multiple graphical representations and self-explanation prompts. J Educ Psychol. 2015;107(1):30–46.
crossref pdf
17. Kim S, Wiseheart R. Exploring text and icon graph interpretation in students with dyslexia: An eye-tracking study. Dyslexia. 2017;23(1):24–41.
crossref pmid
18. Holmqvist MO, Brante EW, Nyström M. The effect of illustration on improving text comprehension in dyslexic adults. Dyslexia. 2017;23(1):42–65.
crossref pmid pmc
19. Hubisz JL. Report on a study of middle school physical science texts. Phys Teach. 2001;39(5):304–309.
crossref
20. McTigue EM. Does multimedia learning theory extend to middle-school students? Contemp Educ Psychol. 2009;34(2):143–153.
crossref
21. Brunyé TT, Taylor HA, Rapp DN, Spiro AB. Learning procedures: The role of working memory in multimedia learning experiences. Appl Cogn Psychol. 2006;20(7):917–940.
crossref
22. Mayer RE. Rote versus meaningful learning. Theory Pract. 2002;41(4):226–232.
crossref
23. Mayer RE. The promise of multimedia learning: using the same instructional design methods across different media. Learn Instr. 2003;13(2):125–139.
crossref
24. Mariano GJ, Doolittle PE, Hicks D. Fostering transfer in multimedia instructional environments. In : Zheng R, editor. Cognitive effects of multimedia learning. Hershey, PA: IGI Global, 2009. p. 237–259.
crossref
25. Geffner D, Ross-Swain D. Auditory processing disorders: Assessment, management and treatment. San Diego, CA: Plural Publishing, 2012. p. 905.

26. Torgesen , Wagner R, Rashotte C. Test of Word Reading Efficiency. Austin, TX: Pro-ED, 1999.

27. Woodcock R, McGrew K, Mather N. Woodcock-Johnson III tests of cognitive abilities. Itasca, IL: Riverside Publishing Company, 2002.

28. Reynolds CR, Voress JK. Test of Memory and Learning-Second Edition (TOMAL-2). PRO-ED, 2007.

29. Moreno R, Mayer RE. Verbal redundancy in multimedia learning: When reading helps listening. J Educ Psychol. 2002;94(1):156–163.
crossref
30. Catts HW, Fey ME, Zhang X, Tomblin JB. Language basis of reading and reading disabilities: Evidence from a longitudinal investigation. Sci Stud Read. 1999;3(4):331–361.
crossref
31. Lombardino LJ, Riccio CA, Hynd GW, Pinheiro SB. Linguistic deficits in children with reading disabilities. Am J Speech Lang Pathol. 1997;6(3):71–78.
crossref
32. Pennington BF, Bishop DVM. Relations among speech, language, and reading disorders. Annu Rev Psychol. 2009;60(1):283–306.
crossref pmid
33. Ramus F, Marshall CR, Rosen S, van der Lely HKJ. Phonological deficits in specific language impairment and developmental dyslexia: towards a multidimensional model. Brain J Neurol. 2013;136(2):630–645.
crossref pmid pmc pdf
34. Mayer RE, Johnson CI. Revising the redundancy principle in multimedia learning. J Educ Psychol. 2008;100(2):380–386.
crossref
35. Microsoft Corporation. Windows movie maker [Internet]. 2001. Available from: www.microsoft.com/windowsxp/using/movie-maker/default.mspx.

36. Kincaid JP, Fishburne J, Rogers RL, Chissom BS. Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel (Research Branch Report 8-75). Memphis, TN: Chief of Naval Technical Training, Naval Air Station Memphis, 1975.

37. Rosnow RL, Rosenthal R, Rubin DB. Contrasts and correlations in effect-size estimation. Psychol Sci. 2000;11(6):446–453.
crossref pmid
38. Sorrell CA, Bell SM, McCallum RS. Reading rate and comprehension as a function of computerized versus traditional presentation mode: A preliminary study. J Spec Educ Technol. 2007;22(1):1–12.
crossref
39. Torgesen JK, Dahlem WE, Greenstein J. Using verbatim text recordings to enhance reading comprehension in learning disabled adolescents. Learn Disabil Focus. 1987;3(1):30–38.

40. Wood SG, Moxley JH, Tighe EL, Wagner RK. Does use of text-to-speech and related read-aloud tools improve reading comprehension for students with reading disabilities? A meta-analysis. J Learn Disabil. 2017;23:1–12.
crossref pmid pmc
41. Clark RC, Mayer RE. e-Learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. 4th ed. Hoboken, New Jersey: John Wiley & Sons, 2003.

42. Dunsworth Q, Atkinson RK. Fostering multimedia learning of science: Exploring the role of an animated agent’s image. Comput Educ. 2007;49(3):677–690.
crossref
43. She H-C, Chen Y-Z. The impact of multimedia effect on science learning: Evidence from eye movements. Comput Educ. 2009;53(4):1297–1307.
crossref
44. Mayer RE. Multimedia learning. 2nd ed.New York, NY: Cambridge University Press, 2009. p. 318.

45. Morett LM, Clegg BA, Blalock LD, Mong HM. Applying multimedia learning theory to map learning and driving navigation. Transp Res Part F Traffic Psychol Behav. 2009;12(1):40–49.
crossref
46. Houts PS, Doak CC, Doak LG, Loscalzo MJ. The role of pictures in improving health communication: A review of research on attention, comprehension, recall, and adherence. Patient Educ Couns. 2006;61(2):173–190.
crossref pmid
47. Carney RN, Levin JR. Pictorial illustrations still improve students’ learning from text. Educ Psychol Rev. 2002;14(1):5–26.
crossref
48. Beacham NA, Alty JL. An investigation into the effects that digital media can have on the learning outcomes of individuals who have dyslexia. Comput Educ. 2006;47(1):74–93.
crossref
49. Massa LJ, Mayer RE. Testing the ATI hypothesis: Should multimedia instruction accommodate verbalizer-visualizer cognitive style? Learn Individ Differ. 2006;16(4):321–335.
crossref
50. Wong A, Leahy W, Marcus N, Sweller J. Cognitive load theory, the transient information effect and e-learning. Learn Instr. 2012;22(6):449–457.
crossref
51. Leahy W, Chandler P, Sweller J. When auditory presentations should and should not be a component of multimedia instruction. Appl Cogn Psychol. 2003;17(4):401–418.
crossref
52. Milani A, Lorusso ML, Molteni M. The effects of audiobooks on the psychosocial adjustment of pre-adolescents and adolescents with dyslexia. Dyslexia. 2010;16(1):87–97.
crossref pmid
53. Wolfson G. Using audiobooks to meet the needs of adolescent readers. Am Second Educ. 2008;36(2):105–114.

54. Kombartzky U, Ploetzner R, Schlag S, Metz B. Developing and evaluating a strategy for learning from animations. Learn Instr. 2010;20(5):424–433.
crossref
55. Van den Broek GSE, Segers E, Verhoeven L. Effects of text modality in multimedia presentations on written and oral performance. J Comput Assist Learn. 2014;30(5):438–449.
crossref
56. Kong KCC. A taxonomy of the discourse relations between words and visuals. Inf Des J IDJ. 2006;14(3):207–230.
crossref
57. Kaltenbacher M. Perspectives on Multimodality: From the early beginnings to the state of the art. Inf Des J Doc Des. 2004;12(3):190–207.
crossref
58. Neto JD de O, Huang WD, Melli NC de A. Online learning: audio or text? Educ Technol Res Dev. 2015;63(4):555–573.
crossref

Appendices

Appendix.

Characteristics of students with reading deficits in Experiment 1

Narration+picture condition

Age (year) TOWRE reading score Prior knowledge questionnaire Verbal ability Visual sequential memory
Student with RD 1 27 79 5 94 11
Student with RD 2 21 80 7 100 10
Student with RD 3 24 83 5 87 11
Student with RD 4 19 82 2 90 11
Student with RD 5 20 81 7 87 15
Student with RD 6 19 79 8 69 15
Student with RD 7 21 81 4 105 13
Student with RD 8 22 73 8 105 11
Student with RD 9 19 83 5 85 12
Student with RD 10 30 83 9 80 9
Student with RD 11 45 78 8 88 10
Student with RD 12 19 78 7 87 5
Student with RD 13 18 83 4 84 8
Student with RD 14 20 84 5 83 11
Student with RD 15 19 83 5 99 8
Student with RD 16 21 81 6 92 6
Student with RD 17 24 84 4 93 10
Narration+picture+text condition

Age (year) TOWRE reading score Prior knowledge questionnaire Verbal ability Visual sequential memory
Student with RD 1 26 79 4 57 6
Student with RD 2 19 78 3 86 13
Student with RD 3 19 79 3 86 5
Student with RD 4 18 84 7 115 10
Student with RD 5 18 77 4 86 9
Student with RD 6 19 69 8 58 14
Student with RD 7 22 63 4 87 9
Student with RD 8 21 80 4 96 10
Student with RD 9 19 77 4 72 8
Student with RD 10 22 69 3 86 15
Student with RD 11 19 81 6 78 12
Student with RD 12 20 82 5 87 10
Student with RD 13 19 69 5 94 11
Student with RD 14 22 80 9 90 7
Student with RD 15 20 81 7 95 9
Student with RD 16 20 82 4 94 9
Student with RD 17 20 78 6 95 10
Narration-only group

Age (year) TOWRE reading score Prior knowledge questionnaire Verbal ability Visual sequential memory
Student with RD 1 20 72 3 69 7
Student with RD 2 22 69 4 94 8
Student with RD 3 20 80 6 115 12
Student with RD 4 18 83 6 96 11
Student with RD 5 18 81 5 75 12
Student with RD 6 23 69 4 99 12
Student with RD 7 22 83 7 90 7
Student with RD 8 30 68 8 82 9
Student with RD 9 20 82 4 92 13
Student with RD 10 19 75 3 90 13
Student with RD 11 18 78 3 95 8
Student with RD 12 19 81 2 80 13
Student with RD 13 20 81 6 94 7
Student with RD 14 23 74 5 94 12
Student with RD 15 25 81 8 84 11
Student with RD 16 20 84 7 98 13
Student with RD 17 19 80 5 86 8
Student with RD 18 19 84 5 92 11
Student with RD 19 20 70 7 76 11
Student with RD 20 20 77 4 83 5
Student with RD 21 18 74 5 83 9
Student with RD 22 20 68 5 82 5

Note. Verbal ability score was measured using WJ-III-COG (Woodcock Johnson III Test of Cognitive Abilities); Visual sequential memory was measured using TOMAL-2 (Test of Memory and Learning, second edition). RD, reading deficits; TOWRE, test of word reading efficiency.

Editorial Office
#409, 102 SK-Hub BULD, 461 Samil-daero, Jongno-gu, Seoul 03147, Korea
FAX: +82-2-795-2726   E-mail: editor@e-cacd.org
About |  Browse Articles |  Current Issue |  For Authors and Reviewers
Copyright © The Korean Association of Speech-Language Pathologists.                 Developed in M2PI