Colleagues: Has anyone had an opportunity to use the NIFL reports published in 2010 entitled Making Sense of Decoding and Spelling: An Adult Reading Course of Study written by Charles A. MacArthur, Judith A. Alamprese, and Deborah Knight? The reports include a teacher’s manual, a course of study, and a learner’s activity book. I think this course of study was a product of one of the six five year research projects funded with $18.5 million in the 2002. I’m wondering if anyone has used the course as it is designed and if so how it worked out.
The reports state that the course of study was tested in an experimental design and produced significantly higher gains than the comparison program. Does anyone know if there is a technical report of this study? Any reference(s) will be appreciated.
Tom Sticht
Comments
I am currently using this course with one of the students I tutor. Both of us are very pleased with the course. My student likes the structure and finds the activities very helpful. He says he is learning things he was never taught before. I highly recommend it.
Hello Tom,
I would like to encourage you and anyone interested to join us on April 16, 2013 on a webinar where Dr. Charles MacArthur will be talking about that exact curriculum. https://community.lincs.ed.gov/notice/save-date-introducing-making-sense-decoding-and-spelling-curriculum-webinar-%E2%80%93-4-16-2013
Dorjan: i can't do the 16 Apr webinar. Do you know if there are technical reports available from the Dept of Ed for the course of reading study? i have found some references to journal articles but no tech report. Would appreciate references to any online research reports so I can judge the design and methodology of the study which I understand provides some evidence for the effectiveness of the course of study.
And Di, thanks for your input. Have you compared the course of study with any other of the available materias for teaching decoding in adult literacy instruction?
Tom Sticht
Hi Tom,
I've used some other strategies but haven't really done a comparison or used any published materials.
Tom,
The main report on the experimental study was published in JREE. Here is the full reference. It is available to the public through the NIH database.
Skip
Alamprese, J. A., MacArthur, C. A., Price, C., & Knight, D. (2011). Effects of a structured decoding curriculum on adult literacy learners' reading development. Journal of Research on Educational Effectiveness, 4, 152-174. NIHMSID: NIHMS337814.
PS. We also published studies on the construct valdity of the measures and on subgroups, both in JLD.
MacArthur, C. A., Konold, T., Glutting, J., & Alamprese, J. (2010). Reading component skills of learners in adult basic education. Journal of Learning Disabilities, 43, 108-121. NIHMSID: NIHMS328878
MacArthur, C. A., Konold, T., Glutting, J., & Alamprese, J. (2012). Subgroups of adult basic education learners with different profiles of reading skills. Reading and Writing, 25, 587-609. (DOI: 10.1007/s11145-010-9287-2) NIHMSID: NIHMS328881
Skip: Thanks for the research citations. I got the article describing the research evidence for the effectiveness of the structured decoding curriculum entitled Effects of a Structured Decoding Curriculum on Adult Literacy Learners’ Reading Development by Judith A. Alamprese, Charles A. MacArthur, Cristofer Price, and Deborah Knight. The Abstract states: “The study found a small but significant effect on one measure of decoding skills, which was the proximal target of the curriculum. No overall significant effects were found for word recognition, spelling, fluency, or comprehension. Pretest to posttest gains for word recognition were small to moderate, but not significantly better than the control classes. “
If I understand the rationale for teaching decoding it is to improve word recognition and based on that recognition one should be able to comprehend sentences and passages better. But the structured decoding curriculum apparently did not lead to significant improvements in word recognition or reading comprehension over that of the control and comparison groups. Is that correct? <?xml:namespace prefix = o />
While I was reading the report I thought that it would have been interesting to see distributions of pre- and post-scores instead of just gain scores. I wondered if those at the lower levels of the distribution on the pre-test gained more than those at the higher ends. Did you notice whether any of the adults had difficulty with comprehending what they read in daily life? Sometimes standardized tests use pretty esoteric prose and that makes it difficult for adults with less extensive vocabulary knowledge to comprehend what they are reading on the tests.
I also thought that it was not really clear what the experimental treatment was being contrasted with. The report describes the diversity of the curricula (or lack of one) in the control and comparison groups. But what the actual curriculum was in the control and comparison groups wasn’t clear to me. This is a perennial problem with the experimental/control comparison research method.
There are a number of other structured decoding programs/curricula available and it would be difficult (and expensive!) to do multiple experimental/control studies for all permutations of such studies among the various programs (e.g., Lindamood-Bell; Barton, McGinness; Moat, structured decoding curriculum, etc).
At the present time, with the data presented in the referenced study, I don’t know how one could decide that he or she should change what they are doing with regard to decoding education and switch to using the structured decoding curriculum.
Tom Sticht
Tom,
Thanks for the questions. It is true that for the full sample, the only statistically significant result was for Word Attack. However, for the US-born/educated sample (67% of the participants), we did get statistically significant gains in Word Recognition and Spelling as well. Thus, we got gains in all areas that the curriculum covered. That's why I tried to be cautious about recommendations for using the curriculum with non-native speakers. (Non-US born/educated also gained less in vocabulary than controls.)
As for the control group, keep in mind that only the treatment and control groups were part of the true randomized experiment. We selected ABE programs that met criteria for overall quality of reading program and randomly assigned them to treatment and control conditions. The controls continued with their business as usual. The comparison group was not randomly assigned; it was a group of programs/teachers that had been using other decoding curricula for some time; they were identified as part of a previous descriptive study by Abt. The randomized part of the study followed rigorous procedures for randomization, including using data from all programs and students regardless of level of implementation of the curriculum (intent to treat analysis).
Re the statistics, we reported standardized gain scores, but we did control for pretest scores. We also looked for interactions with pretest scores, but it is difficult to separate such interactions from the problem of regression to the mean (regardless of statistical procedure).
In the end, if programs have something that is working for them, I would not recommend changing. If not, I think our curriculum is a good option.
Skip
Skip: I agree that no particular recommendation can be made presently from the Making Sense research. But I think the research is consistent with the idea that people are more likely to learn what they are taught than what they are not taught! Dressing this folksy saying up a bit, it seems consistent with what Steve Reder calls more gloriously "practice engagement: theory": i.e., people who engage in certain literacy practices more often are likely to get better at those types of practices! Or, stated otherwise and more briefly, practice makes perfect!! Thanks again for the references.
Tom Sticht
Tom,
I'm not sure what part of my response led you to think that I don't recommend use of Making Sense. For US born students, we found statistically significant and educationally meaningful gains in Word Recognition and Spelling. This in a rigorously designed randomized study on a national scale. Instructors should be cautious in how they use it with non-native speakers, in particular regarding vocabulary instruction. I think it is important to teach decoding, and that our curriculum is a good way to do it. It's not the only way to do it, but as far as I know, it's the only one with this level of evidence with adults.
Skip