Colleagues,
Have you read the September 14, 2012 New York Times Education Issue article "The Machines are Taking Over"? http://tinyurl.com/9dbut9w
It's mostly about the development of one (free) computer tutoring program in math, called ASSISTments.
Here are a few excerpts to whet your interest:
In a 1984 paper that is regarded as a classic of educational psychology, Benjamin Bloom, a professor at the University of Chicago, showed that being tutored is the most effective way to learn, vastly superior to being taught in a classroom. The experiments headed by Bloom randomly assigned fourth-, fifth- and eighth-grade students to classes of about 30 pupils per teacher, or to one-on-one tutoring. Children tutored individually performed two standard deviations better than children who received conventional classroom instruction — a huge difference.
ASSISTments incorporates many of the findings made by researchers who, spurred by the 1984 Bloom study, set out to discover what tutors do that is so helpful to student learning. First and foremost, they concluded, tutors provide immediate feedback: they let students know whether what they’re doing is right or wrong. Such responsiveness keeps students on track, preventing them from wandering down “garden paths” of unproductive reasoning.
The second important service tutors provide, researchers discovered, is guiding students’ efforts, offering nudges in the right direction. ASSISTments provides this, too, in the form of a “hint” button.
Dealing with emotion — helping students regulate their feelings, quelling frustration and rousing flagging morale — is the third important function that human tutors fulfill. So Heffernan [the creator of ASSISTments], along with several researchers at W.P.I. and other institutions, is working on an emotion-sensitive tutor: a computer program that can recognize and respond to students’ moods.
Some researchers, like Ken Koedinger, a professor of human-computer interaction and psychology at Carnegie Mellon University, take a different view from Heffernan’s: computerized tutors shouldn’t try to emulate humans, because computers may well be the superior teachers
What do you think of this article? What are your thoughts about using digital tutors? Have you tried ASSISTments? If so, what do you think?
David J. Rosen
djrosen123@gmail.com
Comments
I applied for an account a while but haven't heard... so I am thinking the NY Times article has 'em swamped.
The title was a little bit misleading; the article included lots of caveats that I appreciated. Human tutors can discern that lost expression that a computer can't. Assistments have "hints" for a person to click on, but different students need different hints.
Also, from the little bit I could tell from what's available to the public, it still uses the instructional philosophy of: "Here's a problem. Try to do it. If you don't know what to do, we'll give you hints."
That is extremely (and painfully) different from teaching.
And, finally, the problems, hints and instructions are all symbolic and procedural... that bridge to the concrete is missing.
Looks like I'd better finish up my little assessment for our "concrete conceptual" math module -- I keep hoping somebody else will do it first ...
When I used to give daily orientation sessions for online learners my constant refrain was, "let the computer be your tutor." Usually, it was in response to a learner saying that they needed more one-on-one help or that they wanted to move toward their study goal more quickly rather than waiting for the whole class to follow the same lesson plans. The same was true with teachers who wanted online learning to lift some of the burden of their workload. Or sometimes both students and teachers thought we were offering live online classes and I'd clafify saying "the computer is the tutor" before talking about how the management systems of our various programs tracked student progress. My slogan never really took off, but maybe it was ahead of its time (or just an anoying rhyme!).
It wasn't until starting with Essential Education and hearing how GED Academy "acts like a personal tutor" that I realized that this framework of a 'computer as tutor' needs some meat on its bones if it's going to resonate. And I think that time is coming. Computer as tutor means adjusting to the learner's needs, and providing real time feedback, both of which I see in this Assistments math program. GED Academy and Assistments share that approach of constantly assessing the learner's abilities (only using very different analytics and algorhythms).
But, I think Susan is getting at the most important part of the computerized tutoring. It's got to feel more personal. The content/instruction has got to be something they an relate to, that addresses misconceptions and validates prior knowledge. And personally, where adult are concerned, it's got to give more control to the learner so they have the freedom to explore. GED Academy saw those needs over the years and tries to meet them. In the long run, I think it's reasonable for adult learners to expect that computer programs will provide tutoring that is customized to their needs. There's a demand. So let's meet it.
I have not tried ASSISTments and would love to hear more from people who have.
However, I've worked with several different math programs in a hybrid classroom setting, and I definitely agree that immediate feedback is essential, and keeps students motivated. A site that provides a good example of this is IXL. Even though this site is geared towards K-8, its feedback, rewards, and reporting mechanisms are working really well for my adult basic math students, and also make it easy for me to see where people are getting stuck. It also varies the difficulty of the questions based on the student's responses, and has a nice algorithm for determining their "score"--as they start a new skill with a 0 score, students gain 10 points for each correct answer and lose only 1 for each wrong answer. As they progress within a skill, this gradually reverses, and as they approach a score of 100, they'll lose 10 points for a wrong answer, while only earning 1 point for a correct answer. Students quickly notice this and begin to develop an understanding of and desire for "mastery," rather than just answering lots of questions as quickly as possible.
Although this site does offer explanations to missed questions, it does not aim to be an online tutor--I tell students to use it as an online practice/homework site to reinforce the skills we're covering through paper and online math curricula (including an NROC Developmental math course that uses videos, worked examples, etc. from Khan Academy).
What I've found working with ABE math students is that they get far more out of computer instruction if they ALSO have access to instructors, or even each other, to ask questions when they are confused or are getting wrong answers. Alot of the things that trip up my students, like substituting a comma for a decimal point in their answers, are not typically "explained" by hints or online explanations, and can lead to a lot of frustration. So what I do is present a lesson and we work some online problems together. Then people get to practice on their own with me and some volunteers circulating to help answer questions and keep an eye on signs of frustration (or of work being too easy). Then we reconvene as a group and I invite students to show any problems they missed that they still don't understand (IXL makes it easy to display any student's missed problems). These group sessions seem particularly effective in that by having someone "think out loud" we can often pinpoint what's confused them much better than an algorithm can. And other students seem to really like this too. We get a lot of issues clarified as a class that would otherwise be stumbling blocks for several participants if they were trying practice new skills online by themselves. And I encourage, but do not require, students to try online practice outside of class. Many students took advantage of this last year, but it helped knowing that they could return to class and ask questions if they started floundering in the online practice.
Thanks for your post, Wendy. Your description of how your students use math tutoring programs, in the context of face-to-face learning where there is an expert instructor, is very interesting to me. I am hoping that this Community of Practice (and perhaps some other new LINCS CoPs) might be able to form sub-groups of experienced teachers -- perhaps like you -- who want to give certain topics a real workout, not necessarily focusing on what external research tells us, but on what teachers' experience tells us. A group like this would need a leader and a few dedicated members who commit to writing about their practice over time, focusing on whatever topics they were especially interested in; it might also have as its goal writing some teacher practice pieces that described clearly, succinct;y and with actual examples, what good practice in that particular area -- for example, math tutoring -- looks like.
One model for this online subgroup, inspired I believe by John Holt, was a face-to-face teacher group that met over many years in the 1970's and 1980's in Cambridge, Massachusetts. These were elementary school teachers who had gathered initially, I think, just to talk and learn about each others' classrooms. The group had teacher leaders who suggested that between meetings each teacher should reflect and write in detail something to share with others about learning challenges and opportunities that they were experiencing in their classroom. After awhile the teachers realized that there were common issues that they wanted to explore together. Some of these explorations led to research, especially to teacher research (action research) in their classrooms. This was a voluntary effort by experienced teachers who wanted to do teacher-to-teacher professional development with their colleagues, to go deep on challenging classroom issues. It resulted in rich, contextual writing about teaching and learning. I believe that this is needed in adult literacy (and of course numeracy and other basic skills and English language) education and I wonder, now that national online communities of practice are possible, if teachers in our field are interested in doing this as an online community.
One of my hopes is that this will lead to a body of writings that might become recognized as adult literacy education professional wisdom, a neglected part of the U.S. Department of Education's definition of evidence-based learning.
David J. Rosen
djrosen123@gmail.com
What I found by Googling ASSISTments leaves me wondering where the "teaching" is? I agree with Susan that assigning a problem, and then coaching as necessary for the student to derive the answer is *not* the first step in teaching. I look at the instructional model as:
1. INPUT: teacher instructs, reviews prior knowledge that is assumed to be known and will be needed to solve this problem, makes the necessary links to this prior knowledge and how this will generalize to this new learning/situation.
2. Teacher Modeling: Teacher models the problem. I see this as the *first* step in the ASSISTments model.... If the student doesn't know how to proceed, s/he can grope around and click on hints or break down the problem to *possibly* connect it to something the student may or may not remember, or something they have mastered from previous learning. Without the first step ( teaching), this can be frustrating for the student. If the student already knows and connects all the prior knowledge necessary, they can simply write the answer and may not need "teaching".
3. Guided practice: The student practices examples with teacher guidance using various types of problems. This is where I think most "electronic tutoring" or "tutoring software" becomes helpful. Students need varying amounts of guided practice to understand the matarial, and they can take as much or as little time as they need to master the learning. Tutoring software is a good tool for this step, in my opinion.
4. Independent practice: Student practices many examples independently, self- monitoring. Does not need coaching. Knows how to proceed. Tutoring software is also good for this step of learning. Overlearning/ mastery.
5. Evaluation: Perhaps the software does a decent job of this, but personally, I like to do my own oral querries about the mastery of concepts to make sure the student is not just rote following examples, and can generalize the learning to varying situations. Are they able to verbally explain what they are doing and why?
In summary.... the "Teaching" is a missing step in the ASSISTment model. Some software programs do have components that at least attempt to state what is being learned, review what is assumed to be known and will be necessary, and show illustrations, but from what I saw with ASSISTments, that piece seems to be missing.... or maybe *I* am the one missing where it is in the program. It all seems to have started with the problem to be solved.
AND... I agree that the keen observation of student's emotional state by a skilled teacher is invaluable to set a positive "affective" climate that facilitates learning. I have seen many a student wildly frustrated for varying reasons when using software without adequate teaching/preparation before the attempt. Metalearning comes about not from isolated study, but from social exchange. Teachers and students actively think about new patterns and rules and make them part of their knowledge base.
There *is* a place for "tutoring software", but let's be mindful of where that is.
Maureen Carro, MS, ET
Academic Learning Solutions
Alamo, CA
I find myself, when describing the first step -- the teacher "input" -- realizing that sometimes the description looks like The Dreaded Traditional Teaching Model. That first part has an awful lot of student input, too, that guides me in where I'm going to go with guiding the student.
I would definitely be interested in participating in something like David Rosen outlines. Although I originally came from an academic background (MA in political theory), I'm particulary interested in where theory meets practice in teaching, and have been enjoying "listening" in on the numeracy online discussions because they often target this intersection. I spend a lot of time listening to learners and soliciting their feedback, as well as using measures like CASAS to try to objectively assess the efficacy of my teaching methods and tools.
I'm already doing some "journaling" of my class' experiences this semester with the NROC pilot--and it is particularly sobering to realize that about 1/2 my students don't have the level of computer literacy needed to participate without a lot of "scaffolding" and help from me, classmates, and volunteers. (Some have never had an email account and are challenged just by the process of getting one--which is a pre-req for using the NROC or Khan Academy courseware.)
How would we go about formalizing something like David suggests? Is there anyone out there who could actually get paid to act as "lead"? Or are there any grants we might be able to get to help support this? Also, is anyone else interested? (I'd love to see a demo of ASSISTments, if anyone is using it.)
- Wendy Hoben, ABE teacher
Berkeley Adult School
Berkeley, CA
OK, I admit, I didn't go try ASSISTments, and it's unlikely I will.
This type of online training was inevitable, but can it replace a face-to-face session with a tutor? In my opinion, no. I'm sure with a subject like Math, there is a point of value to this program, teaching basic skills and equations. There certainly are some people who would favor this type of math instruction, avoiding any possibility of being embarassed or humiliated because they just aren't getting the process.
I seriously question the ability of this program to detect a person's mood, and a comment was made earler about how it cannot possibly detect those small nuances of communication that humans use (body language, facial expressions, voice tone, etc.). Whereas, some people might really do well with this program, getting basic math skills under their belts, I think others are going to miss the human contact and opt for a living, breathing tutor.
Kat Bradley-Bennett
Longmont, CO
Khan Academy is a great resource for math (and other topics)! It includes the human piece by having a real teacher's vioce and simultaneous flash video of what he's writing on the board. He's very personable and engaging, according to my 15-year-old!
I also know a math teacher in ABE who revised her whole online course to incorporate Khan Academy videos in just about all of her lessons. It totally fixed her problem of explaining math concepts with the limitations of online instruction. She was about to create camptasias for all of her lessons, and found that the Khan site had everything she needed.
Julie and others,
I have put together a 21-page list of free numeracy and math instructional videos, organized by math content topics, that includes Khan Academy and many other web sites where one can access free numeracy and math instructional videos for adult learners. It also includes an introduction with some context about how instructional math videos should or shouldn't be used, what they offe,r and what they lack.
You will find the list at:
https://dl.dropbox.com/u/6715575/ABE%20AND%20ASE_HSE_GED%20Math%20Videos%208.21.12.docx
I would, of course, be glad to have your comments about the list.
David J. Rosen
djrosen123@gmail.com
What a wonderful addition to any "teacher tool kit!" I have used this site for several years. Once I introduce it in class, students get hooked and go there frequently at home. Linda
THe videos were done without editing, and actually have rather many errors (my favorite being his frequent statement that "two plus itself times one" is what "two times one" means). THey are also entirely procedural -- so if the teacher in question thinks that the students are learning concepts, she should expect to be disappointed. His basic videos make frequent assumptions that students know material beyond what he's presenting (e.g., in his "averages" lesson, he expects students to be able to solve an algebra equation with fractions -- if I remember right, it even had an X on the bottom of a fraction).
Yes, he's personable... and if students need a procedural review, he can take a lot of the stress out. HOwever, he's not a math teacher -- he claims himself that he just teaches the way he learned (and he also says he was always good at math)... and then goes on to say that he thinks everybody learns like that... and specifically that people learn by doing lots of problems and then... the concepts come to them.
The students I work with, when presented with a bunch of problems to do, figure out a way to do those problems. THey are *not* particularly good at making the connections; most of them already believe they aren't good at math, and so they don't try. The ones who do try generally make "connections" based on their limited knowledge, with predictable misconceptions that good instruction addresses. Khan's videos "address" misconceptions, I'm afraid, by including them early but not explaining. Granted, in his exponents lesson, he states loudly and repeatedly that students should not just multiply!!! And he shows them the multiplication and says it's wrong... and never does show the actual answer to the problem he presents. Nor does he show or explain the difference between regular multiplication and exponents. In my experience, this kind of instruction means students get the problems that come right after the video right, maybe... but later on, when they see an exponent problem, they think: "Oh, it's something to do with multiplication..."
Again, the videos are quite reasonable for most procedures, for review... tho' there are actually much better ones out ther online, just harder to find ( mathtv.com is one , as well as mathvillage.info ). And if all somebody is trying to do is get through their next math test... it has gotten many students over that bridge. Also -- students who really like math often **love** the site, b ecause they get to go further and faster, and not get held back ... Just know its limitations...
In my ABE class, I've used some Khan (and some other) math videos, and I find the best way to use them is to watch them as a group, pausing often to have students ask questions, and to ask students to predict how problems will be worked. I've also found that the Khan videos tend to assume prior knowledge that my students don't have and often gloss over areas of misunderstanding.
I like to do some concept work first. Then present a video to start students on a topic once I know they have the pre-requisite knowledge. Then, we go to other sources for practice, since I don't feel that the few problems presented in the videos are enough for most of my students to master the topic. (Getting an answer right and really "owning" the knowledge are two very different things in my experience.)
Saying that Khan is "personable" and therefore is equivalent to a live teacher completely misses the point in my opinion. But that's not to say that the videos can't be a useful adjunct to instruction.
-Wendy Hoben
It seems relevant to point out that the Secretary of Education just called for the ditching of textbooks as instructional tools. This would seem to necessitate "software that emulates a good human tutor." The NPR story about this described the alternative to textbooks:
I've been thinking about this idea that layers of instructional design techniques and features that individualize learning plans play a big role in motivating learners. I don't think e-readers are the answer, that's for sure. Text-heavy materials is a sure fire way to lose the attention of would be self-directed learners. Technology can do so much more than help us go paperless, it can personalize curricula. But, when will it those features become standard in adult education?
Hi all,
I have a few concerns about the instructinoal concept that we need to refrain from text heavy materials. It is critical that adult learners understand how to interact with long and complicated texts. Technology can be interactive and provide learners with the opportunity to be self-directed, but technology and interactivity does not mean that we must reduce the amount of reading that is necessary for students to thrive in society.
Students must read in daily life. For example, they may need to analyze a mortgage contract, understand terms of credit cards and other loans, read instructional manuals. There is a relationship between the skills of a reader (as readers who have higer skills with complext text) achieve higher paid employment and reach higher levels of educational completion.
While the youtube generation may prefer to 'be engaged' through video - we must, even in online learning, provide depth of content in text based instruction. Students must read - it is necessary for an economically sustainable life - and while we provide diverse learning experineces through e-readers, ibooks, and other interactive / multi media online learning - there must be a balance with expectations of reading.
Kathy Tracey
I also believe our learners have to practice their reading comprehension. But I believe the issue isn't about content, it's instructional design. How can online programs take low reading-level learners through fundamental concepts without subjecting them to lengthy explanations that are written at a level too high for their comprehension? How can computer-based instruction ensure that learners take the time to digest content when only about 20-40% of online text is actually read, once there are more than 100 words on the screen. The most common instinct is to scan the screen looking for the button to advance to the next page.
I agree that online content needs to be designed to meet learners where they are and incrementally increase the rigor. Textbooks aren't much good without teachers. That's why the US Secretary of Ed's call for making textbooks obsolete is supported by descriptions of inspiring alternatives via computer-based instruction:
There is a lot of room for creativity, innovation, and improvement in the way adult education content is delivered through computer-based instruction. The frequent assessements in the Assistments program represent one intriguing approach to a learners' needs. I think GED Academy's computer-adaptive learning plans represent another step forward. But, reading dense pages of text is not the focus for either program, because that's not how most adult students prefer to receive information. Interactivity facilitates knowledge acquisition and engagement better, in my experience.
Hi Jason,
I think we are on the same page with a slightly different approach to instructional design. I absolutely agree that a student must be placed in an instructional environment that is appropriate for their skills and this content continually build on rigorous and relevent instructional materials.
I think digital content (online learning, ibooks, apps) can really enhance education when it is balanced between reading. Imagine the power of a photo story of the civil rights movement that the student is actively moving through while hearing the speech I Have a Dream and asking the students to reflect on what they have just experienced. This is an incredibly powerful use of technology to support instruction.
Yet, I am concerned about the comments only about 20-40% of online text is actually read, once there are more than 100 words on the screen. I would be very interested in reading that research, and I would love to continue a discussion of what it means to be literate in our technolgy rich enviroment.
In my opinion, we can't limit the amount of text our students encounter becuase 60% of online text is not read, but rather - we need to teach them the skills to read the text based material, support it with a rich and instructionally sound use of multi-media when it benefits the overall instructional goal, and give the students the skills needed to move into self-sustaining careers or higher education.
Instructional design does not exclude text based content, nor does it dismiss the intrinsic value of video, animation, audio, and interactivity that can be provided by technology based instruction - but, I think that these two elements must come together to support the intended learning objective.
Respecfully,
Kathy
The folks who think this is the greatest idea since sliced bread haven't really thought it through...
... I think/hope we can help make things "standard" by making things that really do personalize education and really are better than a mess of text stuck on a comptuer (which is, still, at least lighter to carry and you can say "find endoplasmic reticulum" and it will...). Just need to have somebody who understands learning doing the programming, planning and marketing...
I am relatively sure that it was through this list serve that the you-tube video was sent showing how we in the U.S. teach math and how it is done in countries whose standardized test scores are much higher. The upshot was that we teach/tell kids how to do a certain type of problem and then have them do some like that example and in the other countries (Japan) they give the problem to the students and let them wrestle with them alone and in groups to FIGURE out how to solve it. They do far fewer problems, but the struggle they go through to figure it out makes (seemingly) more of an impression as the students understand why something works. It sometimes seems like the long way around, but in the end it works! I have been trying to design lessons like this for years and it is nice to see it has merit, because many colleagues think it takes too long to get through one of my lessons! So it might make sense for the problem to be presented and given hints, but a computer can't see/here what the student is saying or writing, so it can't nudge in the right direction.
That was probably the video that won first prize in the #mtt2k contest; it's at http://www.youtube.com/watch?v=CHoXRvGTtAQ ...
that teacher has a great blog and a site called mathmistakes.org where people discuss what students were or weren't thinking based on their mistakes.
Hello All!
Any time instructors discuss Khan Academy, I cringe a little bit. That is, if that is the only instruction learners are receiving in their adult education classes. The main reason is that it is just content. In 2006, a paper was written, "The Components of Numeracy" that addresses how adults learn mathematics (http://www.ncsall.net/fileadmin/resources/research/op_numeracy.pdf). This report states that there are 3 Components in Effective Numeracy Instruction: 1. Context, 2. Content, and 3. Affective and Cognitive; each one of these components have several subcomponents but the idea is that instructors need to create an avenue for learners to build from their own understanding of mathematics. Khan shows just one way to solve problems, which more than likely is not the way that Khan is thinking. Please consider reading this important article about numeracy instruction.
Best,
Brooke Istas
Subject Matter Expert
Math and Numeracy Community
I agree with Brooke that The Components of Numeracy is a foundational read for math/numeracy instructors. A shorter article by Lynda Ginsburg entitled Designing Numeracy Instruction with the Components of Numeracy in Mind may be a way to "get your feet wet" before reading the longer occasional paper. You can find it on p. 14 of the Numeracy issue of Focus on Basics at http://www.ncsall.net/fileadmin/resources/fob/2008/fob_9a.pdf.