Skip to main content

Does research evidence affect policy and practice?

Hello Colleagues,

In the LINCS CoP English Language Acquisition group Paul Rogers has asked, "if there were evidence / research to prove that using smart phones etc. could enhance learning, then what? Would teachers and administrators change their policies?" This question deserves a discussion thread of its own, so here it is. Please join in.

I would like to unpack Paul's question with several related questions. I'll join in with some answers from my perspective, but first I want to hear from others, from you! Don't feel you have to answer all these questions, and especially not all of them in in one reply. You can reply separately about those that interest you.

1. Good research evidence in our field. Do we have some good research, and research evidence, that adult basic skills (including ESOL/ESL) teachers and administrators can use to improve practices and programs? If so, what are some good examples?

2. Research in our field compared with other fields. How does the research evidence in the adult basic skills (i.e. adult basic education or adult literacy) field compare with research evidence in K-12 education; higher education; and in other fields, for example in medicine, which our field has sometime been compared with?

3. Research that could influence decision-making. As a field, where do we have sufficient research evidence to make decisions about improving programs and practices? Where do we have some evidence? For which topics, questions, or program decision areas is there little or no evidence, and where research evidence is very much needed?

4. Under what circumstances do practitioners or policy makers use research evidence to make decisions? Where we do have adequate evidence, under what conditions or circumstances do teachers and administrators at the program/school and state levels use it in making decisions to implement new models and practices? When do policy makers at local, state and national levels use research evidence in making decisions?

5. Obstacles to using research evidence. Where teachers and administrators do not use existing research evidence, why not? Is it difficult to find relevant research evidence? Is it difficult to understand studies, to interpret their findings and recommendations when they are available? If so, what would make that easier? Do practitioners despair, even if when they know what research evidence suggests as good or best practices, that the funds and professional development support to help make that happen is lacking?

6. Overcoming the obstacles. What are good examples of how practitioners at local and state levels have been able to overcome these obstacles?

7. Other related questions. What questions would you like to add to this discussion?

David J. Rosen, Moderator

LINCS Program Management and Integrating Technology groups.

Comments

David J. Rosen's picture
One hundred

Hello Colleagues,

On July 11th I posted this question, "Does research evidence affect policy and practice?" I broke it down into the seven sub-questions below. So far, over 90 people have looked at the post, but no one has replied to any of the questions. I am not sure why. Perhaps the 90+ are all practitioners who, while they may be interested in the questions, don't feel they have enough experience with adult education research to answer most of them. If this describes you, then let's hear your answers to question 5. Perhaps some who looked at these questions thought, "great questions...I'll set this aside until I have time to give a thoughtful answer." If that's what you were thinking, how about now? But perhaps there are other reasons, so I'll add another question: "What are the obstacles for you in answering these questions?"

Thanks!

1. Good research evidence in our field. Do we have some good research, and research evidence, that adult basic skills (including ESOL/ESL) teachers and administrators can use to improve practices and programs? If so, what are some good examples?

2. Research in our field compared with other fields. How does the research evidence in the adult basic skills (i.e. adult basic education or adult literacy) field compare with research evidence in K-12 education; higher education; and in other fields, for example in medicine, which our field has sometime been compared with?

3. Research that could influence decision-making. As a field, where do we have sufficient research evidence to make decisions about improving programs and practices? Where do we have some evidence? For which topics, questions, or program decision areas is there little or no evidence, and where research evidence is very much needed?

4. Under what circumstances do practitioners or policy makers use research evidence to make decisions? Where we do have adequate evidence, under what conditions or circumstances do teachers and administrators at the program/school and state levels use it in making decisions to implement new models and practices? When do policy makers at local, state and national levels use research evidence in making decisions?

5. Obstacles to using research evidence. Where teachers and administrators do not use existing research evidence, why not? Is it difficult to find relevant research evidence? Is it difficult to understand studies, to interpret their findings and recommendations when they are available? If so, what would make that easier? Do practitioners despair, even if when they know what research evidence suggests as good or best practices, that the funds and professional development support to help make that happen is lacking?

6. Overcoming the obstacles. What are good examples of how practitioners at local and state levels have been able to overcome these obstacles?

7. Other related questions. What questions would you like to add to this discussion?

 

David J. Rosen, Moderator

LINCS CoP, Integratign Technology and Program Management groups

Di Baycich's picture
One hundred

Some of my thoughts on David’s questions. I’d be happy to have folks refute or confirm what I’ve said and continue this discussion.

1. The two research based initiatives that come to  mind are STAR and ANI. But I’m not sure if the research they are based on is from K-12 or adult ed. I have a feeling it may be K-12. Other research, at least to the best of my knowledge, is scant. Most is done on K-12 and postsecondary.

2. Research in adult ed, at least according to some, is not up to the “gold standard”. We have anecdotal information and research that has been done but has often been pooh poohed because it wasn’t “rigorous” research. CSAL has been working to add to our research base but I’m not sure if their results have been released yet. The money seems to be available for studies done in K-12 and postsecondary but not so much in adult ed. Comparing research in medicine to research in adult ed makes me twitchy. Big pharma spends big bucks to fund medical research so of course those studies can measure up to the gold standard.

3. Again, STAR and ANI seem to be two research based initiatives that could make positive impacts on programs.  I think we have some evidence for the success of IETs and other type of career pathways programs. Not that I’m biased (OK, I am) but I think writing is an area in which we could use more research with adult learners.

5. One mind set that sometimes has to be overcome is “I’ve been doing it this way for 10 years and it works just fine.” I think PD is another issue. It’s one thing to tell teachers to use evidence based teaching but another to give them the support they need to do that. Many adult ed teachers are part time and often work another job so don’t have the time to read research reports and try to interpret them. Unless the research has been done with adult learners, some practitioners discount it as not being appropriate.

6. I think that states who implement things like STAR and ANI are making big steps to implementing research based practices in the classroom. These initiatives offer extended training with lots of opportunities to apply what is being learned to the classroom and ongoing support. Unfortunately, budget issues often put the kabosh on extended, embedded PD .

David J. Rosen's picture
One hundred

Thanks, Di for getting us started.

I believe that STAR research was based on both K-12 and Adult Basic Education research. This publication, Research-Based Principles for Adult Basic Education Reading Instruction, by John Kruidenier, Ed.D. and Produced by RMC Research Corporation, Portsmouth, New Hampshire, will give you a fuller answer for what research influenced the development of STAR.

The report describes it as follows:

Most of the principles derived from the ABE reading instruction research are "emerging principles" because they are based on a relatively small body of experimental research. There is much more research focusing on children, as demonstrated in the report of the National Reading Panel. The small size of the ABE reading instruction research base precludes establishing more than just a few principles based solidly on large numbers of research studies that have been replicated. Some of the topic areas reviewed contain no or very few research studies. This does not necessarily suggest that the quality of ABE reading instruction research is poorer than K-12 reading instruction research or other bodies of research, only that there is less of it.

Perhaps someone else could tell us about the research that has been used by the Adult Numeracy Initiative (ANI).

It would be great to hear from many others who have examples of how research evidence has affected practice or policy in their program, or in their state, or what are the obstacles and challenges in finding or using research for this kind of decision making.

David J. Rosen

 

 

Michelle Candy's picture
First

Hi David,

Four years ago I was a new corrections educator, new the world of adult ed. I was just coming off an online MA TESOL program (APU) and was really in the midset for researching best practices, both for corrections and for adult ed. However, I found that much research was behind pay walls. I no longer had access to my university library, so I couldn't do research that way. I have gained access to my state's library, but the resources are still limited.

So as someone trying to find research, I could see abstracts but no articles. I finally gave up.

I have found bits and pieces here and there, and I still search, but there is a lot more blog-type and anecdotal writing than research that is accessible.

David J. Rosen's picture
One hundred

Thanks Michelle.

I have experienced the difficulty you describe, that the research findings that I am looking for are in an article in a prohibitively expensive journal, not accessible for free to those outside an academic community. Sometimes I have found that some of these journals are available online to me as a patron of my public library. That may not be true everywhere, however. Has anyone else experienced this challenge? If so, how have you overcome it?

A related question, has anyone found an up-to-date database of adult basic skills (including ESL/ESL} research articles? If not, what do you do? What are the go-to-databases or journals that you do use, or do you just use a web search engine like Google or Google Scholar? If so, do you find the research that answers your questions?

David J. Rosen

 

Rachel Donelson's picture
First

For us, I'd say the research with the biggest impact on our programs is based on institutional data- enrollment trends, reasons students drop, teacher observations, MSGs, etc.  Basically, what we've found there (at least for our ELA learners) is that students who progress, on average, attend more hours of class, which is no shocker-so we mostly focus on how we can keep students engaged.  If there's big-scale research around best practices for that component, I'd love to read that!  

We're fortunate that we're able to send instructors to regional conferences for COABE, CAEPA, and TESOL- so we access current research that way.  COABE has a useful website for teachers with a section called research to practice- and we see links between good teaching and student engagement.  We don't have consistent access to publications- as another poster noted-the cost/relevance balance doesn't always work out.    

I've only been here for a couple of years, but in my time, our larger institution has also sponsored book clubs related to our field (education, leadership) but those seem to fizzle out after a few meetings.  They seem to do better at the director level in actually finishing books meant to guide reflective educational leadership.  They read Brené Brown this year.     

One of the other issues with research in education is just how difficult it is to isolate variables in order to identify effective practices. The best research on that point could probably be done in a corrections facility since the day-to-day environment is so consistent for all learners.  But even then- there's a slew of factors related to their prior individual experiences that impact students' learning.  And how generalizable is that population to other adult learners?  I'd say that generalizability piece is a challenge with any educational research.  What works in one context may not be that effective in another, so it's important to go back to that institutional data that can help you determine if the practice you implemented based on those articles you read are working for your program.

David J. Rosen's picture
One hundred

Thanks, Rachel.

There is research on adult basic skills learner retention (from the perspective of the program's goal to retain students), sometimes referred to as learner persistence (from the adult learner's point of view.) Both perspectives have a lot to offer practitioners.  For example, you might look at the NCSALL-sponsored studies on adult learner persistence at http://www.ncsall.net/index.php@id=80.html .

Anyone else have studies to recommend on adult learner retention or persistence?

David J. Rosen

 

Susan Finn Miller's picture
One hundred

Hello colleagues, Thanks for raising this important issue, Paul and David. Most of us in adult basic education realize that, unfortunately, there is little research in our field. Diana mentioned the research on adult reading being conducted by the researchers at Georgia State as one example of current research, which has been informative-- with more yet to come.

Some of us who have been around for awhile remember the valuable work of the National Center for the Study of Adult Learning and Literacy (NCSALL). NCSALL conducted research and published many reports and other resources beginning in @ 1997 through 2008. This substantive work is archived online on the NCSALL site. Though some things are likely dated, much still has relevance today.

For instance, Rachel asked about research related to motivation and persistence, and this was one of the main areas of NCSALL research. There are several research reports as well as a study circle on the topic of persistence on the site.

The researchers published the Annual Review of Adult Learning and Literacy featuring research reports and reviews of research each year for seven years. These scholarly research articles are also all archived on the site.

NCSALL was unique in that there was a strong focus on bringing research and practice together to support the work being done daily by practitioners. Toward that end, they also published 34 volumes of Focus on Basics with articles that both summarized research for practitioners and highlighted the practical implications for adult literacy teachers and learners.

You can find all 34 volumes of Focus on Basics (FOB) online. You may want to check out some articles for personal enrichment; some would be useful to discuss during staff meetings. 

Here are a few articles from FOB that may pique your interest:

There's Reading ... And Then There's Reading by Victoria Purcell-Gates

Less Teaching and More Learning by Susan Gaer

Accommodating Math Students with Disabilities by Rochelle Kenyon

Beginning ESOL Learners' Advice to Their Teachers by MaryAnn Cunningham Florez (This article is one of my all time favorites!)

Powerful Motivation by Will Summers

More Curriculum Structure: A Response to "Turbulence" by John Strucker

The work of NCSALL is still valuable, and it attests to the importance of funding additional research. There are so many unanswered questions that research could help to answer about how to support adult learners most effectively. Conducting research is costly, but in my view it is essential. We can all advocate for more resources to support the research we need.

Looking forward to hearing other members' thoughts!

Cheers, Susan Finn Miller

Moderator, English Language Acquisition and Teaching and Learning CoPs

Pam Meader's picture
Ten

Hi Susan,

I certainly do remember NCSALL as I was part of the Practitioner Research and Dissemination Network for the state of Maine. NCSALL built a network of practitioners from several New England states and southern states. We got to meet many of the researchers for NCSALL and tried some of their suggestions as a practitioner research project in our classrooms. Many of the resources you shared are still valuable today. The Focus on Basics issues were wonderful and as your shared, can still be accessed today. Please take time to preview these resources as you won't be disappointed.