What is Evidence-Based Practice & Why Do I Need to Know about It?

Heather, and others,

Thanks, Heather, for pointing out the Discover Corrections web site.

I believe that an important purpose of this LINCS CoP, and others, is to examine, discuss, and debate. So in that spirit, I want to point out what I see as some serious limitations of the article, "What is evidence-Based Practice and Why Do I Need to Know about it?"

The article defines Evidence-based as "Conclusions drawn from rigorous research studies that have been replicated numerous times with defined, measurable outcomes about the effectiveness of an intervention or process. - See more at: http://discovercorrections.com/blog/What-Evidence-Based-Practice-Why-Do-I-Need-Know#sthash.KUa8NVxG.dpuf " In Pre-K - 12 education, and even moreso in the "hard" sciences, this is a reasonable definition, although I am told by doctor friends that is not what good always doctors use in their medical practice. More on that later. In adult education there are many serious problems with this definition:

• Compared with other fields and disciplines there is relatively little rigorous research about the effectiveness of an education intervention or process. The U.S. Department of Education's Institute of Educational Sciences, which in the beginning included adult education in its "What Works Clearinghouse web site" http://ies.ed.gov/ncee/wwc/, later removed it because there were so few (or no ?) studies in adult education that met its standards.
• There are very few adult education studies that have been "replicated numerous times". Interestingly, one area where there have been some replications (although perhaps they are not numerous) is Corrections, particularly on the effects of education on recidivism. I am not sure, however that any of these studies that might meet IES standards review a specific intervention or process and use a gold or silver standard experimental design to determine its effectiveness.
• Gold standard experimental design is ideal. It eliminates variables, studies a single intervention to determine if it works, to what extent, under what circumstances and sometimes with what populations. When there are several rigorous studies like this that show that the "treatment" or "intervention" has positive results, we can have much greater confidence in using it in our practice. That's great. However, these studies are very expensive, which is why we have so few in our marginally-funded field. It is also a waste of these precious resources to do studies on interventions for which we do not first already have other kinds of evidence that they work.
• A serious shortcoming of the definition given for "evidence-based" in this -- and many other articles and papers I have read -- is that other kinds of legitimate education evidence are excluded. The Institute for Educational Sciences definition of evidence-based practice promulgated by it's former director, Grover Whitehurst, in 2002 during a speech to the Student Achievement and School Accountability Conference was “the integration of professional wisdom with the best available empirical evidence in making decisions about how to deliver instruction.”  http://www-tcall.tamu.edu/newsletr/apr09/apr09f.html Professional wisdom, too often overlooked, is an essential part of evidence based practice.
• In our field, we should begin with professional wisdom, building up evidence over time and in different venues, that a promising practice works with its intended population (e.g. incarcerated men between the ages of 16-30, who upon entering an education program read below a third grade level). Then, when we have a strong body of evidence, it may be worthwhile to test the efficacy of an intervention using an expensive silver or gold standard experimental design study, but not before then.

Back to the doctors I have talked with about evidence-based practices. Like teachers, doctors are practitioners. Their job is to prevent illness, help people get better when they are ill, and alleviate pain. Many times, every day, they make decisions about their patients' treatment -- and ideally include the patient in those decisions. They prefer to make decisions based on rigorous gold and silver standard experimental design research that has been replicated many times; in the medical field they have many of these kinds of rigorous, sometimes replicated, studies. My doctor friends, however, said that when they don't have this gold and silver standard information they still have to -- and do -- make medical decisions based on other evidence. That evidence is often (well-respected in the medical field) professional wisdom. They rely on the professional wisdom of their colleagues, and they have formal processes (e.g. Grand Rounds in hospitals, and others) for getting that information. Professional wisdom in adult education, a potentially powerful source of evidence for teachers, is largely neglected. As far as I know, we don't have "grand rounds" in corrections education programs or in other adult education instruction venues. I am not sure that we even have an official definition from IES on what professional wisdom means in education.

So, Heather and others, the evidence-based practices definition in this article, although correct as far as it goes, does a disservice to adult education and adult educators by holding out a standard that is now largely unachievable, and neglecting evidence that would be respectable and useful in practice.

I will be interested to see what others may think about my critique.

David J. Rosen

djrosen123@gmail.com