Digital Badges Discussion Q2: What are the roles for quality & content standards (in awarding badges)? Of accreditation? Development processes?

Welcome to day two of our week long discussion on The Potential and Value of Using Digital Badges for Adult Learners. Yesterday we looked at how badges are different from credentialing methods used in the past and how career pathways, certificates, and stackable credentials are similar and different from digital badges.  You can continue to explore those ideas in the thread on Question 1 (from December 3).

Today we are adding a new thread to explore another question: 

Question 2. If “the technology for issuing badges is available to anyone with access to the Web,” what then are the roles for quality & content standards (in awarding badges)? Of accreditation?  What are the bottom-up or top-down processes needed for developing badges?

While our guest facilitators will chime in on this question, please take a moment to consider your own thoughts and reactions to the concept of digital badges.  You can read the The Potential and Value of Using Digital Badges for Adult Learners, check out Mozilla Backpack and Credly. Do you see a place for digital badges in your own work?  How could digital badges impact quality and content standards etc.?  Do digital badges seem way out in left field from where we are today?







Getting published was a terrifically difficult task 10 years ago. No matter the subject of your book, whether profound or profane, you had to persuade a publisher to accept your manuscript and finance an initial print run. If your book didn't fit into the publisher's mental model of "possible conventional best seller," you simply weren't going to be published - no matter how brilliant your writing. Likewise, if you were conducting research that was unconventional or nontraditional in any way, there was very little chance that the politics of journal editorial boards would work in your favor. If you failed to appropriately appease the holy gatekeepers, there was no way your article would appear in a journal. A decade ago, whether writing on popular or scientific topics, a person with something to say was utterly at the mercy of the establishment. Self-publishing at any scale was so difficult it was essentially a non-option.

Then blogs arrived on the Internet. Overnight, anyone could publish anything - and hundreds of millions of people did. By one estimate, 18.9% of all content on the entire web is published using blogging software called Wordpress. People became fond of saying that "blogs have democratized publishing," by which they mean that publishing is now an option for anyone and everyone, not only the wealthy and connected 1%.

What are the roles for quality and content standards in publishing articles on blogs? Of peer review? What bottom-up or top-down processes are necessary to regulate the so-called blogosphere?

Answers to these problems co-evolve organically with the problems themselves, and the best solutions even leverage the problem against itself. For example, how do you find the high-quality, relevant content you're looking for on the web (over 20% of which is published on blogs)? Initially, humans attempted to catalog every page online and maintain an organized, browsable structure (you can see the remnants of this effort at the Open Directory Project). But the democratization of publishing led to an explosion of pages, and made the task impossible for any group of human beings to accomplish. Enter Google's PageRank algorithm. PageRank leverages the structure of the massive quantity of information published online, by asking questions like 'what percentage of all the other pages on the web link to this page?' Each link to a page is taken as a vote of confidence in the content of that page, and gives Google a sense of the quality of the page. In this extremely clever way, Google leverages the problem (there's just too much content on the web!) against itself - the more webpages people publish the better their solution becomes.

But weren't we talking about badges?

For decades it has been extremely difficult for anyone other than an institution to award a credential. No matter what specialized skill or knowledge you possessed, and no matter how talented a teacher you were, if you weren't employed by a college or university (with the sanction of the educational establishment implied by that employment) you couldn't award a credential. Even for new or aspiring institutions, the holy gatekeepers (aka accreditors) mandate conformance to the orthodoxy in order to receive their sanction. Awarding self-created credentials at any scale was so difficult it was essentially a non-option.

Then badges arrived on the Internet, and this is where we find ourselves today. Many individuals and institutions are now awarding credentials without the sanction or permission of anyone, and the rate of awarding is increasing. I am fond of saying that "badges have democratized credentialing," by which I mean that awarding credentials is now an option for anyone and everyone, not only the wealthy and connected 1% of the world's institutions.

But if "just anyone" can award a credential, we are inevitably lead to ask question 2 of this online conversation: what then are the roles for quality and content standards in awarding badges? Of accreditation? What are the bottom-up or top-down processes needed for developing badges?

Accreditors are essentially the Open Directory Project of credentials. They are a group of human beings who try to vet all the credentials in the world. In the past, this has been possible. However, the democratization of credentialing means that we will soon be in a place where no group of humans can possibly vet all the credentials being offered. An algorithmic approach - something akin to PageRank (BadgeRank?) - will become necessary for us to find the specific high-quality, relevant credential we are looking for. Some people will continue to rely on accreditors, just as some people continue to rely on the Open Directory Project. But had you ever heard of the ODP before reading this article? I didn't think so.

A primary - if unintended - role of standards and accreditation processes is to limit innovation and experimentation. Yes, standards and accreditation have protected unwary learners from many a scam and diploma mill, but they have done so at the cost of protecting us from constructively unorthodox models as well. Democratizing the credentialing process opens the floodgates to a world with millions of credentials spanning the same range as the quality of writing on blogs. While our previous approaches for vetting quality are utterly incapable of dealing with the quantity of badges in the world, some clever person or group somewhere will create the BadgeRank algorithm that uses new standards to help us accomplish our abiding goal - finding high-quality, relevant credentials. We should not be overly concerned today that we cannot yet imagine how this system will work, because it is a characteristic of the best solutions that they co-evolve organically with the problem.

David Wiley makes a great point about the tensions between badges democratizing the means for issuing “credentials”, on one hand, and the needs for standards, quality control and protections from predatory credentialing practices on the other hand.  Without a crystal ball, it’s hard to predict the balances that will “organically” emerge here.  It may be helpful to talk instead about what we as adult educators would *like* to see happen with digital badges in our field and what we should do to help steer things in the right direction.

The balance points brought about by new technologies rarely affect everyone in the same way.  In our field, we often work with economically vulnerable populations at risk of being further marginalized by innovations and technologies.  With each new innovation, certainly including digital badges, opportunities arise to redress historical inequities of access to learning, training and education.  As we begin to realize the full potential of digital badges, how can we remain vigilant about doing this?  Do you think we should try to steer things in the direction of having relatively more “bottom-up” creation and issuing of digital badges, or more centralized versions that better fit with large scale quality and content standards?   What do you think the role of government and commercial interests should be in these badging systems?



   I fully understand the benefits of 'granularity' and 'democratizing' credentials, but also understand the potential for scams.   "Some clever person or group somewhere will create the Badge Rank algorithm..."     ???    "We should not be overly concerned today that we cannot yet imagine how this system will work, because it is a characteristic of the best solutions that they co-evolve organically with the problem.   

I am **all for** innovation and experimentation.   What do the folks at the Open Directory Project know about adult literacy and numeracy?   HOw can we converse with them?   In my experience, the "best solutions" don't co-evolve of their own volition; it takes people moving through the open-ended brainstorming process and designing the solutions and being the "clever person or group somewhere." 

    Even granularity has its issues.   We have learners who will tackle a small grain and successfully demonstrate a skill, but struggle with applying the skills and integrating the skills.  Perhaps we can design separate badges for that ;)   Remedial course design has often struggled with "do we break things into little skill units and practice" (flashing back to the Olive level SRA cards of youth and all those little Reading Comprehension Packets) or do we integrate readign, writing and spoken language... or both?  

    I really like the idea of designing online portfolios so that people in all kinds of fields can have an easy way to communicate accomplishments and skills. I have my little printed out certificate from my "How to Learn Math" course that's taped to my desk... but it's not on my  blog.   Maybe that's another little project for winter break :) 


I'm not able to see that page.     

Breaking things into pieces and getting active engagement aren't the same thing as making sure students can apply what has been granulized, but I don't know wehther that article addresses that or not... the link looks like a valid one but I'm getting "page not available" from Chrome.  

Hi Susan,

If you're referring to the article that David Wiley shared, it did open for me in Chrome, but here is a new link



Susan, I would high recommend Van Merrienboer's work as a place to look for information and guidance about what to break into small pieces and how to support active integration. This article is a great starting place. 

Hi all. This is a timely discussion. As a few of you know, I'm one of a handful of people working to support an online digital literacy assessment tool called the Northstar Digital Literacy Assessment.  We've had the interactive, online assessment modules running for about a year & they have been very popular (over 110,000 assessments given).  

We've just applied for a small grant to figure out how to "badge" our assessments. (Thanks David R for giving us the idea!)  After a conversation with the Mozilla Foundation's Doug Belshaw, we realized that our assessment is a reasonable foundation for many of the skills that are represented in Mozilla's web-literacy standards.  We plan to embed a digital badge into each of our assessments for users who pass (85%). In the past we've offered a certificate; we will continue to do so, but hope that the badge becomes a more popular option because it takes the pressure of maintaining a database of certificate recipients away from us and/or the programs that serve our learners. Once a learner earns a badge, it goes in the backpack & is easily accessed.  This means an employer only has to view the backpack to see all skills and experience a job-seeker wants them to see.  So convenient!

So, why would an employer trust our assessment and believe the badge has value? I think the answer is found partially in a post from the previous day about the meta-information about the badge and the assessment process required to earn the badge.  As a badge issuer, I think it's our responsiblity to be transparent about how the assessments were made, our proctoring process, and the benchmarks that are represented by the badges. Do we need to be "approved" by any large federal or private institutions?  Likely not.  We have obviously not had learners shy away from  taking our assessments nor had programs/teachers reluctant to direct them to our assessment. We are definitely "official", rather we're all practitioners from multiple diverse organizations that serve our learners.  I think crowd sourcing will push badging opportunities that are needed and have proven to have some quality to the forefront.   I guess because of our success with Northstar I favor a grass-roots approach.

Hi all,

For day three of our discussion we are continuing to look at question 2 and I was wondering if either Steve or David could say anything about how CEU's compare to digital badges.  This might be more specific to the use of both in professional development, but CEU's have been around for a while and there is an accreditation component to them so I was wondering how that system could flow into digital badges or why it couldn't.

For a brief overview of CEU's you can go here



Good question, Nell.  Many CEUs for professional development are provided by institutions of higher education, of course, so your question leads right into how higher education will respond (and in some cases, is already responding) to digital badges.  Foundations such as Mozilla's, Gates & MacArthur have funded badge-related projects in higher education.  There have been a lot of exciting developments and ideas emerging from those efforts that bear on your question.  Some have proposed replacing grades & credits -- often the staples of higher education -- with badges.   The fresh look that prioviders of higher ed & continuing education (CEUs) would need to take if they switched over to digital badges would likely push providers to rethinking about learning outcomes and evidence that students have met learning outcomes and the assessment practices built into gathering and evaluating such evidence.  All of that, though it might well get politically and institutionally complicated pretty quickly, would still have a very positive benefit on learning in higher & continuing education.  

But as I'm sure David Wiley and other experts on badging would attest, others see far more ambitious goals for badging in higher education and professional development.  They see badging as a tool for motivating and providing feedback to learners, for curriculum development, improved assessment practices and a way to market specialized kinds of training (for example, CEUs for adult ecucators) within social networks.  As the mission of digital badging creeps in this way into instruction and assessment, digital badges may run into in adult education the kinds of issues that David Rosen referred to in his most interesting comments in the Day 1 thread.

In my own institution, a public university of higher ed that provides quite a bit of CEUs to educators, there has been some resistance to the idea of badging because (IMHO) of the entrenchment and inertia of the financial model that's tied to credits, grades and certificates.  But I believe digital badges have much to offer providers of CEUs (regardless of whether they are based in higher ed institutions), especially as their ability to better support learning is more widely developed, known and appreciated.  

It would be great to hear from others who may have first-hand experience with badging and CEUs/professional development.


One need not channel Sir Isaac Newton to recognize that leveraging previous efforts can be both efficient and effective. The effort most pertinent to Content Standards for Digital Badges is O*NET. This multi-year effort by the US Department of Labor/Employment and Training Administration (USDOL/ETA) provides a validated and living framework upon which taxonomies for Knowledge, Skill, Ability, and required Tool, and Resource (KSART) worker potentials can be located. When quality metrics for performance are thus co-located, a comprehensive dynamic system of "merits" can be envisioned where every worker may hang his or her accomplishments, every employer may seek its requirements, and every educational system may set its Lykert scales for performance measurement. As occupations change and O*NET is updated, some KSARTs resting on this framework will be elevated or migrated by occupational area demands while others will be deprecated. So developers of the Digital Badge must be prepared to accomodate the ebb and flow of skillset requirements through time. A successful Digital Badge System will be able to produce not only an individual's unique KSART fingerprint, but also have the ability to compare it against PIAAC (Program for the International Assessment of Adult Competencies) results, against employer position fulfillment search requirements, and against educational scaffolding and Career Pathways programs to name a few. The potential of this concept for shaping the future of this country and this world is thrilling to contemplate.

Thanks for your comments, Jim.  It sounds like you're suggesting a top-down approach to developing digital badges in relation to broad standards frameworks such as KSART / O*NET, within which various skills metrics and digital badges can be located.  You also mention comparing individuals' digital signatures against the recently released results of the PIAAC framework that includes broadly scaled assessments of literacy, numeracy and problem solving in technology-rich environments.  Can you say a little more about how you see the relationship between the small graularity of digital badges, on one hand, and the rather broad scales of PIAAC and similar assessment frameworks on the other hand.






A bit late for this thread but it seems like the best place to ask. I have been giving thought to the discussion of badges.  I notice that in our newer version of our learning management system, Moodle, they have included options for digital badges. If anyone has been a participant or an online course provider in a course that used badges, could you share your experience? 


Steve Quann

World Education, Inc.