Join the authors of the World Education and Jobs For the Future Report: Leveraging Technology to Increase Opportunity and Economic Security for Adults: Field Testing Tools that Break Barriers to Learning and Employment (TTALE) on Wednesday, May 20th, for an informal conversation about the authors' analysis of the employment opportunity gap.
Join Jen Vanek, Alison Ascher-Webber, Priyanka Sharma and myself as we look at the challenges and proposed solutions for addressing the employment opportunity gap. This report was released in March 2019 and highlights barriers facing many adult learners prior to the added challenges we are facing in light of COVID-19.
We will unpack the authors' field testing of seven digital tools for enhancing adults' skills and supporting greater workforce mobility, and discuss how the key learnings and outcomes are shaping how the authors are thinking about ed-tech for employment in our current environment.
We will use this thread for our conversation, beginning by 9AM EST. Explore the TTALE report now, and join us here with your questions and comments about how we can use technology to help address the employment opportunity gap.
Career Pathways Moderator
Welcome to our conversation with the authors of Leveraging Technology to Increase Opportunity and Economic Security for Adults: Field Testing Tools that Break Barriers to Learning and Employment (TTALE). You don't need to be familiar with the report to join us in learning about its beginnings, and what the authors' research has to tell us about how technology is being used to address the employment opportunity gap. The employment opportunity gap affects 103 million adults in the United States who lack in-demand skills to advance in their careers and earn higher wages.
I want to welcome the report's authors, Jen, Alison, and Priyanka. This paper is based on your field testing of seven employment and education technologies with over 1,500 working learners and job seekers. Would you tell us how you selected these tools, and what makes them a good measure for addressing the employment opportunity gap?
Thanks so much for inviting us to share more about the field testing work and what we learned from watching the ed tech in use in different scenarios.
So, why these tools? In early 2017, four foundations came together to support the launch of The Employment Technology Fund. The Fund aggregates and mobilizes capital to facilitate the growth of companies and/or non-profits working to scale technology-enabled solutions to address the major barriers faced by low-income adults’ employment trajectories and outcomes. The EdTech Center advised the Employment Technology Fund on evaluating new technology solutions for providing low-income adults with Learning & Training, Assessment & Matching, Mentoring & Support and Job Search and Placement. Alison Ascher Webber, Director of Strategic Initiatives for the EdTech Center, serves on the Advisory Board for the Employment Technology Fund. The fund’s first investments were made in the Fall of 2017.
By the end of the project, we had tested seven funded employment and education technologies with over 1,500 working learners and job seekers – CareAcademy, Cell-Ed, Nepris, Northstar, PAIRIN, Skillsmart, and Signal Vine. These tools are still in use and help adults enhance their skills and achieve greater workforce mobility, across four segments: Learning and Training, Mentoring and Support, Job Search and Placement, and Assessment and Matching. The ETF funded these tools because they either had demonstrated effectiveness in use with other audiences or were new and untested with any audience.
Will leave off for my colleagues to add anything I missed. Thanks again for having us!
Yes, the tools we field tested were the ones in which the Employment Technology Fund had invested. So that begs the question as to why each was selected for investment. As Jen said, each had demonstrated effectiveness and showed significant potential to make more impact.
CareAcademy was designed by a former home health care aide and had been successful partnering with long-term care and other employers on training and certifying aides who studied just by smartphone.
Cell-Ed was also a proven mobile learning solution but that was also accessible on simple phones and did not require the Internet as learners can call a phone line to study through intereactive text and audio lessons on a range of topics- or a program can load their own content on the LMS.
Nepris had shown great impact in the K12 space in using live video sessions and chats to bring industry experts directly into classrooms- and storing those videos so they could be accessed later by teachers and students directly, and as expected, adult learners greatly appreciated hearing directly from experts.
Northstar is one of the top online, free digital literacy assessments available and had already shown significant impact and they have recently updated their platform to also include matching curriculum.
PAIRIN is an online assessment that measures personal attributes as a way to screen someone in for work opportunities (given someone's interests, self-assessed strengths, etc.) and help career coaches identify strengths are areas for improvement when preparing learner-workers for opportunities.
Skillsmart offers an incredible way to automatically "screen in" learner-workers to employment opportunities in their area based on their prior experience (including informal experience such as being a parent or volunteer work), which proved quite powerful.
Signal Vine had proven effective as a 'smart' technology to help universities text students at large bulk scale, but including through personalized messages, that we knew could help workforce programs improve their communications with clients and save staff time
Our field testing of these products- that you can read about in the report- proved that the ETF's assessments of these products were correct and these products showed significant impact when used with adult learner workers across different contexts. And then through the field testing we learned of adaptions that could be made for further impact as well as effective features and strategies that other employment technology products should consider using.
Alison, you said that, "...we learned of adaptions that could be made for further impact as well as effective features and strategies that other employment technology products should consider using." Would give us a couple examples of these adaptations, and the features and strategies you propose employment tech developers consider?
Some adaptations included simplifying the interface and the onboarding process (see conversation on that below) both in general but also specifically for students with lower digital literacy as well. Also with some tools it was critical to adjust content a bit so that adults immediately see content that they identify as ‘meant for them’, or reflecting their needs, as opposed to originally meant for youth or other audiences. In some cases small content adaptations or reframing can make a big difference without breaking budgets.
For effective features and strategies that could help other tech tools increase their impact, I’d highlight mobile coaching, adding using interactive text at scale to communicate, support and ‘nudge’ users, mixed media, and effective collating and sharing content such as past video conversations or chats.
Michael, Thank you for posting the question about how the authors went about selecting the tools for this research. I always get excited when I see research directed at helping our student gain the skills to not only get a job, but to keep it. I read through the research and noticed that the free tools were noted, yet the ones with a cost did not denote that a cost was involved. I think for adult educators it is important to know if there will be a cost involved. In my experience that is always the first thing many programs ask. I am interested in hearing the response from the authors.
Hi! Thanks for your question .At the time the research was done, many of these tools were still figuring out what their pricing structure should be in the adult learning / employment space. Part of that would depend on what we learned in the field testing as to possible use scenarios, what features they maybe would need to add, or what other adaptations they might consider making to better meet the needs of adult learner-workers in different contexts. Pricing structures in ed/employment tech change all the time so we can't keep that updated. Additionally pricing can vary greatly depending on volume and features needed. But the tools we feature on our WorkforceEdTech.org site we try to keep updated at least on 'how' they do their pricing (e.g. free/freemium, by license, by amount of use, etc.). Of the tools mentioned, I know Cell-Ed is offering a lot of courses and licenses for free right now and over the next year or more as a Covid-19 response, and Northstar's assessment is free to take, though there is a price to getting a certificate issued.
The tech tools that were tested here were provided to the programs at no cost since they were part of the project and did provide lots of insights and helped enrich our learnings and the report.
The cost of these tools do vary since the pricing structures are sometimes dependent on scale. I want to recommend that you visit www.workforceedtech.org for tool discovery. We developed this website to help programs to decide and compare various tech tools. We also developed a Tool Selection criteria with a general rule being - Goal before Tool. Please take a look and we would love to get your feedback if you end up using the Criteria document.
I hope our overall learnings from the project will help you successfully select a tool that works in your context for your learners. You can some of our recommendations here - https://edtech.worlded.org/ttale-report/#adulteducators.
You talk about 'accessible on-boarding' as part of what is needed to address the employment opportunity gap. Would you tell us more about what you learned from these technologies that programs could apply to their own review of tech they are considering? What are the best practices that programs should look for when building more accessible on-boarding into their remote learning plans?The report also highlights how some of these tools use new ways of collecting and assessing individuals’ past experience, skills, or other attributes for ‘screening in’ candidates for job opportunities. Can you explain how some of these tech tools are replacing traditional resumes and what their impact might be on how we prepare learners for new employment?
Let me address the question about onboarding. Effective tools need to have simple onboarding and their introduction to students needs to be structured to meet the varied needs of different students. Tools that have barriers for getting started, like a complicated log-in process, require more support initially and in the early days of use.
Northstar digital literacy assessment is a great example of a tool with simple onboarding. At the time of the field testing they had given over 3 million assessments in the open version of the assessments.These tests are completed by students who find the link online - they are generally NOT working with the support of a teacher. They have had success in reaching millions of people because there is really no onboarding required. It’s just a click and a short stroll down the front page to reach a test. This simple entry is part of their intended design. They are very aware that a digital literacy assessment needed to be easy to start - or no one would use it!
That is not to say that digital learning resources that require log-in should not be used. It just means that students will require some support getting started and likely more support down the road to remind them of clicks if they forget. If you are adopting a new technology, you should probably run your own field test by observing learners initiate use within the tool. Be on hand to lend support and make note of the points at which they struggle -- those are the points that you’ll need to be sure to support in onboarding more students. They are the points to be highlighted in any materials you put together to help people get started.
In our work with the National Immigration Forum, we experimented with and found ways to do on boarding completely remotely. We designed a completely remote workplace ESL class (read about the project here). We learned that students need clearly laid out materials with simple explanations characterized by proactive support. We also learned that not all students need the same support. These learnings have been demonstrated again and again - in our work with programs in the XPRIZE Adult Literacy Communities Competition and while working with programs shifting their instruction online during the pandemic.
For some students getting started completely at a distance requires beginning the process with a phone call to find out their access to and comfort with different technologies. From there a teacher might graduate to using a simple communication technology that students feel comfortable with. For example, if you are trying to get students to a class held using a webinar tool like Zoom or Go To Meeting, you can send a series of text messages showing the process for logging into it. Something like this from our NIF work (as featured in the report linked above):
From there, once they are comfortable with the webinar tool, you can use that to show them how to log into and use other online resources.
The important thing to remember is that any time you introduce a new technology your support to students needs to be proactive, instructions need to be kept as simple as possible, and you need to reach out using technology that the students already feel comfortable with.
Jen, this breaking down and scaffolding new tech learning processes using text is helpful. Your comment "that any time you introduce a new technology your support to students needs to be proactive, instructions need to be kept as simple as possible, and you need to reach out using technology that the students already feel comfortable with" is also a great reminder.
Next, I want to ask you about the nine traditional barriers to learning and advancement that you list in the report. Let's think through how these barriers may be shifting in the move to all remote instruction. Do you see some of these barriers being reduced, while others could potentially become more problematic? Are there still others that we should be looking out for as we continue developing remote learning plans?
I'll start and let Alison round out the response in a separate post.
I think that any barrier an adult basic skills student faced before the pandemic has only been exacerbated since then. Obviously, employment and access to support services and resources have become much more of a challenge for students. Even those who have the digital literacy skills needed to access resources are finding that there is great demand for them. Those who cannot access the skills either need someone to do it for them, or they go without - at least until social distancing relaxes more universally.
Mobile is still very important. Technologies that play well on mobile devices will reach the most people. Recent Pew data show that 97% of adults in the US have a mobile device and all but 15% of them are wifi enabled. That means that any technology delivered learning needs to be mobile friendly. Learners may not have access to public labs like they did in the past; the closest thing to that might be parking in lot next to a building with open wifi. That makes tech like Cell-Ed, which is useful on a flip phone, and other learning that can be delivered via texting very relevant. There has been great discussion on the LINCS technology integration forum on use of WhatsApp. The forum facilitator, David Rosen, has put together a document reflecting the resources and strategies employed by teachers using WhatsApp.
Demand on time has also been exacerbated. Now sheltering in place with family, possibly, and with school aged children, learners likely have less time to focus on schooling. The micro-learning delivered through WhatsApp and the short lessons on Cell-Ed are very useful in these times. Similarly, in the XPRiZE Adult Literacy Communities Competition, the developers of Learning Upgrade noticed that learners were using the app in binges and late at night, after the other demands of the day had passed.
Jen, you hit on several factors impacting learners' time spent on learning, even when access to devices and wi-fi are given. The demand on time for many learner-workers is high in the current shelter-in-place orders across the U.S. Even as those are lifted, many individuals will still have to balance family, work, and healthcare needs with their education and employment goals. In the report, you outline 'nudging learners' as a strategy for positive outcomes. Would you explain what is meant by that term? How do you imagine our thinking on this may change, both in the present of all-virtual instruction, and in our future return to blended learning and in-class instruction?
In a time of heightened unemployment learners-workers need to work harder to prove their skills. In a tight labor market employers are likely to fall back on traditional credential requirements (diploma, degree), but the larger trend is new digital ways learner-workers can demonstrate their skills. This may be by filling out online applications, surveys or assessments (such as those of Pairin, Skillsmart, or Northstar), or having online portfolios (showcasing work or badges earned). Or other such tools are highlighted at our WorkforceEdTech.org site under the categories of Job Search & Placement or Assessment & Matching. ABE programs can help learners know how to talk about and represent skills that they do have and especially on new digital employment platforms that may require representing their experience and skills in new ways. This support can be offered virtually, for example through mobile coaching.
Mobile/virtual career coaching is all the more critical right now also because many learner-workers will need to reskill and consider shifting to new job classifications or industries. Use of videos, live chats and other ways to orient learner-workers to possible new career directions is especially important as is digital referrals to online learning programs or other supports workers might need to be able to make a career shift.
At at a more fundamental level, if a worker-learner first needs supports finding access to the internet, or a device, or in starting to navigate the internet in the first place, mobile coaches are needed to help them with these needs through what we call “Digital Navigator” services. The EdTech Center is working through the Digital US Coalition to start offering such services virtually right now during shelter-in-place measures but eventually also through drop-in supports provided at locations learner-workers already frequent, such as retail stores or laundromats.
Alison, you mention mobile coaching as another resource to support learner-workers in the transition to new industries, jobs and learning programs. I'd like to talk next about how to best support learners in managing multiple platforms. I know for myself, it's challenging to transition between the multiple video platforms being used by many organizations today. It makes me wonder how learners are dealing with the increasing use of different learning tools and platforms, especially ones requiring increasing digital literacy skills.
The report highlights the need for rich and diverse media in mobile learning. What advice do you have for programs trying to balance providing rich learning opportunities with the need for simple user interfaces? Ideally, we want to see technology developers find ways to make rich and diverse media that employs simple user interfaces, but we're in a period when that isn't yet the norm. How can programs make sure users have access, while also moving them towards more rich and diverse media which requires increasingly greater digital literacy skills?
Some of the best use of mixed media is very simple-- for example a photo or an audio or video recording made by an instructor, another student, or an invited expert. This can be shared through a simple text or message or in a post in a closed social media group such as Facebook or Instagram. Sure, that could also be through a post in LMS or built into a digital lesson designed with an authoring tool, but that’s not necessary at all! Or the LMS can keep a very simple interface such as how Cell-Ed does it through interactive SMS or internet/Whatsapp messaging. Over time, students can then be transitioned to a more sophisticated LMS or other platform so as to become comfortable with them and be able to engage in more involved discussion boards. But we must start by meeting learners where they are, and that might be starting by messaging out multimedia content. There is a huge benefit to that too as if a learner wants to revisit the content, they don’t need to log in to any platform, but instead can just scroll through their text/messaging stream! Despite how much we use informal video and photos in our daily lives to communicate with friends and family, I think it’s one of the most underutilized media formats given its proven power. If I can give one message to teachers, don’t worry about sharing unpolished photos or video or audio recordings with students. Just quickly record something and share it out! And then don’t hesitate to engage your other students (even students in other classes if needed) to help translate it, or transcribe it, or take the other steps needed to make it accessible to all.
In fact it is through less formal use of photos and video that many teachers are supporting learners in this time of shelter-in-place or in distance learning models to help support learners to onboard and start to use more sophisticated LMS systems. A great example is the National Immigration Forum's work training immigrant workers in retail, transportation and entertainment, manufacturing and other industries all at a distance- and how the remote systems we at the EdTech Center helped them develop for supporting these learners on how to onboard and start using Moodle and a video conferencing program for virtual remote classes. Texting out videos and photos and providing just-in-time access to live coaches or technical support is all a part of their model. Learn more about their approach and success in their recent report at: https://immigrationforum.org/article/upskilling-new-americans-innovative-english-training-for-career-advancement/
We field tested SkillSmart to find out what happens when employers start to look at a whole individual (and the actual skills required for a job), not just traditional education and work experience on a resume and how that experience is from a learner-worker perspective. We found that the learner-workers’ were more engaged in “using” the tool because the focus was on them, getting them to capture and reflect on the skills gained from their prior work experience, both paid and unpaid work (like taking care of an elder etc.). The learners really appreciated being “screened in” to jobs that they would not have applied for on their own because of the way the job title is written or other reasons.
For example, a participant at the Chelsea Collaborative, one of our partner sites in Massachusetts, was working as a dishwasher at a restaurant and was looking for similar jobs to apply for at a local resort in Boston. After she completed her SkillSmart profile, she was matched with several jobs that she did not think existed and would really be for someone with her experience. In her words, “It is inspiring for me to see how many skills I actually have after years of working in restaurants. I feel proud every time my Skill Index number goes up.” “I was looking for jobs as a dishwasher but have been encouraged that the SkillSmart system is matching me to jobs that I never even knew existed, like a Casino Porter.” She ended up applying for the job!
Another powerful example from Skillsmart is a woman who had spent 10+ years as a full-time caregiver taking care of a child with disabilities and navigating his health needs. Her experience in managing his health insurance claims and care qualified her for a customer service position with a health insurance provider.
I want to thank Jen, Alison, and Priyanka for joining me today and sharing the TTALE report. Do you have questions or comments about the report, or what was said today? Share them here, and they will be sure to answer as their schedules allow.
I hope you enjoyed the conversation and welcome suggestions for other resources that you're interested in having discussions about in the future.
Career Pathways Moderator