LINCS Discussion on education applications of Artificial Intelligence (AI), Virtual Reality (VR) and Augmented Reality (AR)

Welcome to our week-long discussion on education applications of Artificial Intelligence (AI), Virtual Reality (VR) and Augmented Reality (AR). We will begin today, Monday, October 14th, with an opportunity for our panelists to provide further information about their work and interests in these areas, and with a focus on Artificial Intelligence. On Tuesday we will continue a focus on Artificial Intelligence; on Wednesday we will add a focus on Virtual Reality; on Thursday we'll add a focus on Augmented Reality. The panel discussion will continue through Friday.

I will next re-post in this discussion thread some of the early posts that were included in the bulletin announcing this discussion. My apologies for this duplication to those who get posts and comments by email, but I think this will enable everyone to have the same information in one discussion thread.

David J. Rosen, Moderator

LINCS CoP Integrating Technology and Program Management groups

 

 

Comments

I totally agree with Bob's comment that limited access to data is slowing down progress in developing adaptive instructional systems.  Educational institutions and university researchers are at an extreme disadvantage compared to large corporations in accessing data.  Projects in academia need to have projects approved by an Institutional Review Board, including access to data, whereas the large corporations have little or no similar constraints on their products.  Accessing data collected in schools and high stakes tests is very difficult because of privacy and legal barriers, so it is difficult/impossible to track progress on a student from school to school.  Some people have ethical concerns about their data being tracked in educational settings, but have no concerns about Amazon tracking them to sell products to them -- interesting.  A software corporation can offer a product with the understanding that the users are being tracked (embedded in dense legalese that people miss).  However, a project on a large research grant requires an informed consent process that makes the nature of the decision more explicit. This is why big data is easier to obtain in the corporate sector. 

In work with Squirrel AI in China.  Access to student data is much easier for researchers and the sample sizes are much larger.  Sorting out all of the ethical issues has all sorts of challenges.  .  

Agree with Art and Bob that data sets are limiting the training capabilities of AI to perform what educators and learners need in terms of predicting difficulties and outcomes or selecting the best path for differentiated instruction.

One example: we have a large data set of learner lesson data including scores from our Learning Upgrade system.  If we had within our data set Outcome data including GED/HiSET pass/fail or scores, CASAS / TABE scores, etc. we could train AI to predict outcomes or select an ideal path through our lessons.

Without the outcome data, much of what is called AI within learning systems is tied into optimizing for internal outcomes within a vendor's system, such as completing a course, which may not be effective for overall learner outcomes.

There is an opportunity with systems within Adult Ed such as TOPS, TEAMS, etc to aggregate learning system data such as ours with outcome data they already have to do some of this analysis.  The key is for multiple parties to work together to integrate data and then do the analysis, which can be done without learner personal identifying info.  Perhaps one day, there will be public data sets that can be analyzed with this type of adult ed data.

-- Vinod

 

Sharing the data sets would be a very worthwhile early step along with putting it into a central facility (obviously protecting the privacy of individuals).  LearnSphere (http://learnsphere.org/) is an infrastructure to achieve this goal for AI and other adaptive learning environments.  LearnSphere is a an NSF grant in the cyber infrastructure area that is a collaboration between Carnegie Mellon, University of Memphis, Stanford, and MIT.  The Memphis team has started the ball rolling in adult literacy by putting up AutoTutor data for comprehension training in the Center for the Study of Adult Literacy.  There are also data sets in math for Mathia and possibly ALEKS.  LearnSphere protects privacy, includes documents on research ethics (Institutional Review Board Approval), organizes data in aformat to share data, and has data analysis workflows for statistical analyses.  Our adult learning community could build on LearnSphere with follow up funding. 

 

ALEKS (https://www.aleks.com), Khan Academy, and Spark3000 (https://www.edsurge.com/product-reviews/spark3000) are three examples of adaptive rule-based instructional systems that have been adopted at some scale in adult learning settings.  ALEKS and Khan Academy in math and Spark3000 in adult literacy.  

Hello David,

Great discussion. I caught up on posts.

Last year a read a paper put out by the McKinsey Global Institute (MGI) and found at https://tinyurl.com/ydanzcg7

Their conclusions were that artificial intelligence technologies with their efficiency and effectiveness are well suited to achieving crucial education objectives and developing essential in the 21st century.

A few of the interesting questions and ideas that were written on were not the question of whether AI, VR, or AR will help to engage and help further student learning, but more on "costs, ethical issues, starting with who owns data on students, who can see it, who can use it, and for what purposes."

However, I just wanted to put this read out there because it also had some good examples on AI, VR, or AR can do: bridging the skills gap, attracting students and keeping them, personalizing instruction, and giving teachers more time to mentor and coach.

One of the most encouraging lines in the article and something I will continue to look into is: "Artificial intelligence clearly presents significant opportunities to raise the quality of education to a level that our current standardized-curriculum-and-testing systems have not been able to achieve and allow a shift of teachers’ focus on higher-value creative and interpersonal tasks." 

Hello panelists and Integrating Technology and Program Management members,

Today is the second of five days of discussion with our expert panel of researchers and practitioners in AI, VR and AR. We will continue our focus today on Artificial Intelligence. I hope discussion participants -- teachers, program managers, researchers, professional development people, librarians and others, -- will take advantage of this opportunity to post questions and comments today and throughout the week. If you have posted a question or comment, I encourage you to follow it up, or to post others. Let's have a lively dialogue this week!

In that spirit, I have more questions for our AI panelists:

7. This is a broad question for our AI panelists, and one that many of the members in this discussion may have: What has research shown to be effective in the use of ITS?

8. Another question that many members of this discussion mayl have: What are the limitations of literacy and math ITS in their present state of development?

9. What might be some recent advances in AI development that offer exciting possibilities for education, especially adult basic skills education, including ESL/ESOL?

10. Is the scoring of the GED(R) test, which has been described as completely automated, including the written response items in the RLA and Science tests, an example of a machine learning application in education, of Automated Essay Scoring (AES)?

11. This is a question for panelists and others in this discussion: an intriguing part of AI to me is natural language processing, What are some of the most promising applications for its use in education. For example, I understand that some adult basic skills (including ESL/ESOL) teachers and learners are using virtual personal assistants, such as Amazon’s Alexa,  Apple’s Siri, and others in their classrooms.

David J. Rosen, Moderator

LINCS CoP Integrating Technology and Program Management group

What are the limitations of literacy and math ITS in their present state of development? In this case predominance of. There are a number of AI based math apps and here are some:

  • IXL Learning

  • Maths Formulas Free

  •  BBC Skillswise

  •  Khan Academy

  • Math Brain Booster Games

  • King of Maths

  • GeometryPad

However, some of the most popular math apps using AI can be lumped into a Cheating App Group. Open the app, take a picture and problem solved. Currently, cheating apps are generally focused on math. Math is a good fit both because many people dislike these problems and because solving a math problem is usually straightforward and pattern-based with  a single solution, and only a handful of methods to find it. An ethical and policy making dilemma. Here are some of those apps: 

  • Photomath
  • Mathway
  • SnapCalc – Math Problem Solver
  • Socratic by Google
  • Slader Math Homework Answers
  • FastMath – Take Photo & Solve
  • Brainly – The Homework App
  • MathPapa – Algebra Calculator
  • Chegg Study – Homework Help
  • Cymath – Math Problem Solver
  • MathScript Calculator

 

 

David, in response to your Question #7 about what the research says about the effectiveness of ITSs, in my paper I summarize the results of a 2014 review of the ITS effectiveness research literature....

Ma, W., O. O. Adesope, J. C. Nesbit, and Q. Liu, “Intelligent Tutoring Systems and Learning Outcomes: A Meta-Analysis,” Journal of Educational Psychology, Vol. 106, No. 4, 2014, pp. 901–918.

https://www.apa.org/pubs/journals/features/edu-a0037123.pdf

A 2014 review of the effectiveness research on a variety of ITSs  found that these systems can be relatively effective sources of classroom instruction and support for student learning for topics that are amenable to a rule-based AI architecture. The metaanalysis reviewed the research going back to 1997 (107 studies) and covered a range of content areas in K–12 and postsecondary education  — primarily math, physics, computer science, language, and literacy. The authors found that, when they compared scores on standardized or researcher-developed tests, ITS-based instruction (1) resulted in higher test scores than did traditional formats of teacher-led instruction and non-ITS online instruction and (2) produced learning results similar to one-on-one tutoring and small-group instruction. In general, these results held across grade levels (elementary through postsecondary (Npostsecondary=60)), content domains, ITS type and approach, and study quality (e.g., randomized controlled trials and quasi-experiments).

Unfortunately, the authors did not break out the results for "postsecondary" by 2-year, 4-year, and adult education programs. However, I assume most of the postsecondary research was conducted in 4-year institutions. 

 

 

 

The effectiveness of ITS are indeed promising in several meta-analyses, including the Ma et al. (2015).In addition to a Meta-analysis by Kulik and Fletcher (2015),others are covered in the following two Handbook chapters.

Graesser, A.C., Hu, X., & Sottilare, R. (2018).  Intelligent tutoring systems.  In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, and P. Reimann (Eds.), International handbook of the learning sciences (pp. 246-255).  New York: Routledge. 

Graesser, A.C., Rus, V., Hu, X. (2017).  Instruction based on tutoring. In R.E. Mayer and P.A. Alexander (Eds.), Handbook of Research on Learning and Instruction  (pp. 460-482).  New York: Routledge Press.

Kulik, J.A., & Fletcher, J.D. (2015). Effectiveness of intelligent tutoring systems: A meta-analytic review.  Review of Educational Research,85, 171-204.  

My articles can be downloaded from my website.

 

There are some limitations with current literacy ITS for literacy.  One limitation is that the vast majority of technologies focus on basic reading skills (word decoding, vocabulary) rather than connected discourse and rhetorical structure.  We have tried to fill this gap with AutoTutor to teach comprehension training.  A second limitation is that interests of the adults vary considerably so there needs to be a large repository of examples for training to individual interests.  The interests of individual readers need to be detected somehow, with recommended texts to read that fit their interests (much like the Amazon model that applies machine learning to web exploration behavior).  A third problem is that digital literacy of adult learners is often limited so the technologies are not effectively used.  Fortunately conversational agents can mitigate that problem.  

The problems with current math trainers with AI have also been documents.  First, sometimes the digital interface is difficult to navigate.This can be remedied by a link that accesses a human through chat or video, as is done in tutor.com. Second, the examples of math problems need to be more authentic and relevant to adult lives.  

David - Thank you for facilitating this discussion. At ProLiteracy we are just starting to scratch the surface of using AI in Adult Education, primarily through our partners. One of those partners, Voxy, has a number of AI elements in its digital English Language Learning platform. They use AI speech recognition in the pronunciation practice section of the product. They also use AI in the keyword extraction and text leveling portions of the content processing process. Voxy also uses AI to determine which activity sequence is most appropriate for each learner, given his or her needs / goals / proficiency level / performance.

How AI is defined can be moving target. I came across a presentation from Holon IQ that breaks it into the following five key areas which I found helpful:

  1. Vison - AI used in learning and administrative contexts. Emotion recognition can assist in detecting learners' confusion or engagement while face detection can be used for attendance management. 
  2. Voice - AI using speech to text and voice interface to support learning activities. Application for literacy and language learning (Eg. Voxy) are some of the first to use this.
  3. Natural Language - Deciphering human language is still a difficult AI problem. Some early uses are assessing levels of understanding, providing feedback and plagiarism detection. 
  4. Algorithms - Deep learning and machine learning for "personalized learning" systems. 
  5. Hardware - AI deployed to different devices to reduce latency and lower networking costs. 

HI Kevin,

You wrote that Voxy, uses AI speech recognition in the pronunciation practice section of their product. Do you, or anyone else know how well the speech recognition works? And does it work equally well with dialects/accents?

Daphne Greenberg

Georgia State University

I have used this one on a web based project.

https://tutorialzine.com/2017/08/converting-from-speech-to-text-with-javascript

You can try it here:

https://demo.tutorialzine.com/2017/08/converting-from-speech-to-text-with-javascript/

Pause a half second after clicking Start Recognition before talking or it will miss the first word. Click Pause, or wait two seconds, to see what it has captured.

Will Jones

Unlimited Learning Center

 I'm interested in hearing about panelists' experience with VR and AR application to wrap around services for learners.  My original post highlights an example of using VR to help soon-to-be released women inmates with the transition to post-release reality.  This application of VR as a wrap around support is a novel use of the technology within correctional settings.  More frequent in the news are pieces, like this one, that address how VR is being used to support social awareness and integration of persons on the autism spectrum into  society.  Vocational Rehabilitation, also commonly referred to as 'VR', is a partner to adult education that is also researching VR applications to support individuals with different disabilities.  Virtual Reality for Vocational Rehabilitation (VR4VR) is part of clinical trials designed to assess and train persons with sensory and cognitive disabilities on work-related skills.   VR4VR is a project of the Center for Assistive, Rehabilitation & Robotics Technologies, at the University of South Florida.  

I'm interested in what the panelists here have to say about the potential application of VR/AR/AI in non-academic contexts to support adult learners?  Incarcerated learners and persons with disabilities are already benefitting from these technologies.  Where do you see the technology being applied to help other groups of learners as part of wrap around supports?

Best,

Mike Cruse

Career Pathways and Disabilities and Equitable Outcomes Moderator

michaelcruse74@gmail.com

I'm really enjoying this conversation about the ways AI or AR can be used with adult learners. I've been interested in language, images, and search engines for a long time, mostly how people find pictures online. I don't have as strong of a technical background, but I often think if there isn't way to build a more predictive model to help students see how long it will take them to be HiSET/GED ready based on their self-identified skills. Not too long ago, on NPR's Fresh Air who spoke about the way Cambridge Analytica was able to target individuals, with a high degree of accuracy. 

While a person's online profile is part of a constellation of interests, one thing that helped them was connecting fans of Adam Sandler movies with other data points. That intrigues me. So if we have a knowledge base here, what type of public data or types of data that would help our students? I know there is the NRS data that's there, but it doesn't seem deep enough to be able to really drill down. I'm thinking more of a netflix style predictor where if I like one movie, then it offers me suggestions on other genres I would like. So if I watch Peaky Blinders (which I do), I suddenly might be interested in Dark British Crime Dramas (which I do too). Could something like that be applied to adult learning? As in, if you struggle with multiplying fractions, and you are a visual learner, you might be interested in (insert resource here). However, I'd like to see more of data that would inform it that would be generated from the user, like "I'm tired after work, I learn better in the morning" or "I like listening to music when studying"

Are there any types of specific public data that people like? 

Thanks Lei Lani for these intriguing questions and ideas.  I too have wondered if there might be some greater capacity now with AI to answer the question that so many high school equivalency preparation students have, "How long will it take me before I am ready to take the exam?"  I would love to hear thoughts from our AI experts here and others who have been thinking about this question. My initial thoughts are that "how long?" depends on many variables including, of course, an assessment of the basic skills needed to pass the tests. 

That however is only part of what needs to be considered; here are some other critical variables:

  • How much study and/or class time does the student have per week, and what factors has the student considered in making that estimate?
  • What is the student's level of confidence in being prepared? Sometimes a student is prepared academically to take the test, but lacks the confidence needed.
  • What is the student's motivation for taking the test? For example, is getting, keeping or advancing on on a job contingent on passing it, and is there a deadline involved for these opportunities? 
  • Is the student comfortable studying entirely on his/her own, or does s/he also need a teacher or tutor.
  • Will the student benefit from a cohort of learners studying for the exam, perhaps a peer-study group and or peer support group?

All these variables make it complicated, but perhaps not impossible, to get better predictions of "how long", with the caveat that "life intervenes," i.e. that students' or their family members' sickness, injuries, other emergencies, unexpected family deaths, job changes or losses, and other events in their lives can change the estimated time.

David J. Rosen, Moderator

LINCS CoP Integrating Technology and Program Management groups

David, you list of some of the factors that are likely to be highly predictive of time-to-readiness to pass the HSE. As you correctly note this is a very complicated question that depends on a multitude of factors, the weight of which likely vary for different types of students including by age, prior academic experiences, motivation, availability of academic support, social class, work and family status, presence of learning issues etc. These are the types of complicated questions algorithmic AI is helping to address in other fields. But as I mentioned in a previous post and my paper, one major limitation of the approach is the need for developers to access the necessary historical digitized data from 10s of thousands of students that reflect the target population including a values for each of your prediction variables (or correlated proxy variables) plus the time-to HSE readiness outcome variable. It is my observation that this type of digitized data is extremely rare in education. And if it does exist, education authorities must be willing and able to share this data with third-party developers.