We are just a few short months from the official data release for the Program for the International Assessment of Adult Competencies (PIAAC) on October 8th, 2013. PIAAC is a study that assesses key cognitive and workplace skills in twenty-four participating countries and regions, including the United States. PIAAC provides a new and rich international comparison of the adult workforce that will enable the United States to better understand its global competitiveness and benchmark how well education and training systems are meeting emerging skill demand.
We believe that this data – and other PIAAC tools – will be of great interest to you. To learn more about PIAAC and what it can offer, please visit the NCES website. If you would like to receive regular updates about PIAAC, click here to subscribe to our monthly newsletter. If you sign up for our newsletter, we will not share your contact information with anyone else and you can always opt-out of the mailing list.
PIAAC Project Team
American Institutes for Research
Better Equity and Fewer Unsupported Inferences Coming in the PIAAC
Tom Sticht International Consultant in Adult Education
Definition: eq•ui•ty: The quality of being fair and impartial: “equity of treatment.”
In October of 2013, the Organization for Economic Cooperation and Development (OECD) will release some of the findings from the Program of International Assessment of Adult Competencies (PIAAC). This will include results of the assessment of adult literacy skills using methodologies like those used in the earlier International Adult Literacy Survey (IALS) and the Adult Literacy and Lifeskills (ALL) survey. For adult literacy education advocates there are two things about the PIAAC methodology that will be different and will affect how the results of the survey are interpreted in advocacy programs.
For one thing, in the earlier surveys a standard of having an 80 percent probability of getting items correct was used to assign a person to a literacy level, while in the new PIAAC a standard of a 67 percent probability of getting an item correct will be used to designate an adult’s literacy level. The .67 standard was recommended in a 2005 report from the National Academies of Science, National Research Council (NAS/NRC) and serves to reduce the errors of saying someone cannot perform a certain literacy task when in fact they can. Hence the results are more equitable for advocacy purposes.
The Comprehensive Adult Student Assessment System (CASAS), widely used in the United States to assess adult literacy skills, uses a .50 percent probability level for establishing proficiency. This is the traditional psychometric probability level for creating scales because it produces equal probabilities of saying someone cannot do something when they can, as opposed to saying someone can do something when they cannot. While the PIAAC .67 response probability does not reach this level of equity in error making, it is better than the previously used .80 standard for scaling proficiency.
In the 2005 ALL report it was stated that, "Depending on the country, between one-third and over two-thirds of adult populations do not attain skill Level 3, the level considered by experts as a suitable minimum level for coping with the increasing demands of the emerging knowledge society and information economy. " Now this position has been reversed in reports from the 2013 summer institute of the Centre for Literacy in Montreal which was called Learning from IALS, Preparing for PIAAC. There it was reported that “there will be no official statements from OECD or governments about a specific level, e.g. level 3, required to function in everyday life because the data do not support such normative claims.” This is consistent with the earlier 2005 NAS/NRC report which indicated that the practice of identifying the level of skills adults need in order to function adequately in society, as in the ALL report, is based on “unsupported inferences.”
Because of the foregoing, literacy advocates must use caution in making claims that some 30, 60, or 90 million adults lack the literacy skills needed to function adequately in society. This has been the primary use of the results of past adult literacy surveys. For instance, the report of a National Commission on Adult Literacy in 2008 stated that the National Assessment of Adult Literacy (NAAL) “ found that some 93 million lack literacy at a level needed to enroll in the postsecondary education or job training that current and future jobs require.” Somewhat surprisingly, even a 2011 report by cognitive scientists from the NAS/NRC, who should have known better, especially since an earlier 2005 NAS/NRC report cautioned about making “unsupported inferences” from adult literacy surveys, went ahead and stated that the 2003 NAAL “ estimates that more than 90 million U.S adults lack adequate literacy.”
We can anticipate that the PIAAC will present a distribution of percentages of adults in each of five levels and there will be some very low scoring adults and some very highly scoring adults, and a large percentage in the middle range of literacy scores. For adult literacy education advocates, a learner-centered approach can be taken which points out that the Adult Education and Literacy System (AELS) of the United States offers opportunities for adults across a wide range of literacy abilities to increase their knowledge and skills for meeting their personal goals.
7/29/2013<?xml:namespace prefix = o />
Priming for PIAAC: Historical Trends #1
Tom Sticht International Consultant in Adult Education
In October, 2013 the Program for the International Assessment of Adult Competencies (PIAAC) will release early results of the latest international assessment of adult literacy along with assessments of numeracy and the use of information and communications technology. When these results are released they will take their place in a long history of cognitive skills assessments, including literacy.
In a report entitled Adult Literacy in the United States: A Compendium of Quantitative Data and Interpretive Comments, Sticht & Armstrong (1994, online at www.nald.ca) show samples of items and present general results for ten widespread assessments in the United States, ranging from the 1917 Army Alpha and Beta tests, to the 1993 National Adult Literacy Survey (NALS). The latter formed the basic methodology for several later national and international surveys, including the literacy survey in the PIAAC.
Six Trends From Past Adult Literacy Assessments
A number of trends in adult literacy surveys have remained salient across time and are likely to show up in the results for the PIAAC. As a means of priming adult education advocates and program developers for responding to the PIAAC results, below are six trends along with some ways that I have used the trends for advocacy and program development efforts. The first five trends are discussed as a group.
Trends 1-5 1.Higher educated adults perform better than lower educated adults on literacy tests. 2.Better educated adults tend to engage in more reading than less educated adults. 3.Adults who engage in higher amounts of reading tend to perform better on literacy tests. 4.Adults in professional, managerial, and clerical occupations perform better on literacy tests than adults in laborer, agricultural, and other relatively unskilled occupations. 5.Higher income adults perform better on cognitive (literacy) tests than adults with lower incomes.
Advocacy Use: I have used the five trends to advocate for funding for adult literacy education to improve the economic status of adults and the international competitiveness of their nation. The general argument has been that the test data show that there are a range of scores from low, to medium, to high and that by investing in the education of adults we can increase the use of literacy amongst the lower scoring adult population, that both education and increased use of literacy will increase the literacy skills of less able adults and move more of them from lower to higher levels of literacy. This will help them get a job or a better paying job. It will increase the nation’s human capital and render the nation’s workforce more effective in international competitiveness. See the Compendium mentioned above for data and comments along this line of thought.
Program Use: In a book entitled Reading for Working: A Functional Literacy Anthology (online at www.nald.ca) I discussed a program colleagues and I developed which integrated and contextualized literacy instruction into job training for six different career fields. Later I served on the Secretary of Labor’s Commission on Necessary Skills (SCANS) which recommended integrated and contextualized teaching and learning and today there are many adult education programs providing this integrated type of education.
Trend 6. Adults with higher educated parents score higher on cognitive (literacy) tests than those with less educated parents.
Advocacy Use: I called attention to the intergenerational effects of parent’s education on adult literacy in the Reading for Working book. In 1985 I testified to Congress in support of what later became the Even Start family literacy law. The PIAAC background questionnaire asks for parents’ education levels so there will be data to once again advocate for adult education as a means of improving both adults’ lives and their children’s educational achievement.
Program Use: In 1990-91 I worked with Wider Opportunities for Women and demonstrated that adult basic skills programs could produce changes in parental behaviors that could help their children achieve better in school. In 1995 I worked with colleagues at the San Diego Community College District to use parent behavior checklists to “make every ABE class a family literacy class.” In the last decade I have presented numerous workshops illustrated with educational materials which contextualize basic skills instruction with parenting education to promote the intergenerational transfer of cognitive skills from parents to their children.
I hope this has primed some thoughts about advocacy and program use for those working to advance the field of adult literacy education.