August 25, 3009

Assessing Non-cognitive Skills in Adult Basic Education

Tom Sticht
International Consultant in Adult Education

An online higher education site
(http://www.insidehighered.com/news/2008/05/22/ets) reported on May 22,
2008 that the Educational Testing Service (ETS) was adding a non-cognitive
component called the Personal Potential Index (PPI) to the Graduate Record
Exam (GRE). The web site states, “In the index, three or four professors or
supervisors -- generally those who will also be writing letters of
recommendation -- will answer a series of questions about candidates’
non-cognitive skills in various areas, as well as a more general set of
questions. Applicants will be rated on a scale of 1-5 on questions about
their abilities in these six areas: knowledge and creativity, communication
skills, team work, resilience, planning and organization, and ethics and
integrity. Those filling out the forms would also be able to provide
narrative answers on each of those areas.”


While it is clear that about half of these six factors (knowledge and
creativity, communication skills, planning and organization) are blends of
cognitive and non-cognitive skills, the remaining three factors (team work,
resilience, ethics and integrity) are not generally considered as cognitive
skills and hence comprise the major non-cognitive components of the PPI.


Following up on the discussion of the PPI, above, in the Winter 2009 Policy
Notes newsletter from the Educational Testing Service (www.ets.org) , Dr.
Patrick Kyllonen stated that, “
the next frontier in testing may lie in
assessing the noncognitive skills that influence success in college and the
workplace — such qualities as persistence, integrity, leadership and
motivation
” He goes on to say that a variety of non-cognitive skills are
subject to change across the lifespan. Additional research at ETS has
indicated that assessments of non-cognitive skills significantly improved
the prediction of graduation rates, absenteeism, leadership, and engagement
in college level education.


In the United Kingdom, a recent literature review critiqued the concept of
“non-cognitive” skills and concluded that, “ It is rightly said that ‘many
different personality and motivational traits are lumped into the category
of non-cognitive skills’ (Heckman and Rubenstein, 2001) – too many, we
suggest, for the category as currently understood to be regarded as a
worthwhile resource for policy.” The authors also note that the terms
“cognitive” and “non-cognitive” often appear to refer to overlapping
processes. For these reasons they reject  the term “non-cognitive skills”
in favor of the use of the term “self-regulated learning.” (Duckworth, et
al, 2009)

Still, they indicate that factors such as motivation, self-efficacy,
persistence and others that have been included in the category of
“non-cognitive skills” are included in the category of what they call
“self-regulated learning.”  They go on to conclude that, “There is a
positive overall relationship between self-regulation and academic
achievement. Children and young people with more adaptive personal skills
and learning resources are more likely to succeed academically
” and
“Aspects of self-regulation such as attention, persistence, flexibility,
motivation and confidence can all be improved as a result of effective
teaching and learning practices.” (Duckworth, et al, 2009).


Taken together, these recent reports suggest that more attention is being
given to the assessment of factors other than academically oriented
cognitive tests of oracy, literacy, and numeracy in education for children
and adults. Whether they are called “non-cognitive” or “self-regulated” or
some other name, these aspects of people’s mental lives, appear to have
relationships to success in both academic settings and workplaces.


According to  the research reviewed above, at least some of these
non-cognitive (or semi-cognitive?) factors that are related to academic and
work success are  also modifiable by educational interventions. This
suggests that adult basic education programs might benefit from research to
develop tools for assessing learner’s non-cognitive skills and their growth
in such skills as a result of their participation in adult basic education
programs.


Various studies of ABE outcomes have indicated that based on self-reports,
adults frequently report improvements in self-efficacy (confidence) and
other non-cognitive, affective outcomes of participating in programs. This
provides a baseline literature for new research to find substantive and
validated measures of change in non-cognitive skills to go along with tests
that measure change in the cognitive skills of reading and mathematics that
are presently widely used to evaluate the effectiveness of programs.


Reference

Duckworth, K. et. al (2009). Self-regulated learning: a literature review.
London: Center for Wider Benefits of Learning, Institute of Education,
University of London. (available by Google search)

Tom Sticht
tsticht at aznet.net