Sedlacek, W. E. (in press). Noncognitive measures for
higher education admissions. In B.
McGaw, E. Baker, & P. L. Peterson
(Eds.). International Encyclopedia of Education,
Third
Edition.
Abstract
Standardized tests and grades have serious limitations
in telling us what we need to know about the academic potential of applicants
from diverse backgrounds and cultures. An alternative system of assessment
based on noncognitive variables is presented. How to measure variables such as
self-concept, nontraditional leadership, long-term goals, developing a
community, and handling racism are covered. Implications for admissions in
higher education are discussed.
We appear to have forgotten why tests were created in the first place. While they were always considered to be useful in evaluating candidates, they were also considered to be more equitable than using prior grades because of the variation in quality among preparatory schools. The College Board has long felt that the SAT was limited in what it measured and should not be relied upon as the only tool to judge applicants (Angoff, 1971).
In 1993, the verbal and mathematical reasoning sections of the SAT were
lengthened and the multiple-choice Test of Standard Written English was
dropped. The name was changed from Scholastic Aptitude Test to Scholastic Assessment
Tests, while retaining the SAT initials. Currently it is just called the
SAT-I. In 2003, the College Board
announced that an essay would be added and the analogies item type removed as
of 2005. Despite various changes and
versions over the years, the SAT in essence measures what it did in 1926,
verbal and math ability; it is basically still a general intelligence test
(Sedlacek, 2003, 2004).
However, we
have come to the point where the “Big Test” has become the focal point in our
schools (Lemann, 2000). It has become the standard by which we judge ourselves
and others. Many assume that if an individual has high SAT scores, or if a
school has high mean SAT scores, the students must be learning something, and
the school must be good. To cite that common metaphor; the tail is wagging the
dog.
Test
results should be useful to educators, student service workers, and
administrators, by providing the basis to help students learn better and to
analyze their needs. As currently designed, tests do not accomplish these
objectives. Many teachers tend to teach to get the highest test scores for
their students, student service workers may ignore the tests, and too many
administrators are satisfied if the average test scores rise in their school.
We need some things from our tests that currently we are not getting. We need
tests that are fair to all and provide a good assessment of the developmental
and learning needs of students, while being useful in selecting outstanding
applicants. Our current tests don’t do that.
The world
is much different than it was when the SAT and other tests were developed in
the last century. International students, women, people of color, gays,
lesbians and bisexuals, among others, are participating in higher education in
more extensive and varied ways (Knapp, Kelly, Whitmore, Wu & Gallego, 2002).
Commonly employed tests have not kept up with these changes (Sedlacek, 2004).
We need a
fresh approach. It is not good enough to feel constrained by the limitations of
our current ways of conceiving of tests. Instead of asking; “How can we make
the SAT and other such tests better?” we need to ask; “What kinds of measures
will meet our needs now and in the future?” The purpose of this chapter is to
present the underlying logic and research supporting a method that yields such
measures. We do not need to ignore our current tests, we need to add some new
measures that expand the potential we can derive from assessment.
Noncognitive is used here to refer to variables relating to adjustment,
motivation, and student perceptions, rather than the traditional verbal and
quantitative (often called cognitive) areas typically measured by standardized
tests (Sedlacek, 1998a,b;2004). While noncognitive variables are useful for all
students, they are particularly critical for nontraditional students, since
standardized tests and prior grades may provide only a limited view of their
potential. Below is a discussion of the eight variables recommended to be
included in admissions assessment systems (see Exhibit 1).
--------------
Insert
Exhibit 1
--------------
There is evidence that the way students feel about themselves is related
to their adjustment and success in college (Sedlacek, 2003, 2004). A strong self-concept is particularly
important for students of color, students with disabilities, and women
returning to school.
A number of studies have shown that a positive self-concept correlates
with college grades, retention, and graduation, particularly the later two, for
regularly admitted African American students. O’Callaghan and Bryant (1990)
found self-concept important for the success of Black American students at the
U. S. Air Force Academy. A positive self-concept has also shown to be important
for Asian and Pacific Islanders, Latinos, international students, and women, among others, (Ancis,
Sedlacek, & Mohr, 2000).
Realistic self-appraisal is the
ability to assess one’s strengths and weaknesses and allows for self-development. Realism in self-appraisal by nontraditional
persons does not connote cultural, racial, or gender deficiency or inferiority.
White students may do well pursuing their own interests (internal
control) in a society designed to meet their needs, while students of color
need to also be aware of the external control on their lives which requires
them to negotiate the racism in the system (Sedlacek, 2003, 2004, 2005). Perceived
intrinsic interest in a field has been shown to be a major barrier to achieving
career goals for Whites and Asian Americans, while African Americans, Latinos,
and Native Americans cited personal finances as their major barrier.
Realistic self-appraisal has been found to correlate with college grades,
retention and graduation for students of all races, but the relationships were
particularly strong for African Americans. Women who are able to make realistic
self-appraisals have been shown to get higher grades in a university than those
who have difficulty with such assessments (Sedlacek, 2004).
Understands
and Deals with Racism
The successful nontraditional student is a realist based on a personal
experience with discrimination; is committed to fighting to improve the
existing system and is able to handle a racist system. Institutional
racism is defined as the negative consequences that accrue to a member of a
given group because of the way a system or subsystem operates in the society
(e.g., college admissions) regardless of any other attributes of the individual
(Sedlacek, 2003, 2004). Racism can take
many forms and is used here to cover all types of ‘isms” (e.g., sexism, ageism,
athletism). While racism can be “individual” rather than institutional, the
primary concern here is for dealing with the policies procedures and barriers,
intentional or not, that interfere with the development of people.
For traditional students, the variable takes the form of handling the
system without the addition of racism (Sedlacek, 2003, 2004, 2005)). How one
learns to handle the circumstances with which they are confronted tells us much
about their ability and potential.
Steele’s (1997) work on “stereotype threat” supports the importance of the psychological set with which examinees approach a test. If African Americans are told that they do not usually do well on a test, they do less well than if a more positive set is given. Therefore, for African Americans, the act of taking a test probably involves dealing with the racism that may have been involved in helping to create a stereotype threat in the first place. Hence, part of the variance that is being measured when an African American takes the SAT is likely to relate to how that person handles racism.
Having long-range goals will predict success in college for students. Since role models often are more difficult to find, and the reinforcement system has been relatively random for them, many nontraditional students have difficulty understanding the relationship between current efforts and the ultimate practice of their professions (Sedlacek, 2003, 2004, 2005).
A significant relationship has been found between
setting long-range goals and grades and retention for international students,
including
international community college students (Boyer
& Sedlacek, 1988).
Students who have done well in school tend to
have a person of strong influence who provides advice to them, particularly in
times of crisis (e.g., Sedlacek, 2003, 2004, 2005). This individual may be in the education system,
but may be a relative or a community worker. Evidence is available for women,
athletes of all races, and international students (Bennett & Okinaka, 1990).
This variable involves having someone beyond a role model, although that may be
part of the relationship. A strong support person takes an active role in
advising and directing someone.
Students who are most successful in higher education have shown an
ability to organize and influence others.
The key here is that nontraditional students may show evidence of
leadership in different ways than their White counterparts (Liu & Sedlacek,
1999).
Asian American students have unique and culturally related ways of
expressing their leadership. If an
applicant succeeds in his or her culture and is now ready to “take on” college,
there is evidence that the student has the potential to succeed. There is evidence
of the value of leadership in the retention of Latinos and Native
Americans, and African Americans in undergraduate and medical school programs (Webb, C. T., Sedlacek, W. E.,
Cohen, D., Shields, P., Gracely, E., Hawkins, M., & Nieman, L.,1997. Leadership
has also been identified as a correlate of success for women for female and
male international students (Sedlacek, 2004).
Having a community with which students can identify, and from which they
can receive support, is critical to their academic success. For White students, there tend to be a number
of opportunities to find a community, in or out of school. The community for
nontraditional students often is based on racial, cultural or gender-related
variables. Students of color, women, and other persons with nontraditional
experiences who are active in a community learn how to handle the system,
exhibit leadership, and develop their self-concepts in such groups. For example, it has been found that African
American students who used campus athletic facilities and certain student union
programs were more likely to stay in school than those who did not (Mallinckrodt
& Sedlacek, 1987).
Identification with a community has been shown to be important for Asian
American, Latino, and Native American success in school, and for male and
female athletes of all races. Community also has been shown to be a correlate
of success for undergraduate women, teachers in training and international
students (Sedlacek, 2004).
The ability of someone to learn from experiences outside the classroom
correlates with their success in school. Persons of color are more apt to learn
and develop using methods that are less traditional and are outside the
education system. The methods may be
culture or gender-related and the field itself may be nontraditional students,
and women (Sedlacek, 2004). This learning should be demonstrated in an
admissions or scholarship application, rather than just listed as a job or
internship. Several schools and programs use this method of assessment as
discussed below.
The Noncognitive Questionnaire
(NCQ) was designed to assess the eight noncognitive variables discussed above
and shown in Exhibit 1. (Sedlacek, 1996). Several
forms of the NCQ have been developed and employed in different selection
contexts 2-week test-retest reliability estimates on NCQ scores ranging
from .74 to .94, with a median of .85 (Sedlacek,
2004). Inter-rater reliability on scores from the three open-ended NCQ
items ranged from .73 to 1.00.
The variables shown in Exhibit 1 have been successfully assessed in ways other than a version of the NCQ. In the Gates Millennium Scholars program, a review of an entire application is scored on the noncognitive variables, and makes up about 80% of the weight used in selection. The application includes short- answer questions based on each of the noncognitive variables shown in Exhibit 1, a personal statement by the applicant, letters of recommendation by the nominator and another person, and demographic, background, and activity questions. Raters were trained to identify and consider all this information in scoring each of the eight noncognitive variables. The raters were educators of color, familiar with multicultural issues in education, and in working with the kinds of students that were applying. Inter-judge reliability was estimated at .83 for a sample of raters in the first year (Sedlacek & Sheu, 2004). More than 11,000 Gates Scholars have attended more than 1450 different colleges and universities with a 97% first-year retention rate, an 87% five-year retention rate and a 78% five-year graduation rate. More than 60% are majoring in STEM (science, technology, engineering, mathematics) fields. Their Realistic Self Appraisal score has a significant relationship with their first-year college GPA and their Leadership score has a significant relationship with engaging in academic activities while in college.
Oregon State University (OSU) has developed a selection system based on the noncognitive variables show in Exhibit 1. The OSU application contains six short- answer questions which cover the eight noncogitive variables. Responses are limited to 100 words and are scored independently from other application materials. Raters from many parts of the campus are trained to score the six questions. Inter-judge agreement was estimated at 85. OSU uses its system in selection, academic advising, student services, on and off campus referrals, financial aid, and teaching. OSU noncognitive scores correlate with retention, and since employing noncognitive variables the OSU retention rate is higher, there is more diversity in the applicant pool and first-year class, campus offices are working better together, applicant GPA is up, referrals are better, and new courses and student services have begun based on the noncognitive information.
The noncognitive variables can be
used along with any other variables, models, or techniques that are employed in
whatever role or type of mentoring or advising is involved. Teachers, advisors,
or counselors who use the system can expect to obtain better student outcomes
in terms of grades, retention, and satisfaction, as well as greater
satisfaction themselves in employing something systematic with demonstrated
utility in an area that often produces confusion and anxiety. Sedlacek, Benjamin,
Schlosser, & Sheu (2007) provide examples and case studies of how
noncognitive variables can be used in postmatriculation or precollege programs.
References
Ancis, J. R., Sedlacek, W.
E., & Mohr, J. J. (2000). Student perceptions of the campus cultural
climate by race. Journal of Counseling and Development,
78(2),
180-185
Angoff,
W. H. (1971). The
College Board admissions testing program.
NewYork: College Entrance Examination Board.
Bennett, C., & Okinaka, A. M. (1990). Factors related to persistence among
Asian, Black, Hispanic, and White undergraduates at a predominantly White
university: Comparison between first and
fourth year cohorts. Urban Review, 22, 33-60.
Boyer,
S. P., & Sedlacek, W. E.
(1988). Noncognitive predictors
of academic success for international students: A longitudinal study. Journal
of College Student Development, 29,
218-222.
Fuertes,
J. N., Sedlacek, W. E, & Liu, W.
(1994). Using the SAT and
noncognitive variables to predict the grades and retention of Asian American
university students. Measurement and Evaluation in Counseling and
Development, 27, 74-84.
Knapp, L G., Kelly, J E.,
Whitmore, R W., Wu, S, & Gallego, L M. (2002). Enrollment in
Postsecondary Institutions, Fall 2000 and Financial Statistics, Fiscal Year
2000 (NCES 2002-212).
Lemann,
N. (2000). The big test: The secret
history of the American meritocracy.
Liu, W.M, & Sedlacek,
W.E. (1999). Differences in leadership and co-curricular
perception among male and female Asian Pacific American college students. Journal
of The Freshmen Year Experience, 11,
93-114.
Mallinckrodt,
B, &, Sedlacek, W. E. (1987).
Student retention and the use of campus facilities by race. National
Association of Student Personnel Administrators Journal, 24 (3), 28-32.
O’Callaghan,
K. W., & Bryant, C. (1990).
Noncognitive variables: A key to Black-American academic success at a
military academy. Journal of College Student Development, 31, 121-126.
Sedlacek,
W. E. (1996). An empirical method of determining nontraditional group
status. Measurement and Evaluation in Counseling and Development, 28,
200-210.
Sedlacek,
W. E. (1998a, Winter). Multiple choices for standardized tests. Priorities,
10 ,1-16.
Sedlacek, W. E.
(1998b). Admissions in higher education:
Measuring cognitive and noncognitive variables.
In D. J. Wilds & R. Wilson (Eds.). Minorities in higher education 1997-98: Sixteenth annual status report (pp.
47-71).
Sedlacek,
W. E. (2003). Alternative measures in
admissions and scholarship selection. Measurement
and Evaluation in
Counseling and Development
35, 263-272.
Sedlacek, W. E. (2004). Beyond the big test: Noncognitive assessment in higher education.
Sedlacek,
W. E. (2005). The case for noncognitive measures.
In W. Camara & E. Kimmel (Eds.). Choosing
students:
Higher education admission
tools for the 21st century. (Pp. 177-193).
Sedlacek, W. E.,
Benjamin, E., Schlosser, L. Z., & Sheu, H. B. (2007). Mentoring in
academia: Considerations for diverse populations.
In
T. D. Allen & L. T. Eby (Eds.), The Blackwell handbook of mentoring: A
multiple perspectives approach (pp. 259-280).
Steele,
C. M. (1997). A threat in the air: How stereotypes shape intellectual identity
and performance. American Psychologist,
52, 613-629.
Webb, C. T., Sedlacek, W. E., Cohen, D., Shields, P., Gracely, E., Hawkins, M., & Nieman, L. (1997). The impact of
nonacademic variables on performance at two medical schools. Journal of the National Medical Association, 89 (3), 173-180.
Suggested Further
Ancis,
J. R., & Sedlacek, W. E.
(1997). Predicting the academic
achievement of female students using the SAT and noncognitive variables. College
and University, 72 (3),1-8.
Bennett, C. I. (2002). Enhancing ethnic diversity at a Big Ten
university through project TEAM: A case study in teacher education. Educational Researcher, 31, 21-29.
Fuertes,
J. N., & Sedlacek, W. E.
(1995). Using noncognitive
variables to predict the grades and retention of Hispanic students. College
Student Affairs Journal, 14, 30-36.
Harvey,
W. B. (2002). Minorities in higher education 2001-2002: Nineteenth annual status
report.
McTighe
Musil, C., García, M., Hudgins, C. A., Nettles, M. T., Sedlacek, W. E., &
Smith, D. G. (1999). To form a more
perfect union: Campus diversity initiatives.
Mohr,
J. J., & Sedlacek, W. E.
(2000). Perceived barriers to
friendship with lesbians and gay men among university
students. Journal of College Student Development, 41,
70-79.
Moore,
S. K. (1995). Indicators of academic
success and the student characteristics of international students at
Neville, H. A., Heppner, P., & Wang, L. (1997). Relations among racial identity
attitudes, perceived stressors, and coping
styles in African American
college students. Journal of Counseling and Development, 75, 303-311.
Patterson, A., Sedlacek, W. E., & Scales, W. R. (1988). The other minority: Disabled student backgrounds and attitudes
toward their university and its services. Journal of Postsecondary Education and Disability, 6, 86‑94.
Sandlin,
M. (2008) Insight resume training. http://oregonstate.edu/admissions/2004req/2004profile.html
Sedlacek,
W. E. (1994). Issues in advancing diversity through assessment. Journal of Counseling and Development, 72,
549-553.
Sedlacek,
W. E. (1995). Using research to reduce racism at a university. Journal of Humanistic Education and
Development, 33, 131-140.
Sedlacek,
W. E., & Brooks, G. C., Jr. (1976). Racism in American education: A model for
change.
Ting,
S. R. (1997). Estimating academic
success in the first year of college for specially admitted White students: A
model combining cognitive and psychosocial predictors. Journal
of College Student Development, 38, 401-409.
White,
C. J., & Shelley, C. (1996). Telling
stories: Students and administrators talk about retention. In I. H. Johnson and
A. J. Ottens (Eds.), Leveling the playing
field: Promoting academic success for students of color (pp. 15-34).
Exhibit 1
Description of
Noncognitive Variables
Variable #
|
Variable Name |
|
1 |
Positive Self-Concept ·
Demonstrates confidence,
strength of character, determination, and independence. |
|
2 |
Realistic Self-Appraisal ·
Recognizes and accepts any
strengths and deficiencies, especially academic, and works hard at
self-development. Recognizes need to
broaden his/her individuality. |
|
3 |
Understands and Knows How to Handle Racism (the System) ·
Exhibits a realistic view
of the system based upon personal experience of racism. Committed to improving the existing
system. Takes an assertive approach to
dealing with existing wrongs, but is not hostile to society, nor is a
"cop-out." Able to handle
racist system. |
|
4 |
Prefers Long-Range to Short-Term or Immediate Needs ·
Able to respond to
deferred gratification, plans ahead and sets goals. |
|
5 |
Availability of Strong Support Person ·
Seeks and takes advantage
of a strong support network or has someone to turn to in a crisis or for
encouragement. |
|
6 |
Successful Leadership Experience ·
Demonstrates strong
leadership in any area of his/her background (e.g. church, sports,
non-educational groups, gang leader, etc.). |
|
7 |
Demonstrated Community Service ·
Participates and is
involved in his/her community. |
|
8 |
Knowledge Acquired in or about a Field ·
Acquires knowledge in a
sustained and/or culturally related ways in any field. |