William E. Sedlacek
University of Maryland at College Park
As we move to the 21st century, do we need to examine alternatives to
admissions concepts developed in the early 20th century? The answer is yes! The
measures are already developed and validated, but have not been widely utilized.
Sedlacek (1994) has discussed several reasons for this including what he calls
the "Three Musketters" problem.
The Three Musketeers
The rallying cry of "all for one and one for all" is one that we use often in developing what we think of as fair and equitable admissions measures. Commonly our interpretation of how to handle diversity is to hone and fine-tune our measures so they are equally valid for everyone (see Helms, 1992). However, if different groups have different experiences and different ways of presenting their attributes and abilities, it is unlikely that we could develop a single measure, test item etc. that could be equally valid for all. If we concentrate on results rather than intentions, we could conclude that it is important to do an equally good job of selection for each group, not that we need to use the same measures for all to accomplish that goal. We want equality of results, not process.
Sternberg's (1985, 1986) work on intelligence might prove instructive here. He suggests that there are three kinds of intelligence. Componential intelligence is the ability to interpret information in a hierarchial and taxonomic fashion in a well-defined and unchanging context. People who do well on standardized tests such as the Scholastic Assessment Tests (SAT) or the Graduate Record Examination (GRE) have this type of intelligence. Experiential intelligence involves the ability to interpret information in changing contexts; to be creative. Standardized tests do not measure this type of intelligence, according to Sternberg. Sternberg calls his third type of intelligence contextual; it has to do with the ability to adapt to a changing environment; the ability to handle and negotiate the system.
If Sternberg's types of intelligence are applied to what is typically done in college admissions, there is a heavy concentration on componential intelligence. Applicants who do have traditional White middle or upper-middle class, mostly male-oriented experiences in the society are less likely to show their abilities through componential intelligence. These students will be called nontraditional here and include various racial-cultural groups, international students, women, gay-lesbian and bisexual students, athletes, students with learning disabilities or physical disabilities and older students. The list is intended to be illustrative not exhaustive.
The Noncognitive Questionnaire (NCQ) was developed to assess attributes that are more predictive of success in higher education for nontraditional students than are standardized tests (see Table 1)
Work in assessing nontraditional variables with the NCQ suggests that nontraditional people tend to show their abilities more often through experiential and contextual intelligence. (Boyer and Sedlacek, 1982; Sedlacek, 1989, 1991, 1996a, 1996b; Sedlacek and Adams-Gaston, 1992; Tracey and Sedlacek, 1984, 1985, 1987, 1988, 1989; White and Sedlacek, 1986). Much of this is out of necessity because nontraditional people must learn to be "multicultural" and examine issues from different perspectives, and be able to negotiate a system that was not designed for them. Having long range goals, a self-concept that includes how the system views you and an ability to handle racism are some of the scales on the NCQ. Institutional racism is defined as the negative consequences that accrue to a member of a given group because of the way a system or subsystem operates in the society (e.g., college admissions) regardless of any other attributes of the individual (Sedlacek, 1988). All "isms" (e.g., sexism, ageism) are included under the generic term "racism". The various "isms" take different forms but share a common basis. Thus, if there is a concentration on componential intelligence, less valid assessments will be done for nontraditional persons than for those with more traditional experiences in the system, which would be an example of institutional racism.
It is not that componential intelligence is not important to nontraditional people, it may be that experiential and contextual abilities may be prerequisite (Westbrook and Sedlacek, 1988). If one is struggling with racism in the system, time and energy may not be available for one to show componential talents.
The point illustrates that there is a need to think of measures
differentially in order to achieve equitable assessments for all. There is
probably a classic oxymoron here in thinking that one can assess diversity of
experience with a single measure. Notice that the arguments presented are
positive and proactive. Lowering standards of admission is not being advocated.
The suggestion is to develop and use the most valid measures one can for all
groups that can be operationally defined.
Bias is Bias
Another issue discussed by Sedlacek (1994) he called "bias is bias". Any good methodological reference will tell us to guard against loss of subjects in doing admission research and in developing norms for our instruments ( e.g., Mehrens and Lehmann, 1991). In practical situations, however, there are almost always missing or incomplete data, and the assumption is commonly made that the missing participants do not cause our sample to be biased. When the missing participants are from nontraditional groups, however, that assumption frequently may be incorrect.
When I took my first undergraduate course in statistics, the instructor pointed out the distinction between statistical bias "a consistent overestimate or underestimate of a parameter," and prejudice or social "bias" of some sort. This may not have been a good idea. Much of the work done in admissions has become so esoteric and molecular that many tend to forget or perhaps never think of, the larger implications of the bias issue.
An example of sampling bias was documented by the institutional research office at a university. In order to develop regression equations for use in student selection, data were included for all students whose SAT scores and high school grades were in the university's data base. The equations resulting from this process were found to predict equally well for all races. However, upon further probing, it was noted that students with incomplete data were not included in the equation-generating data set and students in special programs were less likely to be included. The rationale was that the number of exclusions was small and that the researchers were using all the data available.
Analyses of the data from the missing groups showed that the relationships between predictors and criteria were very different for these students. On closer inspection, it was discovered that students for whom the SAT and high school records did not predict as well were less likely to have data in student data base, either because they didn't turn it in, or because they didn't take the SAT and/or were in a special program that generated its own data. Such students were also much more likely to be nontraditional. In this example, the loss of subjects resulted in more sampling bias for nontraditional students than for traditional students. The regression equations produced on the biased data set were used to select students for admission for several years. Consequently, in this case, sampling bias resulted in the use of selection procedures which were less valid or invalid for many nontraditional applicants.
Sampling bias can be reduced by checking the characteristics of population members with missing data, asking ourselves what the effects might be on people with similar characteristics if the missing data are ignored, and working to get data from all members of the sample or population. If it is not possible to get complete data, extraordinary efforts should be made to get data on a well-drawn sample of those missing, so that the extent of the sampling bias can be estimated. Parameter estimates for the population can then be modified or, at the very least, cautions can be given so that the biased equations or norms are treated as more fallible. This should result in less rigid applications of the results of assessment studies to people for whom they may not be appropriate.
As we enter the new century let us continue to search for new ideas and
concepts but let us also remember to put into practice those things we have
already learned.
References
Boyer, S. P., & Sedlacek, W. E. (1988). Noncognitive predictors of academic success for international students: A longitudinal study. Journal of College Student Development, 29, 218-222.
Helms, J. E. (1992). Why is there no study of cultural equivalence in standardized cognitive ability testing? American Psychologist, 47, 1083-1101.
Mehrens, W. A., & Lehmann, I. J. (1991). Measurement and evaluation in education and psychology. Ft. Worth, TX: Holt, Rinehart and Winston.
Sedlacek, W. E. (1988). Institutional racism and how to handle it. Health Pathways, 10 (9), 4-6.
Sedlacek, W. E. (1989). Noncognitive indicators of student success. Journal of College Admissions, 1 (Fall) (125), 2-9.
Sedlacek, W. E. (1991). Using noncognitive variables in advising nontraditional students. National Academic Advising Association Journal, 2 (1), 75-82.
Sedlacek, W. E., & Adams-Gaston, J. (1992). Predicting the academic success of student-athletes using SAT and noncognitive variables. Journal of Counseling and Development 70 (6), 724-727.
Sedlacek, W. E. (1994). Issues in advancing diversity through assessment. Journal of Counseling and Development, 72, 549-553.
Sedlacek, W. E. (1996a). An empirical method of determining nontraditional group status. Measurement and Evaluation in Counseling and Development, 28, 200-210.
Sedlacek, W. E. (1996b). Employing noncognitive variables in admitting students of color. In Johnson, I. H., and Ottens, A. J. (Eds.) . (Pp. 79-91). Leveling the playing field: Promoting academic success for students of color. San Francisco: Jossey-Bass.
Sternberg, R. J. (1985). Beyond IQ. London: Cambridge University Press.
Sternberg, R. J. (1986). What would better intelligence tests look like? In Measures in the College Admissions Process (pp. 146-150). New York: The College Entrance Examination Board.
Tracey, T. J., & Sedlacek, W. E. (1984). Noncognitive variables in predicting academic success by race. Measurement and Evaluation in Guidance, 16, 171-178.
Tracey, T. J., & Sedlacek, W. E. (1985). The relationship of noncognitive variables to academic success: A longitudinal comparison by race. Journal of College Student Personnel, 26, 405-410.
Tracey, T. J., & Sedlacek, W. E. (1987). Prediction of college graduation using noncognitive variables by race. Measurement and Evaluation in Counseling and Development, 19, 177-184.
Tracey, T. J., & Sedlacek, W. E. (1988). A comparison of White and Black student academic success using noncognitive variables: A LISREL analysis. Research in Higher Education, 27, 333-348.
Tracey, T. J., & Sedlacek, W. E. (1989). Factor structure of the Noncognitive Questionnaire-Revised across samples of Black and White college students. Educational and Psychological Measurement, 49, 637-647.
Westbrook, F. D., & Sedlacek, W. E. (1988). Workshop on using noncognitive variables with minority students in higher education. Journal for Specialists in Group Work, 13, 82-89.
White, T. J., & Sedlacek, W. E. (1986). Noncognitive predictors of grades and
retention for specially admitted students. Journal of College Admissions,
3, 20-23.
Table 1: Scales on Noncognitive Questionnaire (NCQ)*
Positive Self-concept or Confidence. Strong self-feeling, strength of character. Determination, independence.
Realistic Self-appraisal, especially academic. Ability to recognize and accept any deficiencies and works hard at self-development. Recognizes need to broaden his/her individuality.
Understanding and dealing with Racism. Realist based upon personal experience of racism; committed to fighting to improve existing system; not submissive to existing wrongs, nor hostile to society, nor a "cop-out." Able to handle racist system; asserts school or organization role to fight racism.
Preference for Long-range Goals over Short-term or Immediate Needs. Able to respond to deferred gratification.
Availability of Strong Support Person. Someone available to turn in crises.
Successful Leadership Experience. Evidence of influencing others in any area pertinent to background (gang leader, church, sports, noneducational groups, etc.)
Demonstrated Community Service. Involvement in his or her cultural community.
*The NCQ is available from the author at Counseling Center, University of Maryland, College Park, MD 20742-8111 or Email: wsed@umd.edu.