Counseling Center

University of Maryland

College Park, Maryland

Construct and Congruent Validity of the

Noncognitive Questionnaire(NCQ)

Paula A. Woods and William E. Sedlacek

Research Report #6 -88

SUMMARY

Although evidence of validity and reliability has been provided on the Noncognitive Questionnaire(NCQ), additional validation studies are needed. In order to further establish construct reliability and validity of the NCQ, new NCQ items were written to establish their relationships to the original 8 NCQ scales.

Stress=coping measures were also identified as another source which would be helpful in establishing congruent validity of the NCQ. As it has been reported that the original 8 noncognitive variables may be related to particular aspects of academic success among minority and non-traditional students, it was felt that stress-coping abilities might be correlated with the NCQ.

Construct and congruent validity of the NCQ was therefore further established by investigating its` correlation with the Perceived Stress Scale (PSS), a stress-coping measure, and correlation of new and original NCQ items.

Retention in higher education has become increasingly important in recent years "for both human value reasons and the continued existence of schools"(Tracey and Sedlacek; 1984). Retention of minority students, in particular, has become an especially vital issue in post secondary education. Research indicates that retention rates for minority students are lower than rates for majority students (Astin, 1982; Sedlacek & Pelham, 1976; Tracey & Sedlacek, 1987).

In order to examine areas that may be closely related to retention for minority students, and ultimately increase retention rates for this group, eight noncognitive variables have been identified (Sedlacek and Brooks, 1976; Sedlacek, 1977, Tracey and Sedlacek, 1984).

Unlike the traditional cognitive predictors of retention and academic success(e.g. G.P.A., SAT scores), the noncognitive variables relate to broader dimensions such as personality, attitudes, and values. The eight noncognitive variables are: (a) Positive self-concept, (b) Realistic self appraisal, (c) Understanding and dealing with racism, (d) Preferring long range goals to short term or immediate needs, (e)Availability of strong support person, (f) Successful leadership experience, (g) Demonstrated community service, and (h) Knowledge acquired in a field.

The eight noncognitive variables have been found to be related to persistence and grade point average, particularly for minority students (Sedlacek & Brooks, 1976; Sedlacek, 1977).In order to better measure these eight variables, Tracey and Sedlacek (1984) designed The Noncognitive Questionnaire (NCQ). The NCQ has been shown to be predictive of continued enrollment for Black students and GPA for Black and White students (Tracey and Sedlacek, 1984, 1985). It has also been predictive of graduation for Black and White students (Tracey and Sedlacek, 1987). In addition, further research has indicated that the NCQ predicted academic success for specially admitted students (White and Sedlacek, 1986), counseling center usage by Black and White U. S. students (Arbona, Sedlacek, & Carstens, 1987), and international students (Boyer & Sedlacek, 1987).

While evidence of validity and reliability of the NCQ has been provided (Tracey & Sedlacek, 1984), the purpose of the present study is to provide additional construct validity and reliability on this instrument.

In order to further establish construct validity and reliability of the NCQ, 65 new NCQ items were written to establish their relationships to the NCQ scales.

Stress-coping measures are one source which may be helpful in establishing congruent validity of the NCQ. The college experience for minorities, particularly at predominantly White institutions, can certainly be a difficult and stressful one. Adaptability, and the ability to persevere and cope with continual stresses in a difficult, often culturally alienating environment is important. In addition, noncognitive variables such as a good degree of self-confidence, understanding of and

ability to deal with racism, and strong social support are a few stress-coping measures needed to get through successfully to graduation. Since it has been shown that noncognitive variables are predictive of certain aspects of academic success among minority and nontraditional students, stress-coping abilities and perception of stress might be correlated with the NCQ.

In order to measure the degree to which life situations are appraised as stressful, Cohen, Kamarck & Mermelstein (1983) designed the Perceived Stress Scale (PSS). The rationale for employing this scale is that the impact of objectively stressful events is determined in large extent by the perception of their stressfulness, not the objective occurrence of the events (Lazarus, 1966, 1977). High correlation of the PSS with certain variables on the NCQ, (e.g. self-confidence, understanding racism), suggests that these variables are related to the subjective experience of stress.

The purpose of the present study will therefore be to further establish construct and congruent validity of the NCQ by investigating its' correlation with the PSS, a stress-coping skills measure, and correlation of new and original NCQ items.

METHOD

Subjects

Subjects employed in this study consisted of 251 incoming freshmen (1987-1988) who attended summer orientation sessions at the University of Maryland, College Park.

Instruments

Noncognitive Questionnaire. (NCQ) The NCQ is an instrument which measures eight non-traditional or non-cognitive variables that have been found to relate to minority student retention. The eight non-cognitive variables on the NCQ are (1) Positive self-concept or self-confidence (2) Realistic self- appraisal (particularly academic), (3) Understands and deals with racism, (4) Prefers long-range goals to short term or immediate needs, (5) Availability of strong support person to turn to in time of crises, (6) Successful leadership experience in an area pertinent to background, (7) Demonstrated community service within ones' cultural community, and (8) Non-traditional or culturally related ways of acquiring knowledge within a field. The NCQ contains 23 items: 18 Likert format items, 2 multiple choice items, and 3 open-ended items pertaining to present goals and past accomplishments.

Test-retest reliability was found to range from .70 to .94 and inter-rater reliability was found to range from .83 to 1.00 for the open-ended items (Tracey and Sedlacek, 1984). Coefficient alpha ranged from .54 to .73 on a modified version of the questionnaire (Lockett, 1980). A principal components factor analysis also showed the NCQ variables to be relatively independent of one another (Tracey and Sedlacek, 1984).

Perceived Stress Scale.(PSS) The PSS is an instrument designed (Cohen, Karmarck, & Mermelstein, 1983) to measure the degree to

which life situations are appraised as stressful by an individual. The PSS consists of 14 items designed to measure the degree to which respondents found their lives unpredictable, uncontrollable, and overloading. Items contained information about perceptions such as feeling nervous and stressed, being upset, being overwhelmed by difficulties, and feeling one has control, or the lack of it, over life in general.

Predictive validity of the PSS is expected to decrease significantly after four to eight weeks given that appraised stress is presumed to be influenced by major life events, daily life hassles, and changes in coping resources over time. Test-retest reliabilities were .85 over a two day interval when examined for college students. Given that the PSS is a state measure, test-retest correlations are expected to be higher for short retest intervals than for long retest intervals.

Analysis

A principal components factor analysis was conducted with the original NCQ items, the new NCQ items, and the PSS, using squared multiple correlations as communality estimates, and varimax rotation of all factors with eigenvalues greater than one.

RESULTS

A total of fifteen factors were identified which accounted for 627 of the common variance (See Table 1). Factor 1 was labeled "Internal-external locus of control" since items dealing with coping resources and perceived ability to control life difficulties loaded highest. Factor 2 was labeled "Achievement and self-improvement". Factor 3 appeared to load highest on items having to do with "Social responsibility and activism". Factor 4 focused on items concerning "Leadership and effectiveness". Factor 5 was labeled "Ethnocentricity". Factor 6 focused on "Professional relationships". Factor 7 appeared most closely related to "Self-awareness issues. Factor 8 concerned items relating to "Resourcefulness". Factor 9 was difficult to interpret and label, but most closely resembled "Community awareness and involvement". Factor 10 related to "Assertiveness and initiative issues". Factor 11 focused primarily on " Social relationships" with others. Factor 12 loaded highest on the "Introversion-extroversion" dimension while Factor 13 focused on items concerning "Social support". Factor 14, also difficult to label, contained items dealing with "Nontraditional learning styles". Lastly, Factor 15 primarily contained items relating to "Self-understanding".

Factors 2, 3, 5, 6, 11, and 14 had their highest item loadings from the new NCQ items.

Construct Validity

Inspection of the factor loadings indicates that the 8 original NCQ scale variables differentiated into several scales and were spread out among 11 of the 15 factors identified in this

study (See Table 1). Three NCQ variables each loaded on Factor 7- "Self-awareness issues" and Factor 10 - "Assertiveness and initiative issues". Two NCQ variables each loaded on Factor 5, which was labeled "Ethnocentricity" and Factor 12, "Introversionextroversion". The remaining 7 factor loadings each contained only one original NCQ scale variable.

The first NCQ variable of "Positive self-concept or confidence" developed into two different scales, revealing loadings on Factor 7 (Self-awareness issues) and Factor 15 (Self-understanding). On Factor 7, this variable had a high positive correlation with another original NCQ variable (Prefers long-range goals to short term needs) and a new NCQ item (Friends all exclusively the same race). On Factor 15, this NCQ variable had a high positive correlation (+.26, +.19) with the new NCQ item (Find I get more comfortable in a new place as soon as I make friends).

Realistic self-appraisal, the second NCQ variable, loaded on Factor 3 (Social responsibility and activism)

and Factor 5 (Ethnocentricity). On Factor 3, this NCQ variable was highly positively correlated with 3 new NCQ items(If I encounter racism, it's up to me to point out and correct; Contact with faculty is important to academic success; and Everyone must work towards improving social conditions). "Realistic self-appraisal also positively correlated with three new NCQ items loading on Factor 5 (See Table 1).

The "Understands and deals with racism" variable did not load on any of the factors. The fourth NCQ variable, "Prefers long term goals to short term or immediate reeds" differentiated into two different scales, loading on Factor 7 (Self-awareness issues) and Factor 12 (Introversion-extroversion). This NCQ variable was positively correlated (+.44 and +.39) with new NCQ item #31(Friends all exclusively the same race) and negatively correlated (-.30) with new NCQ item #47 (Have learned more outside school than in school). On Factor 12, the original NCQ variable loaded with new NCQ item #30 (Enjoy working with others and #43(Know areas where weak and try to improve them).

The "Availability of a strong support person" NCQ variable loaded on three different factors: Factor 5, "Ethnocentricity". Factor 7, "Self-awareness issues" and Factor 13, "Social support". This original NCQ variable was highly negatively correlated (-.35 and -.30) with new NCQ item #47( Have learned more outside school than in school) on Factor 7. It was also positively correlated with 3 new NCQ scale items on Factor 5 (See Table 1). On Factor 13, this NCQ variable was both positively and negatively correlated with two new NCQ items.

The sixth NCQ variable of "Successful leadership experience" did not load on any of the 15 identified factors. The seventh NCQ variable of "Demonstrated community service" also loaded on three factors. The factors were Factor 4 (Leadership and effectiveness), Factor 9 (Community awareness and involvement), and Factor 10 (Assertiveness and initiative issues). On Factor 4, this variable was positively correlated with two new NCQ items: #45-Usually come up with ideas that friends end up doing and

#38-I think many people see racism where it doesn't exist. This variable was correlated with the same two new NCQ items on Factor 9.

The final NCQ variable, "Knowledge acquired in a field" was the most differentiated of all original NCQ variables. This variable loaded on four different factors: Factor 8 (Resourcefulness), Factor 10 (Assertiveness and initiative issues), Factor 12 (Introversion-extroversion) and Factor 14 (Non-traditional learning styles). On Factor 8, this variable was correlated with 3 new NCQ items. It further correlated with 2 new NCQ items on Factor 10, 2 new NCQ items on Factor 12, and 4 new NCQ items on Factor 14 (See Table 1).

Congruent Validity

The Perceived Stress Scale items (PSS) spread out among six different factors (See Table 1), loading highest on Factor 1, which was labeled Internal-External Locus of Control. It should be noted that the top five loadings on Factor 1 were all stress scale items. Two stress scale items, "How often dealt successfully with irritating life hassles", and "Effective coping with important life changes" loaded on Factor 4, which included items concerning leadership and effectiveness. These items loaded with one original NCQ scale item, "Demonstrated community service", and two new NCQ items. Two stress scale items, "How often have you been angered because of things that happened outside of your control" and "How often have you felt on top of

things" also loaded on Factor 13, which focused on items concerning social support. This loading was moderately correlated with the original NCQ scale item of "Availability of strong support person". Factor 15, most closely related to self-understanding, also loaded the two stress scale items, "Have been able to control the way time is spent" and "How often upset because of something that happened unexpectedly", which were both moderately correlated with the NCQ variable of "Positive selfconcept or confidence". One PSS scale item each loaded on Factor 2, "Achievement and self-improvement" (PSS item: How often felt nervous and "stressed") and Factor 12, "Introversionextroversion" (PSS item: "How often thought about things have to accomplish").

None of the 8 original NCQ scale variables clustered with the stress item loading on Factor 2. The two NCQ scale variables of "Knowledge acquired within a field" and "Prefers long-range goals to short term needs" revealed moderately negative correlations with the PSS item which loaded on Factor 12.

DISCUSSION

Based on item loadings, results demonstrated that several of the original noncognitive variables correlated highly with both new NCQ items and the stress questionnaire scale items.

A number of NCQ variables correlated more highly with new items and stress scale items than did others. For example, the noncognitive variable of "Positive self- concept or confidence"

loaded on two factors (Factor 7 and Factor 15) concerning selfawareness or self-concept issues. In addition, it had a strong positive correlation with stress items on one factor (Factor 15). "Positive self-concept or confidence" also demonstrated a strong positive correlation with the new NCQ items. "The Demonstrated community service" variable correlated highly with item loadings on factors having to do with Leadership and effectiveness(Factor 4), Community awareness and involvement (Factor 9), and Assertiveness and initiative issues (Factor 10). "Demonstrated community service" also correlated strongly with both new NCQ items and stress scale items.

The NCQ variable of "Non-traditional knowledge" correlated primarily with the new NCQ items rather than stress scale items. This variable correlated most highly with items pertaining to initiative and openness to non-traditional learning experiences. "Realistic self-appraisal" had high positive correlations with new NCQ items loading under factors of social responsibility and ethnocentric issues. The NCQ variable of "Availability of strong support person" correlated highly with items regarding alienation and self- awareness. This variable correlated with both new NCQ items and stress-scale items.

Three of the 8 original NCQ variables (a) Positive selfconcept or self-confidence, (b) Availability of strong support person and (c) Demonstrated community service in ones'cultural community) correlated most highly with stress-scale items. These variables all appear to be related to stress-coping abilities and adaptability. For example, the high positive correlations found between "Positive self-concept" and stress items concerning self-understanding and self-control may imply that these constructs are closely related. It may further be reasonable to note that lack of self-confidence or positive selfconcept may be presumed to be related to one's ability or the perception of that ability to handle stress situations.

In addition, "Demonstrated community service", which had strong positive correlations with stress scale items concerning effectiveness and leadership issues, and "Availability of strong support person" are both likely to be related to ones' coping resources, particularly for minority students in a culturally alienating environment. Further, "Availability of strong support person" had a strong negative correlation with the stress scale item "How often been angered due to things happening outside control", indicating that the more support one has, the less angry or frustrated one might feel when events occur beyond his/her control. Based on these correlations, one could therefore surmise that the more support the student has, either through ongoing contact with his/her cultural community or one individual, the better will be his/her stress-coping abilities, feelings of effectiveness, and sense of control over the environment.

In total, six of the 8 original NCQ variables correlated with the new NCQ items and 3 of the original NCQ variables correlated with stress scale items.

Results imply that future validation studies reed to be done on the original NCQ scales. However, additional research also needs to further investigate construct validity of two NCQ scales in particular, (a) Understands and deals with racism and (b) Successful leadership experience, as these did not load on any of the 15 factors, indicating no evident correlations with stress scale items or new NCQ items. Perhaps, these two NCQ items should be further explored and more clearly defined in terms of the specific constructs being measured. Excluding these two scales, the remainder of the original NCQ scales appear to be clearly related to the new NCQ items, indicating construct validity for the original instrument.

The fact that results indicated only 3 of the original NCQ scales (a) Positive self-concept or self-confidence, (b) Availability of strong support person and (c) Demonstrated community service in ones'cultural community were clearly related to stress scale items establishes congruent validity for these variables. However, results also imply that further empirical investigation of the remaining 5 NCQ variables is needed in order to establish congruent validity with stress-coping measures or other instruments measuring similar or related constructs.

*ORIGINAL NCQ VARIABLE LOADING

TABLE 1: CONSTRUCT VALIDITY AND CONGRUENT VALIDITY

CONSTRUCT VALIDITY

ITEM LOADINGS FOR ORIGINAL NCQ VARIABLES AND NEW NCQ ITEMS

(5 largest item loadings on 11 factors)

KEY: ITM - PSS items 1-14

TSCR- Original 8 NCQ variables scored as single value

LIK - New NCQ items

FACTOR 3:

+.50 RACE

+.48 LIK34:If encounter racism, up to me to point out and correct

+.46 LIK44: Contact with faculty important to acad. success

+.45 LIK42: Everyone must work towards improving social conditions

+.38 *TSCR2: Realistic self-appraisal up

FACTOR 4:

+.40 *TSCR7: Demonstrated community service

+.39 LIK45: Usually come up with ideas that friends end doing

+.39 ITM 4: How often dealt successfully with irritating life hassles

+.34 ITM 5: Effective coping with important life changes

+.32 LIK38: Think many people see racism where it doesn't

exist

FACTOR 5:

+.42 *TSCR2: Realistic self-appraisal

+.40 LIK31: Friends are all exclusively same race

+.39 LIK36: Uncomfortable interacting with people from other races and cultures

+.32 LIK41: Expect to get picked on by faculty/students because of background

+.31 *TSCRS: Availability of strong support person

FACTOR 7:

+.44 *TSCR4: Prefer long-range goals to short-term needs

+.41 *TSCR1: Positive self-concept

+.39 LIK31: Friends all exclusively same race

-.35 *TSCRS: Availability of strong support person

-.30 LIK47: Have learned more outside school than in school

FACTOR 8:

-.47 LIK46: Often makes lists of things to do

-.38 LIK45: Usually comes up with ideas friends end up doing

-.34 SEX

-.34 *TSCR8: Knowledge acquired in a field

-.31 LIK47: Have learned more outside of school than in school

FACTOR 9:

-.47 RACE

-.38 LIK45: Usually come up with ideas friends end up doing

+.30 SEX

-.27 *TSCR7: Demonstrated community service

-.29 LIK38: Think many people see racism where it doesn't exist

FACTOR 10:

+.40 *TSCR8: Knowledge acquired in a field

+.40 LIK40: When treated unfairly, express anger in no uncertain terms

+.37 *TSCR7: Demonstrated community service

-.29 LIK38: Many people see racism where it doesn't exist

+.28 *TSCR6: Successful leadership experience

FACTOR 12:

+.41 LIK30: Enjoy working with others

-.41 ITM12: How often thought about things have to accomplish

-.30 LIK43: Know areas where weak and try to improve them

+.27 *TSCR8: Knowledge acquired in a field

+.25 *TSCR4: Prefer long-range goals to short-term needs

FACTOR 13:

+.37 *TSCR5: Availability of strong support person

-.35 LIK47: Have learned more outside school than in school

-.31 ITM11: How often been angered due to things happening outside control

+.28 LIK30: Enjoy working with others

+. 27 IMO: How often felt on top of things

FACTOR 14:

-.59 LIK40: When treated unfairly, express anger in no uncertain terms

+.33 *TSCR8: Knowledge acquired in a field

-.33 LIK37: Try to find opportunities to learn new things

+.25 LIK30: Enjoy working with others

+.22 LIK31: Friends are exclusively same race

FACTOR 15:

+.58 LIK139: Have good understanding of strengths and weaknesses

+.33 ITM13: Have been able to control the way time is spent

+.26 *TSCR1: Positive self-concept or confidence

+.19 LIK29: Find I get more comfortable in new place as soon as I make new friends

+.19 ITM 1: How often upset because of something that happened unexpectedly

CONGRUENT VALIDITY

PERCEIVED STRESS SCALE (PSS) ITEM LOADINGS

(5 highest item loadings on 6 factors)

**PSS item loadings

FACTOR 1:

-.70 **ITM10: How often felt on top of things

-.66 **ITM 9: How often able to control irritations

+.66 **ITM14: How often could not overcome difficulties

-.63 **ITM 6: How often felt confident about ability to handle personal problems

-.62 **ITM 5: Effective coping with important life changes

FACTOR 2:

-.49 LIK46: Often makes lists of things to do

+.48 LIK43: Knows weak areas and tries to improve them

+.47 LIK42: Everyone must work towards improving social conditions

-.47 **ITM 3: How often felt nervous and "stressed"

+.44 LIK44: Contact with faculty important to academic success

FACTOR 4:

+.40 TSCR7: Demonstrated community service

+.39 LIK45: Usually come up with ideas that friends end up doing

+.39 **ITM 4: How often dealt successfully with irritating life hassles

+.34 **ITM 5: Effective coping with important life changes

+.32 LIK38: Think many people see racism where it doesn't exist

FACTOR 12:

+.41 LIK30: Enjoy working with others

-.41 **ITM12: How often thought about things have to accomplish

-.30 LIK43: Know areas where weak and try to improve them

+.27 TSCR8: Knowledge acquired in a field

+.25 TSCR4: Prefer long-range goals to short-term needs

FACTOR 13:

+.37 TSCR5: Availability of strong support person

-.35 LIK47: Have learned more outside school than in school

-.31 **ITM11: How often been angered due to things happening outside control

+.28 LIK30: Enjoy working with others

+.27 **ITM10: How often felt on top of things

FACTOR 15:

+.58 LIK39: Have good understanding of strengths and weaknesses

+.33 **ITM13: Have been able to control the way time is spent

+.26 TSCR1: Positive self-concept or confidence

+.19 LIK29: Find I get more comfortable in new place as soon as I make new friends

+.19 **ITM 1: How often upset because of something that happened unexpectedly

REFERENCES

Arbona, C., Sedlacek, W. E., & Carstens, S. P. (1987). Noncognitive variables in predicting counseling center use race. Counseling Center Research Report # 3-87. College Park, Md.: University of Maryland.

Astin, A. W. (1982). Minorities in higher education. San Francisco: Jossey-Bass.

Boyer, S. P. & Sedlacek, W. E. (1987). Noncognitive predictors of academic success for international students: A longitudinal study. Counseling Center Research Report #1-87, College park, MD: University of Maryland.

Cohen, S., Karmarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior,, 24, 385-396.

Lazarus, R. S. (1966). Psychological stress and the coping rocess. New York: McGraw-Hill.

Lazarus, R. S. (1977). Psychological stress and coping in adaptation and illness. In Z. J. Lipowski, D. R. Lipsi, and P.C. Whybrow (Eds.), Psychosomatic Medicine: Current trends. New York: Oxford University Press.

Lockett, G. C. (1980). A study of traditional measures and nontraditional measures used to predict the success of Black college students. Unpublished doctoral dissertation, University of Missouri-Columbia.

Sedlacek, W. E. (1977). Should higher education students be admitted differentially by race and sex? The evidence. Journal of the National Association or College Admissions Counselors, 22, (1), 22-24.

Sedlacek, W. E. & Brooks, G. C., Jr. (1976). Racism in American education: A model for change. Chicago: Nelson-Hall.

Sedlacek, W. E., & Pelham, J. C. (1976). Minority admissions to large universities: A national survey. Journal of Non-White Concerns in Personnel and Guidance. 4, 53-63.

Tracey. T. J. & Sedlacek, W. E. (1984). Noncognitive variables in predicting academic success by race. Measurement and Evaluation in Guidance, 16, 171-178.

Tracey, T. J. & Sedlacek, W. E. (1985). The relationship of noncognitive variables to academic success: A longitudinal comparison by race. Journal of College Student Personnel, 25, 405-410.

Tracey, T. J. & Sedlacek, W. E. (1987). Prediction of college graduation using noncognitive variables by race. Measurement and Evaluation in Counseling and Development, 1, 177-184.

White, T. J. & Sedlacek, W. E. (1986). Noncognitive predictors: Grades and retention of specially-admitted students. The Journal of College Admissions, 3, 20-23.