JVME v20n3: Correlation of Objective and Subjective Admission Criteria with First-year Academic Performance
| Volume 20, Number 3 | 1995 |
|---|
Correlation of Objective and Subjective Admission Criteria with First-year Academic Performance
A. W. Confer, G. H. Turnwald, D. E. Wollenburg
Drs. Confer and Wollenburg (Department of Veterinary Pathology) and Turnwald (Department of Veterinary Medicine and Surgery) are from the College of Veterinary Medicine, Oklahoma State University, Stillwater, OK 74078.
Introduction
In veterinary and medical colleges, admission decisions are often made based on objective criteria such as cumulative grade point average (CUMGPA), GPA for preprofessional required courses (REQGPA) (Table 1), grades in preprofessional science courses, and a national standardized examination such as the Medical College Admissions Test (MCAT), Veterinary Aptitude Test, or Graduate Record Examination (GRE) (1, 2, 3) . Ten of the 31 North American veterinary colleges use the GRE as its sole standardized examination criterion (4) . One of the authors (AWC) found that preprofessional GPAs and scores obtained on various subsets of the GRE (quantitative [GREQUAN], verbal [GREVERB], and analytical [GREANA]) significantly correlated with academic performance in veterinary school (5) . A combination of certain GRE component scores and preprofessional CUMGPA or REQGPA, however, was a better predictor of academic performance than either criteria alone. Similar results have been reported for veterinary and medical college admissions using preprofessional GPA and MCAT scores (6, 7) .
Objective criteria used for selection of professional students are usually augmented with subjective criteria in an attempt to assess nonacademic attributes such as leadership, motivation, career-related experience, work ethic, and interpersonal skills (8) . After interviewing applicants, interviewers may make numerical assessments of applicants, hereafter referred to as an interview score (INTERV). Other information available to the committee, such as letters of reference, extracurricular activities, professional and other work experience, semester course load, and selectivity of colleges and universities attended, are assessed and a numerical score assigned, hereafter referred to as a file score (FILE).
The impact of subjective criteria on predicting academic performance, clinical competency or postgraduation success in medical or veterinary schools has been previously evaluated with variable results; and use of such criteria for admission has been questioned by some medical educators (9, 10, 11) . Meredith et al. (12) demonstrated a positive correlation between interview scores and evaluation of clinical performance in medical school. Kelman (1) found interview ratings to be negatively related to basic science course grades obtained in a veterinary college, whereas Hulland and Ison (13) found that interview ratings did not correlate well with grades in veterinary school. The impact of subjective admission criteria on academic performance in veterinary colleges has not been evaluated in recent years when applicant pools have declined.
Table 1: Abbreviations for admission criteria evaluated in this study.
Cumulative Grade Point Average CUMGPA Required Course Grade Point Average REQGPA Graduate Record Examination REQGPA Verbal test GREVERB Quantitative test GREQUAN Analytical test GREANA Biological test GREBIOL Overall Admission Score ADMISS Interview Score INTERV Score of Admission File FILE Biochemistry BIOCHEM General Biology BIOL 1 & 2 General Chemistry CHEM 1 & 2 Genetics GENETICS Microbiology MICROBIOL Organic Chemistry ORGCHEM Physics PHY 1 & 2 Oklahoma State University (OSU), College of Veterinary Medicine (CVM) uses an admission formula that takes into account both subjective and objective criteria. Similar formulae are used in other veterinary colleges. In the OSU formula, each applicant is evaluated on objective criteria worth up to 600 points [REQGPA, CUMGPA, GREANA, GREQUAN, GREVERB, and the GRE Biology Advanced Test (GREBIOL)]. Up to 200 points are assigned for INTERV and 200 for FILE. The cumulative score each candidate receives is referred to as his/her admissions score (ADMISS). Candidates are ranked by ADMISS, and the final recommendation to the Dean is based on rank.
Although this formula has been in use for several years, evaluation of the predictive quality of ADMISS and its components on academic performance has not been done. This study was undertaken with three objectives:
1. To correlate first-year academic performance with the objective and subjective criteria used in the admission process.
2. To determine the best models of those criteria for predicting first-year academic performance.
3. To determine if INTERV and FILE are of greater or lesser value for students with higher or lower CUMGPA.The cumulative GPA for the first year (CUMY1) of veterinary school was the outcome assessed. First-year academic performance was selected for evaluation for two reasons. In a previous study, there were no significant differences in GPAs among the four years of veterinary school; however, correlations between first- and fourth-year GPAs were significant (5) . Also, 94% of students that withdrew or were dismissed from this college in recent years did so during or immediately after the first year or prior to receiving grades for first semester of the second year of veterinary school (Confer unpublished data).
Methodology
Criteria assessed were for five recent successive classes (designated Classes 1, 2, 3, 4, and 5) consisting of a total of 357 students. For Classes 1 and 2, criteria consisted of: ADMISS, CUMGPA, REQGPA, FILE, INTERV, GREANA, GREQUAN, and GREVERB. For Classes 3-5, GREBIOL became an additional criterion. Required science courses consisting of general chemistry (CHEM 1 & 2), organic chemistry (ORGCHEM), biochemistry (BIOCHEM), general biology (BIOL 1 & 2), physics (PHY 1 & 2), microbiology (MICROBIOL), and GENETICS were evaluated for each class. CUMY1 was the outcome assessed.
A one-way analysis of variance was used to determine differences among the subjective and objective criteria and academic performance for the five classes. Tukey HSD post-hoc multiple comparisons were done using the Tukey-Kramer adjustment of means (14) . A probability (p) <0.05 was accepted as significant.
Linear regression analyses were done for each subjective and objective admission criteria to determine correlation with CUMY1 (15) . Results are expressed as a correlation coefficient (r) and probability that the correlation was by chance alone as determined by analysis of variance through a general linear model (15) .
Stepwise multiple regression analyses were done to determine the best predictor models for each class (5, 7, 14, 15) . The best predictor models are defined as the combinations of subjective and objective admission criteria or grades in required science courses that best predicted academic performance. All possible combinations of criteria were considered in determining these models, except that ADMISS was not included in the analysis because it already represents a combination of the admission criteria. Results are presented as multiple regression coefficients (R) and p values as determined by analysis of variance (14, 15) .
The influence of CUMGPA on predictor models of CUMY1 was assessed. Subsets of admission criteria for each year were created based on CUMGPA >3.00, >3.25, >3.50, <3.50, <3.25, and <3.00. Predictor models of subjective and objective criteria used for admissions were determined for each subset using stepwise multiple regression analysis.
Results
Mean scores for admission criteria and academic performance in year 1 for each class were compared (Tables 2 a and b). With three exceptions, mean values were not different. The ADMISS was significantly greater for Class 4 than for Classes 1-3. FILEs were significantly lower for Class 1 than for other Classes. CUMGPA was significantly lower for Class 5 compared to Classes 1 and 4. There were no significant differences for grades in preprofessional sciences among the classes (data not shown).
Table 2a. Subjective and objective scores used as criteria for admission to the College of Veterinary Medicine and first-year academic performance.
Class Student No. ADMISS* FILE INTERV CUMGPA REQGPA 1 75 758 ± 62a 151 ± 12 148 ± 21 3.32 ± 0.35 3.32 ± 0.32 2 70 760 ± 66a 159 ± 12b 155 ± 22 3.23 ± 0.34 3.27 ± 0.31 3 71 752 ± 72a 159 ± 14b 148 ± 24 3.25 ± 0.35 3.28 ± 0.32 4 74 791 ± 57 158 ± 14b 156 ± 20 3.32 ± 0.35 3.33 ± 0.32 5 67 778 ± 60 162 ± 13b 154 ± 21 3.15 ± 0.37ab 3.21 ± 0.34 Mean ± standard deviations are given for all criteria.
* Admission score (ADMISS, maximum = 1000) is a composite of file score (FILE, maximum = 200); interview score (INTERV, maximum = 200); cumulative preprofessional grade point average (out of a possible 4.00 for all courses (CUMGPA); GPA for required courses (REQGPA); and various Graduate Record Examination subtests.
a Significantly different (p <0.05) than class 4 values.
b Significantly different (p <0.05) than class 1 values.
For each class, correlations between admission criteria and CUMY1 were ranked based on r values. To assess which criteria most consistently correlated with CUMY1 for the 5 classes, how often each criterion for each class had one of the four highest r values was determined. In descending order these were: ADMISS (all classes: r = 0.432 - 0.626); CUMGPA (4 of 5 classes; 0.384 - 0.558); GREBIOL (2 of 3 classes; r = 0.384 and 0.504); FILE (3 of 5 classes; r = 0.327 - 0.506); REQGPA (3 of 5 classes; r = 0.349 - 0.504) GREQUAN (1 or 5 classes; r = 0.599); INTERV (1 of 5 classes; r = 0.503); GREVERB (1 of 5 classes; r = 0.448); GREANA (no classes). The overall admission score (ADMISS) had the highest correlation with CUMY1 for 4 of 5 years.
Table 2b. Subjective and objective scores used as criteria for admission to the College of Veterinary Medicine and first-year academic performance.
Class Student No. GREANA# GREBIOL GREQUAN GREVERB CUMY1§ 1 75 570 ± 93 - 576 ± 90 492 ± 97 2.97 ± 0.61 2 70 563 ± 88 - 559 ± 90 480 ± 97 2.83 ± 0.65 3 71 558 ± 96 549 ± 88 541 ± 88 482 ± 85 2.88 ± 0.63 4 74 576 ± 99 549 ± 87 555 ± 89 490 ± 91 2.90 ± 0.52 5 67 574 ± 107 539 ± 92 556 ± 95 486 ± 96 2.83 ± 0.54 # Graduate Record Examination scores for analytical (GREANA), biologic (GREBIOL), quantitative (GREQUAN), and verbal (GREVERB) subtests.
§ Cumulative GPA (out of a possible 4.00) at the end of the first year of veterinary school.Correlations between grades in 10 preprofessional science courses compared to CUMY1 were ranked based on r values. Grades in BIOCHEM correlated significantly (p <0.05) with CUMY1 for all classes, GENETICS and ORGCHEM for 4 of 5 classes each, BIOL2 and MICROBIOL for 2 of 5 classes each, and CHEM1 for 1 of 5 classes.
Table 3. Best predictor models for first-year academic performance determined by stepwise-multiple regression analysis from among subjective and objective admission criteria and preprofessional science courses.
Class 1* 2* 3 4 5 Subjective and Objective Criteria CUMGPA
GREQUAN
INTERV
R = 0.649
p <0.0001INTERV
FILE
GREQUAN
GREVERB
R = 0.700
p <0.0001FILE
CUMGPA
GREBIOL
R = 0.685
p <0.0001FILE
GREBIOL
CUMGPA
R = 0.515
p <0.0001CUMGPA
GREBIOL
INTERV
R = 0.663
p <0.0001Preprofessional Science Courses BIOCHEM
BIOL2
GENETICS
R = 0.535
p <0.0001BIOCHEM
BIOL2
PHY1
R = 0.540
p <0.0001BIOCHEM
GENETICS
MICROBIOL
R = 0.490
p <0.0001BIOCHEM
CHEM1
R = 0.475
p <0.0001GENETICS
MICROBIOL
ORGCHEM
R = 0.600
p <0.0001*GREBIOL scores not available for these classes. The best predictor models of academic performance based on subjective and objective criteria were determined (Table 3). ADMISS was not included in this evaluation because each criterion evaluated was a component of the admission formula. Each predictor model consisted of 3 or 4 criteria. CUMGPA was a component of the best predictor models for 4 of 5 years, whereas FILE and INTERV were components of the best predictor models for 3 each, GREQUAN for 2, and GREVERB for 1 of 5 years. The GREBIOL was a component of the best predictor model for each of three years it was available. The best predictor models based on preprofessional science courses had lower R values than for the predictor models based on other admission criteria. Grades in BIOCHEM and GENETICS were components in 4 and 3 of the 5 predictor models respectively. MICROBIOL and BIOL2 grades were components in 2 each and PHY1 and CHEM! and ORGCHEM were predictor model components for 1 of the 5 years.
Finally, predictor models for CUMY1 were determined for subsets of students based on their CUMGPA at the time of admission. This was examined to determine if subjective criteria that were part of the predictor models were more important for students with lower than with higher CUMGPA. For each Class except for Class 3, INTERV was most often a component of the predictor models for students with CUMGPA <3.50. For all classes, INTERV was never a component for students with CUMGPA >3.50. FILE, however, was a component of various predictor models with no obvious pattern relative to CUMGPA. FILE was a component of the best predictor model for students with CUMGPA >3.50 in 3 classes.
Discussion
Results presented herein corroborated and expanded several previous findings concerning objective criteria as predictors of academic performance (1, 2, 3, 5, 7, 10, 13) . First, no single set of preadmission predictors of academic performance was uniform among veterinary classes. Second, for OSU CVM, preadmission CUMGPA significantly correlated with first-year veterinary school academic performance better than did REQGPA. Third, selected GRE subtest scores correlated with academic performance. In a previous study, GREQUAN scores often correlated well with academic performance. GREBIOL scores were not available in that study. In the present study, GREQUAN was a component of each model for Classes 1 and 2; however, once GREBIOL became an admission criterion, GREQUAN was no longer a common component of the predictor models but GREBIOL was. In general, GREBIOL scores correlated with first-year academic performance better than did GREQUAN scores. Fourth, combinations of preadmission GPA, selected GRE subtest scores, and subjective criteria were better predictor models than either criterion alone or predictor models based on grades in preprofessional science courses. Fifth, grades in certain preprofessional science courses can correlate with academic performance in veterinary school. Grades in BIOCHEM most consistently had one of the best correlations with CUMY1. However, when linear regression analyses were applied to the data in the present study, ADMISS correlated better with academic performance in the first year for 4 or 5 classes than did any of the individual components of ADMISS. This indicates that a composite score of several admission criteria is a better predictor than any individual criterion alone.
In the present study, there was a significant correlation between FILE or INTERV and first-year academic performance for each class. The FILE and INTERV had one of the four highest correlations with CUMY1 for 3 and 1 of 5 classes evaluated respectively. In addition, for each class, the best predictor models included INTERV or FILE and at least one GRE subtest and CUMGPA. The influence of these subjective criteria on predicting first-year academic performance was unexpected, because historically they are thought to measure nonacademic attributes (8, 16) . However, several nonacademic attributes discerned during an interview could greatly influence first-year academic performance. These might include motivation, commitment to a veterinary career, work ethic, previous experience, and maturity. Also, it may be difficult for an interviewer to not let preprofessional academic performance influence his/her scoring of an applicant during an interview. Therefore, although the correlation between INTERV and CUMGPA was not significant (data not shown), those two criteria were probably not totally independent. Similarly, in evaluation of an application file, a subjective evaluation is made of applicantsπ transcripts. Therefore, FILE and CUMGPA are probably not totally independent.
The interview has been discarded as an admission criterion from several veterinary colleges. That decision probably resulted from the questioned value of the interview by several veterinary and medical educators (1, 8, 11, 13) ; the faculty and staff time required to schedule and conduct interviews; and expense to applicants (especially out-of-state applicants). In addition, a possible lack of well-defined attributes being assessed during interviews could make results seem questionable to admissions committees. Edwards et al. (9) , however, found reliability and validity of the results of interviews of premedical students increased with the degree of structure of the interview process. Results of this study indicate that the interview may have a role in predicting first-year academic performance. It was commonly a component of the best predictor models of first-year academic performance for students with a CUMGPA <3.50. The interview could be of questionable predictive value for a selected subset of students, those with CUMGPA >3.50. Therefore, there is no value added to the selection process by interviewing the higher preprofessional academic achievers. Many of those attributes discerned during an interview (motivation, interest, and commitment) can probably be assumed for students with superior preprofessional CUMGPAs.
In conclusion, the observation that the correlation between ADMISS and first-year academic performance was overall higher than any of its single components for 4 of the 5 classes analyzed and that the best predictor models of first-year academic performance included combinations of both subjective and objective criteria argues for continuation of traditional admission procedures in this college of veterinary medicine. This concept is supported by Miller (17) who recently said, "The most realistic resolution [of the dilemma of determining medical school admission] appears to be that of a quantitative floor for screening and a qualitative superstructure for selection." Based on the current study, modification of the procedure for preprofessional high academic achievers, however, seems justified. It must be remembered, as pointed out by McGaglie (8) , that aptitude-achievement correlations seem to drop as medical students move from lecture halls to clinics and beyond. Predictability of first-year academic performance certainly cannot predict how students will perform professionally after graduation. Finally, results of this study might be applicable to admission procedures of other veterinary colleges and should be considered as educators and administrators attempt to develop a common admissions procedure for North American colleges of veterinary medicine.
Summary
A retrospective analysis was conducted to determine predictability of objective and subjective admission criteria for first-year academic performance for 5 recent classes at OSU CVM. Best predictor models included a combination of 3 or 4 criteria with preprofessional CUMGPA, either GREQUAN or GREBIOL, and either INTERV or FILE scores being common components of the predictor models. Using simple regression analyses, ADMISS, which is a composite admission score that includes objective and subjective criteria, generally correlated with first-year academic performance better than did any of its individual components. Predictor models based on preprofessional science course grades had lower R values than predictor models composed of GPA, GRE test scores, and INTERV or FILE. For the preprofessional science courses, BIOCHEM grade was the major component for predicting first-year academic performance. Predictor models were determined for subsets of students sorted on preprofessional CUMGPA. INTERV was never a component of the predictor models for students with a CUMGPA >3.50, whereas it was nearly always a predictor model for those students with CUMGPA <3.50. The FILE was a component of various predictor models with no obvious regard for CUMGPA.
References and Endnotes
1. Kelman EG: Predicting success in veterinary medical college. Jour Vet Med Educ 8:92-94, 1982.
2. Mitchell KJ: Traditional predictors of performance in medical school. Acad Med 65:149-158, 1990.
3. Render GF, Jackson HD: Preveterinary performance, admissions criteria and personality variables as predictors of success in veterinary school. Jour Vet Med Educ 2:3-6, 1975.
4. Shane SM, Harrington KS: Survey of standard tests as criteria for admission to North American veterinary colleges. Jour Vet Med Educ 16:545-46, 1989
5. Confer AW: Preadmission GRE scores and GPAs as predictors of academic performance in a college of veterinary medicine. Jour Vet Med Educ 17:56-62, 1990.
6. Hall FR, Bailey BA: Correlating studentsπ undergraduate science GPAs, their MCAT scores, and the academic caliber of their undergraduate colleges with their first-year academic performances across five classes at Dartmouth Medical School. Acad Med 67:121-123, 1992.
7. Shane SM, Kearney MT: Evaluation of criteria for admission to a school of veterinary medicine. Jour Vet Med Educ 16:40-44, 1989.
8. McGaglie WC: Qualitative variables in medical school admission. Acad Med 65:145-149, 1990.
9. Edwards JC, Johnson EK, Molidor JB: The interview in the admission process. Acad Med 65:167-177, 1990.
10. Niedzwiedz ER, Friedman BA: A comparative analysis of the validity of pre-admissions information at four colleges of veterinary medicine. Jour of Vet Med Educ 3:32-38, 1976.
11. Taylor TC: The interview: one more life? Acad Med 65:177-178, 1990.
12. Meredith KE, Dunlap MR, Baker HH: Subjective and objective admissions factors as predictor of clinical clerkship. Jour Med Educ 15:231-236, 1981.
13. Hulland TJ, Ison NT: A seven year retrospective study of veterinary admission: II admission characteristics and program performance. Jour Vet Med Educ 9:3-6, 1982.
14. Wilkinson L: SYSTAT: The System for Statistics. Evanston, IL: SYSTAT, Inc., 1988.
15. Neter J, Wasserman W, Kutner MH: Applied Linear Regression Models , ed 2. Boston: Irwin, 1989, pp 444-458.
16. McGaglie WC: Perspectives on medical school admission. Acad Med 65:136-139, 1990.
17. Miller GE: Overview. Acad Med 65:135, 1990.
The authors thank Sherl Holesko for valuable assistance in preparation of this manuscript.