JVER v29n2 - Factorial Invariance of the Occupational Work Ethic Inventory (OWEI)
Factorial Invariance of the Occupational Work Ethic Inventory (OWEI)
Paul E. Brauchle
Illinois State University, Normal
Md. Shafiqul Azam
Illinois Network of Child Care Resource and Referral AgenciesAbstract
This study compared factor structures of the Occupational Work Ethic Inventory (OWEI) for self-perceived work attitudes of manufacturing employees and their supervisors' ratings of those same employees. The purpose of the study was to evaluate construct validity through comparative factor analysis. Four factors were generated through principal components analysis and compared across the groups using the Coefficient of Congruence, the Salient Similarity Index, and the Root Mean Squares methods for comparing factors. Results indicated that the factors measured by the OWEI were virtually the same, for both groups, providing evidence of construct validity for this instrument.
In the rather arid research area of work ethic/work attitudes, the Occupational Work Ethic Inventory (OWEI) developed by Petty (1995) is a notable addition. Funded by a National Science Foundation grant, it was based on prior research works on work ethic, the value of work, and affective work competencies ( Hill, 1997 ). It is a 50 item instrument in which 11 items were reversely stated. The instrument measures work attitudes by asking respondents to indicate on a 7-point Likert-type scale how they can describe themselves at work. Some of the descriptors used are "dependable", "stubborn", "independent", "accurate", and "ambitious". Content validity of the instrument was established through the use of items selected from a review of literature regarding work attitudes, work values, and work habits and through the use of a panel of experts in its construction ( Hill, 1997 ; Petty, 1995 ). According to Hill (1997) , criterion validity was partially addressed in the construction of the instrument by a panel of experts, but he cautioned that further research is needed to establish additional evidence for validity.
We have known for some time that good work attitudes, values, and habits are important to workers and employers. Research has indicated that in the majority of cases where persons lost their jobs or failed to be promoted, the reasons were poor work attitudes and not lack of technical competence (Beach, 1982 ; Custer & Claiborne, 1992 , 1995 ; Gregson & Bettis, 1992 ). Good work attitudes are traits that employers want their employers to have ( Commission on the Skills of the American Workforce, 1990 ), but they may be changing. Yankelovich and Immerwahr (1984) , found in younger workers a decline in the belief that good work would be rewarded, suggesting that the younger workers may have less organizational loyalty then their older colleagues. If Career and Technical Education (the successor to Vocational Education) is to succeed in producing graduates who can gain and hold employment in a competitive world, it must succeed in enhancing their work ethic in addition to their technical skills. In order for work attitudes to be improved in students, these skills must be taught. In order to know the extent to which they exist in students before they enter the labor market, good work attitudes must be measured. The OWEI attempts to do this. It measures four constructs thought to embody the main components of work ethic.
A construct is a non observable trait, such as intelligence, which explains behavior and can explain certain differences between individuals ( Gay, 1996) . For occupational programs, construct validity helps in answering the question, "What does an individual's score on this instrument really mean and how can it be explained in terms of known type of human behavior or psychological constructs?" ( Erickson & Wentling, 1988 ). Logical study of the instrument and intuitive identification of psychological constructs that it measures, and then testing of those constructs, may be a method to establish construct validity. In 1994, Petty and Hill disclosed four dimensions (constructs) of the OWEI using a content analysis and by use of another panel of experts. However, there was no follow-up research study to establish that these four dimensions actually represent the underlying psychological constructs of the OWEI.
One method of evaluating construct validity is through factor analytic techniques ( Anastasi, 1976 ; Reschly, 1978 ). Although several factor analytic studies on the OWEI were available in the literature ( Azam, 2002 ; Hatcher, 1995 ; Hill, 1996 ; Hill & Petty, 1995 ; Petty, 1995 ), none on factor comparison were observed. The purpose of this study was to evaluate the construct validity of the OWEI through comparative factor analysis.
Method
Factor structures compared in this study came from two factor analytic studies. In one study (Study 1), the OWEI factor structure was obtained by conducting a principal component analysis of self perceived work attitudes of employees ( n =454) in medium size manufacturing industries in the central Illinois area. The OWEI was used to measure employee work attitudes. A second principal component analysis was conducted to ascertain the factor structure of the OWEI when supervisors of the employees ( n =581) described earlier rated these same employees' work attitudes with the same instrument (Study 2).
The Principal Components analyses used data without applying any type of transformation or removal of outliers. To ascertain number of factors, both Kaiser's criterion and Cattell's Scree Test were applied. However, Kaiser's criterion retained too many factors (11 for Study 1 and 6 for Study 2). Cattell's Scree Test, which is said to be more accurate than the Kaiser criterion ( Zwick & Velicer, 1986 ), was therefore, used to identify the number of factors. The Scree Test retained four factors in each of the two studies. It could be argued that oblique rotation was called for because the factors would be correlated in the real world. However, benchmark studies using this instrument provided the factors to be compared with ours, and these earlier studies all used orthogonal rotation to generate their factors. Therefore, we rotated our factors orthogonally. With uncorrelated factors, the explanation and interpretation of research results are simpler, less ambiguous, and generally more straightforward ( Kerlinger & Pedhazur, 1973 ). Comrey (1973) suggested that loadings in excess of 0.71 (accounting for 50% variance) are considered excellent, 0.63(40%) very good, 0.55(30%) good, 0.45 (20%) fair, and 0.32(10% of variance) poor. Even though some authors accept loadings as low as 0.3, from a variance perspective Comrey's guidelines make better sense. In this study, a loading of .45, which represents 20% of the variance, was used to include a variable in the definition of a factor. Table 1 and Table 2 gives factor loadings in descending order of loading size for the four factor solutions for Study 1 and Study 2 respectively.
Procedure
Based on the criteria described in the preceding paragraph, four salient and meaningful factors were identified in each of the two studies. Factors obtained in the first study (industrial workers) were named as Teamwork (Factor 1), Dependability (Factor 2), Ambition (Factor 3), and Self-Control (Factor 4). These factors were then compared with four factors obtained in the second study (industrial supervisors). A preliminary matching of factors was made by use of marker variables. Marker variables are those variables that have high loadings (usually 0.5 or above) in the pairs of factors to be compared. Marker variables paired Factor 1 with Factor 2, Factor 2 with Factor 3, and Factor 3 with Factor 1, Factor 4 with Factor 4 of the first and second studies respectively. This method reduced the chance of obtaining spuriously significant results that capitalized on chance relationships. Patternmagnitude similarities of the factor loadings were then compared.
Factor comparisons using the coefficient of congruence are quite common in literature ( Carroll, Houghton, & Baglioni, 2000 ; Cordano, Scherer, & Owen, 2003 ; Ommundsen, Hak, Morch, Larsen, & Veer, 2002 ; and Sakamoto, Kijima, Tomoda, & Kambara, 1998 ). Likewise, factor comparisons using Cattell's Salient Similarity Index are also not uncommon in the literature ( William's & Potter, 1994 ; Zack, Toneatto, & Streiner, 1998 ). Guadagnoli and Velicer (1991) found the Salient Similarity Index to be more reliable than the Coefficient of Congruence when comparing factors. However, Cattell (1978) advised the use of at least two methods of factor comparison when matching factors. For this reason, comparisons were
Table 1
Rotated Component Matrix (Study 1)
Variable Component 1 2 3 4 Dependable 0.125 0.667 0.157 0.127 Stubborn 0.317 -0.025 -0.010 0.361 Following regulations 0.208 0.605 0.012 0.232 Following directions 0.239 0.617 0.054 0.223 Independent -0.001 0.500 0.220 -0.032 Ambitious 0.265 0.460 0.467 0.092 Effective 0.113 0.635 0.322 0.079 Reliable 0.198 0.669 0.183 0.158 Tardy -0.053 0.185 0.079 0.223 Initiating 0.033 0.250 0.527 0.115 Perceptive 0.141 0.449 0.500 0.112 Honest 0.279 0.588 0.044 0.196 Iresponsible 0.065 0.290 0.190 0.350 Efficient 0.032 0.496 0.464 0.147 Adaptable 0.281 0.496 0.289 0.067 Careful 0.289 0.618 -0.063 0.102 Appreciative 0.432 0.379 0.158 0.257 Accurate 0.150 0.510 0.407 0.093 Emotionally stable 0.405 0.222 0.171 0.466 Conscientious 0.363 0.349 0.273 0.318 Depressed 0.152 0.041 0.179 0.556 Patient 0.456 0.080 0.090 0.288 Punctual 0.264 0.302 0.294 0.060 Devious 0.062 0.012 0.072 0.699 Selfish 0.258 0.175 0.032 0.591 Negligent -0.021 0.331 0.151 0.497 Persevering 0.071 0.014 0.445 0.073 Likeable 0.637 0.188 0.137 -0.013 Helpful 0.638 0.297 0.284 0.046 Apathetic -0.175 -0.138 0.086 0.160 Pleasant 0.741 0.083 0.043 0.186 Cooperative 0.627 0.303 0.115 0.155 Hard working 0.365 0.323 0.273 -0.028 Rude 0.379 0.133 -0.001 0.608 Orderly 0.295 0.288 0.361 0.146 Enthusiastic 0.460 0.060 0.562 0.235 Cheerful 0.626 0.107 0.302 0.260 Persistent 0.301 0.221 0.607 0.048 Hostile 0.225 0.090 0.065 0.691 Dedicated 0.503 0.075 0.592 0.195 Devoted 0.564 0.146 0.519 0.161 Courteous 0.723 0.177 0.207 0.214 Considerate 0.650 0.092 0.262 0.301 Careless 0.021 0.269 0.140 0.447 Productive 0.276 0.220 0.485 0.132 Well groomed 0.413 0.268 0.206 0.080 Friendly 0.769 0.122 0.210 0.224 Loyal 0.657 0.108 0.353 0.166 Resourceful 0.437 0.191 0.596 0.143 Modest 0.430 0.154 0.069 -0.067
Table 2
Rotated Component Matrix (Study 2)
Variable Component 1 2 3 4 Dependable 0.369 0.232 0.688 0.110 Stubborn -0.027 0.477 -0.001 0.537 Following regulations 0.282 0.451 0.522 0.251 Following directions 0.411 0.391 0.563 0.191 Independent 0.672 -0.044 0.243 -0.117 Ambitious 0.784 0.230 0.143 0.148 Effective 0.684 0.250 0.433 0.131 Reliable 0.468 0.213 0.683 0.150 Tardy 0.047 -0.023 0.588 0.334 Initiating 0.773 0.131 0.014 0.118 Perceptive 0.725 0.058 0.228 0.168 Honest 0.267 0.485 0.446 0.305 Irresponsible 0.368 0.120 0.390 0.579 Efficient 0.671 0.230 0.433 0.109 Adaptable 0.500 0.443 0.295 0.192 Careful 0.328 0.365 0.572 0.037 Appreciative 0.366 0.617 0.232 0.249 Acurate 0.486 0.265 0.542 0.069 Emotionally stable 0.184 0.518 0.485 0.239 Conscientious 0.465 0.393 0.485 0.256 Depressed 0.181 0.351 0.234 0.472 Patient 0.029 0.524 0.297 0.217 Punctual 0.138 0.226 0.755 0.088 Devious 0.119 0.271 0.140 0.699 Selfish 0.325 0.284 0.005 0.661 Negligent 0.285 0.138 0.312 0.695 Persevering 0.598 0.157 0.155 0.126 Likeable 0.283 0.723 0.272 0.203 Helpful 0.522 0.505 0.296 0.198 Apathetic 0.343 0.044 -0.003 0.216 Pleasant 0.211 0.791 0.195 0.246 Cooperative 0.322 0.682 0.285 0.283 Hard working 0.631 0.340 0.341 0.296 Rude 0.073 0.465 0.071 0.695 Orderly 0.543 0.221 0.277 0.182 Enthusiastic 0.700 0.434 0.010 0.141 Cheerful 0.389 0.734 0.091 0.171 Persistent 0.649 0.289 0.255 0.076 Hostile -0.054 0.388 0.120 0.649 Dedicated 0.717 0.374 0.170 0.175 Devoted 0.698 0.401 0.117 0.166 Courteous 0.290 0.739 0.204 0.320 Considerate 0.244 0.713 0.171 0.367 Careless 0.319 0.084 0.263 0.573 Productive 0.647 0.272 0.342 0.214 Well groomed 0.309 0.368 0.319 0.132 Friendly 0.221 0.811 0.199 0.186 Loyal 0.606 0.485 0.159 0.171 Resourceful 0.745 0.251 0.280 0.090 Modest 0.189 0.428 0.053 0.082
made using the Coefficient of Congruence method as well as the Salient Similarity Index (SSI) method. The Root Mean Square Coefficient (RMS), another stringent method of comparing factors, was also used ( Rummell, 1970 ). Significance values for the coefficient of congruence and Salient Similarity Index were obtained from tables provided by Cattell (1978) .
Results
Harman (1976) recommended that each factor of one study be compared with all the factors of the other study, and paring each factor with the one with which it has the highest coefficient of congruence. Application of this method resulted in factor pairs that were similar to the pairings obtained using marker variables. Therefore, four comparisons were made with factors from studies 1 and 2. Comparisons were made using the Coefficient of Congruence (r c ), the Salient Similarity Index (s) and the Root Mean Square Coefficient (µ). Results of the comparisons are given in Table 3.
Table 3
Factor comparisons by RMS coefficient, Coefficient of Congruence, and Salient Similarity Index
Compared Factors RMS
Coefficient
( µ )Coefficient of
CongruenceSalient Similarity Index 1 st
study2 nd
studyValue (r c ) P Value (s) % hp
CountP Factor 1 Factor 2 0.119 0.960 <0.001 0.900 68 <0.001 Factor 2 Factor 3 0.204 0.940 <0.001 0.690 58 <0.001 Factor 3 Factor 1 0.151 0.903 <0.001 0.960 74 <0.001 Factor 4 Factor 4 0.105 0.956 <0.001 0.940 82 <0.001 The Root Mean Square Coefficient can have any value between 0 and 2, although values of 0 or 2 are extremely unlikely to occur. If µ is zero, the two factors are alike in magnitude and direction. As µ departs from zero, the factors are less alike. In these four comparisons, the values of µ varied from 0.105 to 0.204 (See Table 3). The small values obtained for µ suggested that the four factors were alike across the populations ( Rummell, 1970 ). However, as there is no set standard as to what value of µ indicates an acceptable agreement between factors, it was not used as the primary basis for making a decision on factor similarity.
Values for the Coefficient of Congruence varied from 0.903 to 0.960 (See Table 3). According to Broadbooks and Elmore (1987) , an obtained sample congruence coefficient greater than 0.50 will usually be an underestimate of actual population value. Therefore, the actual population coefficients of congruence may be even higher than the values obtained here. The Coefficient of Congruence ranges from -1.00 (for perfect negative similarity) through zero (for complete dissimilarity) to 1.00 (for perfect positive similarity). The critical value of the coefficient of congruence at P < 0.05 with 50 variables in common and 4 factors is 0.34 ( Cattell, 1978 ). The minimum coefficient value obtained from the four comparisons was 0.903, suggesting a very strong match. However, this does not imply that the factors of the two studies were exactly identical. They are congruent to some extent. The literature recommends different criteria to express the extent of congruence. Ommundsen et al. (2002) treated a 0.80 coefficient of congruence as robust. Sakamoto et al. (1998) described coefficient of congruence values of 0.90 or above as representing very high, 0.80 - 0.89 as high, and 0.70 - 0.79, as moderate agreement. On the other hand, Koschat and Swayne (1991) judged factors to be virtually equal whenever the Coefficient of Congruence was 0.85 or above.
Salient similarity indices and hyperplane counts were calculated for the same four comparisons according to the method given by Cattell (1978) (See Table 3). With 50 variables and a corresponding hyperplane (hp) count, the p value obtained was P < 0.001. These results were consistent with the results obtained by comparing the factors using the Coefficient of Congruence method, although the probability level for the congruence method was different (at P < 0.05).
Table 4 presents subgroup (factor) reliability indices (Cronbach's alpha). Nunnally (1978) indicated 0.7 to be an acceptable reliability coefficient though lower thresholds are sometimes used in the literature. George and Mallery (2003) provided the following rules of thumb: > 0.9 as Excellent; > 0.8 as Good; > 0.7 as Acceptable; > 0.6 as Questionable; > 0.5 as Poor, and < 0.5 as Unacceptable. In both cases, subgroup reliability indices were found to be well within acceptable limits.
Table 4
Subgroup (Factor) Reliability Indices (Cronbach's Alpha)
Factors (Study 1) Factors (Study 2) F1 F2 F3 F4 F1 F2 F3 F4 Cronbach's
Alpha0.93 0.89 0.87 0.80 0.96 0.95 0.92 0.89 Discussion
Four comparisons were made using the Coefficient of Congruence and the Salient Similarity Index and the results obtained by the two methods agreed at P < 0.05. Comparisons made using the root mean square method appear to be consistent with these results. Therefore, we concluded that the OWEI factors are replicable in different populations and that evidence exists for construct validity of this instrument. We believe that others can use these factors with confidence and without fear of population bias in their research. Also, we observed in this study that marker variables could play a very deciding role in the pairing of factors. In particular, we noted that those factors which were paired on the basis of commonality in marker variables yielded the largest coefficients of congruence in every case . Another important feature of this study is that the factor structures compared were not only obtained for different populations (workers and supervisors) but also in terms of the method with which responses were collected. Work attitudes were measured by selfevaluation (Study 1) and evaluation by others (Study 2). This leads us to believe that, irrespective of evaluation method, the constructs of the OWEI are replicable. The comparatively high reliability indices for the factors in both studies indicate that the internal consistency of the factors is high. This is consistent with the evidence for construct validity of this instrument.
This study was conducted in a manufacturing environment. The data used in the analysis were obtained from manufacturing employees in the central Illinois area. A randomized block design was used to select the industries studied because we wanted to increase generalizability of results. However, it cannot be assumed that the same results will occur for other groups. The constructs represented in the OWEI may remain constant for other populations, but this hypothesis should be confirmed by additional research. It is possible that work ethic differs among occupational areas, populations, communities and perhaps even organizations. There may even be multiple work ethics possessed by different people and groups within the same occupational area. We hope that subsequent research will shed light on these and other questions concerning the measurement of work attitudes, values, and habits.
References
Anastasi, A. (1976). Psychological testing (4th Ed.). New York NY: MacMillan.
Azam, M. S. (2002). A study of supervisor and employee perceptions of work attitudes in information age manufacturing industries . Unpublished master's thesis, Illinois State University, Normal.
Beach, D. P. (1982). A training program to improve work habits, attitudes, and values. Journal of Epsilon Pi Tau, 8 (2), 69-74.
Broadbooks, W. J., & Elmore, P. B. (1987). A monte carlo study of the sampling distribution of the congruence coefficient. Educational and Psychological Measurement, 47 , 1-11.
Carroll, A., Houghton, S., & Baglioni, A. J. Jr. (2000). Goals and reputations among young children: The validation of the importance of goals and reputation enhancement scales. School Psychology International, 21 (2), 115-135.
Cattell, R. B. (1978). The scientific use of factor analysis in behavioral and life sciences . New York, NY: Plenum Press.
Comrey, A. L. (1973). A first course in factor analysis . New York, NY: Academic Press.
Cordano, M., Scherer, R. F., & Owen, C. L. (2003). Dimensionality of the women as managers scale: Factor congruency among three samples. The Journal of Social Psychology, 143 (1), 141-143
Commission on the Skills of the American Workforce. (1990, February). America's choice: High skills or low wages! Executive summary . Rochester, NY: National Center on Education and the Economy.
Custer, R.L., & Claiborne, D.M. (1992). Critical skills cluster for vocational education. Journal of Vocational Education Research, 17 (4), 15-40.
Custer, R.L., & Claiborne, D.M. (1995). Critical skills cluster for vocational education: The employer's perspective - A replication study. Journal of Vocational Education Research, 20 (1), 7-33.
Erickson, R. C., & Wentling, T. L. (1988). Measuring student growth. Urbana , IL: Griffon Press.
Gay, L. R. (1996). Educational Research . Englewood Cliffs, NJ: Prentice Hall.
George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update (4th ed.). Boston: Allyn & Bacon.
Gregson, J. A., & Bettis, P.J. (1992, December). Secondary trade and industrial education work values instruction: Emancipatory or indoctrinational? Paper presented at the American Vocational Association Convention, Los Angeles, CA.
Guadagnoli, E., & Velicer, W. (1991). A comparison of pattern matching indices. MultivariateBehavioral Research, 26 (2), 323-343.
Harman, H. H. (1976). Modern factor analysis . Chicago, IL: The University of Chicago Press.
Hatcher, T. (1995). From Apprentice to Instructor: Work ethic in apprenticeship training. Journal of Industrial Teacher Education, 33 (1) , 24-45.
Hill, R. B. (1996). Work ethic differences in vocational education students and fulltime employed workers. Journal of Vocational Education Research, 21 (3), 13-29.
Hill, R. B. (1997). Demographic differences in selected work ethic attributes. Journal of Career Development, 24 (1), 3-23.
Hill, R. B., & Petty, G. C. (1995). A new look at selected employability skills: A factor analysis of the occupational work ethic. Journal of Vocational Education Research, 20 (4), 59-73.
Kerlinger, F. N., & Pedhazur, E. J. (1973). Multiple regression in behavioral research . New York, NY: Holt, Rinehart and Winston, Inc.
Koschat, M.A., & Swayne, D. F. (1991). A weighted procrustes criteria. Psychometrica,56 (2), 229-239.
Nunnally, J. (1978). Psychometric theory. New York: McGraw-Hill.
Ommundsen, R., Hak, T., Morch, S., Larsen, K. S., & Veer, K. V. D. (2002). Attitudes toward illegal immigration: A cross-national methodological comparison. The Journal of Psychology, 136 (1), 103-110.
Petty, G. C. (1995). Adults in the work force and the occupational work ethic. Journal of Studies in Technical Careers, XV (3), 133-140.
Petty, G. C., & Hill, R. B. (1994). Are women and men different? A study of the occupational work ethic. Journal of Vocational Education Research, 19 (1), 71-89.
Reschly, D. J. (1978). WISC-R factor structures among anglos, blacks, chicanos, and native - American papagos. Journal of Consulting and Clinical Psychology, 46 , 417-422.
Rummell, R. J. (1970). Applied Factor Analysis. Evanston, IL: Northwestern University Press.
Sakamoto, S., Kijima, N., Tomoda, A., & Kambara, M. (1998). Factor structures of the Zung self-rating depression scale (sds) for undergraduates. Journal of Clinical Psychology, 54 (4), 477-487.
Williams, D. E. & Potter, A. E. (1994). Factor structure and factorial replication of the children's roles inventory. Educational and Psychological Measurement, 54 (2), 417-427.
Yankelovich, D., & Immerwahr, J. (1984). Putting the work ethic to work. Society, 21 (2), 58-76.
Zack, M., Toneatto, T,, & Streiner, D. L. (1998). The SCL-90 factor structure in comorbid substance abusers. Journal of Substance Abuse, 10 (1), 85.
Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99 , 432-442.
Authors' Notes
Paul Brauchle is Professor of Technology, Director of the Bureau of Training and Development, and Associate Director, The Center for Mathematics, Science and Technology, Department of Technology, Illinois State University, 210K Turner Hall, Campus Box 5100, Normal, IL 61790-5100. Phone: (309) 438-2696. E-mail: pebrauc@ilstu.edu .
Md. Shafiqul Azam can be contacted at the Illinois Network of Child Care Resource and Referral Agencies, 207 West Jefferson Stret, Suite 503, Bloomington, IL 61701. Phone: (309) 829-5327/(800) 649-1884. E-mail: sazam@verizon.net .