Students' Attitudes Toward STEM: Development of an Instrument for High School STEM-Based Programs
Students' Attitudes Toward STEM: Development of an Instrument for High School STEM-Based Programs
Mark Patrick Mahoney
The intent of this study was to develop an instrument to measure the current level of attitude that students' exhibit toward STEM education. The Concerns-Based Adoption Model, Taxonomy of Education Objectives - Handbook II , and other pertinent instruments were utilized as sources of inspiration for the instrument. The selected items were submitted to a panel of experts representative of STEM education. Initial pilot testing refined the instrument through principal components analysis and Cronbach's alpha coefficients. The identified principal components aligned well with reviewed instruments. Reliability coefficients were strong for each of the principal components.
Results of the combined analyses led to revisions of the instrument prior to a larger comparative study - a known-group comparison. A self-identified STEM-based high school program and a conventional college-preparatory program were compared. Principal components analysis and Cronbach's alpha procedures were again applied to the data collected. The two samples were compared using three distinct independent variables - educational location, grade level, and gender. Each independent variable was analyzed for each principal component.
MANOVA procedures were utilized. Male students indicated a statistically significant more positive attitude toward STEM when compared to the female students for the independent variable of gender. The statistical significance was demonstrated specifically for the content areas of technology and engineering. The results of the data analysis supported the proposed hypothesis. Based upon extensive review of the varied data analysis procedures implemented, the students' attitudes towards the STEM instrument demonstrated positive examples of validity and reliability.
In 1983, A Nation at Risk (National Commission on Excellence in Education [NCEE], 1983) established the resurgence for the science, technology, engineering, and mathematics (STEM) movement in education.
The time is long past when American's destiny was assured simply by an abundance of natural resources and inexhaustible human enthusiasm, and by our relative isolation from the malignant problems of older civilizations. The world is indeed one global village. We live among determined, well-educated, and strongly motivated competitors. We compete with them for international standing and markets, not only with products but also with the ideas of our laboratories and neighborhood workshops. America's position in the world may once have been reasonably secure with only a few exceptionally well-trained men and women. It is no longer. (p. 10)
The influence of this report and its recommendations are echoed in the feverish development of national standards produced by academic organizations such as the National Council of Teachers of Mathematics (NCTM), the American Association for the Advancement of Science (AAAS), the National Research Council (NRC), and the International Technology Education Association (ITEA). It is within this process that we the history of STEM can be traced. NCTM (2000) , AAAS (1989) , NRC (1996) and ITEA (2000) documents all suggest the combination or integration of their respective subjects in an attempt to enhance student learning and STEM preparation.
This proposed subject integration has taken many forms since the overall arrival of standards. Programs, modules, packaged curriculums, and even charter schools have aligned themselves with proposed models of what a STEM educational program should represent. A report by the Academic Competitiveness Council ( [ACC], 2007 ) indicates that there are up to 105 government-funded STEM education programs in the United States, ranging from kindergarten to post-graduate education. The report by the ACC also collected information regarding the cost associated with STEM education programs. Overall, estimates indicated a total government expenditure to exceed $3.12 billion during the 2006 fiscal year.
Evaluations of these programs were also collected and reviewed ( ACC, 2007 ). Unfortunately, a majority of the evaluations were below the expectations of the council. In fact, those that did display potential still required revisions to add greater validity to the information provided. This is not a new occurrence. The National Science Foundation (NSF) has been revising its own grant procedures to account for this lack of efficient evaluation. Programs funded by NSF and other organizations have continued for years with government money without providing sufficient information or measurable influence upon the educational community ( ACC, 2007 ).
Added to this condition is the limitless number of private industries that have produced and sold STEM educational products and curricula over the last 20 years. These varied items align themselves with national standards and suggest educational advancement in the form of problem solving, cooperative learning, and subject integration. However, very little research has been conducted regarding the degree of influence such products have had upon education or even student learning ( Bottoms & Anthony, 2005 ; ITEA, n.d. ; PTC-MIT Consortium, 2006 ). A more recent development is the creation of entire educational institutions devoted to STEM development. These schools are not vocational or career and technical institutions, but rather college preparatory programs designed to develop students' abilities and interest in STEM and STEM careers.
In 2005, the report Tapping America's Potential (Business Roundtable) produced a summary of the concerns from a variety of local professional organizations. The report cited warnings in the form of a declining STEM-equipped population, increased foreign competition, low student interest toward engineering, low student achievement, and decline in research funding ( Business Roundtable, 2005 ). The American Electronics Association (AeA) also shared their concern through the following statement in 2005: "America needs to recognize that future innovation is not predetermined to occur in the United States. Even if we were doing everything right, we still face unprecedented competition from abroad" (p. 3).
Large amounts of money and time have already been provided in the hopes that educational institutions will reinforce students' attitudes and abilities related to STEM. However, these donations have yielded little results as demonstrated by the continued reports being constructed each year demanding greater STEM investment and results. The development of an instrument that can accurately measure students' attitudes toward STEM is crucial to STEM-based programs, their intended outcomes, and the companies that aid in their implementation.
In late 2008, the development of an instrument capable of measuring students' attitudes toward STEM began. In order to create this new instrument, the research study was divided into three phases. Phase I consisted of the development of an instrument capable of measuring students' attitudes toward STEM. A panel of experts was assembled and utilized for initial face validity as well as item development. Phase II verified the instrument through pilot-testing and high school student focus group interviews. Results from the pilot test in addition to student responses were then used to revise the instrument.
Phase III completed the intended study by implementing the revised instrument at two high school settings; a conventional college-preparatory school and a STEM-based college-preparatory school. It was hypothesized that students enrolled in the STEM-based high school program would exhibit more positive attitudes toward STEM when compared to students in a conventional college-preparatory high school program. It was also hypothesized that students exposed to STEM education for a longer period of time would exhibit a more positive attitude toward STEM than students who were just entering the program. Finally, it was hypothesized that male students would exhibit a more positive attitude toward STEM than would female students. These hypotheses were tested in an attempt to provide the students' attitude toward STEM instrument with an additional example of construct validity.
Phase I: Instrument Development
To develop an instrument capable of measuring students' attitudes toward STEM, several existing instruments were reviewed. Many of those reviewed are very strong assessments as indicated by their reported statistics. One example is that of the affective instrument located in the Trends in Mathematics and Science Study (TIMSS). According to Chiu (2007) , the TIMSS instrument provides a useful factorial model. Nevertheless, such analysis has come under fire for suffering "from a number of methodological inadequacies;" not measuring up "to those [instruments] that are now expected for these affective attributes by main stream researchers" ( Fensham, 2007 , p. 3).
Other instruments offered a strong basis for an instrument design, but these did not easily make the transition to a scale capable of measuring students' attitude in multiple subjects. The Kuder Occupational Interest Survey, Form DD ( Zytowski, 1973 , 1992 , 1996 ), Ohio Vocational Interest Survey ( 1981 ), and the Thurstone Interest Schedule ( Thurstone, 1947 ) are examples of these types of instruments. In order to construct an attitudinal instrument for STEM, varied affective based documentation, including associated instruments, were sought and reviewed.
To complicate this search, an abundance of definitions can be found in any document or text whose author attempts to grapple with attitude and attitudinal measures. "The concept [attitude] has been plagued with ambiguity," so much so that researchers "may find it difficult to grasp precisely how they [the varied definitions of attitude] are conceptually similar to or different from one another" ( Rokeach, 1968 , p. 110). The assortment of available definitions has been both a strength and a weakness in the creation of attitudinal instruments.
The meaning of a concept is defined in terms of its relations to other constructs in a theoretical network. Thus two investigators may offer different explicit definitions of attitude. However, if their attitude theories revealed that they agreed on the relationships between attitude and other concepts. . . it could be argued that the term "attitude" has the same meaning for the two investigators." ( Fishbein and Ajzen, 1975 , p. 5). It is for this reason that many of the definitions may be interchangeable ( Rokeach, 1968 ).
Schwarz (2007) stated that a "person's attitude is 'stable' when the person provides similar attitude reports at different times and/or in different contexts" (p. 6). This is exemplified when a judge passes similar judgment on cases that share similar attributes and conditions according to the information provided. If the context is the same, the attitude should be stable. If the context of the judgment should change (i.e., by a change in information or condition), the initial attitude demonstrated will no longer fit the model. By this example, it is assumed that the attitude measurements and definitions should be specific to the variables and conditions for which it is to be implemented. If this is followed, then the established concept of attitude created for that situation should remain stable.
It was imperative in this study to establish a definition of attitude that is reflective of the variables and conditions for which it is to be implemented. Materials and instruments that could serve as forms of inspiration were sought and reviewed. An instrument of interest was the Concerns Based Adoption Model (CBAM); specifically the Stages of Concern (SoC)( Hall, 1974 ; Hall, George, & Rutherford, 1978 ). The CBAM model originally was used to understand how a person, specifically a teacher, reacted to a change in instruction or educational format presented during a professional development sequence. The concept was to be able to gauge how a person reacts to a presented change over the course of its implementation. The CBAM concept was closely related to the problem that is presented to a student when engaged with a STEM-based program. Is it possible to gauge how a student may react to a new educational material and format? Does a student accept or reject the change?
However, the CBAM documentation is not nearly enough to base an entire attitudinal instrument upon. To accomplish this, a more thorough review of affective characteristics was required. This would be provided by an established body of work directly associated with attitude and the entire affective domain: the Taxonomy of Educational Objectives, Handbook II (TEOII) by Krathwohl, Bloom, & Masia (1964) .
The affective domain as established by Krathwohl and collegues is a broad and yet applicable interpretation of the subject. The purpose of the affective domain was to establish objectives that "emphasize a feeling tone, an emotion, or a degree of acceptance or rejection" ( Krathwohl et al., 1964 , p. 7). Terms that were discussed included "interest, attitude, values, etc." (p. 27). It was quickly discovered that definitions were "difficult to devise, and their meanings tended to drift into the connotations and denotations which these terms encompassed in common parlance" (p. 27). A more specific assembly of these characteristics would limit the use and flexibility of the objectives intended to be drawn from this taxonomy. It is for this reason that a combination of the affective taxonomy with the CBAM instrument was considered to be most beneficial.
Therefore, the CBAM and the TEOII were utilized as the inspirational models to create measurable categories specific to students' attitudes and their implications toward STEM. Both foundational pieces address key attributes vital to the concerns of the researcher and the desired instrument, the elements of progressive change within an individual and the affective characteristics of such progressive change. The categories were established by observable similarities between the CBAM and TEOII materials in conjunction with measuring the affective domain. A panel of experts in or related to the field of STEM and STEM education was assembled to review these items. Each expert was provided with the four preliminary categories created by the researcher. The list provided to the experts is displayed in Table 1.
|Awareness:||Interest, recognition, knowing, consciousness, attention, curiosity, concern|
|Perceived Ability:||Capability, skill, be able to, confidence, certainty, self-belief|
|Value:||Worth, significance, importance, usefulness, merit, regard|
|Commitment:||Pledge, dedication, devotion, potential, prospective, intention|
After meeting with each of the panel members, the researcher created a list of 50 initial instrument items. Revisions and corrections were offered from these experts, and they were reviewed by the researcher. A final list of 34 initial items for each content area was assembled - 136 items total (see Table 2).
|Awareness:||1. I like to read about:|
|2. My school offers courses in:|
|3. My school does not offer after school programs in:|
|4. I enjoy watching TV shows involving:|
|5. I do not want to learn more about:|
|6. I do not enjoy taking courses in:|
|7. Courses in [subject] are available to me|
|8. I dislike the challenge of:|
|34. I like:|
|Perceived Ability:||9. I am good at projects involving:|
|10. [subject] is difficult for me:|
|11. I perform well in [subject] courses:|
|12. I can not handle advanced courses in:|
|13. [subject] is simple:|
|14. I do not worry about taking tests in:|
|15. I struggle in [subject] courses:|
|16. I do not understand:|
|17. Homework in [subject] is easy:|
|Value:||18. [subject] is important|
|19. What I learn in [subject] has no value to me:|
|20. I believe there is a need for:|
|21. I need:|
|22. Learning [subject] will not help me:|
|23. [subject] is good:|
|24. I care about developments in:|
|25. [subject] is not worth my time to understand:|
|Commitment:||26. I would dislike more/advanced courses in:|
|27. I would like to participate in more after-school programs in:|
|28. I am curious about a career involving:|
|29. I am interested in advanced programs involving:|
|30. I have no interest in discovering new ways to apply:|
|31. [subject] is not a vital part of my perceived future:|
|32. I intend to further develop my abilities in:|
|33. I will continue to enjoy the challenge of:|
The next process was to formulate each item into a scale that could measure across the four content areas of STEM. A variation of a four-level Likert scale was created and implemented in an attempt to avoid central tendency bias. Each level of the scale was arranged to represent all four content areas of STEM. This was accomplished by placing each scale in what is referred to as an "item block" (see Figure 1).
Phase II: Pilot Study
Once the complete instrument was assembled, a high school within a local, metropolitan school district was contacted and used in an initial review of the instrument. The student sample was drawn from an accessible school population that was randomly selected from preexisting homerooms established by the high school administration (see Table 3).
|Grade Level||Provided||% of School||Returned||%||Completed||%|
Pilot Study Results
Three principal components were identified as a result of the principal components analysis: interest, ability , and value . According to the Cronbach's alpha calculations, each identified component indicated very high reliability with alpha ratings above .70 (see Table 4). A focus group of available students was conducted following the study. The students were asked to re-state the items in their own words to demonstrate item clarity and overall communication of the instrument. This was conducted to avoid certain aspects of measurement error. It was expected that these steps would provide greater content and face validity in addition to the computer-based data analysis.
|Content areas||Principal components|
|Alpha||No. of items||Alpha||No. of items||Alpha||No. of items||Alpha||No. of items|
Lastly, a Pearson product moment correlation was established between a semantic differential instrument (SEMDIFF) and the STEM instrument. The SEMDIFF instrument was also given to the participating students. The correlation was .58 ( p = .001), indicating a significant, moderately positive relationship between the two instruments. Significance varied for each content area: science, r = .46, p = .013, technology, r = .41, p = .031, engineering, r = .50, p = .007, mathematics, r = .75 ( p = .000). A collection of bi-polar pairs did not display discernable consistency toward either of the identified principal components. Data from items identified as questionable were removed prior to a second analysis - these were labeled as a modified SEMDIFF.
The Pearson product moment correlation between the overall modified SEMDIFF and STEM instruments was now .63 ( p = .000), indicating a somewhat more significant and moderately positive relationship than the previous score of .58 ( p = .001): r = .48, p = .010, technology, r = .40, p = .034, engineering, r = .63, p = .007, mathematics, r = .76 ( p = .000). The correlation provided an example of concurrent validity for the Student Attitude Toward STEM instrument in its use as a measure of the construct of attitude.
A complete item analysis was conducted for each item for the instrument. Due to the brevity of this document, this analysis has been omitted. However, based upon the collective review of data, 24 items for each content area were compiled for the revised instrument and used in the comparison study: 96 items total. The panel of experts was again contacted and used to review the 24 items for each content area (see Table 5).
|Awareness: (Initial Interest)||1. I do not like|
|2. I enjoy learning about|
|3. I am curious about|
|4. I am not interested in|
|5. I like|
|6. (subject) is appealing to me|
|7. (subject) is difficult for me|
|8. I do well in|
|9. I am not confident about my work in|
|10. I have a hard time in|
|11. Assigned work in (subject) is easy for me|
|12. I can not figure out|
|13. (subject) is important to me|
|14. I feel there is a need for|
|15. I do not need|
|16. It is valuable for me to learn|
|17. (subject) is good for me|
|18. I do not care about|
|Commitment: (Long-term interest)||19. I will continue to enjoy|
|20. I am not interested in a career involving|
|21. I am interested in alternative programs in|
|22. I would like to learn more about|
|23. I do not wish to continue my education in|
|24. I am committed to learning|
Phase III: Known-Group Comparison Study
After completing the initial review of the STEM attitude instrument, a Known-Group Comparison Study was performed. Two high schools within a local metropolitan area were used. One high school consisted of a publicly identified STEM-based program, and the other high school consisted of a state-defined college preparatory program. Two grade levels - the ninth and eleventh grades - from each high school were provided the instrument packets. The collection rate is provided in Table 6. Table 7 describes the distribution of participants by school, grade, and gender in the known-group comparison sample.
|School||Grade Level||Distrib.||% of Pop.||Returned||%||Completed||%|
|STEM-based high school||Ninth||92||100%||37||40%||35||38%|
|College-preparatory high school||Ninth||118||37%||52||44%||48||41%|
|STEM-based high school||College-preparatory high school|
Known-Group Comparison Results
A second principal components analysis was conducted. This was required because of revisions to the instrument. Again, three principal components were identified by the researcher for all content areas: interest, ability , and value . A high percentage of variance was explained by the three identified principal components for each content area: science = 69%, technology = 64%, engineering = 73%, and mathematics = 68%.
Possible intercorrelations between the identified principal components were demonstrated by shared item loadings. Item loadings for all content areas and initial item design intentions were considered prior to assigning principal components. The possibility of intercorrelations between principal components was momentarily overlooked so that further statistical analysis could be performed. This decision was approved for the sake of the study and its preliminary character. Future studies with larger sample sizes and a refined Student Attitude Toward STEM instrument will appropriately address this concern.
Internal reliability was again estimated though the use of Cronbach's alpha internal consistency coefficient. The complete collection of items used in the pilot study provided very strong alpha ratings for each of the content areas (see Table 8).
|Alpha||No. of items||Alpha||No. of items||Alpha||No. of items||Alpha||No. of items|
|Note . Item 22 was removed from the analysis, resulting in an overall total of 23 items.|
According to the results of the data analysis, the STEM-based high school students did not exhibit a statistically significant more positive attitude toward the content areas of STEM when compared to the college-preparatory high school students. It was anticipated that the STEM-based high school students would show a more positive attitude due to the school programs' specific focus and dedication toward STEM, as indicated by public documentation. This proposed difference would have provided an example of construct validity for the Student Attitude Toward STEM instrument. Though this result was not anticipated, it was not believed to have negative implications for the student attitude toward the STEM instrument. Variables or factors that could have influenced this outcome - positively or negatively - have not yet been identified or investigated.
Interestingly, a statistically significant more positive attitude was demonstrated by the college-preparatory high school students when compared to the STEM-based high school students for the content area of mathematics. Review of this analysis could allow for the determination that both high school programs support similar positive attitudes for the content areas of science, technology, and engineering. Also, it may be determined that the college-preparatory high school is supporting a more positive student attitude for mathematics when compared to the STEM-based program students.
Similarly, the students in the eleventh grade did not exhibit a statistically significant more positive attitude for the content areas of STEM when compared to the students in the ninth grade. Like the previous hypothesis, an unexpected and opposite result was demonstrated by the analyses. A statistically significant more positive attitude was demonstrated by the ninth grade students when compared to the eleventh grade students for the content area of mathematics. Review of this analysis could allow for the determination that students at both grade levels exhibit similar levels of attitude for the content areas of science, technology, and engineering. It could also be determined that the ninth-grade students had more positive attitudes for STEM than did eleventh-grade students for the content area of mathematics.
Lastly, the male students did indicate a statistically significant more positive attitude for STEM when compared to the female students. The statistical significance was demonstrated specifically for the content areas of technology and engineering. The results of the data analysis supported the proposed hypothesis for the content areas of technology and engineering, and therefore they provided the Student Attitude Toward STEM instrument with an example of construct validity.
It was anticipated that the male students would provide a more positive attitude for STEM and STEM education due to the gender bias that has been traditionally associated with the STEM content areas. Though not statistically significant, an unexpected and interesting result was revealed in the analyses. Male students did not depict a statistically significant more positive attitude for STEM for the content areas of science and mathematics. This would imply that male and female students do not differ significantly regarding their attitudes for these two content areas.
This study was described as a critical tool for STEM education programs as well as the organizations that support them. The instrument was developed to indicate students' attitudes toward STEM, so that educational institutions that are implementing a STEM-based program can ascertain if their program is having the desired influence on their students.
Levels of student attitude were accurately defined and identified through review of pertaining literature, utilization of a panel of experts, as well as appropriate statistical analysis. The initial analyses demonstrated the foundational construct and content validity for the student attitudinal instrument. They were identified as interest, ability , and value . Items required to address each category of student attitude were defined and identified through review of pertaining instruments, a panel of experts, a student focus group, and appropriate statistical analysis. The combined analyses applied to the instrument items provided strong indications of reliability.
Reliability coefficients collected from the applications of the two versions of the Student Attitude Toward STEM instrument indicated Cronbach's alpha scores above what was anticipated based on established attitudinal instruments; coefficient of .92 alpha. This far exceeded the .70 alpha anticipated from the established research. The Pearson product moment correlation between the Student Attitude Toward STEM instrument and the SEMDIFF indicated an overall moderately positive significant relationship between the two instruments ( r = .63, p = .000). This provided the Student Attitude Toward STEM instrument used for the pilot study with a viable source of concurrent validity.
The instrument was effective in identifying differences between male and female students. The instrument did not detect significant differences between the schools or the grade levels. The lack of detection of difference may not be a deficiency of the instrument, but it could be due to sensitivity provided by small and exclusive samples. Another possible indication could be the actual lack of difference between the independent variable groups of school and grade level. Larger and more varied samples should provide enough information to resolve these concerns.
Further review of the instrument and its associated items will continue through the exploration of larger and varied samples. It is expected that students' attitudes toward the STEM instrument will be exposed to as much research and revisions as are available until it becomes an applicable and reliable attitudinal measurement device. Recommendations for future research include, but are not limited to the following:
- Repeat the study with a larger and more varied sample size.
- Use longitudinal application of the instrument to previously assessed students.
- Conduct individual student interviews following submission of the instrument.
- Review the combined influence of independent variables.
- Investigate other possible independent variables.
An official timeline has not been established for completion of the instrument. Review of other attitudinal instruments revealed that the development and research required for a substantial attitudinal instrument is almost never complete and could continue on indefinitely. This study was an initial step toward what could be a lifelong development of an instrument to measure students' attitudes toward STEM. It was an imperative step in providing what could be a valuable tool for STEM-based educational programs as well as organizations that support them.
Dr. Mark Patrick Mahoney is currently an assistant professor within the Technology and Industrial Arts Department at Berea College in Berea, Kentucky. He is a member of Alpha chapter of Epsilon Pi Tau.
Academic Competitiveness Council [ACC]. (2007). Report of the academic competitiveness council . Washington, DC: U.S. Department of Education.
American Association for the Advancement of Science[AAAS]. (1989). Science for all Americans: A Project 2061 report on literacy goals in science, mathematics, and technology . Washington, DC: Author.
American Electronics Association[AeA]. (2005). Losing the competitive advantage: The challenge for science and technology in the United States . Washington, DC: Author.
Business Roundtable. (2005, July). Tapping America's potential: The education for innovation initiative . Washington, DC: Author.
Chiu, M-S. (2007). Mathematics as mother/basis of science in affect: Analysis of TIMSS 2003 Data. Proceedings of the 31st Conference of the International Group for the Psychology of Mathematics Education, Taiwan, 2 , 145-152.
Fensham, P. (2007). Interest in science: Lessons and non-lessons from TIMSS and PISA. In Pinto, R. & Couso,D. (Eds.), Contributions from science education research (pp. 3-10). Dordrecht, The Netherlands: Springer.
Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research . Reading, MA: Addison-Wesley.
Hall, G. E. (1974). The concerns-based adoption model: A developmental conceptualization of the adoption process within educational institutions . Austin: Research and Development Center for Teacher Education, University of Texas.
Hall, G. E., George, A. A., & Rutherford, W. L. (1978). Stages of concern about the innovation: The concept, verification, and implications . Austin: University of Texas.
International Technology Education Association [ITEA]. (n.d.). Engineering by design: A standards-based model program . Retrieved January 24, 2008, from http://www.iteaconnect.org/ EbD/ebd.htm
International Technology Education Association [ITEA]. (2000). Standards for technological literacy: Content for the study of technology . Reston, VA: Author
Krathwohl, D. R., Bloom, B. S., & Masia, B.B. (1964). Taxonomy of educational objectives: The classification of educational goals, handbook II . New York: McKay.
National Commission on Excellence in Education[NCEE]. (1983). A nation at risk: The imperative for educational reform: A report to the nation and the Secretary of Education, United States Department of Education . Washington, DC: Author.
National Council of Teachers of Mathematics [NCTM] (2000). The principles and standards for school mathematics . Reston, VA: Author
National Science Foundation [NSF]. (n.d.). National science foundation: Where discoveries begin: About funding . Retrieved February 7, 2008, from http://www.nsf.gov/funding/aboutfunding.jsp .
National Science Foundation [NSF]. (n.d.). National STEM education distributed learning (NSDL): Definition of STEM . Retrieved May 25, 2000, from http://www.nsf.gov/pubs/2009/nsf09531/nsf09531.html
Ohio vocational interest survey . (1981). New York: The Psychological Corporation.
PTC-MIT Consortium. (2006). Preparing for the perfect storm . Washington, DC: National Academy of Engineering.
Rokeach, M. (1968). Beliefs, attitudes, and values: A theory of organization and change . San Francisco: Jossey-Bass.
Schwarz, N. (2007). Attitude construction: Evaluation in context. In B. Gawronski (Ed.), What is an attitude? Social Cognition, 25 (5), 638-656.
Thurstone, L.T. (1947). Thurstone Interest Schedule: Manual. New York: Psychological Corporation.
Zytowski, D. G. (1973). Contemporary approaches to interest measurement. Minnesota library on student personnel work (Vol. 10). Minneapolis, MN: University of Minnesota Press.
Zytowski, D. (1992, November). Three generations: The continuing evolution of Frederic Kuder's interest inventories. Journal of Counseling & Development, 71 (2), 245-248. Retrieved May 13, 2008, from Professional Development Collection database.
Zytowski, D. (1996, December). Three decades of interest inventory results: A case study. Career Development Quarterly, 45 (2), 141-148. Retrieved May 13, 2008, from Professional Development Collection database.