JITE v46n1 - Examination of Assessment Practices for Engineering Design Projects in Secondary Education (First in a Three Part Series)

Volume 46, Number 1
Spring 2009


Examination of Assessment Practices for Engineering Design Projects in Secondary Technology Education (First article in 3-part series)

Todd R. Kelley, Ph.D.
Purdue University

Robert C. Wicklein, Ed.D.
University of Georgia

Overview

The following descriptive study was designed to determine the national status of secondary technology education curriculum content and assessment practices as they relate to engineering design. The results of this study were divided into a three-part article series. Although this study focused on the larger construct of the national status of the infusion of engineering design into technology education, three separate sub-constructs emerged. The three sub-constructs were: a) status of engineering design curriculum content; b) the status of assessment practices of engineering design projects, and c) what selected challenges are identified by secondary technology educators in teaching engineering design.

Introduction

Educators within the field of technology education took a great leap forward in establishing a clear direction for the discipline with the publication of Standards for Technological Literacy: Content for the Study of Technology ( ITEA, 2000/2002 ). Additionally, the professional development standards in Advancing Excellence in Technological Literacy ( ITEA, 2003 ), and the call for technological literacy by the National Academy of Engineering and National Research Council in their document Technically Speaking: Why all Americans Need to Know More About Technology ( NAE & NRC, 2002 ) continued to provide focus for the technology education curriculum. Each of these documents clearly established a need to teach technological literacy to all K-12 students. Although none of these documents endorsed a specific method of delivering technological literacy, many in the field of technology education as well as agencies outside of technology education (National Academy of Science) suggested engineering or engineering design as a curricular focus for technology education to achieve technological literacy ( Daugherty, 2005 , Lewis, 2004 , NAE NRC, 2002 , Rogers, 2005 , Wicklein, 2006 ). From an engineering perspective, Douglas, Iversen, and Kalyandurg (2004) also cited the American Society for Engineering Education (ASEE) research results that indicated a strong support for teaching engineering in K-12 public schools.

However, the field of technology education has a history of experiencing curriculum reforms that generate new program titles with little curriculum changes ( Akmal, Oaks, & Barker, 2002 ; Clark, 1989 ; Sanders, 2001 ). Considering this history of resistance to change in the field of technology education, questions arise about the current curriculum shift to move to engineering design as a content focus.

Recently, there have been new curriculums designed to infuse engineering content into technology education such as Project ProBase, Principles of Engineering; Project Lead the Way, Principles of Technology; Engineering Technology; and Introduction to Engineering ( Dearing & Daugherty, 2004 ). Certainly, research was needed to determine the status and degree to which engineering design content was being presented within the field of technology education.

Methodology

Research Design

This descriptive study examined the degree to which technology educators are implementing elements of engineering design in their curriculums. The research collected data about the degree to which engineering design concepts were incorporated into the curriculum content in the secondary technology education. The researchers made a clear distinction between the goals of engineering design and other issues to connecting engineering concepts to the curriculum. One definition for engineering design defined by the Accreditation Board for Engineering and Technology (ABET) states: “as the process of devising a system, component, or process to meet desired needs. It is a decision making process (often iterative), in which the basic science, mathematics, and engineering sciences are applied to convert resources optimally to meet a stated objectives” ( Eide, Jenison, Mashaw, & Northup, 2002 , p. 79).

The research question guiding this part of the research study was:

  1. To what degree does the current curriculum content of secondary technology education programs reflect engineering design concepts?

Population and Sample

A full sample was taken of secondary technology educators who were members of the International Technology Education Association (ITEA) as of September 2007. The identified population of this study consisted of a total of (N=1043) high school technology education teachers in the ITEA membership database. The original research design for this study called for an increase of the initial mailing of the survey by 48.1 percent, the average success rate of an initial mailing ( Gall et al; 2007 ). However, after communication with ITEA personnel that revealed that ITEA survey mailings typically yield a 20-25% rate of return (Price, personal communication), the researchers determined that a full population mailing to all ITEA high school members was necessary to achieve the desired sample of 285.

Instrumentation

Data Collection Procedure

An invitation message was sent through e-mail to all ITEA members in the sample explaining specific instructions for completing the on-line questionnaire and directing participants to access a specific website to obtain and complete the survey questionnaire. The on-line questionnaire was developed using the guidelines and recommendations outlined by Dillman, Tortora, and Bowker (1999) . There was a request to return the survey by a specified date. After waiting three days past the specified date of return, which was three weeks after the initial mailing, the researchers contacted non-respondents by sending a follow-up e-mail delivered letter containing the URL for the on-line survey link. The on-line survey company was Hosted Survey. This has been a proven method used by other researchers to achieve compliance from non-respondents ( Gall et al., 2007 ).

The survey instrument gathered data relating to the degree to which engineering design concepts were incorporated into technology education curriculum content. The curriculum content items were created from the results of Childress and Rhodes (2008) study and Smith’s (2006) study to create the framework for defining engineering design curriculum content in seven categories, see Table 1. Childress and Rhodes (2008) and Smith (2006) used a modified Delphi research method which requires a construct and content validation procedure, thus, providing survey items that were already tested for validity and reliability ( Messick, 1989 ). The results of this research yielded an overall .982 Cronbach’s Alpha for internal consistency. Participants were required to respond to each curriculum content item twice, for frequency of use and for time per typical use using a six-point Likert scale. See Table 2. Assumptions: Traditional schedule meets 5 days a week, 50 minute period, 184 day school year. Typical A/B and 4x4 block scheduling meets for 92 days for 90 minutes.

Table 1. The Seven Categories of Engineering Design Content
Seven Categories of Engineering Design Content
Engineering Design
Engineering Analysis
Application of Engineering Design
Engineering Communication
Design Thinking as It Relates to Engineering Design
Engineering and Human Values
Engineering Science
Table 2. Teaching Style Scale Conversion
How Often? (Frequency)
Likert Wording Traditional (meets 5 days a week) Block
0 Never 0 0
1 A few times a year 5 days 5 days
2 1 or 2 times a month 14 days (1.5*9.1) 7 days (1.5*4.6)
3 1 or 2 times a week 55 days (1.5*36.8) 28 days (1.5*18.4)
4 Nearly everyday 129 days (3.5*36.8) 64 days (3.5*18.4)
5 Daily 184 days 92 days
How Many Minutes? (Time)
Likert Wording Traditional (50 minutes per period) Block (90 minutes per period)
0 None 0 min. 0 min.
1 A few minutes per period 5 min. 9 min.
2 Less than half the period 15 min. 30 min.
3 About half 25 min. 45 min.
4 More than half 37.5 min. 67.5 min
5 Almost all period 50 min. 90 min.

Results

Results from the school demographic section of the survey revealed that 62.4% of respondents worked in schools that use a traditional school schedule with classes meeting five days a week for approximately 50 minute each period; the other 37.6% of those responding to the survey work in schools that implement a block schedule to organize the school day. See Table 3. Of those responding to the survey, 27% teach in schools in a rural setting, 47.4% teach in schools in a suburban setting, and 25.6% teach in schools in an urban setting. School size was also measured in the school demographic section. A total of 14.6% of the participants from this study teach in small (less than 500 students) high schools, 45.1% teach in medium size (500-1500) high schools, and 40.3% of respondents teach in large (greater than 1500 students) size schools. See Table 3 for a detailed breakdown of the general demographics of the respondents.

Table 3. General Demographic Information
Demographic Criteria # of responders % of Total
Which best describes your current position?
Middle/High school teacher 23 10.2%
High School teacher 198 87.6%
Other 5 2.2%
Years of experiences as a technology educator at the start of the 2007-2008 school year
no prior experience 5 2.2%
Less than one year 12 5.3%
1-5 years 36 15.9%
6-10 years 31 13.7%
11-15 years 32 14.2%
16-20 years 25 11.1%
20+ years 85 37.6%
Gender
Male 195 86.2%
Female 31 13.7%
Age at last birthday
Under 25 7 3.1%
25-30 33 14.6%
31-35 20 8.9%
36-40 19 8.5%
41-45 31 13.7%
46-50 34 15.0%
51-55 52 23.0%
56-60 22 9.7%
61-65 7 3.1%
+65 1 0.4%
Highest college degree attained
B.S./B.A. 73 32.3%
Masters 145 64.2%
EdS-Specialist 8 3.5%

The biographical demographic section of the survey revealed that 10.0% of the respondents teach at a middle and high school, compared with 88.0% of respondents indicating they are assigned exclusively to high schools, while 2.0% selected other to describe the grade level they teach. The majority of respondents had multiple years of experience with 62.8% possessing 11 or more years of experience; within that 62.8%, 37.6% have 20+ years of teaching experience. A total of 35.0% of the responses to the survey came from technology education teachers with one to 10 years of experience, and 2.2% of teachers who responded to the survey were in their first year of teaching; see Table 3 for further breakdown of the biographical demographic information. A total of 195 participants were male for a total of 86.3% of responders, leaving 13.7% being female. As mentioned before, the respondents were veterans of the teaching profession, thus, they were deemed as a mature group of professionals. Survey results revealed that 65.0% of the participants are over the age of 40. A total of 32.0% of the teachers who completed the survey are between the ages of 25 to 40. Only 3% of respondents are under the age of 25. The teachers who responded to this survey were not only experienced but were also highly educated with 64.2% holding a Master’s degree, and 3.5% having earned an educational specialist degree. A total of 32.3% have obtained just the required B.S./B.A, a degree necessary to teach technology education in public schools.

The category Engineering Design was the highest ranked category measured by frequency with a mean score of 3.15. See Table 4 for complete list of rankings based upon frequency of use. Engineering Communication was the highest ranked category with a group mean score of 2.80 for time per typical use. See Table 5 for complete list of category rankings based upon time per typical use.

Table 4. Engineering Design Category Rankings for Frequency of Use
Rank Engineering Design Content Category Total Group Mean
f
Total Group SD
f
1 Engineering Design 3.15 1.24
2 Design Thinking Related to Eng. Design 3.00 1.28
3 Engineering Communication 2.89 1.42
4 Engineering Analysis 2.79 1.32
5 Application of Engineering Design 2.77 1.29
6 Engineering Science 2.33 1.35
7 Engineering and Human Values 2.22 1.29
Table 5. Engineering Design Category Rankings for Time Per Typical of Use
Rank Engineering Design Content Category Total Group Mean
Time
Total Group SD
Time
1 Engineering Communication 2.80 1.41
2 Design Thinking Related to Eng. Design 2.74 1.32
3 Application of Engineering Design 2.59 1.33
4 Engineering Design 2.38 1.25
5 Engineering Analysis 2.37 1.32
6 Engineering Science 2.16 1.33
7 Engineering and Human Values 2.08 1.31

Within the Design Thinking Related to Engineering Design category, thinking critically had the highest mean score measured by frequency of use 3.65. See Table 6 for a list of the top five mean scores for individual survey items based upon frequency of use.

In the Engineering Design category, the highest-ranking individual item (measured by time per typical use) use of computer-aided design to construct technical drawings with a mean score of 3.35; see Table 7 for a list of the topic five individual items based upon time per typical use. Also the item use technical drawings to construct or implement an object, structure, or process (mean score of 3.30), received a high mean score. The emphasis of CAD in technology has been discovered in other status studies ( Dearing & Daugherty, 2004 ; Sanders, 2001 ; Warner & Mumford, 2004 )

Another result of particular interest is that the second highest ranked item measured by time per typical use was develop basic student’s skills in the use of tools with a mean of 3.32. It appears that the field of technology education has not moved far from its industrial arts roots. As a matter of fact, a similar survey item, developing skill in using tools and machines, was the highest ranked item in the Standards for Industrial Arts Program Project SfIAP project ( Dugger, Miller, Bame, Pinder, Giles, Young, & Dixon, 1980 ) and Schmitt and Pelly study (1966) according to Sanders (2001) .

Table 6.Top Five Individual Engineering Design Items Mean Scores for Frequency of Use
Top Five Individual Items (category) Mean f SD f
1 think critically (Design Thinking) 3.65 1.10
2 developing basic student’s skills in the use of tools (Application of ED) 3.46 1.26
3 understanding that knowledge of science and mathematics is critical to engineering (Engineering Analysis) 3.44 1.20
4 use computer-aided design to construct technical drawings (Engineering Communication) 3.39 1.52
5 use technical drawings to construct or implement an object, structure, or process (Engineering Communication) 3.34 1.26
Table 7. Top Five Individual Engineering Design Items Mean Scores for Time per Typical Use
Top Five Individual Items (category) Mean f SD f
1 use of computer-aided design to construct technical drawings (Engineering Communication) 3.35 1.49
2 develop basic student’s skills in the use of tools (Application of ED) 3.32 1.34
3 use technical drawings to construct or implement an object, structure, or process (Engineering Communication) 3.30 1.25
4 visualize in three dimensions (Engineering Communication) 3.19 1.32
5 think critically (Design Thinking) 3.15 1.22

A composite score for total hours of teaching time dedicated to the seven engineering content categories was generated using the units of time and frequency identified in the teaching style scale conversion table (see Table 2). This composite score methodology to determine teaching time for curriculum content has been used in previous research. An advantage of using this method is to accurately capture the total instructional time dedicated to a specific curriculum content or to a specific teaching strategy employed the teacher (Mayer,1999; Mullens & Gayler,1999 ; Supovitz & Turner, 2000 ). The composite score was generated by using the units of days per school year for frequency and minutes per class period for duration or time; these numbers multiplied together to generate the final composite score. The researchers split the files; separating traditional and block scheduling results in order to accurately calculate a composite score. Splitting the file was necessary because the units of day and units of duration were different between the groups.

The lowest ranking categories based on composite scores for total instructional time were, Engineering and Human Values (6.21 hours for traditional schedule; 6.06 hours for block schedule), Engineering Science (7.06 hours for traditional schedule; 8.88 hours for block schedule), and Engineering Analysis (14.41 hours for traditional schedule; 14.16 hours for block schedule). See Figure 1.

Composite Score of Total Hours Dedicated to Engineering Design Categories
Figure 1. Composite Score of Total Hours Dedicated to Engineering Design Categories

These results reveal that there is less emphasis on the use of mathematics to predict design results and a low emphasis on optimization techniques, some might question if engineering design is being properly taught when these are key engineering design elements ( Hailey, Erekson, Becker, & Thompson, 2005 ; Hill, 2006 ; Gattie & Wicklein, 2007 ).

Limitation

In order to determine statistical significance for this population size N =1043, the Krejcie and Morgan (1970) method was to locate sample size for a given population size, the required sample size for the size of this population was set at 285 ( Gay & Airasin, 2000 ). Again, the survey was sent out to all secondary education ITEA members in order to increase the chances of achieving an appropriate response rate. The final results of the study yielded a total of 226 respondents; therefore, the results of this study cannot be generalized to the entire population. However, the researchers comparing the demographic data results from this research to similar national status of technology education research ( Gattie & Wicklein, 2007 ) that achieved an acceptable response rate level to generalize to the population. The demographic results of both studies were very similar, thus suggesting that these results were representative to the population. However, the researchers acknowledged that statistical significance was not achieved in this study.

Conclusion

The results of this descriptive study have yielded valuable information for the field of technology education. There has been a body of literature generated regarding the issues related to engineering design as a focus for technology education ( Daugherty, 2005 ; Gattie & Wicklein, 2007 ; Hailey, Erekson, Becker, & Thompson, 2005 ; Hill, 2006 ; Lewis, 2004 ; 2005; Wicklein, 2006 ). Several research studies in technology education have investigated the appropriate outcomes for a high school level engineering design program ( Childress & Rhodes, 2008 ; Smith, 2006 ). This study sought to extend the results of those prior studies by using those results to help describe the current status of technology education regarding the engineering design curriculum content. It is imperative for educational researchers in technology education to have the ability to identify where the field of technology education is, as a whole, regarding issues and needs related to an engineering design focus; this study sought to provide such information.

The evidence from this study provides rationale to conclude that technology education curriculum content currently emphasizes career and technical education skills such as CAD and general tool skills even though the field as a whole wants to assume a more general education focus. Leaders in the field of technology education should embrace these findings and use it as a way to define a clearer mission for the field of technology education, one that provides a career pathway to engineering. Technology education would be best served to embrace the idea that it can provide a logical career pathway for high school students and at the same time provide the universal skills of problem solving used in the engineering profession but which is also applicable to a variety of other important careers.

In recent years, some educators in technology education have endorsed the concept that technology education’s purpose is to foster technological literacy in all students. This purpose for technology education is a noble and worthy mission; however, an equally important mission is to prepare young people to become efficient workers in a global society while at the same time become technologically literate. The U.S. Department of Labor reported that a twenty percent increase in the demand for engineers would occur before the end of the decade, and currently many engineering jobs remain unfilled because of the lack of qualified candidates ( Southern Regional Education Board, 2001 ). Moreover, there are several commissioned reports that describe the job skills necessary for individuals to be prepared to work in a global economy ( Committee on Prospering in the Global Economy of the 21 Century, 2007 ; National Center on Education and the Economy, 2006 ). Within technology education, Dearing and Daugherty’s (2004) study identified the core engineering-related concepts that support a standards-based technology education curriculum. What emerged from the data were outcomes that are job related skills that are also essential skills outlined in global workforce literature. The top five ranked concepts identified were: 1) interpersonal skills: teamwork, group skills, attitude, work ethic; 2) ability to communicate ideas: verbally, physically, visually, etc. ; 3) working within constraints/ parameters; 4) experience in brainstorming and generating ideas; 5.) product design assessment: does a design perform its intended function? (p. 9).

Technology education with an engineering design focus can help equip students with necessary job skills while at the same time prepare students that are technologically literate.

Specific results of this study indicate that technology education is already providing some learning opportunities for high school students to develop necessary job related skills needed of workers in a global economy. The literary works of Friedman (2005) and Pink (2005) not only documented the changes taking place nationally and internationally regarding a global economy, but also describes some attributes of the new kind of problem solver needed to address the complex issues that will emerge from global workforce competition. Some of the highest mean score items in this study addressed these needs including thinking critically (highest mean score item measured by frequency) and worked on a design team as a functional inter-disciplinary unit. These attributes are necessary for a global worker, and according to the results of this research are well supported by current technology education curriculum content.

One particular area of improvement for technology education curriculum content to properly address the needs of a global workforce is the category of Engineering and Human Values (the lowest group mean scoring category by composite score). Some low mean scoring items within the Engineering and Human Values category are those outcomes related to making ethical decisions about engineering problems and outcomes that provide awareness of social, economical, and environmental impacts of technology on our society. The field of technology education would be better served by addressing these issues with improved curriculum content identified in the Engineering and Human Values category as well as implementing a systems thinking approach to problem solving in order to provide a way for students to learn how to address sustainability design issues.

One rationale for the importance of teaching technology education with an engineering design focus is that it can provide a real-world context for the application of mathematics and science ( Daugherty, 2005 ; Wicklein, 2006 ). However, the results of this study indicate that there is little emphasis on the application of mathematics and engineering sciences in current technology education curriculum. As mentioned earlier, a low mean score for time per typical use was the individual item using mathematical models to optimize, describe, and/or predict results (mean of 1.72). In the engineering science category, a low mean score result of 1.58 was determined for use of trigonometry to solve problems and predict results.

If educators within the field of technology education wish to advocate that technology education helps provide a real-world context for the application of mathematics and science, then technology education curriculum must provide more and deeper learning opportunities that include the use of mathematics and science as a part of the design process. However, the results of this study indicate that analysis and optimization stages of the engineering design process are not presently emphasized in technology education curriculum content, which might cause some to question if the engineering design process is being properly implemented. It is important to note that the debate is very much alive about what are the appropriate levels of mathematics and engineering science for teaching engineering design at the secondary level, more research is needed to determine the appropriate levels.

The researcher's desire that the results of this study will be used by those in the field of technology education to help design new engineering design curriculum, assessment strategies, and professional development experiences that will help high school technology educators successfully implement engineering design focused technology programs around the country.

Recommendations for Future Research

This research study has provided great insights into the current national status of technology education regarding engineering design curriculum content, assessment strategies, and challenges facing secondary teachers seeking to infuse engineering design into their classes. From this study, those in the field of technology education will better understand what is taking place in technology education classrooms regarding engineering design. However, more information is needed to help properly inform the field about this construct.

Consequently, the following recommendations are suggested for further research to inform the field of technology education:

  1. Similar descriptive research should be conducted using participants other than ITEA members to compare the results with this study. Moreover, a follow-up study using a different database could yield a larger sample size that would allow the researcher to statistically generalize to the entire population of technology education teachers. One possible database of technology education teachers that could be used for a follow-up study is the Engineering and Technology Education Division (eTED) of the Association for Career and Technical Education (ACTE).
  2. Conduct descriptive research using specific curriculum programs (Project Lead the Way, Probase, etc.) as the grouping variable to examine the student outcomes addressed as they relate to engineering design competencies. A study of this design could provide valuable information about outcomes and competencies achieved by these specific curriculum projects and about curriculum deficiencies.
  3. Conduct qualitative case studies of high school technology education teachers who have successfully implemented an engineering design focused technology education program in order to identify strategies necessary for infusing engineering design concepts into technology education. Furthermore, these types of studies could seek to explore the challenges and constraints facing these teachers as they implement a technology education program focused on engineering design.
  4. Conduct descriptive research using urban, suburban, and rural school settings as a grouping variable to determine if there exists a statistical difference in the challenges facing teachers seeking to infuse engineering design into technology education when grouped by school setting.
  5. Replicate this study using the same instrument and a sample of ITEA members five years in the future. A comparison of the results of this study and a study five years out could help identify the progress made with the infusion of engineering design in technology education curriculum content.
  6. Conduct qualitative and quantitative research to determine the levels of mathematics and engineering science that are appropriate for teaching engineering design at the secondary level in order to remain authentic to the engineering design process and remain manageable for technology education teachers.

Todd R. Kelley, Ph.D., is an Assistant Professor in the Department of Industrial Technology at Purdue University. He can be reached at trkelley@purdue.edu.
Robert C. Wicklein, Ed.D., is a Professor at the University of Georgia. He can be reached at wickone@uga.edu.

References

Akmal, T., Oaks, M.M., & Barker, R. (2002). The state of technology education: A national report on the state of the profession. Journal of Industrial Teacher Education, 39(4), 1-20.

Childress, V. W., & Rhodes, C. (2008). Engineering Outcomes for Grades 9 -12. The Technology Teacher. 67(7), 5-12.

Committee on Prospering in the Global Economy of the 21st Century (2007). Rising above the gathering storm: Energizing and employing America for a brighter economic future. National Academy of Science. Washington, DC: National Academies Press.

Clark, S.C. (1989). The industrial arts paradigm: Adjustment, replacement, or extinction? Journal of Technology Education, 1(1),1 -9.

Daugherty, M.K., (2005). A changing role for technology teacher education. Journal of Industrial Technology Education, 42(1), 41-58.

Dearing, B.M., & Daugherty, M.K. (2004). Delivering engineering content in technology education. The Technology Teacher, 64(3), 8-11.

Dillman, D.A., Tortora, R.D. & Bowker, D. (1999). Principles of constructing web surveys. Retrieved December 2, 2006, from http://survey.sesrc.wsu.edu/dillman/papers/websurveyppr.pdf

Douglas, J. Iversen, E. & Kalyandurg, C. (2004). Engineering in the K-12 classroom: An analysis of current practices and guidelines for the future. A production of the ASEE Engineering K12 Center.

Dugger, W. E., Miller, C. D., Bame, E. A. Pinder, C. A., Giles, M. B., Young, L.H., and Dixon, J. D. (1980). Report of the Survey Data. Blacksburg, VA: Virginia Polytechnic Institute and State University.

Eide, A.R., Jenison, RD., Marshaw, L.H., Northrup,L. (2001). Engineering fundamentals and problem solving (4th ed.) Boston: McGraw-Hill.

Friedman, T.L. (2005). The world is flat: A brief history of the twenty-first century. New York: Farra, Straus, and Giroux.

Gall, M.D., Gall, J.P., & Borg, W.R.(2007). Educational research: An introduction (8th ed.). Boston: Pearson Education, Inc.

Gattie, D.K. & Wicklein, R.C. (2007). Curricular value and instructional needs for infusing engineering design into K-12 technology education. Journal of Technology Education, 19(1), 6-18.

Gay, L.R. & Airasian, P. (2000) Educational research: Competencies for analysis and application (6th ed.) Columbus, OH: Merrill

Hailey, C. E., Erickson, T., Becker, K., & Thomas, T. (2005). National center for engineering and technology education. The Technology Teacher, 64(5), 23-26.

Hill, R.B. (2006). New perspectives: Technology teacher education and engineering design. Journal of Industrial Teacher Education, 43(3), 45-63.

International Technology Education Association. (2000/2002) Standards for technological literacy: Content for the study of technology. Reston, VA.

International Technology Education Association. (2003). Advancing excellence in technological literacy: Student assessment, professional development, and program standards. Reston, VA.

Krejcie, R.V., & Morgan, D.W. (1970). Determine sample size for research activities. Educational and Psychological Measurement, 30(3) 607-610.

Lewis, T. (2004). A turn to engineering: The continuing struggle of technology education for legitimatization as a school subject. Journal of Technology Education, 16(1), 21-39.

Messick, S. (1989). Validity. In R.L. Linn (Ed.) Educational measurement 3rd ed., pp 13-103. New York: Macmillan.

Mullens, J. E., & Gayler, K. (1999). Measuring classroom instructional processes: Using survey and case study fieldtest results to improve item construction. U. S. Department of Education. Office of Educational Research and Improvement. National Center for Education Statistics.

National Academy of Engineering and National Research Council. (2002). Technically speaking: Why all Americans need to know more about technology. Greg Pearson and A. Thomas Young, (Eds.), Committee on Technological Literacy. Washington, DC: National Academy Press (http://www.nae.edu/techlit).

National Center on Education and the Economy. (2006). Tough choices or tough times: The report on the new commission on the skills of the American workforce. Hoboken, NJ: Jossey-Bass

Pink, D. H. (2005). A whole new mind: moving from the information age to the conceptual age. New York: Riverhead Books.

Rogers, G.E. (2005). Pre-engineering’s place in technology education and its effect on technological literacy as perceived by technology education teachers. Journal of Industrial Technology Teacher, 42, (1), 6-22.

Sanders, M. (2001). New paradigm or old wine? The status of technology education practice in the United States. Journal of Technology Education, 12(2), 35-55.

Schmitt, M. L., & Pelley, A. L. (1966). Industrial arts education: A survey of programs, teachers, students, and curriculum. U. S. Department of Health, Education, and Welfare,. OE 33038, Circular No. 791. Washington, DC: Office of U.S. Government Printing Office.

Smith, P.C. (2006). Essential Aspects and Related Academic Concepts of an Engineering Design Curriculum in Secondary Technology Education. Unpublished doctoral dissertation, University of Georgia, Athens.

Southern Regional Education Board. (2001). High schools that work presents a pre-engineering program of study. Atlanta, GA: Author.

Supovitz, J.A., & Turner, H.M. (2000). The effects of professional development on science teachers practices and classroom culture. Journal of Research in Science Teaching. 37(9), 963-980.

Wicklein, R. C. (2006). Five good reason for engineering design as the focus for technology education. The Technology Teacher, 65(7), 25-29.

Warner, S.A.,& Morford, L.L. (2004). The status of design in technology teacher education in the United States. Journal of Technology Education, 15(2), 33-45.