Engineering Professional Development Design for Secondary School Teachers: A Multiple Case Study

Jenny Lynn Daugherty

The effectiveness of teachers has been regarded as crucial to the success of standards-based reform ( Fishman, Marx, Best, & Tal, 2003 ). Research, particularly within science and mathematics, has underscored the need for professional development to help teachers understand (a) subject matter, (b) learners and learning, and (c) teaching methods ( Loucks-Horsley, 1999 ). In addition to focusing on teacher professional development, national reform efforts have also emphasized science, technology, engineering, and mathematics (STEM) education (i.e., Rising Above the Gathering Storm , NRC, 2006 ). While substantial work has been conducted in mathematics and science, the efforts in technology and engineering education are much less mature. This makes sense given the relatively recent development of the Standards for Technological Literacy ( ITEA, 2000 ) and recent calls for integrating engineering into the K-12 classroom as both an avenue to technological literacy and as a way to enhance the engineering pipeline ( Erekson & Custer, 2008 ; Lewis, 2005 ; Wicklein, 2006 ).

The complexity of engineering and its integration into K-12 education, however, have resulted in a variety of issues requiring sustained empirical research ( Johnson, Burghardt, & Daugherty, 2008 ). One particular area of need given the emphasis on teacher effects on student learning is to research engineering-oriented teacher professional development. A lack of publication on the effective practices of engineering-specific professional development projects makes a study investigating mature efforts necessary. Thus, the purpose of this qualitative study was to explore professional development elements for secondary school engineering education. The research questions that guided this study were:

1. What are the primary design elements used to deliver engineering-oriented professional development (logistics, format, activities, instructors, and instructional strategies) and why were these elements selected?
2. How do the projects define and evaluate effectiveness?

The focus on the professional development design decisions and determinations of effectiveness for secondary school engineering education are particularly important because they are the elements that “designers of professional development have immediate control over and can modify in order to increase their impact on teachers’ knowledge, beliefs, and attitudes, and subsequent enactment” ( Fishman et al., 2003 , p. 646). Each design decision is typically connected to a distinct purpose and level of impact ( Speck & Knipe, 2005 ). By understanding the design decisions of specific projects, the connection to secondary school engineering and its impact on teaching and student learning can be better understood.

Review of the Literature

Teacher professional development has been conceptualized in various ways; from a systematic attempt to bring about change ( Guskey, 1986 ) to a continuous process ( Clement & Vandenberhe, 2000 ). Professional development can include practitioner-development, formal education, training, and informal support. Despite the different types, there is a growing demand for professional development that is more closely linked to the genuine demands and resources of teachers; that contains a greater coherence and link to curriculum policy; and that justifies the tremendous expenditures dedicated to it ( Evans, 2002 ; Shaha, Lewis, O’Donnell, & Brown, 2004 ). Researchers have estimated that professional development costs approximately $19 billion annually ( Bredeson, 2003 ).

A consensus has emerged concerning a set of principles and processes that differentiate effective teacher professional development (i.e., Darling-Hammond & McLaughlin, 1995 ; Garet, Porter, Desimone, Birman, & Yoon, 2001 ; Loucks-Horsley, Love, Stiles, Mundry, & Hewson, 2003 ). Guskey (2003) conducted an analysis of many of these lists and concluded that the most frequently cited characteristic of effective professional development was the enhancement of teachers’ content and pedagogical knowledge. In addition, many of the lists included research-based approaches; having a well-defined image of classroom learning and teaching; and continuous evaluation and improvement. Effective processes included collaborative participation; in-depth, active learning opportunities; and engaging teachers as adult learners and in leadership roles.

In addition to an emphasis on integrating these effective practices, the research on teacher professional development in science and mathematics has evolved into addressing specific teacher needs. Professional developers in science education have largely focused on the need for science teachers to increase their content knowledge and experience using inquiry in the classroom ( Johnson, 2006 ). Likewise, within mathematics education, professional developers are being called upon to develop teachers’ knowledge of the content and effective pedagogy, as well as “to provide opportunities for teachers to develop their own identities as teachers of mathematics” ( Peressini, Borko, Romagnano, Knuth, & Willis, 2004 , p. 67).

Technology teacher professional development, however, has been less well explored. Compton and Jones’ (1998) study of two technology education projects led them to conclude that there should be a focus on teachers’ conceptualizations of technology education, pedagogy, and technological practice. Bybee and Loucks-Horsley (2000) articulated four key components of technology teacher professional development. Technology teachers need: (a) to develop technology skills; (b) to learn about how to teach technology; (c) tools and motivation to continue their own learning; and (d) long-term professional development to support standards-based reform. With K-12 engineering education being a relatively new phenomenon, research specifically on engineering professional development is lacking. Although initiatives have emerged to assist teachers in this endeavor, little has been documented of their approaches or effects. It is critical to understand the professional development design decisions that lead to effective professional development experiences for teachers preparing to teach engineering.

Method

This study consisted of five case studies of projects designed to prepare secondary teachers to deliver engineering education. Multiple case studies allow comparative analysis so that similar cases can be compared and contrasted ( Stake, 2006 ). This research design was appropriate for this study because of the nature of the research questions. The focus was on describing the design decisions and practices involved in the professional development of teachers for secondary school engineering education. By coming to know each project through an in-depth analysis, this study was able to answer the research questions, as well as draw significant comparisons across the cases and against the research literature.

A discriminate sampling technique, where the researcher deliberately selects persons, sites, and documents to maximize comparative analysis ( Strauss & Corbin, 1998 ), was used to select the five cases for analysis. Based on the review of literature and research questions, the following criteria were developed to help guide the case study selection process.

1. Engineering-Oriented Content : The cases had to contain elements that were interesting, applicable, and useful for engineering-oriented professional development at the 9-12 (secondary school) level. Engineering was defined as including a focus on: (a) preparing students for postsecondary engineering education or (b) providing a broad base of technological literacy for all students. A focus on secondary school level projects was included because of the predominance of initiatives targeting this grade level.
2. Illuminative Professional Development Design Practices : The initiatives needed to have a reputation for attempting to include “best practices” (e.g., standards based, pedagogically sound, and assessment based), as well as creative design practices, that could illuminate and inform future professional development in this area.
3. Maturity : Priority was given to mature initiatives with an established track record for delivering professional development over a sustained period of time (at least two years).

In order to identify the cases, the researcher asked acquaintances, who were actively involved in technology and engineering education, to identify individuals who had national reputations in K-12 engineering education and who would be knowledgeable about teacher professional development projects. Interviews with 15 of these individuals were conducted to assist the researcher in identifying projects. The individuals were asked to identify projects and rank the top three sites that best fit all of the criteria. It was assumed that the process for identifying and selecting the sites was appropriate given the lack of publication and the limited advertisement of engineering-oriented professional development projects.

The identified projects were rated based on the number of times mentioned and by the rankings provided. A total of five projects were selected for inclusion in the study because they were ranked highly by multiple informants. Five cases were deemed sufficient enough to be able to analyze different approaches to engineering-oriented professional development and allow for in-depth comparisons across projects without being too cumbersome. The cases selected for inclusion into this study were Engineering the Future: Science, Technology, and the Design Process™, Project Lead the Way™, Mathematics Across the Middle School MST Curriculum, The Infinity Project ^sm , and INSPIRES .

The data collection process for each case study consisted of the following phases: (a) pre-visit, (b) on-site, and (c) post-visit. The pre-visit data collection phase consisted of two elements: (a) structured telephone interviews with the project’s leaders, and (b) an analysis of the project’s documents. The structured hour-long telephone interviews with the project’s leaders were conducted to collect factual data about the project to help provide the “back story” and inform the on-site data collection. The project leaders were also asked to supply evaluator reports, curriculum, and related documentation of the project. These documents were reviewed to better understand the project’s development, philosophy, and approach to professional development. The data gathered from the interviews and documents were synthesized and developed into the foundation of the case study report prior to the on-site visit.

The on-site data collection was conducted over the span of two days. The rationale for conducting on-site visits was to (a) obtain first-hand reports from the projects’ participants, (b) directly observe the professional development activities and interact with project leaders and participants, and (c) document and validate information obtained from the pre-site interviews. In order to ensure the triangulation of data, the on-site data collection for this study consisted of the following three methods: (a) observations conducted during the Summer of 2008, (b) teacher questionnaires, and (c) interviews (teacher focus groups, instructors, and project leadership).

The first on-site day consisted primarily of observations guided by an observation form. The researcher and a co-observer independently documented the day’s activities with field notes and compared these notes at the close of each day. Eisenhardt (1989) outlined two key advantages for the use of multiple investigators: (a) it adds to the richness of the data, and (b) the task of converging observations enhances the confidence in the findings. At the end of the first day, a survey questionnaire was also administered to all of the teachers. The questionnaire was developed based on the need to better understand the teacher participants’ demographic characteristics, their motivations to attend, and what they had learned. The same questionnaire was administered at all of the sites, providing data for comparison across the cases.

On the second day, focus group interviews of the teachers, interviews with the professional development instructors, and follow-up interviews with the project leadership were conducted. When possible, the focus groups were comprised of existing small groups of teachers. All of the teachers were asked to be in a focus group, with all but a few electing to participate. Teachers were asked about what they were learning, how it would influence their teaching, and strengths and weaknesses of the experience. The interviews with the instructors were intended to provide information about the materials, the delivery of the instruction, and their training. By the end of the second day, if unanswered questions remained, informal interviews of the project’s leadership occurred. All interviews were audio-recorded and transcribed by a professional transcriptionist. The background report prepared from the pre-visit data, the observation notes, and the analysis of the transcripts were compiled into case study write-ups.

During the post-visit data collection phase, member checking was conducted to ensure the accuracy of the case study write-ups. The project leaders were asked to examine their project’s case study report and provide feedback on any inaccuracies related to the project’s history and development. Inaccuracies were reconciled via a telephone conversation. Afterward, full descriptive case studies were prepared for each case. The background report, the analysis of the transcripts, and a descriptive narrative of each on-site visit were integrated into separate case studies. This approach allowed the researcher to gain a rich familiarity with each case, resulting in the emergence of unique patterns within each case before pushing “to generalize patterns across cases” ( Eisenhardt, 1989 , p. 540).

About the Cases

The five case studies are presented in an abbreviated form in the order they were visited. The findings from the individual case studies are then compared and summarized across the research questions. As Stake (2006) pointed out, what multiple case studies “have most to offer is a collection of situated case activities in a binding of larger research questions” (p. 90).

Engineering the Future: Science, Technology, and the Design Process™ (EtF)

The National Center for Technological Literacy (NCTL) at the Museum of Science, Boston began the EtF project to develop a full-year course designed for all students in their first years of high school. Professional development emerged from the field testing of the curriculum. Currently the professional development is comprised of two designs: (a) in-person workshops and (b) online courses. The in-person workshops are structured around a combination of mini-lectures, hands-on activities, and reflections. Each of the four days of the workshop observed for this case study was devoted to one of the four projects in the course. The instructor structured the professional development experience around the five E’s (engage, explore, explain, elaborate, and evaluate). The evaluation included daily plus-delta activities and a summative feedback form. In addition, online courses enable the project to introduce, train, and support teachers using the curriculum nationally.

Project Leader the Way™ (PLTW)

PLTW is an instructional project that is designed to prepare students to be successful in post secondary engineering and engineering technology programs. There are three elements of PLTW’s professional development design: (a) self-assessment and pre-core training, (b) core training in the form of summer training institutes, and (c) continuous training. Teachers take a skills self-test and questionnaire to determine their readiness for core training in three basic areas: (a) mathematics, (b) science, and (c) computer literacy. The two week (80 hours) Summer Training Institutes (STIs) are conducted at an affiliate training center; typically a university. There are STIs for each of the PLTW’s courses within the middle school (Gateway to Technology) and high school programs (Pathway to Engineering). Master teachers and affiliate university professors lead the STIs. The master teachers assist in developing the “scope and sequence” of the workshop that will be used at all STIs across the country. Continuous training is provided to the teachers in the form of university based level II training and a virtual academy.

Mathematics Across the Middle School MST Curriculum Project (MSTP)

MSTP is a National Science Foundation (NSF) Mathematics Science Partnership (MSP) project. The primary focus of the project is mathematics infusion into technology education classrooms through engineering design problems. There have been three distinct phases of the MSTP project’s approach to professional development. The first phase utilized a train-the-trainer approach. The second phase had teachers meeting twice (A workshop and B workshop), and between implementing a mathematics-infused lesson, bringing examples of student’s work to the second workshop. The third phase, which was observed for this case study, was to result in an experimental control group research study designed to measure the impact of a mathematics-infused design lesson. The workshop, facilitated by a lead teacher and supported by a mathematics education expert, provided the teachers with the experience of working through the lesson, which also emphasized virtual and physical modeling.

The Infinity Project ^sm

The Infinity Project ^sm is a partnership between Southern Methodist University (SMU) and Texas Instruments that resulted in a year-long, upper level high school course titled Engineering Our Digital Future , and an adapted version for 9 ^th and 10 ^th grades. The instructional materials include a textbook, lab materials, an instructor’s guide, and a technology kit utilizing LabVIEW software. Classroom support is provided through the project’s professional development institutes, which are week long (40 hour) sessions hosted by SMU or other university partners. Institutes include hands-on instruction by master instructors in the use of the hardware, software, and textbook features of the curriculum. Open lab time was built into the format of the institute, which was structured around the textbook’s chapters. The primary focus of the institute is on learning how to use the LabVIEW software. The evaluation component of the institute included a pre-test/post-test assessment. Teachers are asked to complete the assessment before they attend the institute and then at the end of the institute they are asked to complete it again.

INSPIRES

INSPIRES is an NSF-funded project with the purpose of “Increasing Student Participation, Interest, and Recruitment in Engineering and Science.” The INSPIRES curriculum targets core engineering skills and concepts in order to better prepare students to pursue engineering and technology related careers. At the time of the on-site visit for this case study, the INSPIRES project had completed three of its five stand-alone modules, which are centered on specific engineering design challenges. As they completed a module, the project’s leaders conducted two-day teacher workshops. The observations conducted for this case study were completed at a workshop focused on the Engineering Energy Solutions: A Renewable Energy System Case Study module. The workshop consisted of an overview of the project and then experiencing the curriculum in the same order and format that it is to be implemented in the classroom. The teachers also work through the web-based tutorials and interactive simulations that are included in the module. The workshop begins and ends with an evaluation survey.

Cross-Case Analysis

In order to address the study’s research questions, the complete case studies were synthesized by conducting a cross-case search for patterns of design elements and determinations of effectiveness. It was assumed that the triangulation of data, validation measures, and the member checking process were appropriate to generate accurate and valid case studies from which to address the study’s research questions.

Research Question 1

The first research question was focused on the primary design elements used to deliver engineering oriented professional development and the reasons these design elements were selected. The relevant categories that emerged as a result of the cross-case analysis included: philosophy towards engineering, format in number of days, the online component, teacher recruitment, design model, instructional design, and instructors. Table 1 provides a side by side comparison of each project across these design elements.

The five projects involved in this research study had philosophies guiding their approach to engineering-oriented education at the secondary school level, which impacted their design decisions. The philosophy of EtF and MSTP was oriented toward engineering as an avenue toward technological literacy for all students. For example, the EtF course is designed for students, whether college-bound, whether they plan to attend a tertiary education institution, or enter the workforce directly. Although there were elements within these projects of increasing all students’ awareness and interest, the philosophy of PLTW, The Infinity Project ^sm , and INSPIRES , was oriented more toward developing students’ aptitudes toward pursuing post-secondary engineering. For example, The Infinity Project ^sm is advertised as an “early college engineering education project.”

The length of the in-person aspects of the professional development differed among the projects, including two and four day; one and two week formats. In addition to in-person workshops, all of the projects included an online component from courses to blogs. The online component of most of the projects was designed to provide additional follow-up support to the teachers after they had attended the in-person workshop. Teacher recruitment, another important design decision, differed among the projects. EtF, MSTP, and INSPIRES sent direct mailings marketing their workshops to area schools so teachers could self select. PLTW and Infinity required an agreement to be completed by the school district administrator, who identified the teachers to attend the professional development. Table 2 summarizes the characteristics of the teachers who attended the workshops and completed the survey across two dimensions: (a) subjects taught and (b) gender. Across the five projects, the majority of the teachers were male (71%) and taught technology education, industrial technology, pre-engineering, or computer science subjects ( n = 47).

All but one of the five projects in this study pursued curriculum-linked instructional design models, focusing on the knowledge, skills, and abilities deemed necessary to implement a specific set of curriculum materials. For example, these projects devoted significant amounts of time to providing training on specific software or tools used in the curriculum. However, there were different decisions made concerning how much of the curriculum to cover. For example, the EtF project devoted each day to a module, covering all of the modules in the curriculum. On the other hand, INSPIRES devoted an entire workshop to just one of its modules. PLTW and The Infinity Project designed their workshops around having the teachers experience the entire scope of a course. MSTP was the only project not based on a set of specific curriculum materials but did focus on the implementation of a specific lesson.

In addition, there were two patterns of instructional design that emerged: (a) scaffolded problem solving and (b) self-guided learning. EtF, PLTW, and MSTP’s approach was to scaffold problem solving activities on top of developing skills and knowledge related to the hands-on activities. For example, PLTW’s overall approach was to provide instruction on specific tools using a demonstration and lecture, and then teachers would move on to more open-ended design problems after the “basics” were learned. The other instructional design pattern observed could be categorized as self-guided learning. Teachers were introduced to the content of the curriculum and then given time to work through the activities at their own pace. For example, The Infinity Project ^sm instructor briefly reviewed PowerPoint presentations and then the majority of time was spent on computers, completing the labs within the curriculum at the teachers’ own pace.

An important decision related to instructional design is the selection and preparation of the instructors used to deliver the professional development. There were three types of instructors, with some projects using a combination: (a) master teachers, (b) project leaders, and (c) higher education faculty. PLTW, MSTP, and Infinity had master teachers, who had implemented the curriculum for an extended period of time, deliver the professional development instruction. The project leaders for EtF, INSPIRES , as well as MSTP, served as instructors. In addition, engineering faculty served as instructors on PLTW and INSPIRES . MSTP also included a mathematics consultant as part of its team of instructors.

Research Question 2

The second research question was oriented toward how projects defined and evaluated effectiveness. All of the projects included a summative evaluation by distributing surveys to the teachers, asking feedback about the delivery of the workshop. PLTW, Infinity , and INSPIRES administered surveys to the teachers prior to and at the conclusion of the workshop. All of the projects incorporated formative evaluations into their format, though it was obtained mostly informally through discussions. A formal process was pursued by EtF with the daily completion of plus/delta comment cards. All of the projects created online environments to provide a venue for teacher support during implementation. In addition, PLTW, Infinity , and MSTP had a formal plan in place to follow up with the teachers during the school year.

Despite these measures, the projects did not articulate or had not completed comprehensive evaluation plans that accounted for multiple stakeholders and that carried through to implementation in the classroom to measure impacts on student learning. The primary focus was on the teachers’ perceptions of the experience and their ability to train the teachers to implement the curriculum as intended. For example, all of the projects designed their professional development approach around teachers experiencing aspects of the curriculum or lesson, as well as learning specific tools, to improve implementation. This contributed to the project’s ability to evaluate the effectiveness of their project by maximizing the likelihood that teachers would implement as intended.

The teachers across the five projects, who participated in the focus groups, largely agreed on three aspects that contributed to effective professional development experiences: (a) hands-on activities, (b) teacher collaboration, and (c) instructor credibility. All of the projects devoted a majority of their time to hands-on activities. This was appreciated by the teachers when asked about what was particularly effective about the workshops. In addition, the hands-on activities allowed the teachers to work together. The ability to collaborate with other teachers both at the in-person workshops and via the online environments was consistently commented on as effective aspects of the professional development. Many of the teachers also commented on the credibility of the instructors, both the master teachers and engineers, as effective elements.

Discussion

Based on the five case studies, consistent decisions concerning design elements emerged, which were linked primarily to curriculum implementation. The assumption being that that “good” curriculum translates into “good” professional development and “good” teaching. Although this focus is one of the oldest professional development strategies, it has been criticized as a “deskilling” process in that teachers are not developed beyond the curriculum. As Ball and Cohen (1996) argued, the “adoption of new materials is rarely seen as one component of a systemic approach to professional development” (p. 7). With little to no extensions of learning beyond the curriculum, the transfer of training to other aspects of teaching is assumed to be low. What do teachers learn and can implement into their particular community of practice beyond, or in addition to, the curriculum?

In terms of effective professional development practices, across the projects there was an emphasis on active engagement and collaborative learning. This focus aligns with the literature, which points to the need for adults to be actively engaged, as well as for teachers to develop a sense of collegiality and collaboration ( Gordon, 2004 ; Guskey, 2003 ). The research literature, however, indicates the need for the design of more comprehensive experiences for teachers, with an emphasis on what happens before an in-service training event and afterwards ( Craft, 2000 ). Comprehensive experiences include a focus on content and how students learn that content, meet for an extended duration of time, and include teachers as partners in its design and implementation.

Engineering at the secondary school level is a new and emerging phenomenon. As is apparent in this study, there are different ways to approach its inclusion into the high school classroom. The projects articulated two different philosophies: (a) technological literacy and (b) pre-engineering, which greatly impacted the professional development design. Those projects that aligned with technological literacy indicated that the emphasis was on developing critical thinking and problem solving capabilities in all students. Engineering was seen as one avenue to help accomplish this goal, with little connection to the engineering discipline or engineering-specific content. A pre-engineering philosophy was also evident, with strong connections to postsecondary engineering, designed to encourage students into the “pipeline.”

These two distinct philosophies are important because it gets to the heart of what is meant by engineering at the secondary school level and has implications for how teachers should best be prepared. How engineering is conceived impacts the design of curriculum, instruction, teacher preparation, and professional development. For example, the instructional design decisions made by the projects, whether to scaffold learning or provide self-guided learning experiences, appear to be connected to these different approaches to engineering. The pre-engineering projects mirrored post-secondary engineering education approaches, emphasizing self-guided learning. Technological literacy projects mirrored K-12 technology education pedagogy, providing scaffolds to learn tools and knowledge to complete hands-on activities. Research needs to be conducted to better understand how teachers and students best learn engineering so as to effectively design the professional development instruction.

In addition, the philosophy of engineering may impact where in the secondary school curriculum engineering is best suited. The engineering projects explored in this study attracted science, technology, and mathematics teachers. Due to the discrepancies in their pre-service teacher education, teachers’ capabilities vary across and within these three disciplines; for example, in their mathematics abilities and skills. However, the professional development projects in this study lacked any overt attention to these discrepancies and focused little on reflecting on engineering related content, skills, or abilities. If pre-college engineering moves toward an engineering content focus, professional development would need to face the challenge of meeting the needs of teachers with varying levels of science, technology, engineering, and mathematics backgrounds.

Implications and Recommendations

Based on the findings of this study, there are important implications for secondary level engineering professional development. Even when anchored on curriculum, the professional development design should include more comprehensive needs assessments, evaluation, and follow-up. Projects should incorporate rigorous evaluation into the design of their professional development so that they can provide a better understanding of how teachers learn engineering, change, and impact student learning. Secondary level engineering-oriented professional development should also move toward more comprehensive designs to account for the minimal teacher preparation in engineering at the pre-service level. A clear vision of teaching and learning engineering needs to drive the design of the professional development. Teachers’ needs, whether mathematics, science, technology, or a combination, should inform the design and should be continuously monitored. The design should be a collaborative venture between professional development providers and the teachers so as to account for the particular contexts within which the teachers operate. This process should include key stakeholders such as school administrators, guidance counselors, and parents.

In terms of recommendations for research, a study of engineering-oriented professional development projects that are not curriculum-based and inclusive of the entire K-12 spectrum is warranted. Another recommendation is to study the link between teacher participation in engineering professional development and student learning outcomes. As Fishman et al. (2003) pointed out, to “create excellent projects of professional development, it is necessary to build an empirical knowledge base that links different forms of professional development to both teacher and student learning outcomes” (p. 643). This link has not been thoroughly explored and with increasing calls for the integration of engineering, it is important that this be emphasized in future research.

Jenny Daugherty ( jldaughe@purdue.edu ) is an Assistant Professor in the Organizational Leadership & Supervision Department at Purdue University, West Lafayette, Indiana.

References