Formulating a Concept Base for Secondary Level Engineering: A Review and Synthesis

Rodney L. Custer, Jenny L. Daugherty, and Joseph P. Meyer

Introduction

In recent years, there has been growing interest in science, technology, engineering, and mathematics (STEM) education across the K-12 spectrum (e.g., Borgman, Abelson, Dirks, Johnson, Koedinger, Linn, Lynch, Oblinger, Pea, Salen, Smith, & Szalay, 2008 ; National Commission on Mathematics and Science Teaching for the 21st Century, 2000 ; National Mathematics Advisory Panel, 2008 ; National Research Council, 2006 ). In part, this interest has been triggered by a “growing concern that the United States is not preparing a sufficient number of students, teachers, and practitioners in the areas of science, technology, engineering, and mathematics” ( Kuenzi, 2008 ). While much of the focus on STEM has concentrated on science and mathematics, engineering and technology are emerging as disciplines in their own right at the K-12 level ( Coppola & Malyn-Smith, 2006 ). A significant part of this emphasis on engineering and technology can be attributed to a concern that insufficient numbers of students are being attracted into and prepared for post-secondary engineering education ( Brophy, Klein, Portsmore, & Rogers, 2008 ). There is also a growing awareness that an engineering presence within the K-12 curriculum provides an authentic contextual base for mathematics and science concepts ( Daugherty, Reese, & Merrill, in press ; Lewis, 2005 ; Wicklein, 2006 ).

One large scale initiative focused on pre-college engineering is the National Center for Engineering and Technology Education (NCETE) funded through the National Science Foundations’ (NSF) Centers for Learning and Teaching program ( Hailey, Erekson, Becker, & Thomas, 2005 ). One key problem that emerged from a multiple case study project of engineering teacher professional development funded by NCETE was the lack of a well-defined conceptual base for K-12 engineering ( Daugherty, 2009 ). The development of meaningful learning, teaching, and assessment is problematic in the absence of a clear understanding of the conceptual base of the subject matter—in this case K-12 engineering ( Bransford, Brown, & Cocking, 2000 ). Given the current ambiguity of the conceptual base of secondary engineering, and the need for conceptual clarity in curricula, professional development, and research, this study was designed to coalesce a body of engineering concepts for the secondary level.

Purpose of the Study

The purpose of the study was to identify and refine a conceptual base for secondary level engineering education. Specifically, this study addressed the following research questions:

1. What engineering concepts are present in the pertinent literature including: philosophy of engineering; secondary level science, technology, engineering, and mathematics standards; secondary level engineering-oriented curriculum; and the related research literature?
2. What engineering concepts are deemed core for secondary level education by practicing engineers and engineering educators?

Literature Review

Many have targeted the engineering design process as the avenue for integration ( Lewis, 2005 ; Wicklein, 2006 ). The implementation of engineering design into technology education has largely centered on process through a step-by-step approach ( Hill & Anning, 2001 ). This approach however has been increasingly criticized because it contradicts both expert and novice designers’ approaches to the problem solving and design process ( Lewis, Petrina, & Hill, 1998 ; Mawson, 2003 ; Welch, 1999 ; Williams, 2000 ). In addition, a focus on process may not lead to conceptual learning ( Eisenhart, Borko, Underhill, Brown, Jones, & Agard, 1993 ; Rittle-Johnson, & Alibali, 1999 ; Rittle-Johnson, Siegler, & Alibali, 2001 ). As Antony (1996) argued, teachers “may be lulled into a false sense of security by providing students with numerous investigations, open-ended problem-solving experiences, and hands on activities with the expectations that students are successfully constructing knowledge from these experiences” (p. 351).

The lack of a defined conceptual base is a concern. As Erickson (2002) argued, attempting to “teach in the 21st century without a conceptual schema for knowledge is like trying to build a house without a blueprint” (p. 7). Conceptual knowledge is essential for learning as it requires understanding the operational structure of something and how it relates to associated concepts. Conceptual knowledge can be “thought of as a connected web of knowledge, a network in which the linking relationships are as prominent as the discrete pieces of information” ( Hiebert & Lefevre, 1986 , p. 3-4). Concepts are organizing ideas that are timeless, universal, abstract and broad, represented by one or two words, and examples of which share common attributes ( Erickson, 2002 ; Tennyson & Cocchiarella, 1986 ).

There have been several studies, largely utilizing a modified Delphi and/or survey approach, in the past few years that have sought to identify K-12 engineering outcomes ( Childress & Rhodes, 2008 ; Childress & Sanders, 2007 ; Dearing & Daugherty, 2004 ; Hacker, de Vries, & Rossouw, 2009 ; Harris & Rogers, 2008 ). As Katehi, Pearson, and Feder (2009) pointed out, there are common concepts that appear on most of these lists including systems, modeling (representational and mathematical), predictive analysis, specifications, constraints, optimization, and trade-offs. However, these studies focused on something other than articulating the concept base for engineering at the secondary level (i.e., engineering outcomes, dispositions, skills) through a process of consensus. This study aims to identify the conceptual base particular to engineering education at the secondary level by consulting multiple sources including philosophy, curriculum, standards, and experts.

Method

Operating under an emergent qualitative research design, an adaptive approach to data collection was utilized ( Schwandt, 2001 ). This type of emergent strategy is characteristic of situations where researchers are attempting to extract and interpret meanings from within a larger context and where strategies are needed to retain an emergent quality ( Patton, 1990 ). In addition, multiple methods were utilized to achieve triangulation and “secure an in-depth understanding of the phenomenon in question” ( Denzin &, Lincoln, 2005 , p. 5) (i.e., engineering concepts). The initial data collection plan included a review of secondary level engineering curriculum materials and STEM curriculum standards, as well as focus groups of engineering education experts. After a review of the conceptual learning literature and consistent with emergent qualitative designs, the research team realized that a more in-depth understanding of conceptual literature was necessary. To address this concern, an in-depth review of the engineering and technology philosophy literature was added to the methodology in order to help fully define the domain. The decision to include literature from both engineering and technology was made due to the substantial conceptual overlap in the philosophical and historical literature. For example, a review of work published in a variety of sources including Technology and Culture (Society for the History of Technology) and Techné (Society for Philosophy and Technology) includes substantial treatment of both technology and engineering, both for illustrative and analytical purposes.

Ultimately, four sets of documents were reviewed and three focus groups were conducted. The documents, in the order they were reviewed, included: (a) engineering and technology philosophy writings, (b) curriculum materials focused on secondary level engineering, (c) curriculum standards documents developed for the STEM disciplines and National Academy of Engineering reports, and (d) survey research studies relevant to K-12 engineering. Following the compilation and analysis of the focus group and document review data, a peer debriefing ( Schwandt, 2001 ) with engineering and technology education experts was convened to review and discuss the study’s methods and outcomes. Extant Document Review

The goal of the document review was to systematically identify and review key documents to identify core engineering concepts (see Table 1). The selection of documents for analysis varied depending on type. The philosophy documents were selected based on the work of one of the researchers whose doctoral dissertation included a thorough treatment of engineering and technology philosophy literature. That study included a systematic document selection process, which included nominations, discussion, and, ultimately, a vote by a national panel of experts ( Custer, 1991 , 1995 ).

Curriculum materials were drawn from those identified in the K-12 engineering study conducted by the National Academy of Engineering ( Katehi, Pearson, & Feder, 2009 ). For the purposes of this study, with guidance from the NAE project curriculum analysis consultant (Dr. Kenneth Welty), only those units within the high school curricula that were directly related to engineering were reviewed. The standards documents included in the study were those developed by the professional organizations representing the STEM disciplines. The research studies were identified through electronic database searches based on their relevance to secondary level engineering.

A standard process was developed and used to review each set of documents. Two of the three researchers, alternating the pair of researchers, independently reviewed each set of documents and identified “engineering themes” in the narrative. To ensure adequate coverage, each document was reviewed by two of the three researchers. Engineering themes were those elements in the narrative that were described as important to engineering and applicable across various engineering disciplines, as informed by the philosophy of engineering and technology literature. At this stage, the decision was made to be inclusive, retaining themes that would later be analyzed and refined through a systematic, analytical procedure employed by the research team.

Criteria were used to evaluate each theme according to an agreed upon understanding of how it met definitions of core, engineering, and concept. From the list of engineering themes, all three researchers independently identified what they considered to be core engineering concepts using the following specified definitional criteria:

• Engineering: defined by the Accreditation Board for Engineering and Technology (ABET) as the knowledge of the mathematical and natural sciences—gained by study, experience, and practice—, is applied with judgment to develop ways to use, economically, the materials and forces for the benefit of mankind ( Gomez, Oakes, & Leone, 2006 ).
• Concepts: Abstract labels ( Erickson, 2002 ), organizing ideas ( Hiebert & Lefevre, 1986 ), typically represented with one or two words ( Sigel, 1983 ), and take on meaning in the knowledge-rich contexts in which they are applied ( Tennyson & Cocchiarella, 1986 ).
• Core: The center of an object, a small group of indispensable things, and the most essential or most vital part of some idea or experience ( Wordnet, 2009 ).

The research team applied the criteria to all of the themes that emerged from the analysis. The criteria were applied individually in the order presented above. If a theme “failed” to meet any of the criteria, it was eliminated from consideration. In order to be included in the listing of core engineering concepts, the theme was required to meet all three criteria by all three researchers on a consensus basis.

With the “engineering” criterion, the focus was on whether the theme focused specifically on the study, expertise, and practice specific to engineering. With the “concepts” criterion, the team’s deliberations concentrated on the perceived robustness and complexity of the ideas and the extent to which they could be “unpacked,” as well as the extent to which they extended well beyond processes and procedures. The “core” criterion focused on the extent to which the ideas were perceived to be essential to engineering as well as their appropriateness to secondary level education.

To the extent possible, the review identified concepts distinct from the more procedural aspects and interpersonal dispositions of engineering. Procedural items consist of those where the primary emphasis is on the more technical aspects of accomplishing an engineering design. For example, a set of heuristics or technical stages or steps used to optimize a particular design was excluded from the study due to its lack of conceptual robustness. Similarly, while social/interpersonal dispositions such as communication skills, teamwork, and time management skills are central to engineering, they focus more on the attributes needed to succeed in engineering rather than on the discipline’s core ideas.

Focus Groups

In addition to the thorough document review, the researchers conducted three focus group sessions with a total of 21 engineering educators and practicing engineers from selected departments of engineering and local engineering firms. These individuals had a recognized interest in and expertise with the broader, conceptual aspects of engineering. One focus group session was conducted at Colorado State University and two at Virginia Tech University. A point person at each of the universities, both of which are actively engaged in secondary level engineering education, identified participants based on guidance from the researchers. The point persons had been engaged in research and professional activities associated with engineering education and were well-equipped to select participants based on the study’s selection criteria.

The purpose of these sessions was to capture participants’ thinking about engineering concepts. The sessions were facilitated using an affinity group process. Participants were provided with an overview of the three criteria used to define core engineering concepts and then asked to brainstorm and record concepts onto sticky notes. As a group, the participants then clustered the concepts into categories and named each of the categories on a consensus basis. They were then asked to classify the categories into three columns: (a) those core to engineering, (b) those very much on the fringe, and (c) those undecided or somewhere between core to and on the fringe of engineering. Finally, the participants were asked to conduct one final review of their lists against the three selection criteria that were used for the study. This process generated a set of core engineering concepts from the perspective of practicing engineers and engineering educators.

Peer Debriefing

The culminating activity of the study consisted of a peer debriefing conducted by a panel of six engineering and technology education experts for a half-day discussion. The purposes of the activity were (a) to review the process used to conduct the study and (b) to discuss the findings. Peer debriefings allow qualitative researchers the opportunity to confide in “trusted and knowledgeable colleagues and uses them as a sounding board for one or more purposes” ( Schwandt, 2001 , p. 188). For the purposes of this study, colleagues were selected based on the researchers’ views of their recognized ability to think conceptually, knowledge of secondary level education, and understanding of engineering education.

Findings

The synthesis of the 5 major analyses yielded over 100 themes judged by the research team to be pertinent to engineering. The themes consisted of ideas, terms, and constructs that were judged by the researchers to be important to engineering. As noted earlier, the approach during this phase of the analysis was to be broad and inclusive. The next step of the refinement process consisted of subjecting the set of themes to the three criteria that were established for the process—that the themes were “core," “engineering,” and “conceptual." Each member of the research team independently applied the three criteria central to the analysis to each of the themes. Subsequent to these individual analyses, the team engaged in extensive discussions to achieve consensus until a composite listing of concepts, across all five inputs, was compiled. In those cases where consensus was not achieved, the item was not included in the listing of core engineering concepts.

Table 2 depicts the set of thirteen concepts that were generated through this process. In addition to the list of concepts, column two contains a set of descriptive terms associated with each concept. These terms were drawn directly from the document sources and were used to define, clarify, or illustrate the concepts. The remaining columns provide an indication of where the concept was located within the five sources of input. Careful records were maintained to track the sources of themes and concepts derived from all five sources throughout the analysis, which provided the documentation needed for the information presented in the “sources of input” columns in Table 2.

The listing of concepts presented in column one of Table 2 represents a distillation of over 100 themes. A substantial number of themes were deemed to have met the “core” and “engineering” criteria, but not the “conceptual” criterion. While these are important ideas, the goal of this study was to carefully identify ideas judged to be conceptually robust. Examples of items classified as non-conceptual included technical research, refinement, testing, and reverse engineering. Of those that met all three criteria, remarkable conceptual consistency was observed across the study’s five major inputs. Ten of the thirteen concepts were represented in all five inputs and two additional concepts were represented in four of the inputs. It is also clear that considerable conceptual overlap exists among the concepts. For example, many of the concepts represent aspects of the engineering design process.

A brief comment should also be made about the items presented in the “Terms” column. Prior to applying the three criteria, the approach was to be inclusive, identifying and retaining a broad range of ideas generated through the process. As the three criteria were applied to the ideas, the terms associated with those concepts were tracked and retained in order to maintain fidelity. The decision was made to include a representative sample of the terms associated with the core engineering concepts to provide a broader contextual perspective on the analysis. However, due to the nature of the analysis, the representative terms are not intended to be conceptually homogeneous. Some terms are essentially synonyms and descriptions, while others represent classifications or types.

Although some of the items on the list are phrased as verbs (i.e., prototyping) or represent identifiable processes (i.e., design), the researchers concluded that these ideas represent a depth of understanding beyond procedural knowledge. The list, irrespective of phrasing, contains ideas that can be generalized from particular instances (i.e., concepts) ( Rittle-Johnson & Koedinger, 2009 ) to the broader context of engineering. Using an example from mathematics, there is a procedural element to subtracting, as well as a conceptual component of understanding subtraction (e.g., what these functions mean within larger contexts, as well as within specific instances). A conceptual understanding is needed to situate ideas within the larger context and certainly extends beyond knowledge of procedures or processes. Instead of following steps, individuals understand what is occurring during and as a result of those steps.

Discussion

The outcomes of the study consisted of much more than a list of core engineering concepts and are thus worthy of discussion to shed light both on the researchers’ method and in terms of implications. Although not an original purpose of the study, the process used to identify the concepts raised a number of questions and issues important for secondary level engineering education that the researchers felt necessitated discussion. In addition, the peer debriefing participants were asked to reflect on these issues as they were deemed by researchers as being just as important as the list of concepts.

Problematic Concepts

The research team struggled with two particular themes: (a) problem solving and (b) experimentation. After considerable discussion, consensus was achieved to include experimentation as an engineering concept. The team was, however, unable to achieve consensus on problem solving, even though it emerged as a substantial theme across the five data sets. Engineering activities, such as the clarification of design parameters relative to design constraints, involve solving problems. Thus, at a practical implementation level, a compelling case was made for including problem-solving as a core concept. At a conceptual level, however, problem-solving extends far beyond engineering activity into all realms of human existence. Custer (1995) addressed these issues, classifying problem-solving into three major categories: (a) personal/social, (b) scientific, and (c) technological. Specific to engineering, the concept of problem solving can be seen to represent an overarching concept subsuming design, invention, and trouble-shooting ( Custer, 1995 ) thus confusing its conceptual distinctiveness. Given these challenges, the research team did not include problem-solving on the list of concepts.

As with problem solving, issues were raised by one of the researchers concerning the inclusion of experimentation as a core engineering concept in that the term “experimentation” is closely identified with the scientific method. Within a scientific context, experimentation connotes a specific methodology designed to establish and test hypotheses. Within an engineering context, it deals more generally with informed and incremental trial and error activities involved in making a design work. The argument could be made that the term experimentation is more appropriately associated with science than engineering. However, engineering can be viewed as engineering science, triggered in large part by increased federal funding for engineering research following World War II ( Seely, 1993 ). From this view, experimentation represents a formal analysis of applications of engineering theory. Although the term experimentation may connote other meanings beyond engineering, the researchers decided that experimentation met the inclusion criteria.

Engineering Education Ontology

As evidenced by the discussions of the two “problematic” concepts, the distinctions made to generate a list of core engineering concepts were important to the study. The overarching issues related to this endeavor are linked to the development of an engineering ontology for secondary level education. An ontology is a theory or representative vocabulary about the objects, their properties, and relationships within a specific domain of knowledge ( Chandrasekaran, Josephson, & Benjamins, 1999 ). The identification of a representative vocabulary requires careful analysis and typically begins with clarifying the terminology for coherence and consistency. This involves devising a syntax for encoding knowledge in terms of concepts and relations. This study furthered this process for secondary engineering education in one important area, by identifying core concepts.

As with other domain-specific ontologies (e.g., Borst & Akkermans, 1997 ; Guarino & Poli, 1995 ; Newell, 1982 ), this field’s concepts are not discrete and exhibit substantial conceptual overlap. An example is the number of concepts subsumed by or intertwined with engineering design. Functionality, efficiency, systems, and optimization could be considered to be subsumed by design, but in many documents they were also seen as distinct areas of investigation or focus. Design can be considered a primary engineering concept or even a threshold concept. Threshold concepts are distinguished from core concepts in that they are “akin to a portal, opening up a new and previously inaccessible way of thinking about something” ( Meyer & Land, 2006 p. 3). Engineering design could provide the “portal” for all other engineering concepts and themes appropriate for the secondary level.

Related to defining an engineering ontology, the research team struggled with the extent to which a body of concepts and knowledge can be said to be unique or distinct to engineering. The notion of distinctiveness is problematic for two primary reasons. First, the engineering field is comprised of a spectrum of disciplines, each with a specific set of knowledge. Given these separate fields, the question was raised whether the disciplines share a common and generalizable conceptual core. The second problem with formulating an engineering ontology is that much of engineering is interwoven with knowledge from other academic disciplines, particularly science and mathematics. This leads to the perception that engineering knowledge is essentially the application of knowledge from other disciplines.

These should not be construed as arguments against the existence of an engineering ontology. Rather, we argue that it is critically important to situate discussions of core engineering concepts, such as those identified in this study, within the broader context of an ontology. Furthermore, an engineering ontology should be developed with full realization of the complexity, richness, and challenges associated with such an endeavor.

Social Context of Engineering

The issue of engineering knowledge extends beyond ontology to issues of engineering practice and dispositions. This issue emerged particularly from discussions of the focus group, who encountered difficulty in making these distinctions given the applied and socially grounded nature of engineering. Throughout the analysis of the documents, social issues continually emerged as important to engineering. Primary among these were ethics and interpersonal skills, such as communication and teamwork. As Herkert (2000) pointed out, spurred in part by the standards promoted by ABET, engineering educators “take seriously the challenge of educating professionals who are both technically competent and ethically sensitive” (p. 303). This is not surprising given that engineering is inherently a social construct ( Bijker, Hughes, & Pinch, 1989 ). These contextual issues however are important if core engineering concepts are to be understood in a meaningful way.

Pedagogical & Curricular Implications

Another important issue raised most directly by the peer debriefing participants was the pedagogical and curricular implications of teaching the engineering concepts identified in this study. Many of the panelists questioned how these concepts could appropriately inform curriculum and instruction at the secondary level. However as Donovan and Bransford (2005) indicated, concepts are only a piece of the puzzle. Concepts provide a framework for students to understand factual knowledge and use that understanding in different ways. Concepts do not stand alone, but “take on meaning in the knowledge-rich contexts in which they are applied” ( Donovan & Bransford, 2005 , p. 6). Thus, the list of concepts generated through this study is not intended to be implemented in isolation or in an abstract manner in the classroom.

Additionally, procedural knowledge should not be taught abstracted from content or concepts. An understanding of process requires the learning of content; each “piece of subject matter is a way of knowing, a way of representing, or a way of solving problems” ( Costa & Liebemann, 1997 , p. 14). Within a technical domain such as engineering, this view of learning requires that teachers identify the possible knowledge requirements of tasks, ascertain students’ relevant prior knowledge, and provide adequate support for conceptual development ( McCormick, 1997 ). The concepts generated in this study provide a base for understanding engineering that can transfer across contexts. However, the domain knowledge specific to a context is equally important for understanding and reflecting upon the meaning of the concepts. This awareness of the need for conceptual, procedural, and domain knowledge should be reflected in curriculum and specifically addressed in teacher professional development contexts.

Conclusion

Given the framework of an ontological approach for secondary level engineering education, it is important that these concepts be seen as the initial phase of research. As Chandrasekaran, Josephson, and Benjamins (1999) pointed out, constructing an ontology is an ongoing research enterprise. They recommended sharing the knowledge representation language generated through careful analysis with others who have similar needs for knowledge representation in that domain, so as to eliminate the need for replication. This can then lead to building specific knowledge bases for specific situations (e.g., curriculum). It is recommended that this study be used to further that process. Specifically, the interrelationships between the concepts should be more fully explored. An excellent model to help guide this type of work is the Atlas of Science Literacy ( AAAS, 2001 ).

This study concentrated on identifying a conceptual base for secondary level engineering education. It should be apparent that this represents a daunting task, triggering a number of conceptual and practical issues. These issues have important implications for education if engineering is to be seriously considered as an integral part of the K-12 curriculum. These issues could significantly impact educational policy at the pre-collegiate level, where the case remains to be made for including engineering content, as well as at the post-secondary level with a growing call for reform in engineering education. Additional areas that warrant further investigation include the possible need for K-12 engineering standards, curriculum, and teacher pre-service and professional development. The central premise of this study is that these issues are best addressed after the conceptual base has been thoughtfully developed.

Acknowledgments

A version of this paper was prepared and publicly presented under contract to the National Academy of Engineering for a study on K-12 engineering education standards.

NCETE was funded by the National Science Foundation Under Grant No. ESI-0426421. This study was supported by the Center.

Rodney L. Custer ( rlcuster@ilstu.edu ) is a Professor and Associate Vice-President of Research, Graduate Education, and International Studies at Illinois State University. Jenny L. Daugherty ( jldaughe@purdue.edu ) is an Assistant Professor in Organizational Leadership & Supervision at Purdue University. Joseph P. Meyer ( jpmeyer3@illinois.edu ) is a physics and chemistry teacher at PBL High School in Paxton, IL.

References