Selected Core Thinking Skills and Cognitive Strategy of an Expert and Novice Engineer

Raymond A. Dixon
University of Illinois at Urbana-Champaign

Introduction

Understanding how students learn engineering design concepts and the subsequent instructional interventions that are directed to improve their performance is also contingent on understanding how experts in the various engineering disciplines solve engineering problems. Naturally, a part of the engineering and technology educators’ research agenda is aimed at gaining a better insight of how student and expert engineers solve specific engineering problems. This hopefully would lead to a larger body of knowledge that is accessible for administrators and teachers to make informed decisions about the teaching of engineering design concepts.

Over the past two decades a steady proliferation of studies in engineering problem solving have focused on the differences between expert and novice designers ( Cross, 2002 , 2004 ), design reasoning and thinking ( Goldschmidt & Weil, 1998 ), creativity and design ( Christiaans & Venselaar, 2005 ; Dost & Cross, 2001 ) and the design processes and strategies of engineering students ( Atman & Bursic, 1998 ; Cardella, Atman, Turns & Adams, 2008 ; Merrill, Custer, Daugherty, Westrick, & Zeng, 2007 ). In a recent study by Merrill et al. (2007) three core engineering design problem solving concepts were identified as important to teach at the high school level—constraint, optimization, and predictive analysis (COPA). These concepts reflect the processes that the professional designer uses in the real world to solve design problems. The ill-structured nature of engineering design problem solving also demands the use of high level thinking skills such as analyzing and generating skills ( Ullman, 2003 ; Atman & Bursic, 1998 ). These skills are inextricably linked to the strategies used by experts and novices as they work with the constraints of a design problem, find an optimal solution, and use various analytical procedures.

When engineering design is examined from the perspective of the problem space and solution space ( Dorst & Cross, 2001 ), these two mental spaces represents spaces of association between core thinking skills and core engineering design processes such as COPA. The problem space includes activities such as defining the problem, identifying constraints, specifying evaluation criteria, and gathering information about various solutions. The generation of solutions and the execution of problem solving strategies define the solution space. Specifically, this includes activities such as making decisions about various possible solutions, performing analysis, optimizing the selected solution, and determining specifications.

Purpose of the Study

The purpose of this study was to use verbal protocol analysis to explore qualitatively how the analyzing and generating thinking skills of a student and an expert engineer differ, and to determine how these core thinking skills influences their overall cognitive strategy in the problem and solution spaces as they solved a common engineering design problem.

Research Questions

The following research questions guided this study:

1. How do the analyzing skills of a student and an expert engineer differ when using core engineering design concepts in solving a design problem?
2. How do the generating skills of a student and an expert engineer differ when using core engineering design concepts in solving a design problem?
3. What are the dominant cognitive strategies used by the engineering student and the expert engineer?

Thinking Skills, Design Concepts, and Design Strategies Core Thinking Skills

Researchers identified several core thinking skills that are used by individuals in cognitive processing and for creative and critical thinking. These skills are valued by educators as important for learning and problem solving ( Marzano et al., 1988 ). They are grouped into eight categories: focusing skills, information gathering skills, remembering skills, organizational skills, analyzing skills, generating skills, integrating skills and evaluating skills. For the purpose of this study, analyzing and generating thinking skills were examined because these skills constitute key cognitive activities of engineers when they solve design problems.

As engineers solve problems, they generate various types of conceptual solutions and perform analyses using different mathematical strategies and heuristics ( Ullman, 2003 ). According to Marzano et al. (1998) , “analyzing skills are used to clarify existing information by examining their parts and relationships” (p. 91). To analyze, one must be able to identify attributes and components, identify relationships and patterns, and identify errors.

In engineering design, the ability to identify attributes and components helps students to focus analytically on the structure of objects, their systems, and forms. Identifying relationships and patterns allows students to recognize and articulate the interrelationship and the functionality of component parts. Several studies have examined the importance of identifying relationships and patterns. In a study conducted by Egan and Schwartz (1979) , they found that the recall of circuit drawings by skilled technicians was remarkably similar to the recall of chess positions by expert chess players. They referred to this aspect of pattern relationship as chunking. In addition, they verified that experts are able to improve their memory and problem-solving capability by identifying conceptual relationships. This is in contrast to novices, who because of a lack of domain-specific experience arrange patterns according to positional relationships.

Ball, Omerod, and Morley (2004) conducted think-aloud protocols of expert engineers with a minimum of 7 years of academic and commercial design experience, and novices who were master’s engineering students with limited design experience. Each participant received an identical brief that related to the design of an automated car-rental facility. This brief was designed “to be complex, multifaceted, and ill-defined in the traditional sense of a prototypical design problem but tractable enough to be tackled to a satisfactory level by designers with only a few years of design experience” (p. 502). They found that experts displayed greater evidence of analogical reasoning than do novices, irrespective of whether such analogizing is “schema-driven” or “case-driven.” Schema-driven analogizing involves “the recognition-primed application of abstract experiential knowledge that could afford a design solution to a familiar problem type” while case-driven analogizing entails “the invocation of a concrete prior design problem whose solution elements could be mapped onto the current problem.” They also found that the expert designers showed more evidence of schema-driven analogizing than case-driven analogizing, while the novice designers showed more evidence of case-driven analogizing than schema-driven analogizing. In other words, expert designers are more proficient in recognizing design problems with similar underlying conceptual relationships than do students or novice engineers.

Identifying errors, the third analyzing skill, involves detecting flaws that may exist in knowledge, logic, calculation, or procedure. This analysis also extends, where possible, to actions that identify the causes and make corrections where necessary ( Marzano, et al., 1988 ). It is often postulated that people use mental models or their own naïve theories to help them understand how complex systems behave ( Gentner 2002 ; Collins, 1985 ). These mental models and theories assist in the diagnosing of error. Because of their experience, experts have more sophisticated causal mental models and theories that are governed by concepts from several related domains ( Kempton, 1986 ) and so they are naturally more efficient at identifying errors in their design conceptualizations.

Generating skills enables an individual to use prior knowledge to add information beyond what is given ( Marzano et al., 1988 ). The student therefore uses his/her knowledge of the sciences, technical drawing, and the function of mechanisms to generate or construct a new device or system. This cognitive process involves connecting new ideas with prior knowledge to build a coherent organization that houses both new and old knowledge structures, which represents the interpretation of a new situation. Generating skills involves inferring, predicting, and, elaborating. Inferring involves deductive and inductive reasoning. Predicting is making a statement or expectation anticipating the outcomes of a situation based on prior knowledge of how things usually turn out. Elaborating involves adding details, explanations, examples, or other relevant information from prior knowledge to improve understanding. Elaborating can be complex and involves constructing mental models and analogies to understand the structural and functional features of objects and systems.

According to Bedard and Chi (1992) , experts are more efficient and superior in classifying problems according to relevant features. They are also efficient in their inference about additional aspects of the problem. Experts represent problems according to their conceptual features, and spend a considerable amount of time developing their representation by adding domain specific and general constraints. In contrast, novices’ representations are largely based on literal features and they may attempt to solve problems directly without properly defining them. The complex mental models that experts are able to generate make it easier for them to make accurate inferences and predictions, and elaborate about their solutions performance and the functionality of the designed component. Cross (2004) indicated that experienced designers use more generative reasoning in contrast to less experienced designers who use more deductive reasoning. In addition, expert designers select features of the problem space to which they chose to attend (naming) and identify areas of the solution space which they chose to explore (framing). Some expert designers (architects) approach to problem solving was characterized by strong paradigms or guiding themes, while novices had weaker guiding themes.

Core Engineering Design Concepts

Although engineers use various approaches and methods to solve design problems, three core engineering design problem solving concepts were identified as important to teach at the high school level—constraint, optimization, and predictive analysis (COPA) ( Merrill et al., 2007 ).

Various types of analytical methods and models are used in engineering design to predict the performance of artifacts and systems. The applicability of an analysis method is dependent on the level of accuracy needed and the availability of sufficient methods. General analytical methods are less expensive and faster to implement than physical modeling methods ( Ullman, 2003 ). Ullman gave an example of how the stiffness of a diving board can be determined by using a method from the strength of materials:

…the board is assumed to be a cantilever beam made of one piece of material of constant prismatic cross section, and with known moment of inertia. Further, the load of a diver bouncing on the end of the board is estimated to be a constant point load. With this analysis, the important variable—the energy storage properties of the board, its deflection, and the maximum stress—can be estimated. (p. 264)

Predictive analysis should be carried out in the planning environment and not the task environment because moves made in the planning environment can be easily undone, while the task environment actions cannot be reversed.

In the initiation of a new design problem, the design requirements such as specification and conditions for function, effectively constrain the possible solutions to a subset of all possible product designs. As the design process continues, other constraints are added to further reduce the potential solutions to the problem, and potential solutions are continually eliminated until there is only one final product design ( Ullman, 2003 ). Beyond the constraints of the original problem, constraints that are created during the design process come from the designer’s knowledge of mechanisms, devices, and systems, and also from the design decisions made during the designing process. Other sources of constraints are costs, economics, feasibility, time, material, and environmental implications.

The purpose of optimization is to achieve the best design based on prioritized constraints and criteria. This includes maximizing factors such as productivity, strength, reliability, longevity, cost, efficiency, and utilization ( Merrill et al., 2007 ). The functional parameters of designed components can be converted to some mathematical formula to determine the optimal functioning capability of the designed component.

Cognitive Strategy

Designers use different strategies to solve a design problem. For example, some designers may begin solving a problem by deciding whether the process should be one of design or redesign. Another group of designers may prioritize certain stakeholders and strategize their solutions around the high-priority stakeholders. Other designers may arrange their design assignment to be new and challenging in order to provoke a creative response ( Dorst & Cross, 2001 ).

In a protocol analysis of nine experienced designers solving a design problem, Kruger and Cross (2006) categorized the strategies used into four general approaches. These are:

• Problem-driven design. The designer focuses closely on the problem at hand and uses only information and knowledge that are strictly needed to solve the problem. The emphasis lies in defining the problem and finding a solution as soon as possible.
• Information-driven design. The designer focuses on gathering information from external sources that have not yet been processed and develops a solution on the basis of this information.
• Solution-driven design. The designer focuses on generating solutions and gathers only information that is needed to further develop a solution. The emphasis lies on generating solutions, and little time is spent on defining the problem.
• Knowledge-driven design. The designer focuses on using prior highly structured, individual knowledge and develops a solution on the basis of this knowledge. Only minimal necessary information from external sources is gathered.

The Conceptual Framework

Figure 1 represents the framework that was used in this study. Analyzing and generating skills are used by designers in varying degrees as both mental spaces (problem space and solution space) evolve and as they work with various constraints, and carry out predictive analysis and optimization. This is illustrated in the diagram by the two ellipses. The design strategies used by each engineering designer will also vary and may include one or a combination of problem, information, solution and knowledge driven strategies.

Verbal Protocol Analysis

A protocol is a “description of activities ordered in time, in which a subject engages while performing a task” ( Hayes, 1989 , p.51). Verbal Protocol Analysis (VPA), also known as “think-aloud” protocols, are often collected during (concurrent protocols) and after (reflective or retrospective protocols) problem solving episodes, to obtain a record of the knowledge used by the problem solver, and the succession of mental states through which he or she passes while working on the problem ( Proctor & Dutta, 1995 ). When conducting a verbal protocol, the participants are asked to say aloud everything they think, while performing the task, no matter how trivial it seems. The obvious benefits of this type of analysis include the relative ease with which participants typically verbalize their thoughts, and the potential for insight into their cognitive processes. Once the verbal protocols are collected by audio and/or video, they are transcribed, segmented into codable units of subject statements, coded according to a coding scheme, and analyzed to answer specific research questions.

Think-aloud protocol has been used extensively in reading and comprehension studies ( Donndelinger, 2005 ). Atman and Bursic (1998) argued that concurrent report is a valid method that can be used to collect data about someone’s thinking process. However, some have expressed concern that think-aloud protocols may distort or interfere with the mental processes that we seek to observe ( Proctor & Dutta, 1995 ). Others contend that when protocols are collected properly it does not distort or interfere with the participant’s thinking and performance, because information is being collected from the short term memory, while subjects are prompted to “keep talking” with minimal interference from the experimenter (see Christensen & Yasar, 2007 ; Ericsson & Simon, 1993 ).

Verbal protocol analysis has also been used by several researchers in engineering design to understand the cognitive process of experts and novice designers (see Atman & Bursic, 1998 ; Ball, Ormerod, & Morley, 2004 ; Christensen & Schunn, 2007 ; Christiaans & Dorst, 1992 ; Cross, 2002 ; Dorst & Cross, 2001 ). A more recent study by Cardella, Atman, Turns, and Adams (2008) investigated the changes in individual engineering students design process over their course and how these changes might prepare them to become global engineers. Verbal protocol analysis was used to gain insight of the design behavior of engineering students as well as faculty members. A total of 61 students from various engineering disciplines participated. Some of their findings revealed that the more experienced designers (seniors) tend to spend more time in design activities such as evaluating design alternatives, making design decisions, and communicating design decisions. Senior engineering students had more complete design solutions. Their solutions also had additional mechanical and technical features. Finally, they found that differences in “the structure of the task may affect students’ use of ‘analytical skills’, their ‘holistic, multidisciplinary thinking’, their tendency to ‘exhibit creativity’, the extent to which they exhibit ‘high ethical standards and a strong sense of professionalism’ and their use of ‘the principles of business management’” ( p. 257).

Methodology

Purposeful sampling was used with a multiple-case study design. According to Gall, Gall, and Borg (2007) , in purposeful sampling, the goal is to select cases that are likely to be information rich with respect to the purposes of the study. In a multiple-case design, the unit of analysis is two or more individuals, or two or more instances of phenomena that are collectively studied in one case. For the purpose of this study two participants with significant differences in their years of experience were used—an experienced engineer and a engineering student. Verbal or “think aloud” protocols were collected from each participant as they solved a design problem within an artificial context.

The Participants

The novice was a senior mechanical engineering student who worked as a teaching assistant for a computer-aided design course. The expert had a Bachelor’s degree in mechanical engineering and over 28 years of work experience. This included working as a manufacturing engineer and a locomotive engineer. He is also recognized as an expert builder of large size locomotive models. That located him well above the number of years that it normally takes one to achieve expertise. According to Simon and Chase (1973) , experts needed about 10 years of intense involvement in domain-specific activities before they can reach international levels of performance.

The Task

Each participant was given an ill-defined mechanical engineering design problem to solve. The engineering design problem was taken from a mechanical design text and modified to make it more ill-defined and more challenging. The task was then sent for validation to two engineering instructors, each with over 25 years of faculty experience teaching mechanical engineering. Feedback provided by each of the instructors resulted in minor modifications of the design task (see Figure 2).

Figure 2
The Design Task

Procedure

Concurrent verbal protocols were collected from each participant as they solved the design problem. Verbal protocol analysis requires participants to “think aloud” while solving a problem or performing a task ( Atman & Bursic, 1998 ). The novice (engineering student) selected a computer lab on campus to solve the design task while the expert used a drafting office at his home. Both were given pencils and sketch pads, and both were allowed to use the drafting software on their computer. They were each given an hour to complete the problem and were prompted to continue to speak aloud what they are thinking whenever they became silent. Each “think aloud” session was recorded digitally.

Data Analysis

After each participant completed the design task, the audio recordings of their concurrent protocols were transcribed. The transcribed protocols were then segmented into think-aloud utterances, divided into sentences, and coded. The quality of the sketches was not evaluated since the objective of the study was to examine the mental processes of the engineering student and engineers while solving the design task. The sketches and notes however, acted as a reference to clarify some sections in the protocols.

The purpose of segmenting is to break the transcribed verbal protocol text into units (or segments) that can be coded with a pre-defined coding scheme. Segmenting took place in two stages. In the first stage, larger units of analysis called think-aloud utterances were identified and segmented from each other. Think-aloud utterances are comprised of those words spoken aloud by a participant that were followed by some period of silence ( Hartman, 1995 ). These periods of silence or pausing had to have duration of five or more seconds. A total of 70 utterances were segmented (40 for the engineer and 30 for the engineering student). The think-aloud utterances were further segmented into sentences.

Codes were provided for eleven predefined constructs. The constructs, their codes, and meanings are described in Table 1. Reliability coding was conducted by having one additional person code five pages of the first transcript. A reliability kappa coefficient of 0.86 was calculated which was well within the range recommended for interrater reliability ( Miles & Huberman, 1994 ).

Table 1.
Constructs, Codes and Definitions

Construct	Code	Definition
Analyzing Skill
Indentifying attribute and component	IAC	Make distinction between parts that together constitute a whole. e.g., The various parts of a nail gun.
Indentifying relationship and pattern	IRP	Articulate the interrelationships between components. e.g., rotating wheel and a gear mechanism.
Indentifying Error	IE	Detecting mistake in logic, procedure, calculation. Identifying an error in design e.g., recognizing a component won’t work because of a design flaw.
Generating Skills
Inferring skills	IS	Going beyond available information to identify what reasonable may be true.
Predicting skills	PS	Assessing the likelihood of an outcome based on prior knowledge of how things usually turned out. E.g., Shaft will wear because tolerance is too small.
Elaborating skills	ES	Adding detail explanations, examples or other relevant information from prior knowledge to improve understanding. e.g., using knowledge of typewriter to know how a computer works.
Engineering Design Concept
Constraint	C	Specifications, criteria and conditions that constrains an engineering designing task. e.g.,
Predictive analysis	PA	A device to clamp soft surfaces. Methods analyzing the functionality of a designed component. e.g., Calculating the maximum stress a device can withstand.
Optimization	O	Obtaining the best design based upon prioritized constraints, and criteria.
Solution Space	SP	Generating solution
Problem Space	PS	Framing the problem

Results

Research Questions One

Table 2 illustrates that the student performed twice as much analysis as the expert. However there were differences between the student and the expert in where the analyses were primarily performed. The expert did most of his analysis (15) when using engineering science and mathematical formula. The student, on the other hand, did most of his analysis (34) when he was resolving issues relating to the various constraints in the design problem. Both the student and the expert’s analysis mainly focused on the relationships between the component parts of their design.

Table 2
Analyzing Skills Frequency Table

Analyzing skills of Expert
	Constraint	Predictive Analysis	Optimization	TOTAL
Identifying attribute and component	5	4	2	11
Identifying relationship and pattern	5	8	2	15
Identifying error	0	3	3	6
TOTAL	10	15	7	32
Analyzing skills of Novice
Identifying attribute and component	14	4	2	20
Identifying relationship and pattern	14	7	4	25
Identifying error	6	6	6	18
TOTAL	34	17	12	63

Research Question Two

Table 3 illustrates that the expert used almost twice as much generating skills as the student. Most of these skills were used primarily when he carried out analysis and optimization. In contrast the student did most of his generating when he performed analysis. The expert also spent more time making inferences and elaboration about the quality of his solution while the student spent more time predicting how each component will function.

Table 3
Generating Skills Frequency Table

Generating skills of Expert
	Constraint	Predictive Analysis	Optimization	TOTAL
Inferring	2	13	12	27
Predicting	2	10	6	18
Elaborating	12	6	8	26
TOTAL	7	31	26	64
Generating skills of Novice
Inferring	2	7	2	11
Predicting	6	8	4	18
Elaborating	2	4	4	10
TOTAL	10	19	10	39

Research Question Three

Verbatim reports from the transcripts of both participants’ “think-aloud” session highlighted the strategy used by each to solve the design challenge. The comments are preceded by a timestamp.

The Expert

While the expert had over 25 years of experience, the type of task he was required to solve did not fall within the general type of problems that he was accustomed to solving. Immediately after reading the problem, the expert formed a mental image of the solution. He used this mental image to frame and plan the strategy he would use:

(00:33) What I would probably do is I see a mental picture of that hold down device in my head. I would probably go with a design that was similar to that because I knew that was the design that has been proven to work so in a way. I'm not creating something from nothing. I've already got an idea what that would look like.

The expert used his experience and knowledge of moments and forces to resolve positional and functional issues and made decisions about the relationship and structure of the component:

(03:03) I know that from experience there should probably be a lower pivot point. Just guessing at this point in time, so I'm going to sketch a lever that comes up that's pivoted and then there should be another connection from this lever, the lever on my arm that I'm using to raise and lower it. So now what I got here is some rough idea of the linkage. I think would work and then of course to hold this down to a clamping device. I have to realize that my lever here needs to go over center so the item that you're holding down cannot be the force, because it would not pull back and release itself. For instance, if I pivot this point here and I pivot this point back to here. What it is going to do to the lever that I install here is that it is going to actually pull back and lift this thing, or is it going to bind or whatever. I may have to have some spacers or something to bring it out around the other point and ah something like this.

The expert strategized his approach from a system perspective. He seemed to identify the most critical element of the system, and then focused his cognitive effort into solving the function of those components to meet his personal goal and the solution criteria:

(10:00) …once I know my linkage is going to work and if I pull the lever, this thing is going to open up and whatnot like I want it to. Then I'll go to more of a design process where I actually may come up with, I guess, the second design of the parts that actually would work.

He also evaluated and reflected on his progress. If there was conflict with his approach and solution he considered alternatives, but his motivation to achieve his goal impelled him to use his experience or knowledge of physical principles to resolve the conflict:

(25:00) …if I use these parameters I'm going to be able hold the part down or would the part just push back and just push these lever arms up? That I don't really know at this point in time. Should I start this thing all over again or is this going to be a good design? Should I go back to the paper or should I come up with an alternative holding device?

The Novice

The novice used more time (approximately 10 minutes) when compared to the expert (approximately 3 minutes) to analyze the problem, clarify elements of the specifications that he could not understand, and gather data, before he started generating solutions. His solutions were primarily influenced by the specifications identified in the problem.

(00.34) …specifications are to hold three inches of material, and have eight inches of reach which is somewhat ambiguous. I'm interpreting that as meaning that it can extend eight inches from the material so that it can be gripped. To be able to solve this next part, to release the work piece quickly, easy to position, and easy to move to other work areas so it's not too overly bulky or too heavy …the holding strength should be considered. Okay so now the holding strength would depend on what type of material that we're going to be holding.

The solutions generated by the novice appeared initially to be limited to the concept of two moving parts. He then started generating solutions around the two moving parts using mental pictures and imposing constraints as he progressed.

(03:23) …first thing that I'm going to do, I guess for something like this we're going to have moving parts. So I'll be designing probably at least two parts …would probably be the holding mechanism some sort of screw, that's what I'm interpreting … going to be able to adjust because we want to be able to let it grip below three inches, so it needs to have some sort of adjustment.

As the novice progressed, he used analogies that were case specific to decide on the structure of the component. This was evident at various points during his solution. He also spent time determining the relationship of one component with the other. As his solution progressed, it showed a component-by-component pattern of solution which can be compared to using bottom-up generation of solutions rather than a top-down:

(06:33) …some kind of gripping mechanism on the top of the lever like some claws which is what we see on monkey wrenches…I'm going to design something similar for the bottom now except that this has to be able to interact with the screw to let it move up and down… am I thinking that this other part is actually going to be sliding up and down the long side of the part.

Like the expert, the novice also used knowledge of moments and forces to evaluate and resolve positional and functional issues and also to make decisions about the structure and relationship of the components. He also performed evaluation at various stages of his solution.

(18:50)…I try to make use of symmetry so that it fits directly in the center. The last thing I would want to do is create offset forces in moments that could really mean it introduces bending stresses and that's just not something we want… Albeit a quick release mechanism something like this might be a little bit dangerous to have, which is perhaps why there aren't a lot of them....

Figures 3 and 4 illustrate the cognitive strategies used by the expert and novice respectively, as they navigate cognitively between the problem space and the solution space. The time spent in the problem space was determined by statements made by the participants when framing the problem and gathering data about specifications and constraints, while time spent in the solution space was determined by statements made by the participants when generating solutions, performing analysis, and optimizing solutions. The network diagrams show that the novice spent more time in the problem space than the expert. It also shows that the expert framed his solution around his mental model, while the novice framed his solution around the problem specifications. The novice depended mainly on the information provided by the problem, thus proving that he used the problem-driven strategy. In contrast the expert used the solution-driven approach, spending most of his time generating solutions and little time defining the problem.

Conclusions

This exploratory study highlights certain differences in the way an expert and a novice engineer used their analyzing and generating skills while solving a fairly ill-structured design problem. The expert tends to use more inferences and elaboration when solving the design problem and the novice tend to use analysis that is focused on the functional relationship between the parts of the designed component. This difference might be attributable to the mental models or analogies that they generate. The mental representations used by the expert not only allowed him to go beyond merely predicting the performance of his conceptualization, but to also improve his solution by adding additional details that the novice, who because of his limited experience, was unable to add.

The novice behavior was associated with a “depth-first” approach to problem solving. This approach is characterized by sequentially identifying and exploring sub-solutions in detail. He approached the solution component-by-component, focusing on the adjustable jaws, then the screw, locking device, and lever. The expert in this protocol study showed a “top-down” “breadth first” approach, which is consistent with what research confirms of experts in some knowledge domains. He quickly recognized the system requirements and then focused his efforts on designing what he perceived to be the most critical element of the system—the lever locking mechanism. His approach might also be attributable to the fact that experts are exposed to a large number of examples, problems, and solutions that occur in their domain. Therefore a key competency of the expert is the ability to stand back mentally from the specifics of the accumulated examples, and form a more abstract conceptualization pertinent to the solution ( Cross, 2004 ). This allows him to focus on the critical elements of the problem, rather than on superfluous details.

Both the expert and novice carried out frequent monitoring and evaluation of their strategy and solutions to identify errors. The novice however, evaluated against the specifications and constraints dictated in the question, while the expert evaluated against his perceived functional goals. The engineer used his initial mental models to guide his solution and displayed more complex inferring and elaborating skills. He also spent more time in the solution space. Lloyd and Scott (1994) , reported that experienced designers used more generative reasoning, in contrast to deductive reasoning employed by less experienced designers. In addition, their protocol studies of experienced designers found that they were more solution focused. The expert’s approach clearly was solution-driven. He selected a feature of the problem-space to attend (naming), and from there explored the solution space ( Schon, 1983 ). The expert spends more time within the solution-space, while the novice spends more time within the problem-space (see Figure 3 & 4).

Finally, both the expert and novice in this study used knowledge of mechanics to evaluate and also predict functions and positional locations, as well as to identify errors. This was consistent with findings which showed that one of the commonalities between designers was their implicit or explicit reliance on ‘first principles’ in both the origination and detailed development of their design solutions ( Cross, 2002 ).

Implications for Engineering and Technology Education

The findings from this study would suggest that a proper grasp of systems concept is necessary to raise the problems solving ability of students to be reflective of experts. How components interrelate with each other and to the entire system, whether it be a simple or a complex system, are important for students to understand in order to increase their ability to generate conceptual solutions and solve functional issues. This will enable students to spend more time in the solution space, like experts, rather than in the problem space. Requiring students to solve design activities in the classroom that are different in surface features, but may have similar underlying operational concepts, would broaden their schema of problems that fall within the same category, and allow them to generate solutions more fluently. The ability to make inference and elaborate is a critical skill that distinguishes the expert from the novice in this study. Curriculums that integrate scientific enquiry skills in the engineering design process may be a step in the right direction to develop student’s ability to infer and elaborate. According to Crismond (2007, p. 27) , “Students can develop their own guidelines based on tests they conduct by formulating design rules-of-thumb.” As they evaluate scientific findings in relationship to their engineering solutions, they will grow in their ability to identify and add missing details of a solution as experts are able to do.

As efforts to infuse engineering design in technology education continues, the presence of these thinking skills and cognitive strategies emphasize that they cannot be ignored in the instructional process. Clearly, more studies need to focus on the variety of thinking skills and cognitive strategies used in engineering design and the most effective instructional techniques to develop these skills and strategies.

---

Raymond Dixon is a Research Associate at the Center for Mathematics, Science and Technology at Illinois State University. He can be reached at rdixonenator@gmail.com .

References