JITE v41n3 - Teacher Attitudes Toward and Perceptions of Student Performance on the Design Component of the Caribbean CXC Examination

Volume 41, Number 3
Fall 2004


Teacher Attitudes Toward and Perceptions of Student Performance on the Design Component of the Caribbean CXC Examination

Philbert J. Crossfield
Michael K. Daugherty
Chris Merril
Illinois State University

The Caribbean Examinations Council (CXC) is a regional examining body that provides examinations for secondary and postsecondary students in Caribbean countries. The CXC administers the mechanical engineering technology examination for secondary students in the Caribbean, which includes Barbados, Trinidad and Tobago, Guyana, Jamaica, Antigua/Barbuda, Belize, St. Lucia, and St. Kitts/Nevis. After conducting appropriate examinations, the CXC awards certificates to successful students. As one of the compulsory technical subjects in the Caribbean, students completing the mechanical engineering technology curriculum complete an examination once per year. A compulsory question on the engineering design component of the curriculum comprises 40% of the final score on this examination, and this compulsory design question has proven to be a continuing problem for students taking the examination.

During the June 1999 examinations, only 49% of the candidates who took the examination were able to score above 45% ( Caribbean Examinations Council, 1999 ). Similarly, the report for the year 2000 indicated that only 33% of the candidates who attempted this examination scored 50% or more ( CXC, 2000b ). This trend in performance on the design component of the examination has caused concern among teachers and members of the CXC.

As stated in various reports, this problem persisted despite the fact that teachers had been given suggestions in successive reports that could possibly have helped to correct the situation. Suggestions made in the various mechanical engineering technology reports included the following.

  1. Continuous improvement in design skills will be achieved if students are regularly introduced to design elements such as fasteners, joints, etc.; having students complete projects using these features can provide reinforcement.
  2. Mechanisms used to convey movement in machines, such as chain drives, gear drives, and belt drives, should be introduced regularly to students; this process may involve taking down machine guards.
  3. The ability to sketch is very important in answering questions in mechanical engineering technology; creating practice sessions for students will develop this skill ( Caribbean Examinations Council, 2000b ).

Suggestions and comments made by examiners in various reports with respect to the curriculum objectives suggest that if teachers involved in the program satisfied the requirements as outlined in the syllabus, then the performance of students would improve, which would lead to better results overall on the examination. It was with this view in mind that the researchers sought to examine the attitudes and perceptions of technology teachers concerning the performance of students in design and the requirements of the mechanical engineering technology curriculum.

Problem and Purpose

The purpose of this study was to identify teacher attitudes toward and perceptions of student performance on the design component of the CXC Mechanical Engineering Technology examination. While the overall performance of students on the CXC Mechanical Engineering Technology Examination has been favorable since the test was initiated in 1991 (76% of the candidates obtained passing grades), the candidates continue to perform poorly on the design component of the examination. These poor results on the design portion of the examination prompted an investigation to isolate teacher perceptions concerning student performance. Therefore, the problem for this study was to investigate why candidates taking the design component of the CXC Mechanical Engineering Technology Examination continued to perform poorly on the test.

Comments made by examiners on various reports suggested that the candidates taking the examination needed to be exposed to the required content on the syllabus to prepare them for the examination. If this exposure was not provided to students, then it could be a contributory factor to the poor performance displayed by students. On the other hand, exposure might not have been a factor, as there could be various other underlying reasons. In a study carried out by Atkinson (1999) on key factors influencing pupil motivation in design and technology, it was discovered that there was a positive correlation between students' drawing, writing, design, and manufacturing skills, and their levels of motivation. The results indicated that students who were motivated achieved higher mean scores for drawing, writing, and designing. Those students who were unmotivated achieved low mean scores in the same areas. This question of motivation being a factor and other questions related to student performance could only be answered if an investigation were done regarding the attitudes and perceptions of teachers on these factors. It was with this view in mind that the researchers isolated teacher attitudes toward and perceptions of student performance on the design component of the CXC Mechanical Engineering Technology Examination.

Research Question

The following research question was developed for this study: What are teacher attitudes toward and perceptions of student performance on the design component of the CXC Mechanical Engineering Technology Examination?

Research Assumptions

Based on this study, the following assumptions were made.

  1. The technology teachers were representative of all technology teachers in Jamaica.
  2. The responses given to questions asked on the survey were independent expressions by each teacher.
  3. The responses were honest and a true representation of the beliefs of the teachers surveyed.

Review of Related Literature

A review of related literature on factors that affect student performance on design and technology revealed that very little had been written on the subject in Jamaica. Because of this, related journals, books, and publications of professional societies in Australia, the United States, and the United Kingdom were reviewed; more emphasis was placed on literature from the United Kingdom, as the educational system in Jamaica and other Caribbean nations was originally based on and continues to be modeled on the British educational system.

Design in the Curriculum

While the majority of the technology curriculum in the Caribbean would seem very traditional by U.S. standards (foundry working, bricklaying, etc.), the emphasis on design is a notable exception. In fact, the CXC has been including design-based curriculum materials and assessments for almost 10 years. Design and technology is one of the newest curricular subjects in the Caribbean, and it is known by different names throughout the world. For example, it is known as technology in Scotland, South Africa and other parts of the world; design and technology in England and Wales; technology and design in Northern Ireland ( Owen-Jackson, 2000 ). Owen-Jackson suggested that one of the important aspects of studying design and technology, especially if one teaches it, is being clear about what it means, what is taught in such a course, and why the course is a part of the curriculum. McCracken (2000 ) suggested that teachers should understand the nature of design if they are to optimize learning through design. McCracken further suggested that, design is not a one-shot process and that it should be recognized as such. McCracken contended that design should be seen as an ongoing process; if design is not seen in this manner, the educational value will not be achieved. Williams (2000) asserted that design is a significant process in the development of technology and that this can be seen in many disciplines from engineering to architecture, and when examined from an educational perspective, it is an ideal methodology to use as a vehicle to achieve the desired competencies.

The International Technology Education Association (ITEA) Standards for Technological Literacy: Content for the Study of Technology (2000) referred to design and implied that design can be regarded as the core problem-solving process of the technological development of humankind and is as fundamental to technology as inquiry is to science and reading is to language arts. This standards document further suggested that for students to become literate in design and the design process, they must acquire the cognitive and procedural knowledge that is needed to create a design and be familiar with the procedure used to carry out the design process. Design in technology differs significantly from designing in art. For example, technological designers work within certain requirements to satisfy human needs and wants. Artists, on the other hand, display their mental images and ideas with few constraints. As a result of this difference, efficiency is a major consideration in technological design, while the beauty or appearance of the product is often less important.

The Ministry of Education and Culture (1999) in Jamaica also focused on design in the Resource and Technology Curriculum Guide, in which it was suggested that the design process could be seen as a problem-solving approach used to solve practical problems. The Ministry of Education and Culture further suggested that this process was similar to other problem- solving processes, but the process employed in resource and technology ensures that students use physical materials in problem solving. The Jamaican curriculum guide lists five aims in the design process: helping students identify contextual problems, resources, planning work, producing products/systems, and evaluating the results. The Ministry of Education and Culture illustration of the stages in the design process is presented in Figure 1.

Figure 1 . Ministry of Education and Culture Design Process.

Ministry of Education and Culture Design Process - 5 Stages

The Standards for Technological Literacy ( ITEA, 2000 ) supported the idea of the process being cyclic and implied that the design process was a systematic, iterative approach to design problem solving that promoted innovation and yielded design solutions. These standards further implied that the design process for grades 9 to 12 should include ". . . defining a problem, brainstorming, researching and generating ideas, identifying constraints, exploring possibilities, selecting an approach, developing a design proposal, making a model or prototype, testing and evaluating the design, creating or making it, and communicating processes and results" ( ITEA , p. 97).

Williams (2000) asserted that the appeal of design problem solving as a methodology lies in the fact that it is a natural activity; humans have always been faced with problems and have tried to solve them. On the other hand, design problems are seldom presented in a clearly defined form; to systematically seek an optimum design solution, engineers and other design professionals use experience, education, established design principles, creative intuition, imagination, and culturally specific requirements ( ITEA, 2000 , p. 98). Williams also suggested that despite the fact that the terms "design" and "problem solving" were often used interchangeably, problem solving was different from design in that design dealt with ill-defined problems and did not begin with a problem, while problem solving did. McCracken (2000) supported this assertion when the suggestion was made that design could be described as the process of creating something useful. It brought a sense of order where before there was only the randomness of nature. According to McCracken, design could be considered to be the creative soul of technology: design is to technology as the human soul is to the body. It is therefore important to understand the interdependence and complimentary nature of technology and design. Technology would be incomplete without design, and design cannot be fully appreciated without an understanding of technology. If teaching is to be fully understood, the concepts of design must be understood.

Teaching Design and Problem Solving

Woods, Felder, Rugarcia, and Stice (2000) suggested that the goal of educators in engineering should be to equip students with design problem solving, communication, teamwork, self-assessment, change management, and lifelong learning skills. Woods et al. further suggested that process skills were important in design, and that the degree to which students developed these skills determined how they solved problems, wrote reports, functioned in teams, carried out self-assessment and performance review of others, learned new knowledge, and managed stress when they had to cope with change. The development of these skills was best facilitated by giving students practice and not by simply talking about or demonstrating what was to be done. The instructor's role should be primarily that of a coach, encouraging the students to achieve the target attitudes and skills, and providing constructive feedback on their efforts. To develop the desired skills, Woods et al. suggested the following eight basic activities.

  1. Identify the skills you wish your students to develop, include them in the course syllabus and communicate their importance to the students.
  2. Use research, not personal intuition, to identify the target skills, and share the research with the students.
  3. Make explicit the implicit behavior associated with the successful application of the skills.
  4. Provide extensive practice in the application of the skills, using carefully structured activities, and provide prompt constructive feedback on students' efforts using evidence-based targets.
  5. Encourage monitoring.
  6. Encourage reflection.
  7. Grade the process, not just the product.
  8. Use a standard assessment and feedback form (p. 2).

In addition to these eight basic steps to be used in developing design and problem-solving skills, Woods et al. further listed the following as additional activities: (a) Use a standard research-based problem-solving strategy across several (and ideally, all) courses in an instructional program; (b) Solve some problems in depth; and (c) Help students make connections between statement, the identification of required technical knowledge, and the problem solution (p. 4).

The steps outlined by Woods et al. (2000) suggested that the goals of educators in engineering should be to equip students with the desired problem-solving skills and outlined some basic, as well as other, activities that could be used to develop the skills. They also supported suggestions made by the CXC to technology teachers for improving the problem-solving skills of students, which would improve performance on the mechanical engineering technology examination.

The procedure followed by teachers could be dependent on their perception of what was involved in the teaching of the design process. Mallett (1996) addressed the following issues in a study relating to the perceptions of lower secondary design and technology teachers about the utilization of the design process.

  1. (a) What is a design process, and (b) how do design and technology teachers view the different elements of a design process?
  2. How does the inclusion of a design process affect the teaching and learning process?
  3. What do teachers look for when they assess students' achievements in design and technology?
  4. How can a design process be taught in lower secondary school? (p. 2).

Further research discovered that the design process was being used in an environment which was structurally inappropriate for the teaching of methodologies suggested by other researchers. For example, it was suggested that in using a design process, students should be given the opportunity to use, in a safe way, any material they wanted to include in their design; but this was not the case. In practice, the teachers were still using, in most cases, an approach based on limited materials.

Teaching Approaches

Welch and Lim (2000) suggested that most of the current schoolwork on design presented tasks to students in a form that assumed there was only one correct way to do it and often only one correct solution. This, they contended, should not be the case, as design and technology presented tasks with many possible solutions. Mallett (1996) suggested that in his study teachers experimented with different approaches through trial and error, and listed the following as areas of major concern: (a) the provision of a selected repertoire of basic skills to students, (b) the development of the graphical abilities of students, (c) the designing of the curriculum material, and (d) the introduction of technology via a design process.

The idea of teachers experimenting with different methods to teach design was supported by Welch (1996) , who suggested that there was good reason to doubt the efficacy of requiring students to follow any form of a linear or sequential design process model as outlined in some textbooks and curricular documents. This, they suggested, came from findings in a study done on problem-solving strategies used by novice designers. The study revealed that untutored designers engaged themselves in many of the subprocesses of theoretical design models, but did not prioritize or sequence those subprocesses as suggested by the models. As a result of this, they suggested that teachers explicitly taught process skills that would assist students' designing but which should not impose a strict sequence in which the skills were applied.

On the other hand, Davies (2000) suggested that there were many factors that contributed to anxieties when teaching design and technology. He further suggested that teachers were not universally multi-skilled in all areas of the subject and were most confident when supporting work with which they were most familiar through their personal expertise. The idea that teachers were not universally multi-skilled was supported by Kimbell (1982) , who suggested that excellence could be expected in the realization of design solutions only through the establishment of confidence and competence with tools and materials.

According to Davies (2000) , teachers knew it was important to keep their knowledge and skills updated and relevant to student needs and interests, and were for the most part frustrated because of the difficulty in doing so. There was also uncertainty about certain aspects of their professional role and for the most part a degree of role conflict between the interests of their department and their relationships with students. According to Davies, teachers who felt that they made most progress in life through being risk-takers were better prepared to challenge learners at a high level and support them in risk-taking with their projects. Davies also contended that teachers felt creativity was important in designing and that building students' confidence was crucial to the development of creativity.

Atkinson (1999) examined the relationship that existed between teacher and student motivation, and student performance in design and technology project work. Atkinson contended that there was a positive relationship between student motivation and student performance. It was evident that students who were motivated achieved a high mean score in their examination project work, while those who were unmotivated achieved a low mean score. On the issue of the relationship among student skills, motivation, and performance, it was suggested that there was a positive correlation between students' drawing, writing, design, and manufacturing skills, and their level of motivation. Atkinson contended that this was evident, as those students who were motivated achieved high mean scores for their drawing, writing, and designing, while those who were unmotivated achieved low mean scores.

Atkinson (1999) further suggested that the lack of skills and understanding regarding processes and materials were found to present a major problem for many of the students during both the design and manufacturing stage of their project work. According to Atkinson, an examination of the relationship among creativity, motivation, and performance indicated that there was a positive relationship between creativity and performance. The relationship among teaching strategy, motivation, and performance indicated that students attained higher levels of achievement in schools that adopted an interventionist approach; however, the students for the most part were unmotivated, while schools that employed a collaborative approach had more motivated students. The relationship between student motivation and teacher motivation indicated that a significant number of the sample of students being taught by unmotivated teachers were unmotivated themselves ( Atkinson ).

Designing Achievement Tests

Student performance on standardized tests may be caused by any number of factors, including what was taught in schools, a student's native intelligence, and out-of-school learning opportunities that are heavily influenced by a student's home environment ( Bauer, 2000 ). On the other hand, Popham (1999) asserted that standardized achievement tests are quite remarkable, as they do what they are supposed to do; however, they should not be used to evaluate the quality of education. To emphasize this point, Popham contended that standardized achievement tests would always contain many items that were not aligned with what was emphasized instructionally in a particular setting, thus there would always be a mismatch occurring between what was taught locally and what was tested through standardized achievement tests. The CXC examinations could be referred to as a high-stakes examination in the Caribbean, as it was the regionally accepted examination for students leaving high schools in the Caribbean.

One key issue revealed in the literature was that there was a disjuncture between what was expected by the National Curriculum in Jamaica and what was happening in the classroom; this, the literature contended, was being addressed, as attempts were being made to rectify deficiencies and to deliver a comprehensive design and technology curriculum ( Caribbean Examinations Council, 2000a ). There were still problems in dealing with this disjuncture, however, as some aspects of the curriculum were being omitted because of lack of staff expertise or inadequate resources. It was also suggested that there were inconsistencies in the procedures used to assess student activities in design and technology. Some teachers felt that the final product was the important factor and that marks should be awarded accordingly, while others felt that the process was more important, especially when problem-solving activities were addressed.

On the issue of standardized tests, it was thought that they were not necessarily the best means of assessing student performance. However, standardized tests were thought to be the most efficient means of assessing students; and that students' performance on these tests was at least in part dependent on the school system. The CXC examination, as outlined in the CXC syllabus, satisfied the characteristics outlined in the literature as characteristics of curriculum-based external exit examinations and thus was seen as an efficient means of assessing student performance in specific subject areas across the Caribbean.

Methodology and Procedure

Research Design

The design of this study was descriptive. The selected research participants consisted of technology teachers in Jamaica who prepared students for the CXC Mechanical Engineering Technology Examination. Data for this study were collected from the selected sample using a questionnaire which was based on a review of related literature as well as the stated objectives of the mechanical engineering technology curriculum. To ensure confidentiality and maintain anonymity, respondents were asked not to put their name or the name of their institution on the questionnaire. They were also asked to complete and mail the questionnaire in the self-addressed envelope provided. The respondents completed the questionnaires and mailed them to the researcher's address in Jamaica, where they were collected and taken back to the United States for analysis.

Population and Sample

The population for the study was taken from institutions in Jamaica that prepare candidates for the CXC Mechanical Engineering Technology Examination. There were 46 such institutions identified by the Overseas Examinations Office in Kingston, Jamaica. The population consisted of 80 technology teachers from these institutions in Jamaica.

The sample consisted of 46 technology teachers from schools in Jamaica that prepare candidates for the examination; this was approximately 58% of the population. It should be noted that while the CXC administers the Mechanical Engineering Technology Examination for secondary students in many Caribbean nations (including Barbados, Trinidad and Tobago, Guyana, Jamaica, Antigua/Barbuda, Belize, St. Lucia, and St. Kitts/Nevis), the overwhelming majority of the teachers are Jamaican. Most of the other island nations have only one or two technology teachers. It was for this reason and the limitations of collecting research in the Caribbean that the sample was drawn from Jamaican teachers.

A cluster sampling technique was used to select the sample. This method of sampling was feasible, as the researchers were able to randomly select schools from the list of schools in Jamaica, instead of selecting individual teachers. The selection of the sample was conducted using a random numbers table; all schools on the list had an equal chance of being selected. From the list of schools, 20 schools were selected, which was approximately 43% of the schools that prepared candidates for the examination. The technology teachers used as the participants in the sample were contacted through the principal in each school. An envelope containing a cover letter to the principal, questionnaires, informed consent forms, and return envelopes was sent to each principal. The cover letter explained the procedure for completing and returning the questionnaires and made suggestions detailing the importance of the study to the participants. The questionnaires were completed by the participants and mailed in the self-addressed return envelopes provided.

Instrumentation

To answer the research question, a four-part questionnaire was developed. This questionnaire was pilot tested, and a reliability coefficient alpha of .8439 was achieved for the instrument. Following the pilot study, the formal study was conducted to collect the relevant data. A total of 46 questionnaires were sent to secondary schools in Jamaica that prepared candidates for the CXC Mechanical Engineering Technology Examination. Each mailing consisted of a three-page questionnaire, an informed consent form, and a cover letter addressed to the principal of each school. A total of 25 (54.35%) of the respondents returned the instrument.

The questionnaire was divided into four parts. Part one consisted of seven questions that were used to collect demographic information related to the personal, professional, and educational characteristics of participants on gender, age, educational experience; the location of the school; and whether it was in an urban or rural setting. Part two consisted of 21 questions designed to elicit responses from teachers regarding their attitudes and perceptions related to student performance on the design component of the CXC Mechanical Engineering Technology Examination. This part of the questionnaire employed the use of Likert-type responses on a five-point scale to answer the questions, with 5 being the highest (5 - Strongly Agree SA, 4 - Agree A, 3 - Undecided U, 2 - Disagree D, and 1 - Strongly Disagree SD). The data for part two of the questionnaire can be found in Table 1. Part three had five questions which addressed issues relating to the CXC mechanical engineering technology report that was published annually with copies made available to the schools that provided candidates for the examination. These five questions were used in this section of the instrument to elicit responses from the respondents pertaining to (a) whether they


Table 1
Percentage Data from Part Two of Questionnaire
Item SA A UD S D M
Exposing students to the features of machines helps when teaching design. 64 36 0 0 0 4.64
Designing and constructing projects motivate students to learn concepts. 64 32 4 0 0 4.60
I am highly motivated to teach design. 60 36 4 0 0 4.56
Solving design problems improves interpretation of questions in exam. 60 36 0 4 0 4.52
Maintaining and repairing equipment enhance design capabilities. 52 48 0 0 0 4.52
Assessing and critiquing design features assists students in learning design. 52 48 0 0 0 4.52
Design should not be excluded from the syllabus. 56 40 4 0 0 4.52
Encouraging students to think critically helps to solve design problems 48 52 0 0 0 4.48
Sketching design ideas in class helps produce similar sketches in exams. 48 48 4 0 0 4.44
Problem solving activities aids learning in design. 48 48 4 0 0 4.44
Using design process in class improves student performance on exam. 48 48 0 4 0 4.40
Familiarization with production in engineering helps in teaching design. 48 44 8 0 0 4.40
My training in College/University prepared me to teach design. 28 56 4 8 4 3.96
Most students I teach perform well on design. 20 60 8 12 0 3.88
Reading and writing capabilities of students affect performance in design. 32 40 4 24 0 3.80
Unavailability of tools and equipment de-motivate students. 28 44 16 4 8 3.80
Design should not be optional in the syllabus. 32 32 8 20 8 3.60
Students motivated to participate in design activities 8 52 24 8 8 3.44
Administration does not provide necessary resources for teachers. 12 48 8 24 8 3.32
Time allotted to teach design is enough to prepare students for exam. 8 16 20 40 16 2.60
Students' ability to learn is not limited by the facilities available. 0 32 8 32 28 2.44

were aware of the report published by CXC, (b) their access to the reports, (c) their reading of the reports, (d) trying suggestions in the report to improve performance, and (e) whether they thought application of suggestions in the report could improve performance on the examination. The respondents were asked to indicate "yes" or "no" to answer the questions (see Table 2).


Table 2
Representative Data from Part Three of Questionnaire
Item Response
Yes No
ƒ P ƒ P
Are you aware of the Mechanical Engineering Technology Report that CXC publishes and sends to your school each year? 22 88 3 12
Have you had access to these reports? 20 80 5 20
Do you often read the reports? 17 68 8 32
Have you tried any of the suggestions included in the report to improve design skills? 18 72 7 28
Do you think an application of these suggestions could improve the performance of your students on the exam? 22 88 3 12
P (%) = ƒ (Frequency)/Total number of respondents (25).

Finally, part four of the instrument provided open-ended questions for the respondents to make comments on the Mechanical Engineering Technology Examination. There were four questions in the section, along with an option for the respondents to make any additional comments that they perceived to be necessary.

Summary of Data

The analysis of the data indicated that teachers were in general agreement on most issues, but there were disagreements on some very critical factors that could have affected student performance on the examination, including student motivation, time allotted to teach the course, availability of tools and equipment, and teaching methodology. The suggestions given in the various reports for improving student performance were not seen by some teachers, as the reports were not made available to them, while others saw the reports but did not read them. The data analyzed might have indicated that the teachers' perceptions on the issues mentioned were varied and that there was a need to address the issues if student performance on the examination were to improve.

The data collected in this study were used to examine teacher attitudes toward and perceptions of student performance on the design component of the Caribbean Examinations Council Mechanical Engineering Technology Examination. The data implied that there was agreement among teachers on most issues relating to items on the questionnaire; but, at the same time, there were also some disagreements. The variance of response indicated that some teachers do not support the requirements of the design component of the CXC Mechanical Engineering Technology Examination curriculum.

The major picture emerging from the results suggests that teachers agreed that factors such as exposing students to features of machines, giving them the opportunity to maintain and repair equipment, and practice in designing and constructing projects would assist in improving the students' performance on the exam. This picture is evident, as the data indicate that the mean score for all responses relating to the factors mentioned were above 4.50. This may suggest that there was strong support for the importance of these factors to the improved performance of students on the exam. Encouraging students to think critically and giving them practice in sketching and solving problems were also seen as being important in assisting students to perform better on the exam.

The teachers' perceptions on the issue of time allotted for preparing students for the exam, as well as whether the students' ability to learn was limited because of the available facilities available, were varied, as the means for the responses were 2.60 and 2.44 respectively, which suggests that support for the item was weak, and there was no clear consensus on the issues.

A majority of the respondents thought that most of the students they taught performed well on the design component of the examination. This was very interesting, as the figures in the various CXC mechanical engineering technology reports indicated that students were performing poorly on that component of the examination. This was a case in which the teachers' perception of what was happening with respect to the performance of their students was not accurate. Was this a case of the teachers' not knowing the capability of their students, or were their perceptions of what was adequate not meeting the requirements of the examination? These are some of the questions that must be answered to address the problem of poor performance among the students on the design component of the examination. Responses indicated that the majority of the respondents were highly motivated to teach the course, while at the same time the majority of respondents in the 35-to-54-years-of age categories indicated that students were highly motivated to participate in the design component of the examination. This may suggest that teachers who were more mature were better able to motivate students to participate in design activities. The data indicated that the majority of respondents in the sample were in the 26-to- 34-years-of-age category, which may suggest that students were not being sufficiently motivated to participate in design activities, or that these teachers need to be provided with training related to motivation.

The majority of the respondents indicated that the lack of adequate tools and equipment made it difficult to motivate students, and that this affected their performance on the design portion of the exam. This may suggest that adequate tools and equipment were not available in some schools, and that students were therefore not motivated to participate in design activities. It may also suggest that if the desired tools and equipment were made available to students, they would more likely be motivated to participate in design activities, thus possibly improving their performance on the exam.

With respect to part three of the questionnaire, 88% of the respondents indicated that they were aware of the mechanical engineering technology school reports published by the CXC annually; 88% also indicated that they thought an application of the suggestions made in the reports could improve the performance on the examination. However, only 68% of the respondents indicated that they had read the reports; and 72% indicated that they had tried some of the suggestions for improving design skills included in the reports. This may indicate that respondents thought that the suggestions made in the report were important and could improve performance on the examination. On the other hand, if teachers did not read the reports, they may not have been aware of the suggestions made to assist in improving performance on the examination. The results also indicated that the reports were not available to all the respondents. If the reports were sent to each school that participated in the examinations, it stands to reason that the administration in the schools concerned were not making the reports available to teachers. The majority of the respondents indicated that the suggestions made in the reports could assist in improving performance on design; if teachers were not exposed to these reports, they would not be aware of the suggestions made for improving performance on the examination.

The majority of the respondents indicated that they were satisfied with the purpose, quality, and relevance of the examination. Their responses indicated that the examination was meeting its objectives and was preparing students for further training or for the workplace. This may indicate that issues relating to the teachers' perception of the examination and what it had to offer students were good. This may also suggest that the poor performance of students on the examination was not a result of a poorly designed examination.

Conclusions and Recommendations

The study revealed that the methodology used to teach students was different in many respects and to a large extent was determined by what was available for use by teachers, and their perceptions of what should be taught and how it should be taught. It was further revealed that in many cases there was a disjuncture in the classroom, as what was expected to be taught was not being taught for various reasons, and that regardless of what was taught, students were expected to satisfy the requirements of standardized achievement tests.

Therefore, based on the interpretation of data relative to this study, the following conclusions were drawn.

  1. Student potential on the Caribbean Examinations Council mechanical engineering technology examination cannot be fully reached unless they are motivated to participate in design activities and are provided with adequate tools and equipment for satisfying the requirements of the syllabus.
  2. Teachers believe that they cannot prepare students' with the experiences necessary to satisfy the requirements of the syllabus unless they have adequate tools, training, and equipment.
  3. The CXC reports will not significantly assist teachers in improving scores unless the teachers receive these reports, the reports are read, and the suggestions for improved performance are observed in the classroom.
  4. Teachers are not receiving adequate feedback from the Caribbean Examinations Council. The perception of some teachers as to how well their students performed on the examination is flawed, as teachers did not seem to be aware of how poorly their students were performing on the design component of the examination.
  5. Teachers believe that the time allotted for teaching the course was not adequate to prepare students for success on the design component of the examination.

Based upon the results of this study, the following recommendations are suggested for further research.

  1. Follow-up research should be conducted to more clearly analyze three questions from the original instrument: (a) Is it important to encourage students to use critical thinking to solve design problems?; (b) Can students learn design by assessing new equipment and critiquing the design features?; and (c) Is it important to expose students to the mechanical features of machines to teach design? The researchers believe that additional follow-up questions that pertain to these original items may provide further information regarding student performance and teacher attitudes on design-based examinations, and that these results may provide useful information for the industrial teacher education field.
  2. An investigation should be done on administrators' awareness of students' performance on the examination. The researchers believe that there may be a perception barrier among administrators, faculty, and students regarding the design-based examination. Results from this study may help administrators in industrial teacher education make informed decisions on examination scores rather than on perceptions of what they believe to be happening in curricular programs.
  3. An investigation should be conducted to determine whether the unavailability of tools and equipment has an affect on student performance on the CXC examination. Respondents from this study believed that the unavailability of tools may have caused poor performance among students—research should be conducted to validate or discredit this claim.
  4. Research should be conducted to determine how teachers actually teach design, both in and out of this geographical region. It would most likely be beneficial to develop a list of "best practices" for teaching design at the postsecondary level. The pedagogy employed in the Caribbean may have been a contributing factor to the low test scores on the design component of the CXC examination.

References

Atkinson, S. E. (1999). Key factors influencing pupil motivation in design and technology. Journal of Technology Education 10 (2) , Spring 1999. Retrieved October 08, 2001. from http://scholar.lib.vt.edu/ejournals/JTE/v10n2/pdf/

Bauer, S. C. (2000). Should achievement tests be used to judge school quality? Education Policy Analysis Archives, 8 (46). Retrieved October 17, 2001, from http://epaa.asu.edu/ epaa/v8n46.html

Caribbean Examinations Council. (1999). Report . The Garrison, St. Michael, Barbados: Author.

Caribbean Examinations Council. (2000a). Industrial technology syllabus . The Garrison, St. Michael 20, Barbados: Author.

Caribbean Examinations Council. (2000b). Report . The Garrison, St. Michael, Barbados: Author.

Davies, T. (2000). Confidence! Its role in the creative teaching and learning of design and technology. Journal of Technology Education 12 (1) . Retrieved October 16, 2001, from http://scholar.lib.vt.edu/ejournals/JTE/v12n1/davies.html

International Technology Education Association. (2000) Standards for technological literacy . Reston, VA: Author.

Kimbell, R. (1982). Design education: The foundation years . London: Routledge & Kegan Paul Ltd.

Mallett, D. (1996). Perceptions of lower secondary design and technology teachers about the utilization of the design process. Proceedings Western Australian Institute for Educational Research Forum. Retrieved October 10, 2001 from http://eduction.curtin.edu.waier/forums/1996/mallett.html
Note : The website provided could not be accessed. The link above goes to http://education.curtin.edu.au/waier/forums/1996/mallett.html .

McCracken, J. R. (2000). Design - The creative soul of technology. In E. Eugene Martin (Ed.), Technology education for the 21 st century (pp. 85 - 90). New York: McGraw-Hill.

Ministry of Education and Culture. (1999). Resource and technology curriculum guide . Kingston, Jamaica: Ministry of Education and Culture.

Owen-Jackson, G. (2000). Learning to teach design and technology in the secondary school . London and New York: Routledge Falmer Taylor & Francis Group.

Popham, J. W. (1999). Why standardized tests don't measure educational quality. Educational Leadership, 56 (6). Retrieved October 18, 2001, from http://www.ascd.org/ readingroom/edlead/9903/extpopham.html
Note : The website provided could not be accessed. The link above goes to http://www.ascd.org/publications/ed_lead/199903/popham.html .

Welch, M., & Lim, H. S. (2000). The strategic thinking of novice designers: Discontinuity between theory and practice. The Journal of Technology Studies, 26 (2) . Retrieved October 10, 2001, from http://scholar.lib.vt.edu/ejournals/JTS/ Summer-Fall-2000/welch.html

Welch, M. (1996, April). The problem-solving strategies of young designers . Paper presented at the Annual Meeting of the American Educational Research Association, New York, NY. (ERIC Document Reproduction Service No. ED401491).

Williams, J. P. (2000). Design: The only methodology of technology? Journal of Technology Education, 11 (2) . Retrieved October 17, 2001, from http://scholar.lib.vt.edu/ ejournals/JTE/v11n2/pdf/williams.pdf

Woods, D. R., & Felder, R. M., Rugaracia A., & Stice, J. E. (2000). The future of engineering education III: Developing critical skills. Chemical Engineering Education, 34 (2), 108-117. Retrieved October 10, 2001, from http:// www2.ncsu.edu/unity/lockers/users/f/felder/public/Papers/Quartet3.pdf


____________________
Daugherty is Professor and Merrill is Assistant Professor in the Department of Technology at Illinois State University in Normal, Illinois. Crossfield is Lecturer at Mico Teachers College in Kingston, Jamaica. Daugherty can be reached at mkdaugh@ilstu.edu. Merrill can be reached at cpmerri@ilstu.edu .