JOTS v24n1 - Assessment Modes, Learning Styles, and Design and Technology Project Work in Higher Education

Assessment Modes, Learning Styles, and Design and Technology Project Work in Higher Education

Clive Mockford and Howard Denton

Mr. Mockford is a Lecturer in the Department of Design and Technology at Loughborough University. Dr. Denton is Program Leader for the Industrial Design and Technology with Education degree in Design and Technology at Loughborough University.

---

Effective assessment strategies enable staff to tailor learning to individuals, to evaluate the appropriateness of the work, and to evaluate their teaching. The management of learning within technology project work is particularly challenging, and so assessment at various stages is needed to obtain the necessary feedback. A significant problem, however, is the effect of the assessment regime on the learning strategies adopted by students. As assessment strategies are made more explicit, there may be a tendency for students to "work the system" rather than focus on the development of the project.

This article reports part of an ongoing project on assessment strategies and the influence on learning styles within technology project work, particularly the effects of peer-group review and self-assessment strategies. The work looks at the potential for encouraging the development of more effective learning strategies at a transitional point in a student's education.

The target group included students in a program of Industrial Design and Technology at Loughborough University, United Kingdom. Fifty students completed a self-reported profile to establish their learning styles as related to technological project work prior to joining the university. From this initial sample 10 students who exemplified certain styles were interviewed. The interviews examined the influence of assessment on learning styles, particularly an assessment strand concerned with peer-group review.

Conducted in the context of a series of technological design projects, we examined changes in learning style and attitudes towards assessment among the student cohort. Three areas of this study were preferred student learning styles in project work prior to university, student motivation in relation to three critical areas of project work activity, and student perceptions of the benefits of using peer review and self-assessment techniques towards modifying their learning styles.

Mechanisms to Assess Learning Outcomes

A number of innovative mechanisms for assessing learning outcomes within technological project work have been developed. These have moved towards including students in the process of assessmentusing peer group assessment, peer group review, or individual peer appraisal (Hodskinson & Patel, 1995) while maintaining staff in the overall position of control. In such a framework the student's contribution to assessment may be valued through its inclusion as a component in the final grade. It is suggested that learning opportunities are more likely to be evident when peer review of technology project work is utilized, conducted in a reflective manner, and set against a non-absolute set of judgments. In this framework the connection between learning and assessment is more likely to be communicated through the interactions of the group and through cognitive processes.

The transition from lower schools to the university is one period of educational change that may correspond with modifications to teaching and learning styles. Managing this transition is important if learners are to adopt increasingly mature and productive styles of working. This is especially so in relation to preparation for employment, where skills of independent thinking, taking responsibility, and time management are highly valued. An understanding of the effects of assessment on learning styles, therefore, becomes particularly important to instructors. There is always a danger of only dealing with easy-to-assess work and failing to take cognizance of more difficult aspects such as group working ability.

Researchers have sought to describe clearly identifiable, qualitative distinctions in student learning styles. The most basic of these use the classifications of "deep learning" and "surface learning" (Entwistle & Ramsden, 1983 ; Marton & Säljö, 1984) . A third classification is "strategic learning" (Entwistle, 1992).

Deep learning is based on high levels of intrinsic motivation, pursuing new ideas and materials through a variety of strategies in the search for understanding. The deep approach is the ideal model for learning, although student performance may not necessarily be recognized in the award of high marks during assessment. The deep approach to learning within a design context would involve:

• Concentration on developing a viable design.
• Taking considerable risks when exploring and developing ideas.
• The use of research strategies to expand knowledge of materials and processes.
• Using organizing principles to integrate ideas.
• Evaluation of the process of design as well as the product throughout the project.
• Relating design decisions to evidence based on the requirements of the task.

Surface learning occurs when the student simply puts in the minimal effort to avoid failure. There is a focus on assessment requirements and an early move to final prototype modeling on the basis of limited design decision making. In a design context this would mean:

• The use of known materials and processes.
• The use of a simple, essentially linear methodology for designing.
• Evaluation of the design prototype largely on the basis of function.
• Limited reflection on design decisions.

Strategic learning is focused towards the product of learning rather than the process and the achievement of high grades. Concentration is towards planning to use time and effort efficiently to achieve specific predetermined outcomes. Organizational behaviors and activities commonly referred to as study skills become increasingly important in the pursuit of a specific goal. The element of competition among students as a key motivator is very strong. In a design context this would mean:

• Concentration on developing a design that gains approval by tutors.
• Developing ideas that match tightly the product requirements.
• The use of research to collect materials rather than to acquire and reflect on information.
• Using a tightly focused design strategy to produce outcomes that match assessment criteria.
• A focus on the outcome of the design process as opposed to the process of designing.
• Relating design decisions to the likely success of the proposal in terms of assessment.

If students move towards surface and strategic learning styles in reaction to assessment systems, there can be a degradation in the learning experience. Opportunities for creative thinking can be reduced or even lost if the focus of learning moves towards assessment and attainment is measured only against stated performance criteria. What can emerge is a student who seeks to please staff by judging what is the preferred design style or practical outcome required. In this learning framework students are unlikely to engage their minds deeply in an active, yet considered, reflective exploration for new ways of doing things: they will stay within the guidelines of what output is required to satisfy the instructor and the stated assessment criteria. In the search for more effective design and technology teaching, assessment strategies that encourage students towards the opposite of this characteristic, namely a deep approach to learning, can offer considerable gains in learning.

Of specific importance in this small scale study was the transitionary learning periods between lower secondary, upper secondary, and tertiary education. First year undergraduate students arrive at the university having established what they perceive to be successful learning strategies. Incorporated within the individual learning strategy is the equally complex and largely inaccessible strategy that is used when approaching technological design problems. The correlation between learning style and design style could be considered to be significant. Hence, influencing learning style could affect the design strategy adopted and the quality of the design outcome. The corollary could also be valid: developing a more mature style of designing may promote a deeper learning style.

The Approach Taken

This study sought to investigate the learning style, as applied to design project work, that had developed among specific students across transitionary phases of education. Three phases of educational experience were covered as part of the interviews with researchers. These are shown in Figure 1.

The method involved two stages: a self-report profile completed by 50 students and interviews with 10 students selected from the results of the profiles and who completed a questionnaire.

Figure 1 . Chronological periods of education and transitions investigated during interviews with students.

Age band	Examination Period
School 14 to 16	General Certificate of Secondary Education Design, Technology subject
	Learning Transition
School 16 to 18	General Certificate of Education Advanced Level Design, Technology subject
	Learning Transition
University 18 to 22	Undergraduate program Bachelor of Arts/Science Industrial Design and Technology with Education

Table 1
Selection of Students for Interview (Data sourced from a goal orientation index)

	Better in these aspects of goal orientation (higher score)		Worse in these aspects of goal orientation (lower score)
Group 1	ACTION		PLANNING	REFLECTION
A B C D E F	133 132 127 118 127 126		94 99 99 77 114 117	118 108 86 63 109 99
Group 2	PLANNING	REFLECTION	ACTION
G H J K	107 116 124 97	125 140 143 111	122 126 133 113

The self-reported profile examined student perceptions of their competencies in three key technological areas: the action of starting projects; moving ahead with projects to formulate a technological product proposal; and reflecting upon the technological process and outcome of a project. This instrument was used as the primary selection mechanism. It was developed from work examining the goal orientation of pupils in school (Atman, 1994) . Two types of students were sought according to the two selection bands: those good at acting, poorer at planning and reflecting, and those good at planning and reflecting, poorer at acting. In this way only students who represented the two extremes regarding self-reported preferences for different stages of design activity were interviewed. The results of the target group of 10 students is shown in Table 1. Note that the students in Group 1 were selected as those showing high action scores by comparison to planning and reflection whereas the students in Group 2 were selected as those showing higher scores on planning and reflection in relation to action.

Data from interviews gave a picture of the general pre-university position in relation to student learning styles in technological project work. The interviews also concentrated on examining the influence of assessment on learning styles. Discussion was stimulated through the use of an outline form, Figure 2, on which both the researcher and the student were encouraged to write and draw. It proved a valuable research instrument, particularly when used in conjunction with transcripts of the individual interviews. Of specific benefit was the diagrammatic additions and annotations that confirmed an overall approach to the subject of design and technology during the three transitionary phases considered.

A questionnaire was administered to the target group of 10. It aimed to identify their perception of the benefits of peer appraisal of practical work and design folio work. The results are shown in Figure 3 (see PDF version of article).

The Meaning of the Results

Profiles obtained from theself-report profile (goal orientation data) for the target group of 10 students are shown in Table 1. Individual students are identified as A to K.

Questionnaire results concerning the perceptions among the target group of the benefits of peer appraisal of practical work and design folio work are shown in Figure 3.

The results from interviews are more difficult to quantify. A table summarizing the significant and consistent features of each of the three phases of technology education covered is shown in Table 2.

The process of interviewing subjects was intended to provide insights concerning predominant learning styles. This information was sought by an examination of approaches to designing across three transitionary periods of education. Through discussion about design activity, observations were made concerning general learning style, although further work would need to be conducted to validate these connections.

A distinctive feature of the design work, and possibly the learning style, was a lack of strategic activity. At General Certificate Secondary Education (age 16) and Advanced Levels (age 18) of study, students reflected characteristics that could largely be considered to be nonstrategic in their orientation. Students did not spend time scrutinizing or discussing the assessment framework for the technology project work, nor the criteria used for assessment.

However, an element of strategic activity could be identified at the school level. This appeared to be orchestrated by teaching staff, without the tacit agreement of pupils. All interviewees reported teachers showing exemples of technology project work and directing the on-going work to meet these guidelines. It was clear that the teacher was often acting in a strategic manner, directing pupil operations towards a specific goal that would be successful in gaining marks for assessment.

Figure 2 . Approaches to designing: strategies and changes. (see PDF version of article)

Figure 3 . Questionnaire responses indicating perceived benefits in learning as a result of engaging in peer appraisal of technology project work. (see PDF version of article)

When discussing the transition between school and university, it was clear that the design style that had developed could usually be classified as deep in nature. This might be expected for students who were now studying a vocationally oriented course in industrial design. However, an element of strategic activity was evident. This was linked to two key indicators of strategic thinking: student effort to be invested in a project and student time allocation for a project. This makes good sense from the student perspective when used as a mechanism for managing a very considerable workload.

From the results of the interviews, a tentative link between preferred learning styles and design styles is proposed. This is based on the two learning styles described earlier in this paper as deep and strategic. A deep learning style can be linked with positive project engagement, limited referencing to assessment criteria and the desire to produce workable design proposals. Indicative of a strategic style of learning can be the matching of output to individual assessment criteria, where a workable design proposal is less likely to be generated.

In connection with peer appraisal, the data collected from questionnaire responses gave a clear picture of the learning benefits of this assessment dimension. The general indication is that it can exert a direct influence on student learning styles, encouraging a deep approach. Peer appraisal and review can be seen to have a significant effect on student perceptions of their learning, and as a result influence their individual learning style. However, it is acknowledged that a balanced framework for assessment is required; no single strand or strategy can dominate the process.

Major Concerns

The interview results show that initial student learning styles in a new educational context largely reflect strategies that have been successful in achieving high examination grades during previous periods of education. These styles of learning are not necessarily appropriate to either in-depth study of a subject area or a more independent learning environment. The strategies that students adopt in technology project work are subject to considerable change during this transition. A period of adaptation can result, reducing the effectiveness of learning. Peer appraisal of folio work during the activity of designing and also the critical evaluation of practical outcomes can be considered to be a very significant catalyst in compressing this period of change.

It is also evident that design strategies that students use during design project work should be revised and developed to become more individually tailored as students move through the age-related phases of education. This maturation cannot necessarily occur in a steady progression; rather, specific changes tend to occur following the transition between primary, secondary, and tertiary levels.

As the assessment style and criteria for assessment change become more prominent in the process of education, a corresponding reduction in overall creative freedom and thinking can result. Assessment can dominate the activity, with the result that learning is focused towards largely surface and strategic approaches that allow students to obtain positive grades for their work. Indeed, assessment is frequently seen by students as disconnected from the process of learning through technological project work. Including peer appraisal and review into the assessment framework can stimulate a move to what can be considered to be a deeper style of designing and learning.

Table 2
Summary of Consistent Interview Comments

Age/Phase	Summary of interview comments from students
GCSE Age 14 to 16	Students report that assessment is not overtly stated or studied during this phase Staff often show work of previous candidates to indicate "the standard" required The work is very structured by staff: a sequential framework of research,           design, make, and evaluate Student perception of effort put into design work                           = 20% Student perception of effort put into making (30% on other aspects) = 50% Actual time students spend on design work                                     = 20% Actual time students spend on making (20% on other aspects)          = 60%
Advanced level Age 16 to 18	Students report an increase in detail as opposed to complexity of technology   projects There was an increase in the amount of research conducted Staff emphasize "get the design right before making" Students report "design approach rammed into you" by staff Assessment made little impact on individual student design strategies
Undergraduate level Age 18/19 in sample	Students report less differentiation between stages of designing Design work seen as more continuous, more iterative Assessment balance provided by staff and used by students to inform selves on   the speed for each aspect of a project Peer appraisal seen by students: Enables you to see what others are doing Enables you to pick up on what others are doing Feels good to know what others feel of your work An opportunity to reflect on staff assessment

References

Atman, K. S. (1992) . Curriculum implications of goal accomplishment style for design technology education. In J. S. Smith (Ed.), IDATER94 International Conference on Design and Technology Education Research and Curriculum Development: Keynote lectures . Loughborough, United Kingdom: Loughborough University of Technology.

Entwistle, N., Thompson, S., & Tait, H. (1992). Guidelines for promoting effective learning in higher education. University of Edinburgh, Scotland: Centre for Research on Learning and Instruction.

Entwistle, N. J. (1992) . Influences on the quality of student learningImplications for medical education. South African Medical Journal, 81, 596­606.

Entwistle, N. J., & Ramsden, P. (1983) . Understanding student learning . London: Croom Helm.

Hodskinson, M., & Patel, R. (1995, July) . More experience with design teams and peer assessment. Paper presented at the second National Conference on Product Design, Coventry University, United Kingdom.

Fisher, R. (1990). Teaching children to think . Oxford, United Kingdom: Blackwell.

Marton, F., & Säljö, R. (1984) . Approaches to learning. In F. Martin, D. Hounsell, & N. J. Entwistle (Eds.), The experience of learning . Edinburgh, Scotland: Scottish Academic Press.

Richardson, J. T. E. (1994). Cultural specifity of approaches to studying in higher education: A literature survey. Higher Education , 27 , 449-468.

TS