Analysis of Five Instructional Methods for Teaching Sketchpad to Junior High Students

Geoffrey Wright, Ronald Terry, and Scott Bartholomew

This manuscript addresses a problem teachers of computer software applications face today: What is an effective method for teaching new computer software? Technology and engineering teachers, specifically those with communications and other related courses that involve computer software applications, face this problem when teaching computer software designed to assist in graphic design, web design, programming, robotics, etc. The question of what instructional method would prove most effective is one that affects not only teachers but also IT trainers, as computers and computer software applications continue to be the primary tools of work and leisure. Despite the increase in computer software application use, the associated literature on instructional techniques used to teach computer software is inconclusive in regard to which instructional methods are the most appropriate for teaching new software, especially in junior high technology and engineering classrooms.

A review of literature related to technology teaching, software application instruction, instructional methodologies, and other related topics was performed as part of this research study. However, because the focus of this paper is to describe and analyze teaching a new computer software application to junior high students using five different instructional methods, the literature review will focus on a description of the five instructional techniques used in this study.

Related Literature

A review of literature related to technology teaching, software application instruction, instructional methodologies, and other related topics was performed as part of this research study. However, because the focus of this paper is to describe and analyze teaching a new computer software application to junior high students using five different instructional methods, the literature review will focus on a description of the five instructional techniques used in this study.

Instructional Methods

Throughout history, there have been many instructional methods documented ( Egal, 2009 ). The literature associated with technology education reveals five commonly used, cited, and recommended instructional methods. These methods include: (1) direct instruction , (2) problem-based learning , (3) video-based tutorial learning , (4) cooperative/collaborative learning , and (5) book/written script tutorial learning .

Direct Instruction

Direct instruction, a term first coined by Rosenshine (1976) , is a teacher-centered instructional method ( Schuman, 1998 ) that typically follows a process in which teachers present new information followed by classroom activities that incorporate structured, guided, and independent student practice. While many research studies have found direct instruction to be an effective instructional strategy ( Bock, Stebbins, & Proper, 1977 ), the recent push towards hands-on, student-centered curricular activities has resulted in direct instruction becoming less popular with many teachers ( Magliaro, Lockee, & Burton, 2005 ).

Problem-based Learning

Problem-based learning is an instructional strategy in which problems form the organizing focus and stimulus for student learning. Distinguishing features of problem-based learning include teachers accepting the role of facilitators and students assuming major responsibility for their learning as they engage in problem-solving activities. Students are typically presented with problems and then work in small, self-directed learning groups to investigate and develop solutions to given problems ( Barrows, 1996 ). While problem-based learning can be difficult to implement in the classroom ( Liu, 2004 ), benefits from problem-based learning include development of higher-level thinking skills ( Duch, 2001 ), long term content retention ( Norman & Schmidt, 1992 ), better attitudes toward learning, higher motivation ( Albanese, 1993 ; Norman & Schmidt, 1992 ), and the development of students problem solving skills ( Gallagher, 1997 ; Hmelo & Ferrari, 1997 ).

Video-based Tutorial Learning

With the popularity of computer-based instruction, video-based tutorials as a means to learn various software programs have become commonplace as is evidenced with a quick YouTube search of most major software programs. The perceived advantage of this instruction method is that students are able to watch, review, and utilize lesson recordings in whatever manner best suits their educational needs. Some studies have reported positive findings in relation to video-based tutorials, reporting a greater ability for students to construct, or discover, their own knowledge ( Bork, 2000 ) or that foreign students with weaker language skills prefer Web-based tutorials to traditional class lectures ( Sweeney & Ingram, 2001 ). However, Merino and Abel (2003) reported findings, which are consistent with other studies, that there was no statistical significant difference in student learning when comparing video tutorials and traditional lectures.

Cooperative/Collaborative Learning

In a cooperative or collaborative learning structure, students work in small groups to accomplish a task and are usually rewarded based upon the performance of the group. Deutsch (1962) first conceptualized the three types of interpersonal goal structures that are typically used in classrooms: cooperative, competitive, and individualistic. These goal structures specify the type of interdependence that exists among students as they strive to accomplish learning goals. While many teachers agree that there is a time in which each of the goal structures should be appropriately used, research ( Johnson, & Johnson, 1995 ) indicates that students participating in cooperative learning environments perform as well or better than students in competitive and individualistic learning environments on measures of achievement and attitudes toward learning.

Book/written Script Tutorial Learning

In typical text tutorials, students are expected to read the text, answer key questions posed to them in the text, and retain the knowledge for future use. The addition of images, graphs, and iconic cues has increased the effectiveness of textbook learning ( Winn, 1987 ; Kamil, 2010 ; Guri-Rozenblit, 1988 ).

Methodology

This study included using each of the instructional techniques described above to teach a new computer software, Sketchpad , to a sample of technology education junior high students and then analyzing the impact that each technique had on student learning by giving them an assignment to use Sketchpad to design a CD cover of a band or artist of their choice.

Students

The students in our study were between the ages of 11 and 13 and were registered in a public junior high school 7th or 8th grade Intro to Technology class. Intro to Technology is part of the Utah CTE (Career and Technology Education) core classes that are designed to introduce students to technology and allow exploration of technological systems and their impacts on society ( Utah State Board of Education, 2010 ). Demographic information such as grade point average, socioeconomic status, computer experience, and computer-based multimedia program experience was collected. This information was used to ensure that the sample size was homogeneous.

Teachers

Schools and teachers were selected because teachers had a similar number of years teaching, facilities were comparable, student demographics were similar, and teachers had multiple periods of the Intro to Technology course. Each teacher was assigned one of the methods identified in the literature review as the method of instruction to use when teaching the new program to the students. Teachers were asked to adhere strictly to their assigned instructional method while involved in this study. Teaching styles were assigned randomly to teachers who were provided an explanation of the teaching style, definitions, examples, outlines, and associated procedures as a guide for their teaching experience. The authors recognize that there may be a reliability issue or limitation in assigning and expecting a teacher to properly use the assigned teaching style. However, video recordings of the teacher using the method were made and later analyzed by three education professors who verified that the teaching methods were appropriately implemented. Teachers were provided with cameras and recorded for approximately 90 minutes. Teachers positioned the camera such that the majority of the class was visible and teacher-student interactions were captured digitally. This verification process helped reduce this limitation.

Software Program

The software program to be taught needed to be new and unfamiliar to all student participants. Sketchpad is an online image creation and editing software developed by Mugtug, an online community dedicated to the development of free online programs for image editing and creation. Sketchpad was chosen because it: (a) is a program similar to other image-editing programs typical to the multimedia industry; (b) is advertised as easy to use; (c) has buttons, effects, and options similar to other multimedia programs; and (d) has a relatively small number of tools and options, which provided for a smaller learning curve. Sketchpad is a strictly online program, requiring no download, and allows for an easy download of the finished product upon completion.

Prior to the study, it was confirmed that this software had never been taught to the student participants. The software was taught for two 90-minute class periods. Although some might argue this is insufficient to establish a cause and effect relationship between instructional methodology and the outcome, this time allotment was appropriate for this study, as it fit within the typical amount of time that each teacher reported they used for introducing software. For example, in the reporting of the demographic information, each teacher reported that they usually spend 1-2 class periods (60 minutes each) introducing the basics of a software. Concerning this, one teacher clarified, “I usually spend only 1-2 classes teaching the students the basics of the new tool (ex. Adobe Illustrator), and then in following classes, students work on their projects using the tool. I find a brief intro suffices for my students.”

Data Collection

Data was collected in multiple ways: (1) students and teachers completed a survey regarding their perceptions of the effectiveness of different types of instructional methods, (2) teachers were video recorded while teaching the software ( Sketchpad ) using the assigned method of instruction, (3) students created a CD cover for an artist or band of their choice using the program taught in class, and (4) student work was graded by a panel of 20 graders. The panel consisted of students and professor from a college-level graphic design course. The average grade of each product was used as the reported data point.

Surveys

Students completed a Likert-based survey prior to creating the CD cover. The survey questions included items such as: How much experience do you have with multimedia programs on the computer? How familiar are you with computers?

One key element of the self-report survey was asking students what they believe is the most effective method for teaching a new computer software application. Students were given options and definitions for each method of instruction. Students were asked to differentiate effectiveness of teaching methods for themselves and for their classmates. Students were also asked to identify the method that they perceived the teacher in their class used most frequently.

Teachers also completed a survey prior to teaching the students. The teacher survey consisted of 20 questions about issues regarding teaching experience, class size, technology equipment use, teaching style, education, and multimedia program experience. Teachers were asked to answer each question while considering only the specific class the study was being conducted in. These responses were analyzed to ensure that teachers were similar and that each teacher had a broad base of technology education experience to draw from.

Teachers were also asked to identify personal tendencies, preferences, and effectiveness in using different methods for their classroom. Teacher responses were compared with student responses to determine what relationship teacher–student perceptions have in regard to instructional methods.

The teacher survey also contained questions relating to their students’ grade point average, socioeconomic status, computer experience, multimedia program experience, and average class assignment grade. These results were cross-analyzed with similar questions posed to students to verify data validity and reliability. These results also helped ensure that items such as student computer experience and average grade on assignments were comparable for different classrooms involved in the study.

Classroom Instruction

Each teacher was assigned an instructional method to teach Sketchpad . Teachers were provided with a definition of the method of instruction and asked to strictly adhere to this method of instruction. Teachers introduced the assignment to students, introduced the associated rubric, and gave the students a timeline for completion. Teachers were given a copy of the rubric outlining how the final CD covers were to be graded. Teachers then taught Sketchpad to the students. Each teacher completed the study during the course of two class periods (90 minutes), while video recording the instruction.

Teachers video recorded themselves while teaching. The video recordings were watched by three education professors who used a specific rubric to ensure that the assigned teaching techniques were indeed the actual technique used. The professors unanimously reported that there were no deviations.

Student Assignment

Students were given the grading rubric and description of the assignment before working on the computer. Then students were taught Sketchpad before they did the assignment. Students were given 60 minutes to produce the CD cover either by themselves or in a group, depending on the assigned method of instruction. As part of the study, students were informed that their participation in the survey and study would have no impact on their grade. Student work was graded at a later date according to the provided rubric by a panel of graders with design background.

Grading

Twenty students and a professor from a college-level design course graded the student work. Graders received a copy of the rubric and assignment instructions to assist them in grading. Each student-produced CD cover was assigned a grade on a Likert scale from 1-5 by each of the graders. Graders were blind to the student name, class, and instructional method. Student scores were compiled from each grader, and an average score for each student and then each class was obtained. The average grade received by students from each class was compared with the instructional method used in that class in an attempt to identify effectiveness of each method.

Data Analysis

Student demographic information was analyzed to ensure similar populations, similar familiarity with technology and computers, and similar experience in multimedia classroom settings. The average scores for student work in each class was collected and compared with the method of instruction provided, resulting in an average score for each method of instruction. Additionally, surveys for teachers and students were collected and cross-analyzed. The student’s perceptions of methods used in the classroom were compared with the methods identified by the teachers in an effort to identify similarities and disparities in perceptions of instructional methods. Data was aggregated for statistical analysis. Two specific measures of significance were performed with regards to the data—a t-test and an effect-size test. Although the authors believe the t-test and effect-size test were appropriate for this study, they recognize that statistical power is directly related to sample size. Because the sample of this study was limited to 87 student participants, the authors believe the findings are limited to helping create only an understanding regarding teaching Sketchpad to 7-8th grade junior high technology education students, but is not telling for a larger population. Consequently, additional research should be done to further corroborate these findings.

Findings

The most prevalent findings of this study are: (a) teachers and students have different perceptions about the effectiveness of different instructional techniques, (b) teachers and students have different perceptions regarding frequency of use of instructional methods in class, and (c) student perceptions of higher instructional effectiveness did not correlate with higher grades received for the assignment.

Student and Teacher Perceptions about Effectiveness of Instructional Techniques

There is a disconnect between what teachers and students perceive as effective instructional techniques: (1) students perceive book learning to be the most effective method of instruction for themselves and their classmates, and (2) teachers perceived direct instruction to be the most effective method of instruction and book learning to be the least effective method of instruction.

Student Perceptions

Students perceive book learning to be the most effective method of instruction for themselves and their classmates, ranking book learning above all other forms of learning in effectiveness for their classmates’ learning (Table 1). The variance between responses showed statistical significance ( t = 2.57, 4.01, 4.06, 3.6).

Table 1
Student Ranking of Effectiveness of Instructional Methods for Their Classmates’ Learning

Instructional Method	Mean Score
Book/Written Script Tutorial Learning	3.04
Problem-Based Learning	2.76
Direct Instruction	2.63
Collaborative Learning	2.57
Video-Based Tutorial Learning	2.55

The difference between the two highest ranked methods (book learning and problem-based learning) was .28 (3.04-2.76), suggesting statistical significance ( t = 2.57). This means that students not only perceive book learning as most effective for their classmates but the gap between book learning and the next most effective method (problem-based learning) is significant—suggesting an important difference for respondents between the effectiveness of each method of instruction (Table 2).

Table 2
Statistical Analysis of Student Ranking of Instructional Methods for Their Classmates’ Learning

Data Sets Compared	Mean	Std. Deviation	t	d	r	r 2
Book/Problem-Based	3.04; 2.76	1.285; 1.02	2.57	.24	.12	.014
Book/Video	3.04; 2.55	1.285; 1.33	4.01	.37	.18	.032
Book/Collaborative	3.04; 2.57	1.285; 1.19	4.06	.38	.19	.032
Book/Direct	3.04; 2.63	1.285; 1.148	3.6	.34	.17	.029
Problem-Based/Direct	2.76; 2.63	1.02; 1.148	1.28	.12	.06	.003
Problem-Based/Video	2.76; 2.55	1.02; 1.33	1.90	.18	.09	.008
Problem-Based/Collaborative	2.76; 2.57	1.02; 1.19	1.83	.17	.09	.008
Video/Direct	2.55; 2.63	1.33; 1.148	.69	.06	.03	.001
Video/Collaborative	2.55; 2.57	1.33; 1.19	.17	.02	.01	.002
Direct/Collaborative	2.63; 2.57	1.148; 1.19	.55	.05	.03	.001

When book learning was compared with each of the other identified teaching methods, it was the only method to show statistical significance in the average mean difference in every comparison (i.e., book learning compared with video tutorial learning, book learning compared with direct instruction, and so forth). No other method had such statistical significance.

Several possible reasons could be cited for this perception. First, books often include images, graphs, screenshots, step-by-step instructions, and other tools that may assist the learning of a new computer software application. Although video tutorials can provide similar media content, books allow students the ability to tangibly hold the instructional material and go at their own pace of learning. A book can be easily consulted for questions and can help the reader to access needed information quickly and repeatedly if needed ( Kamil, 2010 ).

Second, it is possible that student perception is skewed by the common practice of book learning, and students simply assume that book learning is the best way because that’s what they perceive most of their teachers use. Up through and including junior high, textbooks are the “primary mediator of learning” for students in and outside of the classroom ( Kamil, 2010 ).

The third possibility for this finding is that, developmentally, junior high students are not quite ready to be self-learners (where they no longer need as much teacher-led learning). In Perry’s (1970) theory of intellectual and moral development, Perry states that students begin their development “trusting authority figures” at a young age, but they later seek to know the “right answer” on their own as they mature. At the junior high level, students are still in the very beginning stages of intellectual and moral development, which may be the reason students perceive book learning as so effective—it’s a built-in authority figure that they can reference whenever needing to find the “right answer.”

Students were also asked to identify the effectiveness of instructional methods for their own learning. Although learning styles were not taken into account in this research study, this question did allow students to independently identify which method(s) of instruction they believe is (are) most effective for their own learning. Students were not instructed to think about any one particular class or subject in reference to this question.

Students reported that they believed book learning is the most effective method of instruction for their own learning (Table 3). Similar to the previous question, students were not asked what method of instruction they preferred, but rather what method of instruction they perceive as most effective for their own learning. The difference in average scores of effectiveness for book learning when compared with each other method was statistically significant ( t = 2.64, 4.54, 3.17, 2.93). Additionally, when compared for educational significance (Table 4, next page) each variance for book learning compared to other forms of learning showed educational significance ( d = .25, .43, .3, .27).

Table 3
Student Ranking of Effectiveness of Instructional Methods for Their Own Learning

Instructional Method	Mean Score
Book/Written Script Tutorial Learning	3.02
Problem-Based Learning	2.71
Direct Instruction	2.66
Collaborative Learning	2.63
Video-Based Tutorial Learning	2.45

Table 4
Statistical Analysis of Student Ranking of Effectiveness of Instructional Methods for Their Own Learning

Data Sets Compared	Mean	Std. Deviation	t	d	r	r 2
Book/Problem-Based	3.02; 2.71	1.34; 1.16	2.644	.25	.12	.014
Book/Video	3.02; 2.45	1.34; 1.34	4.54	.43	.21	.044
Book/Collaborative	3.02; 2.63	1.34; 1.29	3.17	.3	.15	.02
Book/Direct	3.02; 2.66	1.34; 1.29	2.93	.27	.14	.02
Problem-Based/Video	2.71; 2.45	1.34; 1.34	2.07	.19	.1	.01
Problem-Based/Direct	2.71; 2.66	1.34; 1.29	.41	.04	.02	.0004
Problem-Based/Collaborative	2.71; 2.63	1.34; 1.29	.651	.06	.03	.001
Video/Direct	2.45; 2.66	1.34; 1.29	1.71	.16	.08	.006
Video/Collaborative	2.45; 2.63	1.34; 1.29	1.46	.14	.07	.005
Collaborative/Direct	2.63; 2.66	1.34; 1.29	.244	.02	.01	.0001

This is an important finding because in the high-tech, fast-paced, and increasingly digital world, students still perceive book learning as more effective than learning from a video tutorial. The availability of video tutorials and online videos in general has increased dramatically in the past 10 years (Tew, 2007), but, despite the increased availability, students in this study ranked book learning as more effective than video tutorials. Not only did students rank book learning as more effective than video tutorials but students ranked video tutorials as the least effective method of instruction.

Although students believe working alone in a book based environment for the purposes of learning a new software application is most effective, students do not appear to think working in groups is completely ineffective. The data suggests that group work (collaborative learning) is considered effective as long as they are working with a common problem (problem-based learning) in mind.

It is equally important to note that students in this study ranked the effectiveness of instructional methods for themselves in the exact same order as they reported for their classmates. Although no learning style preferences were considered in this study, the data suggests that students perceive personal and peer learning styles to be similar.

Teacher Perceptions

Teachers perceived direct instruction to be the most effective method of instruction and book learning to be the least effective method of instruction. In addition to student perceptions regarding most effective learning methods, teacher’s perceptions were recorded and analyzed. Teachers were asked to rate the identified methods according to their perceived level of effectiveness in their class. Teachers used a 5-point Likert-type scale when ranking each method of instruction from 1 ( not effective ) to 5 ( very effective ).

The findings reveal that teachers believe direct instruction is superior to the other methods of instruction; not surprisingly, the teachers also reported that they most commonly use direct instruction in class (see table 5).

Table 5
Teacher Ranking of Effectiveness of Instructional Methods for Student Learning

Method of Instruction	Mean Score
Book/Written Script Tutorial Learning	2
Problem-Based Learning	2.6
Collaborative Learning	2.6
Video-Based Tutorial Learning	2.8
Direct Instruction	4.6

Converse to what students reported to be the most effective instructional style, teachers believed that book learning is the least effective method of instruction for students. The difference in mean score for direct instruction when compared with other forms of instruction (Table 6, next page) returned a t-test value of 5.09, 4.27, 3.53, and 2.55—suggesting a statistically significant teacher preference towards direct instruction. The effect size for each comparison was likewise significant ( d = 3.22, 2.7, 2.23, 1.61).

Table 6
Statistical Analysis of Teacher Ranking of Effectiveness of Instructional Methods for Student Learning

Data Sets Compared	Mean	Std. Deviation	t	d	r	r 2
Direct/Book	4.6; 2	.55; 1	5.09	3.22	.85	.72
Direct/Problem-Based	4.6; 2.6	.55; .89	4.27	2.7	.8	.64
Direct/Collaborative	4.6; 2.6	.55; 1.14	3.53	2.23	.75	.56
Direct/Video	4.6; 2.8	.55; 1.48	2.55	1.61	.63	.4
Collaborative/Book	2.6; 2	1.14; 1	.88	.56	.27	.08
Collaborative/Video	2.6; 2.8	1.14; 1.48	.24	.15	.08	.01
Collaborative/Problem-Based	2.6; 2.6	1.14; .89	0	0	0	0
Book/Video	2; 2.8	1; 1.48	1	.63	.3	.09
Book/Problem-Based	2; 2.6	1; .89	1	.4	.2	.04
Video/Problem-Based	2.8; 2.6	1.48; .89	.26	.16	.1	.01

Student and Teacher Perceptions about Instructional Methods Used in the Classroom

A comparison was performed of student perceptions of instructional methods used in class and teacher perceptions of instructional methods used in class. Two themes were discovered: (1) students perceived book learning to be the most commonly used method of instruction used in class and direct instruction to be the least commonly used method; (2) conversely, teachers reported using direct instruction the most and book learning the least. This finding is interesting because it shows a disconnect between student and teacher perceptions. Each of these issues is discussed below.

Student Perceptions

Students perceived book/written script tutorial learning to be the most commonly used instructional method in class (Table 7, next page). Strangely, students perceived direct instruction, which provided the highest grades for students, to be the least commonly used method of instruction.

Table 7
Statistical Analysis of Student Ranking of Frequency of Use of Different Instructional Methods in Class

Data Sets Compared	Mean	Std. Deviation	t	d	r	r 2
Book/Problem-Based	3.078; 2.939	1.401; 1.156	1.158	.11	.054	.003
Book/Direct	3.078; 2.52	1.401; 1.315	4.395	.411	.2	.04
Book/Collaborative	3.078; 2.86	1.401; 1.33	1.71	.16	.08	.01
Book/Video	3.078; 2.73	1.401; 1.33	2.73	.254	.13	.02
Problem-Based/Direct	2.939; 2.52	1.156; 1.315	.88	.56	.27	.08
Problem-Based/Video	2.939; 2.73	1.156; 1.33	.24	.15	.08	.01
Problem-Based/Collaborative	2.939; 2.86	1.156; 1.33	.68	.06	.03	.001
Video/Direct	2.73; 2.52	1.33; 1.315	1.7	.16	.08	.01
Video/Collaborative	2.73; 2.86	1.33; 1.33	1.05	.1	.05	.003
Collaborative/Direct	2.86; 2.52	1.33; 1.315	2.75	.26	.13	.02

Students perceived teachers as using book learning more than any other method of instruction in class ( t = 1.158, 4.39, 1.71, 2.73) and much more than direct instruction ( t = 4.39). Also, students perceived their teachers as using books to teach materials far more frequently than videos or other multimedia.

Teacher Perceptions

While students reported book learning to be the most commonly used method of instruction in class and direct instruction to be the least commonly used method, teachers reported the opposite—reporting using direct instruction far more than any other method of instruction (see table 8, next page). Teachers ranked book learning, which was ranked by the students to be the most used technique, to be the least used method.

Table 8
Teacher Ranking of Frequency of Use of Different Instructional Methods in Class

Method of Instruction	Mean Score
Book/Written Script Tutorial Learning	2.4
Problem-Based Learning	2.4
Collaborative Learning	2.8
Video-Based Tutorial Learning	2.4
Direct Instruction	4.6

Instead, the teachers ranked direct instruction as being used significantly more than any other method (4.6 average rating compared with 2.8 for collaborative learning, which was ranked second). When compared with the other methods (Table 9) of instruction the variance was statistically significant in each comparison ( t = 4.7, 4.7, 3.29, 2.8). When compared for an effect size, educational significance was also found in each scenario ( d = .83, .83, .72, .66). The difference in student and teacher perceptions is alarming when considering that students and teachers both show strong leanings about which method of instruction is most effective.

Table 9
Statistical Analysis of Teacher Ranking of Frequency of Use of Different Instructional Methods in Class

Data Sets Compared	Mean	Std. Deviation	t	d	r	r 2
Direct/Book	4.6; 2.4	.55; .89	4.7	2.97	.83	.69
Direct/Problem-Based	4.6; 2.4	.55; .89	4.7	2.97	.83	.69
Direct/Collaborative	4.6; 2.8	.55; 1.09	3.29	2.09	.72	.52
Direct/Video	4.6; 2.4	.55; 1.67	2.8	1.77	.66	.44
Collaborative/Book	2.8; 2.4	1.09; .89	.64	.4	.2	.04
Collaborative/Video	2.8; 2.4	1.09; 1.67	.45	.28	.14	.02
Collaborative/Problem-Based	2.8; 2.4	1.09; .89	.64	.4	.2	.04
Book/Video	2.4; 2.4	.89; 1.67	0	0	0	0
Book/Problem-Based	2.4; 2.4	.89; .89	0	0	0	0
Video/Problem-Based	2.4; 2.4	1.67; .89	0	0	0	0

Effectiveness of Instructional Methods in Teaching Sketchpad

Each student produced a CD cover using the software application taught in class. Students were given approximately 60 minutes to create their CD cover and turn it in electronically (85 CD covers were graded in total). A panel of 20 graders with design background graded the student work. Graders were blind as to the method of instruction received and graded student work on a 1-5 Likert scale. A grading rubric was provided to the graders.

Student grades for each group were combined and a class average grade was obtained (Table 10). Each class average was compared and analyzed to determine how effective each method of instruction proved to be in respect to the grade given. Student perceptions of higher instructional effectiveness did not correlate with higher grades received for the assignment. In fact, the data show that students receiving direct instruction scored higher than any other method of instruction. When compared with other methods of instruction (Table 11, next page) a significant difference in variance between scores for students receiving direct instruction and those receiving other instructional methods was shown for multiple comparisons ( t = 2.65, .45, 2.63, .95).

Table 10
Average Grade Received by Students—Separated by Instructional Method Used

Instructional Method Received	Average Grade
Direct Instruction	3.02
Problem-Based Learning	2.95
Book/Written Script Tutorial Learning	2.87
Video-Based Tutorial Learning	2.49
Collaborative Learning	2.43

Table 11
Statistical Analysis of Average Grade Received by Students—Separated by Instructional Method Used

Data Sets Compared	Mean	Std. Deviation	t	d	r	r 2
Direct/Collaborative	3; 2.43	.48; .52	2.654	1.13	.49	.232
Direct/Problem-Based	3; 2.95	.48; .32	.447	.122	.06	.002
Direct/Video	3; 2.48	.48; .70	2.63	.866	.40	.016
Direct/Book-written	3; 2.86	.48; .24	.9478	.369	.18	.032
Book/Collaborative	2.86; 2.43	.24; .53	2.41	1.05	.46	.211
Book/Problem-Based	2.86; 2.95	.24, .32	.774	.32	.16	.025
Book/Video	2.86; 2.48	.24; .70	1.71	.73	.34	.116
Video/Collaborative	2.48; 2.43	.70; .53	.195	.08	.04	.002
Video/Problem-Based	2.48; 2.95	.70; .32	2.63	.86	.40	.16
Problem-Based/Collaborative	2.95; 2.43	.32; .53	3.08	1.19	.51	.26

The combined validity of multiple tests ( t -test, Cohen’s d ) adds weight to the assertion that direct instruction appears to be more effective than collaborative learning or video-based tutorials in helping students score higher when taught a new computer software application at the junior high level. In summary, despite teacher and student perceptions regarding effectiveness and frequency of use of different instructional methods, direct instruction proved to produce the best grades for students when taught a new computer software application.

Conclusions and Recommendations

Based on the findings from this study, several conclusions and recommendations can be generalized for application by teachers of computer software applications like Sketchpad . Three are discussed below.

Use of Book Learning at the Junior High Level

Teachers need to involve the use of book learning—especially at the junior high age level. At the junior high level, students are still in the early stages of their own intellectual development (Perry, 1970), and students want (or are used to) an authority for everything they do. Students want to have someone tell them the “right way” of doing each thing and the “right answer” for each question they encounter (Perry, 1970). A book is also another authority figure in the classroom—the book can be a source of “right answers” and “right ways” for students when the teacher is not available. Books can provide a constant stream of hints, tips, tricks, and steps for students to follow as they learn new software programs. Because students can take books home, students can use them to learn on their own time, at their own pace, and in any desired location.

Another aspect of learning that is critical to students at the junior high level is praise and positive feedback. At the junior high level, as at all age levels, there is a need for reinforcement and praise—often this praise and reinforcement comes as a confirmation that one is doing the right thing, following the steps correctly, and has achieved a short-term goal along the way. When student methods, answers, or products resemble what is outlined in the book, the student receives a small measure of “praise” as they reaffirmed that their learning corresponds with what was intended.

It may be difficult for many teachers to institute and effectively use books in their classrooms; lack of books, lack of excitement for books (by the teacher or the students), and other factors may make book learning difficult in some settings. A possible alternative to a textbook is a packet for each assignment. A packet of instructions could be copied for each student and used as a reference for students to refer to throughout instruction and the process of learning.

Understanding Student Perception of Classroom Teaching Practices

Teachers need to understand the perceptions of their students in regard to the teaching practices used in the classroom. Teachers must consciously and consistently evaluate their own teaching practices and seek to understand the perceptions of their students. An understanding of student perceptions will help inform teachers regarding their instructional effectiveness and teaching methods used (Hicks, 2010). As shown in this study, frequently teacher perceptions of instructional methods being used do not match with methods perceived by students.

Teachers should explicitly ask their students about techniques used in class to discover student perceptions and not rely solely on self-evaluation techniques for discovering effectiveness of instructional methods. Video recordings and post-teaching analysis ( Wright, 2008 ) have been shown as effective in improving teacher cognition of methods used and improving teaching effectiveness. A simple survey, questionnaire, or even an open discussion with students could also provide such feedback for a teacher.

Improving Direct Instruction Techniques

In this study, direct instruction provided the highest average student grade for the assignment and was reported by teachers to be the most effective instructional method. Teachers also reported using direct instruction significantly more than any other method of instruction. Conversely, students perceived direct instruction to be the least used method of instruction in class. Students also ranked book learning and problem-based learning as more effective than direct instruction for their own learning and their classmates learning. Teachers must find ways to improve the perception of direct instruction in the eyes of students by improving their own direct instruction techniques

---

Geoffrey Wright ( geoffwright@byu.edu ) is Assistant Professor, Steve Shumway ( steve_shumway@byu.edu ) is Assistant Professor, Ronald Terry ( ron_terry@byu.edu ) is Professor and Program Coordinator, and Scott Bartholomew ( sbartholomew@alpinedistrict.org ) is M.S. in the College of Engineering Technology and Engineering Education at Brigham Young University.

References