JITE v35n4 - A Comparison of College Students' Achievement Following Traditional and Integrated Media Presentations
A Comparison of College Students' Achievement Following Traditional and Integrated Media Presentations
William L. Havice
Clemson UniversityMany schools are currently in the process of changing from industrial arts curricula, centered in wood and metal shops, to technology education courses taught in labs with technology resources such as computer-aided-drafting (CAD), computer-aided manufacturing (CAM), and computer-aided presentation hardware and software. Teachers in this new computer technology-intensive environment need to continue to develop their computer skills in order to be successful.
As more and more activities in the classroom are delivered with the use of computers, teachers are realizing that these tools are more complex and more capable than other media such as filmstrips or overheads. Computer systems are increasingly being used to integrate what once were stand-alone devices (e.g., video and audio). Educators are introducing more and various forms of software and computer-driven media into their classroom activities ( Tolhurst, 1995 ).
Hypertext, hypermedia, multimedia, and integrated media are examples of the kinds of organizational, retrieval, and presentation systems that are being incorporated. These systems make information available at the stroke of a computer key. These innovative presentation systems are creating a great deal of enthusiasm among educators. At the same time, there is confusion as to how to best incorporate these systems into existing pedagogy ( Allred & Locatis, 1989 ; Anglin, 1995 ; Balajthy, 1990 ; D'Ignazio, 1989 ; Marchionini, 1988 ).
Hypertext, hypermedia, multimedia, and integrated media were developed to provide ways of storing, retrieving, organizing, and presenting information. Thus, information can be presented in a non-sequential or non- linear manner. Furthermore, presenters or readers can move through a complex network of information including references, graphics and text in the manner best suited to their needs. In integrated media presentation systems, users can read linearly across the surface of the information, as well as move non-sequentially into the material to define new terms, clarify misconceptions, or search for additional information. Hofstetter ( 1993 ) stated that hypermedia can enhance classroom activities by enabling students to explore and discover new information while being guided by their teacher. The basic concept of non-sequential (hypertext), computer-based information retrieval is not new; it was first envisioned more than 40 years ago ( Bush, 1945 ).
New opportunities for pedagogical change abound as computers are becoming a component of classroom learning environments. Moreover, computers, with their related peripheral devices, have become more accessible to increasing numbers of students. Using integrated media systems for presentations allows teachers to literally bring the world into the classroom with the stroke of a key. Integrated media systems can link text to other text (hypertext), still pictures, dynamic video, audio clips, or networks anywhere in the world, just to name a few possibilities ( Hofstetter, 1993 ).
Instructional Methods
Teachers have a variety of methods from which to choose that enable them to actualize instructional objectives (Hensen, 1988 ). Kim and Kellough ( 1987 ) identified two "avenues" to be considered when selecting methods for teaching content or processes. The first is the "delivery mode," whereby the instructor sees that information is delivered to the students. This is referred to as "the traditional or didactic mode where knowledge is passed on to the learners via the teacher, or from content reading in a textbook, or both" ( p. 202 ). The second avenue identified for selecting an instructional methodology is the "access mode," where the teacher provides students access to "information and experiences" so "they develop knowledge and skills" ( Kim & Kellough, 1987, p. 202 ).
Kim and Kellough ( 1987 ) defined the strengths of the traditional delivery mode as (a) time efficient, (b) controllable by the teacher, c) predictable, and d) manageable student learning. These four strengths can also be viewed as weaknesses of the access mode. The weaknesses of the delivery mode are that it tends to stifle creative thinking, does not address students' self-concepts, minimizes student involvement in decision-making, and lacks intrinsic sources for student motivation. Again, as the characteristics of both teaching modalities are compared, the weaknesses of the traditional delivery mode tend to be strengths of the access mode.
Traditional Instruction
Zenger and Zenger ( 1990 ) defined the traditional lecture as "an oral presentation given to a class by the teacher" ( p. 31 ) while Ericson ( 1960 ) noted that the lecture or "telling" method is the method of teaching outside of manipulative work. Teachers are often comfortable with the traditional method because it enables them to remain in control of content and time ( Kim & Kellough, 1987 ).
Advantages of pedagogical techniques are (a) ability to maintain teacher control, (b) usefulness in introducing new materials, (c) can be used with other teaching techniques, and (d) is efficient for presenting to large groups as well as content areas containing many facts ( Kim & Kellough, 1987 ; Zenger & Zenger, 1990 ).
Some distinct disadvantages are also associated with traditional instructional techniques. These include (a) student boredom, (b) difficulty in accounting for individual differences, (c) the necessity of teachers having superior speaking skills, and (d) the difficulty of producing learning transfer to new situations. Other problems include (a) one-way communication (teacher to students), (b) lack of enthusiasm and student involvement, (c) lack of motivation for extra or advanced learning, and (d) lack of development of concepts and other aids leading to true understanding ( e.g., Kim & Kellough, 1987 ; Zenger & Zenger, 1990 ). For most of their lives, college and university students have been in educational systems that have used the lecture method of delivering instruction.
Integrated Media
Considerable confusion has surrounded the precise meaning of the terms multimedia, hypermedia, and integrated media since there is no generally accepted definition for any of them ( Tolhurst, 1995 ). Often working definitions are developed that are dependent on the perspective or disciplinary background. For example, cognitive psychologists may define these terms based on the effects on human learning, while computer scientists will tend to focus on user interfaces. As is the case with new and rapidly emerging fields, the terms and their applications are still evolving. The following definitions and applications of these terms were designed to frame the discussion from the perspective of educators. Multimedia. The term multimedia is not new. In fact, according to Kozma ( 1991 ), it has been used for several decades and only recently has been linked to the use of computers. At a basic level, multimedia simply specifies the use of multiple media formats to present information. Multimedia definitions do not include nonlinear links between information and it becomes interactive only when it is orchestrated using a computer.
Hypertext. Theodor Nelson, a self-proclaimed computer visionary, coined the term hypertext. As a young Harvard graduate student, Nelson used the term to describe a concept of non-sequential writing. He believed that the computer was too restrictive, lacking the ability to link the information on the screen or the ability to branch in other directions. He envisioned hypertext as a solution to speedy access of all the world's literature using a network link that would be available from any computer ( Fiderio, 1988 ). Tripp and Roby ( 1990 ) define hypertext as a nonlinear, multidimensional, semantic structure in which words are linked by associations. According to Norman ( 1976 ), memory and hypertext have some commonalties. Associations that we make about an idea are composed of a set of attributes. This set of attributes could be described as "schema." Schema relate to memory in the same way that nodes are a part of hypertext.
Hypermedia. In addition to multimedia and hypertext solutions, an innovation being incorporated into classrooms today is hypermedia. Hypermedia implies multiple forms of communications media, which are all controlled, coordinated, and integrated by the microcomputer. Hypermedia is an extension of hypertext that integrates graphics, animation, audio, and video with text ( Heller, 1990 ; Marchionini, 1988 ; Tsai, 1989 ). The most important characteristic of hypermedia is its potential for encouraging students to be proactive learners. Hypermedia provides "a tool that…immediately gratifies students' intellectual curiosity" ( McCarthy, 1989, p. 27 ).
Integrated Media Instruction
Boone and Higgins ( 1991 ) referred to multimedia, hypertext, hypermedia, and integrated media as terms with overlapping concepts. From their perspective, these new terms and definitions are best presented and examined collectively. As one examines the definition of multimedia and hypermedia, this overlap becomes obvious. The Cognition and Technology Group at Vanderbilt University ( 1993 ) defined integrated media as the linkage of text, sound, video, graphics, and the computer in such a way that the user access to these various media is non-linear and virtually instantaneous.
Integrated media instruction is used in higher education in many different ways. For example, integrated media instruction is used at the front of the classroom to support presentations, as a more flexible and versatile upgrade of traditional audio-visual media. This technology can also be used by student groups for collaborative learning as well as by individual students for independent learning ( Lynch, 1993 ).
Statement of the Problem
As we look to the 21st Century, educators will be challenged to develop new ways of managing, presenting, and utilizing information. One of the electronic systems that is being incorporated into classrooms today to assist educators is integrated media. Integrated media implies multiple forms of communication media, controlled, coordinated, and integrated by the microcomputer. This could include a combination of text, graphics, writing, still or motion video, and audio as well as hypermedia ( The Cognition and Technology Group of Vanderbilt Learning Technology Center, 1993 ). At the present, rapid changes are taking place without sufficient time to develop or analyze trends or to research the effectiveness of traditional methods of instruction compared with new integrated media delivery methods. This study was designed to measure the effectiveness of integrated media as an instructional presentation tool.
Purpose of the Study
The special purpose of this study was to explore whether or not there was a difference in achievement of college students when integrated media were used as the method of presentation, compared to the traditional instructional methods during an introductory computer information systems class.
Method
This study utilized a quasi-experimental design, known as the non- equivalent control group design ( Campbell & Stanley, 1963 ). The study involved comparisons between a treatment group, which in this study received integrated media instruction presentations as a treatment, and a control group, which received traditional instruction presentations. Both groups were given a pretest and a posttest to measure achievement of the course objectives. To strengthen the design of this study, the groups were compared on pretest scores. The dependent measure was the gain between pretest and posttest achievement scores for students enrolled in Introduction to Computer Information Systems #101. The independent variable was the treatment.
Subjects
All participants in this study were enrolled at a small state university in the Midwest. The course, Introduction to Computer Information Systems, was distributed across seven sections. The total enrollment was 367 students. Each section had approximately 52 students and represented a variety of majors.
The course was a required general education course for all undergraduate students. The prerequisite was (a) a Mathematics ACT score of 12 or better, or (b) one university-level mathematics course. No previous computer experience was required. The basic purpose of the course was to introduce students to computers and information systems. The course content included the history, operation, and potential use (or misuse) of computers in diverse fields. Students were introduced to common software applications programs. Control and treatment groups were established for this study. The control group consisted of 206 students, while the treatment group contained 161. Both groups were from intact classes. The treatment group was composed of students who were instructed using integrated instructional media materials. The control group was instructed utilizing traditional instructional materials. Participants were expected to complete all of the same course requirements for the introductory computer information systems course.
Computer Literacy Assessment
The faculty in the department of computer information systems and quantitative methods at the university developed and used a pretest and posttest to assess student competencies in the course. The test was composed of 60 questions, evenly divided among general computer knowledge, vocabulary, word processing, database, and spreadsheets. In assessment terms, the proficiency examination was a minimum competency measurement designed to differentiate between those who had mastered basic requisite skills from those who had not. The scores reported for the three semesters prior to this experiment, for both the control and treatment group instructors, indicated an average standard deviation of 5.0. No report of reliability was provided by the department. However, the investigator calculated a split-half reliability score including a Spearman-Brown correction, which yielded a reliability coefficient of .72 ( Gay, 1992, p. 169 ).
Treatment
Integrated Media Presentation Group Subjects in the integrated media group received instruction from the instructor who used an IBM-compatible computer in conjunction with a transparent active matrix color liquid-crystal (LCD) panel. The LCD panel was placed on an overhead transparency projector, and the PC display shown on the screen. A VCR was used to display video segments via the color panel. Computer application problems were reviewed in class on the computer prior to each assignment. These included applications for Procomm, Internet, DOS, LOTUS 1-2-3, Wordperfect 5.1, and dBASE III+. Selected students performed computer application problems on the computer during reviews. Class presentation concepts included the use of Asymmetrix ToolBook, Microsoft PowerPoint for Windows and Microsoft Visual Basic 3.0. This software was selected because the instructor was familiar with the products and because technical support was available. The presentations were interactive in terms of student requests to "repeat" information or to "see more" information. Field trips to selected computer facilities were videotaped by the instructor and viewed in the class. Some material was presented in computer-based video and some viewed via VCR.
The DOS operating system, MS windows, MS PowerPoint, and PODIUM Presentation Manager software ( Hofstetter, 1993 ) were used to manage the integrated media presentations. PODIUM is a hypermedia presentation software that enables presenters to use a text editor or word processor to develop hypermedia presentations. This software allows the instructor to place any picture on the computer screen, including 35 mm slides, flat art, video, computer graphics, and clip art. Text files, pictures, waveform audio files, animation sequences, digital video files, and application programs can be linked to one another. Furthermore, linkages are possible between any multimedia object, including color images, audio sound-tracks, and full-motion video. This presentation system has distinct hypertextual capabilities by providing linkages between any line of text to any other text ( Hofstetter, 1993 ). Students also utilized the textbook and particated in laboratory activities.
During class presentations the instructor typically used the Windows environment to have more than one application program operating at a time. For example, when demonstrating the electronic mail system, the instructor would have e-mail running in one window, while at the same time, text and graphics were being displayed in another. This provided the instructor with the ability to do an activity on Internet while simultaneously displaying text and graphics that supported the presentation.
Traditional Instruction Group
The traditional instruction group prepared for the Introduction to Computer Information Systems posttest using traditional textbook, lecture, and lab activities. Course presentations were not supported with or delivered by any electronic integrated media.
The instructors for the traditional instructional group utilized the lecture format with only limited opportunities for active student participation. The chalkboard was the only tool that was used to illustrate the class presentations. The instructors spent most of the time lecturing and presenting material from the text. Considerable time was also spent going over study questions and reviewing test materials before and after the tests.
Data Analysis
A chi-square goodness of fit analysis was computed to mathematically examine the homogeneity of the study groups. No statistically significant difference was detected (p<.05) between the control and treatment groups on any of the demographic factors. These demographic factors included age, gender, ethnic background, employment status, classification in school, major field of study (technical or non-technical), self-reported grade point average, level of education, self-perceived level of computer knowledge, and attitude toward computers. This lack of significant differences indicated that the groups were statistically equivalent.
An analysis of variance (ANOVA) general linear model for non- equivalent groups was used to test the significance of difference between the means of the control and treatment group results on the pre and post achievement tests ( Gay, 1992 ).
Achievement
The research question asked: Is there a statistically significant difference in the achievement between those students instructed in a traditional manner and those students instructed through integrated media in Introduction to Computer Information Systems classes? To address this question, an achievement difference was calculated as the difference between the pretest and posttest scores on the Computer Literacy Assessment tool. The independent variable was the treatment.
Results
There was a statistically significant difference in achievement between those students instructed in a traditional manner (control group) and those instructed through integrated media (treatment group) (p<.05). To measure this, pre- and posttest scores on a Computer Literacy Assessment were compared for both groups.
Tables 1 through 5 show the results of the ANOVA that was used to analyze the performance differences (gain) between the treatment and control groups. Table 1 shows that the students in the treatment group (N = 118) attained a mean of 27.72 and a standard deviation of 6.85, whereas those in the control group (N = 149) attained a mean of 33.39 with a standard deviation of 6.13 on posttest scores.
Tables 2 and 3 show that both groups of students attained similar pretest scores. Students in the treatment group attained a mean of 23.74 with a standard deviation of 5.89 and the control group attained a mean of 24.93 with a standard deviation of 5.80.
Table 4 reveals that there was a significant difference between the control and treatment groups' gain in their respective pretest and posttest achievement scores. Table 5 shows that the mean gain for the treatment group was 3.98 with a standard deviation of 4.77 and the mean gain for the control group was 8.46 with a standard deviation of 5.75. Therefore, the control group showed higher achievement gain between the pretest and posttest scores (p<.05).
Table 1 Means and Standard Deviations of Computer Literacy Assessment Posttest Scores
Group N M SD
Treatment 118 27.72 6.85 Control 149 33.39 6.13
Table 2 ANOVA: Computer Literacy Assessment Pretest Scores, for Treatment and Control Group
Source of
VariationSS df MS F
Between Groups 94.13 1 94.13 2.76 Within Groups 9034.18 265 34.09 Total 9128.31 266
General Linear Models Procedure, for unequal groups p=n.s.(0.098)
Table 3 Means and Standard Deviations of Computer Literacy Assessment Pretest Scores
Group N M SD
Treatment 118 23.74 5.89 Control 149 24.93 5.80
Table 4 ANOVA: Computer Literacy Assessment Pretest Scores, for Treatment and Control Group
Source of
VariationSS df MS F
Between Groups 1317.70 1 1317.50 46.20* Within Groups 7558.93 265 28.52 Total 8876.64 266
General Linear Models Procedure, for unequal groups *p<.0001
Table 5 Means of Computer Literacy Assessment Gain Scores
Group N M SD
Treatment 118 3.98 4.77 Control 149 8.46 5.75
Discussion
This study was designed to examine the effectiveness of two instructional methods in teaching Introduction to Computer Information Systems on the measure of student achievement. Previous research suggests that integrated media presentations can be superior to traditional instruction. However, the results of this study did not support these findings. The posttest scores indicated that students instructed in a traditional manner showed a higher achievement gain between the pretest and the posttest scores (p<.05). The differences in achievement test scores found in this study support previous research. Schramm ( 1977 ) stated that "learning seems to be affected more by what is delivered than by the delivery system" ( p. 273 ). Clark and Sugrue ( 1995 ) found in their meta-analyses that the method of instruction rather than the choice of medium leads directly and powerfully to learning. In other words, it is the method not the medium that influences the psychological processes that allows learning to take place. Furthermore, in a review of classroom technology since 1920, Cuban ( 1986 ) found that many researchers felt the teachers' inability to adjust their teaching styles to get the most out of the new technologies was the reason for technology's failure. Early attempts to reform education with the use of technology failed because reformers underestimated the importance of the teacher in the classroom ( Hannafin & Savenye, 1993 ). According to Hannafin and Savenye ( 1993 ), changes in teaching and learning are prerequisite for changes in technology integration. Hofstetter ( 1993 ) supports these observations from the perspective of student interest, noting that it is insufficient to simply use technology to access information related to the topic being discussed. Sophisticated technology will not work if it is not directed at engaging students' interest beyond the hardware. This may necessitate smaller class sizes in order to facilitate the integrated media-supported presentation. It is important at this juncture to comment on some factors that contributed to the outcomes of this study, including those having to do with the classroom environment, preparation, and setup, as well as procedure used to deliver the mediated approach.
Environment
One of the key environmental factors that affected the quality of the mediated approach had to do with the lack of light in the room. When the instructor would dim the lights so that the presentation system would work, the room was so dark the students had a difficult time seeing to write notes. While the design of this study was not specifically focused on identifying the effects of environmental factors, it appeared that a lack of sufficient lighting negatively impacted the effectiveness of the mediated approach. It is important to note that computer projection technologies are changing rapidly to address this problem, through the development of high intensity, projection devices that can be operated under normal lighting conditions.
Preparation and Setup Procedures
Some preparation and setup factors also appeared to affect the study. The logistics were such that the instructor using the mediated approach had to bring in (wheeled in on a cart) and configure the hardware at the beginning of every session. There were a significant number of times when minor problems were experienced with the setup, which led to some frustration for both the instructor and students. A related problem occurred during instruction when there were delays as the instructor searched for information stored in the computer.
While it is likely that these problems with the treatment are reflected in the outcomes of this study, it is important to note that setup and configuration difficulties are important factors that must be addressed in order for mediated instruction to work effectively. As software systems become more intuitive and as instructors become more comfortable with and knowledgeable about media-driven systems, problems of the type experienced in this study will likely be reduced. At the same time, the dynamic nature of instructional technologies is likely to require solid computer support services for some time to come.
Industrial and technology education programs should be designed to prepare students for careers in teaching and industry. To be appropriately prepared, given the major advances that are being made with information technologies, it is critically important that students learn about and develop skills in the use of multimedia-based technologies in the schools. Teacher educators that model the use of computer integrated technology in their classrooms will help their students develop the skills necessary to future success.
Computer skills are rapidly becoming a necessity for today's industrial and technology educators. Faculty in these programs are in a perfect position to take a lead role in developing successful applications and using computer technology for classroom activities. Using integrated media for presentations can allow teachers to bring the world to the classroom with the stroke of a key and link text to other text, still pictures, dynamic video, audio clips, or to networks anywhere in the world, just to name a few possibilities ( Hofstetter, 1993 ). To make this a reality, educators must continue to investigate appropriate ways to introduce new technologies into the classroom.
Author
Havice is Professor, Department of Technology and Human Resource Development, Clemson University, Clemson, South Carolina.
References
Allred, K., & Locatis, C. (1989). Research, instructional design, and new technology. Journal of Instruction Development , 11(1), 2-5.
Anglin, G. (Ed.). (1995). Instructional technology: Past, present, and future (2nd ed.). Englewood, CO: Libraries Unlimited, Incorporated.
Balajthy, E. (1990) . Hypertext, hypermedia, and meta-cognition: Research and instructional applications for disabled readers. Reading, Writing, and Learning Disabilities , 6, 183-202.
Boone, R., & Higgens, K. (1991). Hypertext/hypermedia information presentation: Developing a hypercard template. Educational Technology , 31(2), 21-30.
Bush, V. (1945). As we may think. Atlantic Monthly , 176, 106-107.
Campbell, D., & Stanley, J. (1963). Treatment and quasi-treatment designs for research . Boston, MA: Houghton Mifflin Company.
Clark, R., & Sugrue, B. (1995) . Research on instructional media, 1978-1988. In G. J. Anglin (Ed.), Instructional technology: Past, present, and future (2nd edition). Englewood: Libraries Unlimited, Inc. (pp. 348-364).
Cognition and Technology Group at Vanderbilt University. (1993). Examining the cognitive challenges and pedagogical opportunities of integrated media systems: Toward a research agenda. Journal of Special Education Technology , 12(2), 118-124.
Cuban, L. (1986) . Teachers and machines: Classroom use of technology since 1920 . New York, NY: Teachers College Press.
D'Ignazio, F. (1989) . Welcome to the multimedia sandbox. The Computing Teacher , 17(1), 27-28.
Ericson, E. E. (1960) . Teaching the industrial arts (3rd ed.). Peoria, IL: Chas. A. Bennett Company, Incorporated.
Fiderio, J. (1988, October) . A grand vision. Byte , 237-243.
Gay, L. R. (1992) . Educational research competencies for analysis and application (4th ed). New York, NY: MacMillan Publishing Company.
Hannafin, R. D., & Savenye, W. C. (1993) . Technology in the classroom: The teachers' new role and resistance to it. Educational Technology , 33 (6), 26-31.
Heller, R. S. (1990, Summer) . The role of hypermedia in education: A look at the research issues. Journal of Research on Computing in Education , 431-441.
Hensen, K. T. (1988) . Methods and strategies for teaching in secondary and middle schools . New York, NY: Longman.
Hofstetter, F. T. (1993) . Multimedia presentation technology . Belmont, CA: Wadsworth.
Kim, E. C., & Kellough, R. D. (1987) . A resource guide for secondary school teaching: Planning for competence (4th Ed.). New York, NY: Macmillan.
Kozma, R. B. (1991) . Learning with media. Review of Educational Research , 61(2), 179-211.
Lynch, P. J. (1993, March/April) . Interactive media enlivens learning. Computing Technology for Higher Education , 2(3), 8-11.
Marchionini, G. (1988) . Hypermedia and learning: Freedom and chaos. Educational Technology , 28(11), 8-12.
McCarthy, R. (1989). Multimedia: What the excitement's all about. Electronic Learning , 8(8), 26-31.
Norman, D. A., Gentner, S., & Stevens, A. L. (1976). Comments on learning schemata and memory representation. In D. Klahr (Ed.), Cognition and instruction . Hillsdale, NJ: Lawrence Erlbaum Associates.
Schramm, W. (1977). Big media, little media . Beverly Hills, CA: Sage.
Tolhurst, D. (1995). Hypertext, hypermedia, multimedia defined? Educational Technology , 35(2), 21-26.
Tripp, S. D., & Roby, W. (1990). Orientation and disorientation in a hypertext lexicon. Journal of Computer Based Instruction , 17(4), 120-124.
Tsai, C. (1989) . Hypertext: Technology, applications, and research issues. Journal of Educational Technology Systems , 17(1), 3-14.
Zenger, S. K., & Zenger, W. F. (1990) . Strategies and techniques for teaching . Saratoga, CA: R & E Publishers, Incorporated.
Reference Citation: Havice, W.L. A comparison of college students' achievement following traditional and integrated media presentations. Journal of Industrial Teacher Education, 35(4), 29-43.