JITE v41n4 - Considerations for Embedding On-Line Components into Traditional Classroom Environments

Considerations for Embedding On-Line Components into Traditional Classroom Environments

Klaus Schmidt
Dan Brown
Illinois State University

The use of on-line instructional delivery techniques such as instructional web sites, on-line communication tools, and multimedia productions to deliver classes is a rapidly growing practice in education. It has been suggested that most universities offer some courses using on-line technology ( Beller & Or, 1998 ). Recent literature related to effectiveness of and satisfaction with on-line instruction focused on reasons to implement on-line instruction as a replacement for traditional face-to-face instruction and on comparing student achievement and attitudes toward this approach to delivery. Various reasons have been noted for delivery of instruction on-line.

Beller and Or (1998), and Johnson, Aragon, Shaik, and Palma-Rivas (2001) discussed the possibility that on-line learning increased enrollments due to the convenience encountered by students at a distance. Expectations of cost effectiveness for both students and universities were discussed by Beller and Or , and by LaRose, Gregg, and Eastin (1998) . It has also been suggested that on-line learning has the potential to enhance individual student learning ( Kandies & Stern, 1999 ; LaRose, Gregg, & Eastin, 1998 ). Lastly, Sanders and Morrison-Shetlar (2001) discussed the importance of on-line learning as a means to assure student familiarity with computer technologies.

Research thus far has primarily focused on identifying factors that lead to success in on-line classes as an alternative to traditional instructional delivery. However, what has not occurred to date is sufficient exploration of those factors in the context of on-line instruction embedded in an otherwise traditional instructional environment.

Findings of previous studies have indicated that students participating in classes entirely delivered through electronic distance learning formats have no significant differences in learning when compared to students taking classes in traditional classroom settings ( Johnson, Aragon, Shaik, & Palma-Rivas, 2001 ; LaRose, Gregg, & Eastin, 1998 ; Swan & Jackman, 2000 ). Students participating in studies of an entire class delivered on-line seemed to enjoy the convenience of asynchronous class delivery. Further studies indicated that students have been mixed in their level of satisfaction with teacher interaction when enrolled in on-line courses ( Chester & Gwynne, 1998 ; Johnson, Aragon, Shaik, & Palma-Rivas; LaRose, Gregg, & Eastin ) and with their ability to create communities of learning with other dispersed students through tools used in on-line delivery ( Benbunan-Fich & Hiltz, 1999 ; Johnson, Aragon, Shaik, & Palma-Rivas ).

Further, findings on courses delivered electronically indicated that enjoyment and control of pace were significant factors ( Fitzelle & Trochim, 1996 ). Christensen, Anakwe, and Kessler (2001) found that students whose attitude toward the usefulness of technology was positive were significantly more receptive to on-line instruction than those students with a negative attitude toward the usefulness of technology. They also found that technology familiarity was positively related to receptivity for on-line instruction, and that technology accessibility was not significantly related to receptivity for on-line instruction.

Studies examining student attitudes and learning related to web-enhanced instruction found that students participating in a web-enhanced class had mixed levels of satisfaction with teacher interaction and with reactions to their ability to create learning communities through on-line tools ( Sanders & Morrison-Shetlar, 2001 ; Benbunan-Fich & Hiltz, 1999 ). Sanders and Morrison-Shetlar found females to be significantly more positive toward web-enhanced instruction than males. In neither study did age relate significantly to attitude toward web-enhanced instruction. Benbunan-Fich and Hiltz found that asynchronous learning networks enhanced the product quality in collaborative learning projects.

Bandura (1994) described self-efficacy as beliefs about personal capabilities to perform in ways to exercise influence over events that affect our lives with respect to cognitive, motivational, affective, and selection processes. Self-efficacy in this study was operationally defined as representing student sense of being personally in control of factors such as time, pace of instruction, and the technology (software and computer hardware). Self-efficacy has been suggested as an important factor in students' efforts to use and succeed with electronic technologies ( Eastin & LaRose, 2000 ; Kandies & Stern, 1999 ).

Purpose of Study

Knowledge of how student a priori experiences with computer technologies and applications correlate to student perceptions of and success with embedded on-line learning experiences could enhance decisions about the design and delivery of on-line components embedded in traditional classroom instruction. For the purpose of this study, student success was operationally defined as achieving learning objectives as indicated by test and project scores. This knowledge could also begin to provide the basis for development of a useful pre-assessment instrumentation to assist instructors in identifying which students are most likely to be successful in achieving learning objectives with embedded on-line instruction. More specifically, it could be useful to better understand how students react when selected portions of course material are delivered on-line to support instruction in an otherwise traditional classroom. With that in mind, the following research questions were asked.

1. What student characteristics and a priori experiences correlate with student perceptions of satisfaction with on-line learning experiences embedded in a traditional classroom environment?
2. What student characteristics and a priori experiences correlate with student success in achieving learning objectives when participating in on-line learning experiences embedded in a traditional classroom environment?

Methodology

At the beginning of each semester, prior to introduction of the embedded on-line instructional segment, a pretest instrument was administered to university technical students who volunteered to participate in the on-line instructional activity. The opportunity to participate in this study was offered to students during two consecutive semester sessions of an introductory industrial computer systems class.

The pretest instrument consisted of four parts. Part one consisted of the 10-item scale developed by Compeau and Higgins (1995) to measure computer self-efficacy. Computer self-efficacy has been described as the individual's judgment of or belief in his/her own capacity to use a computer and is believed to influence personal decisions about what behaviors to attempt, as well as persistence ( Compeau & Higgins ). This scale had been validated in a regression study involving more than 1000 participants. Individuals who were higher in computer self-efficacy were found to experience less anxiety about attempting a new computer application, and to derive more enjoyment from use of the computer ( Compeau & Higgins ).

Part two measured the construct described as cognitive playfulness utilizing the seven-item Microcomputer Playfulness Scale developed and validated by Webster and Martocchlo (1992) , involving more than 400 participants over multiple studies. The scale resulting from this research measures the trait of cognitive playfulness in the context of microcomputer use. The computer playfulness scale (CPS), defined as representing "the degree of cognitive spontaneity in microcomputer interactions" ( Webster & Martocchlo , p. 201), focused on seven characteristics of cognitive spontaneity: sense of feeling spontaneous, imaginative, flexible, creative, playful, original, and inventive when using a personal computer. This research suggested that individuals who were higher in the trait microcomputer playfulness would be more spontaneous, imaginative, and inventive in their microcomputer interactions. It was further suggested that the trait described as high cognitive computer playfulness can be motivating and relate positively to learning in the context of computers; significant positive correlations were found between learning outcomes and computer playfulness ( Webster & Martocchlo ).

Part three consisted of two scales of six items each designed to assess the constructs of perceived usefulness and perceived ease of use. These scales were developed, refined, and validated by Davis (1989) . Cronbach alpha reliability for the usefulness scale was reported at r = .98 and for the ease of use scale at r = .94. Perceived usefulness is defined as the degree to which individuals believe a technology will help them perform better. Perceived ease of use was defined as the degree of difficulty an individual believes would be required in order to use a technology. Both scales exhibited high convergent, discriminant, and factorial validity; thus, construct validity was demonstrated. Both ease of use and usefulness scale results were found to have strong correlations with actual use of technologies, but the usefulness scale was found to be a better predictor of future use ( Davis ).

Cronbach alpha scores were calculated for the scales comprising parts one, two, and three of the pretest instrument. Cronbach alpha for part one of the instrument, computer self-efficacy, was r = .907; and for part two, computer playfulness, r = .914. Cronbach alpha for the first half of part three, computer usefulness, was r = .892, and for the second half of part three, computer ease of use, r = .948.

Part four was created by the researchers and collected information about previous student familiarity with computers and specific software applications. Some of these software applications included web browsers, word processors, e-mail, and presentation software.

Following the completion of the embedded on-line instructional segment, a second questionnaire, previously designed by Ryan, Hodson Carlton, and Ali (1998) to specifically assess student perceptions about on-line instruction, was administered to the students who participated in the three-week on-line learning experience. Ryan, Hodson Carlton, and Ali determined the reliability of this instrument using Cronbach's alpha to be r = .82 for the WWW module scale and a test-retest procedure. The Likert-type items on the second questionnaire ranked from (1) strongly agree to (5) strongly disagree. This questionnaire solicited student perceptions of the on-line learning environment. Specifically, the questionnaire included items about student perceptions on how well and the extent to which:

• Content was covered,
• Interaction was evident (among students and between students and faculty),
• Student participation was facilitated,
• Communication skills were required,
• Critical thinking was required,
• Time was appropriate for assignments,
• Faculty preparation and expertise was important, and
• Technical skills were required

In addition, learning outcomes were assessed via both on-line tests and performance-based exercises throughout each three-week on-line block.

Information received from both the pre- and post- assessment questionnaires was matched for individual students, along with grades on tests and exercises. Only students who provided data in both the pre- and post-assessments were included in the final analysis. The pre-assessment was administered at the beginning of the semester. The three-week on-line instructional block took place in the middle of the semester. Correlations (Pearson's R) were used to identify relationships within the data. An Alpha level of p > .05 was applied.

Population

For two semesters (fall 2001 and spring 2002), students enrolled in an introductory industrial computer systems class were eligible to participate in the project. For each semester, student volunteers agreed to participate in one on-line instructional module to support the study. During the remainder of the semester, students were taught using traditional instructional methods. All nonparticipating students were taught entirely using traditional classroom-based methods. The data received from the two on-line cohorts were combined, allowing for a sample of 57 students.

Findings and Discussion

This section presents and discusses correlations between pre-assessment survey items/constructs and post-treatment learning outcomes, as well as students' perceptions of their on-line learning experience as assessed through the post-instruction assessment instrument. These correlations are reflected in Table 1. The pre-assessment instrument asked for indications of a priori computer technology experiences as well as responses to items designed to assess the constructs of computer self-efficacy, computer playfulness, perceived computer usefulness, and perceived ease of use.

Contrary to suggestions in the literature, no statistically significant correlations were found. The most surprising finding here was that self-efficacy did not emerge as a significant characteristic. Since computer self-efficacy was believed to relate to the individual's belief that he/she can master a challenge, this result may indicate that students did not view learning in the context of interactive web-based instruction embedded within an otherwise traditional course as more challenging or difficult that the traditional instructional method.

In previous research, ease of use scale results were found to have statistically significant correlations with actual use of technologies ( Davis, 1989 ). In this study, the ease of use construct was found to have significant correlations with three post-treatment variables: student perceptions that during the web-based instructional unit (a) content was adequately covered (r = .369), (b) interaction was evident (r = .313), and (c) communication skills were required (r = .291). This may suggest that students found the use of synchronous chats, asynchronous bulletin boards, e-mail, and voice-over PowerPoint presentations adequate communication tools in this teaching/learning environment.

The relationship between a priori familiarity with computers and specific software applications, and student perceptions of satisfaction with on-line learning experiences embedded in a traditional classroom environment, was positively supported in one variation. It had been speculated that prior experience and perceived proficiency with such computer applications as Microsoft Windows, Internet browsers, electronic

Table 1
Correlations between Pre and Post-online Instruction Items

	Power Point	e–mail	Cognitive playfulness	Computer ease	Computer usefulness
Quiz	.275*	.143	.100	.097	.272*
Time	.026	.395*	.032	.214	.168
Interaction	.208	.071	.284*	.313*	.374*
Content	.067	.034	.244	.369*	.341*
Communication	.158	.026	.079	.291*	.291*
Critical thinking	.137	.237	.153	.150	.358*

* significant at the .05 level

mail, and PowerPoint would correlate with the student's perceptions about the web-based instructional unit that incorporated elements similar to those in each of these applications. One positive statistically significant correlation of r = .395 was observed between student experience with use of e-mail and post-treatment perceptions that the time available during the web-based instructional unit was appropriate for the assignments. This would suggest that proficiency with e-mail may be an important prerequisite skill for students participating in asynchronous web-based instructional units.

The relationship between cognitive playfulness and success in achieving learning objectives with on-line learning experiences embedded in a traditional classroom environment was supported. Based on previous findings that high cognitive computer playfulness was potentially motivating and related positively to learning ( Webster & Martocchlo, 1992 ), it was expected that there would be positive statistically significant correlations between this construct and learning outcomes. The playfulness construct was found to significantly correlate with only one post-treatment measure. The correlation between computer playfulness and student perceptions that interaction was evident (among students and between students and faculty) during the web-based instructional segment was r = .284. This suggests that students who enjoy using computers may also value interpersonal communications facilitated by the technology.

The relationship between perceived usefulness of personal computers and student success in achieving learning objectives with on-line learning experiences embedded in a traditional classroom environment was supported. Previous research suggested that the computer usefulness construct was a useful predictor of future use of computer technologies. The computer usefulness construct was significantly correlated with five post- treatment variables: learning outcomes as measured by cognitive quiz and unit test scores (r = .272), as well as student perceptions that while completing the web-based instructional unit (a) content was adequately covered (r = .341), (b) interaction was evident (r = .374), (c) critical thinking was required to successfully complete the assignments (r = .358), and (d) communication skills were required (r = .291).

The relationship between a priori familiarity with computers and specific software applications, and student success in achieving learning objectives with on-line learning experiences embedded in a traditional classroom environment, was supported in one variation. A statistically significant correlation was found relating a priori student experience in use of PowerPoint with cognitive learning outcomes as demonstrated through scores of unit tests and quizzes. A positive statistically significant correlation of r = .275 was found. Since the delivery format for the quizzes on line was via PowerPoint presentation slides saved to the instructional web site, it appears that students believed that proficiency with PowerPoint may have also been an important prerequisite skill for students participating in these web-based instructional units.

Conclusions

Typical reasons noted for delivery of whole course instruction on-line are convenience for students resulting in increased enrollments, expectations of cost effectiveness, enhancing individual student learning, and assuring student familiarity with computer technologies. Increasing enrollments and cost reductions do not seem to be relevant to decisions on how to, or even whether to, embed on-line components within a traditional classroom environment. However, the potential for enhancing learning for some or even all students and assuring student familiarity with computer technologies may be more compelling. Further study seems warranted to determine the degree to which embedded on-line instruction may enhance student motivation and learning for select topics and/or students who exhibit specific characteristics. The question also arises whether there are effective pre-assessment themes and variables that could indicate which, if any, students will find that on-line instruction enhances learning.

The primary implications of this study are that it is essential for on-line component instructors to consider student characteristics and preferences before delivering on-line instructional segments. It must be remembered that both learning and using computers to aid learning and productivity can and should be enjoyable. Nevertheless, students learning new skills with computer technology and using a new software package to complete new tasks face numerous challenges. When developing on-line components embedded within traditional instructional environments, instructors need to reinforce student perceptions that the technology is useful and make learning activities and use of computers to support learning enjoyable.

The findings of this study suggest several possible areas for further study.

1. Some of the variables selected for inclusion in this study, specifically, computer self-efficacy and computer playfulness, do not appear to be important factors related to students' perceptions about imbedded web-based instruction.
2. The constructs of computer ease of use and computer usefulness may warrant further study to see if they have any predictive value in anticipating student learning success and attitudes toward web-based imbedded instructional units.
3. Further exploration is needed if constructs which have adequate predictive value when anticipating student learning success and attitudes toward web-based imbedded instructional units are eventually to prove useful.

Students who believe that computers are useful, easy to learn, and flexible seem more likely to be computer literate and have sufficient knowledge to focus on the instructional content rather than becoming preoccupied with learning to use the on-line technology. Finally, on-line instruction may provide opportunities to teach with a variety of different learning style preferences and to shape instructional experiences that are beneficial to learners in a world of increasing diversity in communication and learning styles.

References

Bandura, A. (1994). Self-efficacy. In V.S. Ramachaudran (Ed.), Encyclopedia of human behavior (pp. 71-81). New York: Academic Press.

Beller, M., & Or, E. (1998). The crossroads between lifelong learning and information technology: A challenge facing leading universities. Journal of Computer Mediated Communication, 4 (2). Retrieved August 1, 2002, from http://www.ascusc.org/jcmc/vol4/issue2/beller.html

Benbunan-Fich, R., & Hiltz, S. R. (1999). Educational application of CMCS: Solving case studies through asynchronous learning networks. Journal of Computer Mediated Communication, 4 (2). Retrieved July 11, 2002, from http://www.ascusc.org/jcmc/vol4/issue3/benbunan-fich.html

Chester, A., & Gwynne, G. (1998). Online teaching: Encouraging collaboration through anonymity. Journal of Computer Mediated Communication, 4 (2). Retrieved July 11, 2002, from http://www.ascusc.org/jcmc/vol4/issue2/chester.html

Christensen, E. W., Anakwe, P. U., & Kessler, E. H. (2001). Receptivity to distance learning: The effect of technology, reputation, constraints, and learning preferences. Journal of Research on Computing in Education, 33 (3), 263-279.

Compeau, D. R., & Higgins, C. A. (1995). Computer self-efficacy: Development of a measure and initial test. Managing Information Systems Quarterly, 19 (2), 189-211.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Managing Information systems Quarterly, 13 (3), 318-340.

Eastin, M. S., & LaRose, R. (2000). Internet self-efficacy and the psychology of the digital divide. Journal of Computer Mediated Communication, 4 (2). Retrieved July 11, 2002, from http://www.ascusc.org/jcmc/vol6/issue1/eastin.html

Fitzelle, G. T., & Trochim, W. M. K. (1996). Survey Evaluation of web site instructional technology: Does it increase student learning? Retrieved May 9, 2002, from http://trochim.human.cornell.edu/webeval/webques/webques.htm

Johnson, S. D., Aragon, S. R., Shaik, N., & Palma-Rivas, N. (2001). Comparative analysis of online vs. face-to-face instruction . Retrieved August 1, 2002, from http://www.hre.uiuc.edu/online/comparison.pdf

Kandies, J., & Stern, M. B. (1999). Weaving the Web into the classroom: An evolution of Web enhanced instruction . Paper presented at the Teacher Education International Conference, San Antonio, TX. (ERIC Document Reproduction Service No. ED 432270).

LaRose, R. L., Gregg, J., & Eastin, M. (1998). Audiographic telecourses for the web: An experiment. Journal of Computer Mediated Communication, 4 (2). Retrieved July 11, 2002, from http://www.ascusc.org/jcmc/vol4/issue2/larose.html

Ryan, M., Hodson Carlton, K., & Ali, N. (1998). Evaluation of traditional classroom teaching methods versus course delivery via the world wide web. Journal of Nursing Education, 38 (6), 272-277.

Sanders, D. W., & Morrison-Shetlar, A. (2001). Student attitudes toward web-enhanced instruction in an introductory biology course. Journal of Research on Computing in Education, 33(3), 251-261.

Swan, M. K., & Jackman, D. H. (2000) . Comparing the success of students enrolled in distance education courses vs. face-to-face classrooms. The Journal of Technology Studies , 26 (1) , 58-63.

Webster, J., & Martocchlo, J. J. (1992). Microcomputer playfulness: Development of a measure with workplace implications. Managing Information systems Quarterly, 16 (2), 201-226.

____________________
Schmidt is Assistant Professor and Brown is Associate Professor in the Department of Technology at Illinois State University in Normal, Illinois. Schmidt can be reached at kschmid@ilstu.edu . Brown can be reached at dcbrown@ilstu.edu .