JITE v37n3 - Effects of an On-Line Computer-Mediated Communication Course, Prior Computer Experience and Internet Knowledge, and Learning Styles on Students' Internet Attitudes

Volume 37, Number 3
Spring 2000

Effects Of An On-Line Computer-Mediated Communication Course, Prior Computer Experience and Internet Knowledge, and Learning Styles On Students' Internet Attitudes Computer-Mediated Technologies and New Educational Challenges

John G. Wells
West Virginia University

Coupling the 1980s advances in computer technology with those made in communication/information technology throughout the 1990s has created a tremendous potential for improving the quality and effectiveness of the educational process. Networked computers gave rise to the Internet and World Wide Web (WWW), which, in turn, made accessible a seemingly endless array of educational resources. The lure of these resources, the establishment of advanced telecommunication infrastructures to support distributed learning, and growing economic pressures brought about a rush by institutes of higher education to accept and incorporate new information technology-based means of instruction ( Beller & Or, 1998 ).

Of the broad spectrum of communication/information technologies, computer-mediated communication (CMC) technologies are unique in providing the individual with access to information resources not before possible and the capacity to participate in the educational process anywhere, anytime ( Romiszowski, 1997 ). Communication/information networks are influencing the very essence of the educational process, and nowhere is this more apparent than in the proliferation of on-line education courses delivered primarily and/or entirely using CMC technologies ( Schrum, 1995 ). From past experiences in delivering on-line instruction, much has been learned regarding the essential ingredients for a successful teaching/learning experience. The purpose of this research was to investigate the effects of an on-line course on students' internal and external concerns or using the Internet for instructional purposes, taking into account prior computer and Internet knowledge, as well as individual learning styles. The investigation sought to provide insight and guidance for educators designing and delivering on-line instruction by focusing on a set of student variables that might influence their acceptance of this mode of education, and their ultimate achievement of stated learning outcomes.

Teacher Education Programs

Initial development of the Internet was for the express purpose of military communication. The Internet and associated WWW resources found their way into the realm of education due to unique features that create tremendous possibilities for delivery of instruction. The Internet is primarily a delivery vehicle for information and a communication system with significant potential for facilitating the teaching/learning process. However, utilizing CMC technologies to access and incorporate WWW resources for instructional purposes presents a drastic departure from traditional methods of instructional design and delivery.

According to the report issued by the National Council for Accreditation of Teacher Education's Task Force on Technology and Teacher Education ( 1977 ), faculty in teacher education programs across the country, faced with accreditation pressures for preparing future techno-savvy teachers, are being challenged to create and model effective methods of instruction that integrate CMC technologies in pedagogically sound ways. With limited empirical guidance on effective methods of integration, teacher education faculty are encouraged to experiment with CMC applications to determine their effectiveness in promoting a quality learning experience. Infusion into instructional practice requires educators to take advantage of CMC tools readily available for teaching and learning in this new environment, and to improve their ability to use and integrate them into their educational practice ( Green, 1999 ). In addition, results from studies such as the one presented here can provide insight into design and delivery methods that proved effective.

On-Line Courses: Theoretical Framework

Facilitating good learning experiences in on-line courses results from taking into consideration those criteria that have been found to promote pedagogically sound instruction in this environment ( Schweizer, 1999 ). Primarily, those criteria center on the CMC technologies used ( Hassenplug & Harnish, 1998 ), structure and navigation of the course web site ( Jones & Farquhar, 1997 ; Starr, 1997 ), modes of interactivity throughout the course ( El-Tigi & Branch, 1997 ; Moore, 1989 ), and student variables such as computer/Internet skills, learning styles, and internal/external concerns about application of knowledge gained ( Takacs, Reed, Wells, & Dombrowski, 1999 ; Wells & Anderson, 1997 ).

The delivery of instruction using an on-line method is heavily dependent on network technologies and, ultimately, determines the design criteria by which such courses are developed. Those CMC technologies selected for delivery of instruction (email, chat, web board, listserv, MOOs/MUDs, etc.) play a critical role in the quality of communication and information exchange in on-line courses ( Hassenplug & Harnish, 1998 ). Instructional sessions, whether designed for delivery in a traditional classroom, geographically distant classroom, wireless reception on the campus lawn, or electronically to individual students around the world, should all provide a teaching/learning environment that supports (1) interactivity (instructor-learner, learner-learner, learner-content), (2) student-centered control of pertinent information, and (3) mechanisms for students to discuss the ongoing construction of their knowledge ( El-Tigi & Branch, 1997 ; Moore, 1989 ). Such interactivity lends itself to a constructivist approach to learning, facilitated by a course web site where students have navigational freedom.

Despite a growing understanding of those criteria affecting positive learning experiences using CMC modes of delivery, accounting for learner characteristics still presents significant challenges to faculty preparing instructional materials for delivery in an on-line, web-based environment. It is the influence of learner characteristics on the design and delivery of instruction via CMC that forms the basis of this study.

Guiding Questions

The intent of this study was to assess the influence on student attitudes toward (a) using the Internet to deliver a computer-mediated communication course, and (b) requiring the integration of Internet applications into the design of instruction for learners enrolled in the Computer-Mediated Communication in Education course (TE 365). The research questions guiding this study were the following. (1) What effect does the on-line delivery of TE 365 have on students' Internet stages of concern? (2) What is the relationship between students' prior computer experience and their Internet stages of concern? (3) What is the relationship between students' prior Internet knowledge and their Internet stages of concern? (4) What is the relationship between students' learning styles and their Internet stages of concern?

Design of the Study

Moving toward instruction delivered on-line brings with it different expectations from students taking the course and questions concerning a teaching/learning process that is mediated by the computer. Influences on course design and effectiveness will differ depending on students' entry-level skills and experiences, personal learning styles, and attitudes toward using CMC as an instructional medium. This study was, therefore, designed using independent variables chosen to report on course and student characteristics that would later be correlated with attitudinal changes (dependent variable) assessed at pre, mid, and post data collection points.


A single group of learners participated in this study. They were graduate students enrolled in the Computer-Mediated Communication in Education course (TE 365) offered spring of 1998 at West Virginia University. The number of participants was 13, all technology education graduate students who, themselves, were practicing community college faculty.

Independent Variables

Treatment: TE 365 Course. Computer-Mediated Communication in Education is a graduate course in the College of Human Resources and Education at West Virginia University that was first designed and delivered in a traditional classroom environment during Fall 1994. The overall intent of the course was to address the fundamental mechanics of using computers to access information networks and understand their associated infrastructures for application in a teaching/learning environment. Course objectives were developed to address three major areas related to computer-medicated communications (CMC): basic Internet enabling skills, fundamentals of network infrastructures, and the integration of Internet applications into instructional design. Each major area was addressed by a set of assignments designed to increase students' ability to use a given Internet technology and to increase their knowledge about its application in a variety of teaching/learning environments. A variety of virtual and face-to-face instructional methods were employed in delivering the course, including lecture, discussion, application demonstrations, student presentations, and weekly web-based assignments. All students had university or personal mainframe accounts required for emailing, accessing web resources, and for completing the course assignments. Students met face-to-face (F-2-F) with the instructor three times during the semester (beginning, middle, and end). The remainder of the course was conducted via the Internet in a virtual (textual and web-based) environment (see Appendix A for course syllabus, objectives, and individual assignments).

Prior Computer Experience. Students enrolled in the course were asked to self-report their level of expertise on nine different computer experience categories: general computer experience, content area software, word processing, data base, spreadsheet, programming, authoring, hypermedia, and the Internet. Responses were recorded using a 0 to 9 point Likert scale, with 0 indicating no experience and 9 indicating expert. Their self-reported computer skills indicated the group was composed of individuals with skill levels ranging from very low to high.

Prior Internet Knowledge. Using an existing instrument developed by Anderson ( Wells, J. & Anderson, D., 1995 ), students' Internet knowledge prior to taking the course was measured. The Internet Knowledge instrument consisted of 30 items related to the Internet, each worth a single point. Composite scores were determined by adding correct responses. The instrument has been shown to be reliable, with alpha ranges from .79 to .90.

Learning Styles. Participants were asked to respond to the Group Embedded Figures Test (GEFT), which measures a subject's field dependence on a range from 0 to 18 by having them identify and outline a simple shape within a more complex one. At the low end, a score of 0 reflects an individual who is highly field-dependent and finds it difficult to recognize a simple shape within (the surrounding field) a more complex one. At the high end, a score of 18 indicates the individual is field-independent and has little difficulty recognizing a simple shape embedded within the surrounding complex field. The GEFT has been shown to have good reliability (.89 on re-test over 3-year period) and validity (.82 correlation between two major sub-sections) ( Witkin, Oltman, Raskin, & Karp, 1971 ).

Determining field dependence preferences for students participating in the study provides indicators of the effectiveness of the instructional design strategies employed and of the basic web-course structure. It has been postulated ( Witkin & Goodenough, 1981 ) that the cognitive style of field-dependent students makes them more sensitive to social cues and criticisms and show a preference for less structured learning environments that allow for interpersonal exchanges. Conversely, field-independent students are more self-directed learners, preferring to work alone and in a more structured learning environment. Designing for web-based delivery of instruction must take into account the various learning style preferences ( Foell & Fritz, 1997 ).

Dependent Measure - Stages of Concern

Instrument Description. There are a number of computer attitude instruments available to researchers, though arguably none more robust than the Stages of Concern (SoC) instrument developed by Hall, George, and Rutherford ( 1977 ). The strength of the SoC instrument lies in its ability to measure, over time, a continuum of attitudes an individual may develop for a given innovation and to divide those attitudes among internal and external concerns. When applied to educators, the typical developmental phases (pre-teaching, early-teaching, late-teaching) one would go through in preparation for and the delivery of instruction can be seen to parallel the stages addressed by the SoC. Specifically, during the pre-teaching and early-teaching phases an instructor would be likely to have concerns related to self (internal), followed in the late-teaching phase with a shift in concerns that then focused on student learning and personal professional development (external concerns). For this study, the participants were educators enrolled in the technology education graduate program, and whose jobs were increasingly demanding the integration into their teaching of instructional technologies. In this case the innovation was the Internet, encountered both as the primary mode of delivery for the Computer-Mediated Communication in Education (TE 365) course and as a tool to be integrated into the student's own instructional practices.

The first four stages of the SoC instrument address internal concerns and relate to how an individual perceives the innovation to be affecting them directly. The last three stages of the instrument deal with the individual's external concerns and relate to how one's learning about the innovation might influence others who would become the recipients of their newly gained knowledge. For those who find the innovation to be truly novel, one would expect their internal concerns to decrease throughout the instructional period as their initial anxieties were mitigated. Likewise, an increase in external concerns would be expected toward the end of the instructional period as the participants, now less anxious about the innovation, were able to focus on their new roles as integrators and users of the innovation. Furthermore, those participants in a given target group having greater familiarity with the innovation would be expected to display an increase in external concerns earlier in the treatment ( Reed, 1990 ).

The 35-item Stages of Concern (SoC) instrument developed by Hall, George, and Rutherford ( 1977 ) employs a 0 to 7 response-range (Likert) to measure differences in attitudes toward a given innovation. The modifiable word in the original instrument is "innovation," and is replaceable with any term that describes a specific innovation. Innovation, defined in this study as any instructional tool or novel approach applied in a learning situation, was replaced by the term Internet in the Stages of Concern instrument. With Internet as the innovation used in the SoC instrument, it was possible to measure changes in concerns toward Internet applications; either its use in delivering the course or its integration into course assignments. The instrument has been shown to be a reliable measure (coefficient alpha = .91) of attitudinal changes ( Reed, 1990 ).

Seven Stages of Concerns. The SoC instrument uses five items to address each of the seven stages of concern. Awareness is the first stage and seeks to measure the extent to which an individual is informed about an innovation; a sample statement is I am not concerned about the Internet. The second stage is Informational, which addresses the individual's desire to become more informed about the innovation; a sample statement is I would like to know more about the Internet. The third stage is Personal, referring to the way an individual perceives how the innovation might affect them personally; a sample statement is I am concerned about how the Internet will affect me. Stage four, Management, focuses on how the innovation may impose time constraints on them; a sample statement is I seem to be spending all of my time getting instructional materials ready when using the Internet. Consequence, stage five, looks to gain insight into how an individual perceives the innovation will have influence as a result of their using it; a sample statement is I am concerned about how the Internet will affect my students. The sixth stage is Collaboration, and indicates the level of comfort toward the innovation an individual has reached and, thus, the extent to which they can then direct attention toward broadening its application; a sample statement is I am concerned about relating what I am doing with the Internet to what other instructors are doing with the Internet. Refocusing is the final stage and the point where the individual begins to explore more global benefits from the innovation; a sample statement is I have some ideas about how something other than the Internet might work better (see Appendix B for the Stages of Concern instrument).

Analysis of the Data.

To answer the first question, "What effect does the on-line delivery of the Computer-Mediated Communication in Education course (TE 365) have on students' Internet stages of concern?", seven one-way analyses of variance with a repeated measure on one factor (data collection at pre-, mid-, and post-treatment) were conducted. The independent variable was time of data collection and for each of the seven stages of concern the means served as dependent measures.

Research question two, "What is the relationship between students' prior computer knowledge (general, content-area software, word processing software, database software, spreadsheet software, programming languages, authoring languages, hypermedia software, and Internet knowledge) and their Internet stages of concern?" was answered by performing nine simple regressions. Means for each of the seven stages of concern served as the dependent measures and the various prior computer knowledge responses were the predictors.

In answering the third research question, "What is the relationship between students' prior Internet knowledge and their Internet stages of concern?", and fourth research question, "What is the relationship between students' learning styles and their Internet stages of concern?", simple regressions were run using Internet Knowledge scores and GEFT scores as predictors against the SoC scores. The SoC scores were regressed on the Internet Knowledge and GEFT scores to determine any possible relationships.


Awareness There was a significant effect of TE 365 on Awareness concerns between the pre/mid [t(13) = -3.14, p = .0085] and pre/post [t(13) = -2.201, p = .0480] data collection points. There was no significant effect between the mid/post data collection points (see Table 1 ). Results with p < .05 are reported as significant, and because the Internet is so new in educational research, results where .05 < p < .15 are reported as trends.

Table 1
Stages of Concern Group Mean Results

Stage of Concern Pretreatment Midtreatment Post-treatment

Awareness 24.69 b 45.62 c 50.46
Informational 86.15 a 84.77 59.77 d
Personal 82.00 a 77.62 1 47.92 d
Management 45.38 56.08 2 33.54 d
Consequence 60.46 a 56.77 33.92 3
Collaboration 90.15 a 79.08 1 53.38 d
Refocusing 90.31 a 87.69 64.85 d

a Significant pre-post decrease. b Significant pre- post increase
c Significant pre- mid increase. d Significant mid-post decrease.
1 Negative trend pre- mid. 2 Positive trend pre-mid.
3 Negative trend mid-post.

Initial awareness concerns regarding the course indicated high levels from the start of the course through to the mid-point of the semester. During this portion of the course Internet enabling skills (minimally required skills) were presented and learned. Student anxiety was elevated in anticipation of the tasks required to gain the necessary enabling skills and, generally, from what the course would demand of them.

At the mid-point of the course the anxiety surrounding the course requirements and gaining the necessary enabling skills were mitigated. Most students reached an acceptable comfort level with respect to those initial concerns, allowing new concerns to be focused elsewhere in the course. This is reflected in the non-significant awareness concerns between the mid/post treatments. These results indicate that the learning activities developed and the instructional strategies employed were effective in teaching the necessary enabling skills, and by mid-semester allowed students to focus on methods of integration.


There was a significant effect of TE 365 on Informational concerns between pre/post [t(13) = 3.383, p = .0054] and mid/post [t(13) = 3.342, p = .0059] data collection points. There was no significant effect between the pre/mid data points (see Table 1 ).

Significance found between the pre/post and mid/post treatments indicates informational concerns were higher near the end of the course. Though this contradicts expected results from the Stages of Concern instrument, it can be explained from a course design standpoint. End of the semester assignments were designed to combine and apply skills gained by students in the first half of the course through the development of actual on-line teaching/learning modules (see syllabus Appendix A , on-line activity 5). Doing so most likely generated more questions and concerns than were specifically addressed in the course, elevating their informational concerns at the end of the semester.


There was a significant effect of TE 365 on Personal concerns between the pre/post [t(13) = 4.858, p = .0004] and mid/post [t(13) = 3.669, p = .0032] data collection points. A significant trend [t(13) = 1.609, p = .1337] was observed between pre/mid data collection points (see Table 1 ).

Comparisons between the pre/post and mid/post treatments were found to be significant, again showing personal concerns increasing at the end of the course. As with the informational concerns, this is likely attributable to applying skills/knowledge gained at the front end of the course to actual teaching/learning situations in the end. Students developed, as part of the course, a small instructional module as a capstone activity. Anticipating applying their new skills and knowledge for development of an entire curriculum or course would pose advanced design issues not addressed in the TE 365 course.


There was a significant effect of TE 365 on Management concerns between the mid/post [t(13) = 3.217, p = .0074] data collection points. No significant effect occurred between the pre/post data collection points, though a trend was observed between the pre/mid [t(13) = -1.743, p = .1068] data collection points (see Table 1 ).

Self-reported student information gathered on the first day of class indicated a fairly low level of Internet skills for the majority of participants, but average general computer skills. In light of this, the lack of significant management concerns at the start of the course (pre/post treatment) might have resulted from students not knowing what to expect. As the course progressed, students became increasingly aware of management requirements involved with on-line course delivery and Internet access, causing their level of concern to rise over time, and peak at the end (mid/post comparisons). This is further strengthened by the indication of a trend toward increased management concerns occurring between the pre/mid treatments.


There was a significant effect of TE 365 on Consequence concerns between pre/post [t(13) = 2.309, p = .0395] data collection points, and a trend observed between mid/post [t(13) = 1.793, p = .0982] data collection points. There was no significant effect found between pre/mid data collection points (see Table 1 ).

Student concerns toward consequences of the innovation showed a gradual mitigation as the course progressed (significance between pre/post; trend between mid/post; no significance between pre/mid). Although this mitigation goes contrary to expected outcomes regarding consequence concerns, a plausible explanation may be related to the clearer understanding of on-line assignments and the course design gained by students as the course progressed. As students completed the activities and interacted with the on-line course, their familiarity and ease in using it increased to a point that they gained a greater level of comfort, resulting in reduced concern for its individual consequences.


There was a significant effect of TE 365 on Collaboration concerns between pre/post [t(13) = 3.346, p = .0058] and mid/post [t(13) = 2.677, p = .0202] data collection points, with a trend observed between pre/mid [t(13) 1.614, p = .1325] data collection points (see Table 1 ).

Coordinating and cooperating with others regarding the innovation indicated a gradual increase in collaborative concerns throughout the course. Significant concerns appeared between the pre/post and mid/post treatment stages, with a trend apparent at the pre/mid treatment. This would indicate that the level of collaborative concerns increased throughout the course, but with a small drop during the mid-point. The course design was such that enabling skills were taught/learned during the first half of the semester. After that time assignments shifted toward encouraging group work, as well as, partnering to review teaching/learning modules under development. At the start of the course students would not have understood the amount of, or need for, collaboration. By mid-semester however, they would begin to recognize and better anticipate these requirements in pending assignments. Between the middle and end of the semester, as collaborative activities were being completed and integrated into instructional modules, this stage of concern showed a significant increase.


There was a significant effect of TE 365 on Refocusing concerns between[t(13) = 2.823, p = .0154] and mid/post [t(13) = 2.503, p = .0278] data collection points. No significant effect was found between pre/mid data collection points (see Table 1 ).

Results from the refocusing stage demonstrated significance between the pre/post and mid/post treatments, with no significance between the pre/mid treatments. This is interpreted to mean that refocusing concerns steadily increased throughout the course. The course was designed to focus first on building basic Internet skills and gradually move toward the integration of these skills into their own instructional modules by the end of the semester. By mid-semester the navigational skills are mastered, no longer presenting impediments to using the Internet, and thereby freeing the student to focus on the application of newly acquired skills for instructional design purposes. As a result, concerns over finding better, novel, or more powerful alternatives for, in this case, integrating the Internet into instruction, increased. With fundamental Internet skills intact, it is reasonable that concerns toward their application in a broader, more complex teaching/learning environment would increase.

Prior Computer Experience and Stages of Concern: Simple Regression

An assessment was made of all participants' prior computer experiences at the first class meeting. Data indicate that most students in the course entered with average or above computer familiarity. Mean scores (standard deviation in parentheses) for each of the 9 categories were as follows: general computer experience, 5.92 (1.85); content area software experience, 4.38 (2.75); word processing experience, 6.38 (2.47); data base experience, 4.85 (2.19); spread sheet experience, 5.00 (2.61); programming experience, 2.69 (2.32); authoring experience, 2.46 (2.57); hypermedia experience, 3.00 (2.89); and Internet experience, 5.15 (1.95).

Results pertaining to the entire set of regressions performed to analyze the relationships between the various categories of prior computer experience are listed in Table 2 . Significant results and those results indicating trends toward significance are outlined below.

Internal Concerns: Awareness/Informational/ Personal Management

For these four stages of concern there appeared to be only minimal relationships to prior computer experience. Those relationships that were revealed indicated mostly mid- and post-treatment negative trends, occurring predominantly at the awareness and management stages (see Table 3 ). Given that student computer familiarity was average or above, it is reasonable to expect that the first four stages of concern would not initially indicate elevated concerns, and that any relationships appearing would be of a negative nature; those with more computer experience would have an inverse level of concern. Self-reported student scores of prior computer experience indicate the group as a whole had good general computer experience, and greater than average experience with word processing, spreadsheets, and the Internet. As a result, the relationships between prior computer experiences and internal stages of concern (Awareness, Informational, Personal, Management) showed mostly negative trends or negative significance occurring at mid and post data collection points ( Table 3 ). Where these trends appeared, it can be interpreted to mean that as the course progressed, student levels of internal concern experienced mitigation.

Table 2
Stages of Concern Regression Results

Stage 0
t (13) p t (13) p t (13) p t (13) p t (13) p t (13) p t (13) p

General Computing Experience
Pre -.79 .48 -1.07 .31 -.40 .70 -2.38 .04* .158 .88 -.29 .78 .87 .40
Mid -1.90 .08** .15 .88 .17 .87 -2.04 .08** 2.46 .08* 5.04 .001* 4.90 .001*
Post -1.59 .14* 1.37 .20 1.37 .20 -.76 .46 1.90 .08 1.12 .29 1.27 .23
Content-Area Software Experience
Pre -.16 .88 -.58 .58 1.07 .31 -1.43 .18 1.03 .33 -.16 .88 1.24 .24
Mid -90 .39 1.06 .31 1.22 .25 -.35 .74 3.00 .01* 2.78 .02* 4.53 .001
Post -1.09 .30 1.05 .31 1.24 .24 -.12 .90 1.93 .08** 1.26 .23 1.61 .14**
Word Processing Experience
Pre -.46 .65 -.43 .67 .07 .94 -2.00 .07** .17 .87 -.37 .72 .96 .36
Mid -2.05 .07** .68 .51 .16 .88 -1.63 .13** 3.20 .01* 5.47 .001* 4.74 .001*
Post -1.73 .11** 1.16 .27 1.89 .09* -1.12 .28 1.63 .13** .97 .86 1.10 .80
Database Experience
Pre -.20 .84 -.27 .80 .41 .69 -1.82 .10** -.58 .57 -.53 .61 .86 .41
Mid -1.44 .18 .56 .59 .43 .68 -1.30 .22 2.68 .02* 3.37 .01* 2.94 .01*
Post -2.61 .02* 1.03 .33 2.5 .03* -1.54 .15** 1.61 .13** 1.75 .11** 2.04 .07**
Spreadsheet Experience
Pre -.28 .79 .64 .80 .72 .49 -1.09 .30 .62 .55 .36 .72 .13 .90
Mid -1.98 .07** 1.45 .17 .43 .67 -1.08 .30 2.95 .01* 2.09 .06** 2.56 .03*
Post -1.36 .20 1.19 .26 1.53 .15** -.29 .78 1.67 .12** .69 .50 1.11 .29
Programming Language Experience
Pre -1.50 .16 -83 .43 .81 .43 -2.03 .06** .50 .63 .11 .91 1.15 .27
Mid -1.32 .21 .19 .86 1.05 .31 -1.08 .30 1.65 .13** 1.95 .08** 2.94 .01*
Post -1.01 .33 1.49 .16 .22 .83 -.33 .75 1.40 .19 .83 .43 .70 .50
Authoring Experience
Pre .06 .96 -1.54 .15** .61 .55 -2.19 .05* .67 .52 .43 .68 1.15 .28
Mid -.65 .53 .28 .79 1.10 .29 -.83 .42 1.53 .15** 1.39 .19 1.89 .09**
Post -1.06 .31 1.53 .15** .59 .57 .18 .86 1.57 .15** 1.62 .13** 1.79 .10**
Hypermedia Experience
Pre -.97 .35 -1.85 .09** .46 .65 -1.36 .20 1.34 .21 .35 .73 1.88 .09**
Mid -.39 .71 .16 .88 1.34 .21 -.20 .84 1.49 .17 1.32 .21 2.25 .05*
Post -.20 .85 .91 .38 -.36 .73 .28 .79 .96 .36 .86 .41 .74 .48
Internet Experience
Pre -.93 .37 -2.02 .07** -.62 .55 -3.03 .01* -.32 .76 -.98 .35 1.16 .27
Mid -1.28 .23 -.63 .54 .02 .99 -1.96 .08** 1.64 .13** 3.86 .002* 3.53 .005*
Post -1.56 .15** 1.73 .11** 1.32 .21 -1.73 .11** 1.06 .31 .89 .39 .45 .66
Individual Learning Styles
Pre .09 .93 .25 .81 1.53 .15** .41 .69 -.47 .65 -.12 .91 3.41 .006*
Mid -.53 .61 1.39 .19 2.76 .02* 1.06 .31 2.70 .02* .91 .38 1.21 .25
Post -.72 .49 -.94 .37 1.02 .33 -1.63 .13** -.76 .46 .16 .88 .18 .85
Internet Knowledge
Pre -1.32 .21 -1.40 .19 -.13 .90 -.64 .53 .75 .47 -1.16 .27 .98 .35
Mid -.84 .42 -.10 .93 .44 .67 -.35 .73 1.77 .11** 2.29 .04* 3.82 .003*
Post -.29 .77 1.05 .32 .78 .45 -.69 .50 .86 .41 -.38 .71 -.58 .57

* = significant result ( p <.05) ** = trend (.05< p <.15).
Pre = pretreatment Mid = midtreatment Post = posttreatment

Table 3
Internal Stages of Concern Regression Summary

Awareness Informational Personal Managment
Pre Mid Post Pre Mid Post Pre Mid Post Pre Mid Post

General Content -T -T -S -T
Word Processing -T -T +T -T -T
Data Base -S +S -T -T
Spreadsheet -T +T
Programming -T
Authoring Hypermedia -T +T -S
Internet -T
Learning Style +T +S -T
Prior Internte +T +S +T

+S = significant positive result ( p <.05).   -S = significant negative result( p <.05).   +T = positive trend (.05< p <.15).
-T = negative trend (.05< p <.15).   Pre = pretreatement. Mid = midtreatment. Post = posttreatment

External Concerns: Consequence/Collaboration/Refocusing.

Typically these three stages of concern would show low levels of concern at the pre-treatment point with a steady increase throughout the time period when the individual is interacting with the innovation. The post-treatment data would then be expected to show a positive significant relationship. In this study, data analyzed on the relationships between prior computer experience and student external concerns indicated significance at the midpoint of the course and a positive trend at the post data collection point in those areas students had higher computer experience (see Table 4 ). An explanation for low external concerns at the end of the course can be found in the design of the on-line course itself, and ,specifically, when certain types of assignments were completed.

The course syllabus (see Appendix A ) shows that assignments completed during the initial face-to-face (F-2-F) class meeting were designed to address basic email capabilities. These skills were fundamental for participating in the course in the weeks following the first F-2-F meeting. During the month prior to the second F-2-F meeting assignments required students to learn, practice, and become confident in performing basic Internet interactions of emailing, attaching files, navigating, searching, and file transfers. Concurrent with these hands-on assignments were readings (see syllabus Appendix A ) that provided further detail and insight into the types of Internet activities they were performing, as well as methods of integration into instruction.

Table 4
External Stages of Concern Regression Summary

Consequence Collaboration Refocusing
Pre Mid Post Pre Mid Post Pre Mid Post

General +S +T +S +S
Content +S +T +S +S +T
Word Processing +S +T +S +S
Data Base +S +T +S +T +S +T
Spreadsheet +S +T +T +S
Programming +T +T +S
Authoring +T +T +T +T +T
Hypermedia +T +S
Internet +S +S
Learning Style +S +S
Prior Internet +T +S +S

+S = significant positive result (p<.05).  +T = positive trend (.05 < p < .15).
Pre = pretreatment.  Mid = midtreatment.  Post = posttreatment

As the mid-point of the course approached, assignments were becoming more collaborative and integrative. Assignments increasingly required students to apply their new skills toward an instructional purpose that entailed on-line interaction with other students or the instructor (see on-line activities 1 & 2 in syllabus Appendix A ). In addition, the mid-semester assignment required student teams to develop and prepare, via the Internet, a group presentation for delivery at the second F-2-F meeting (see mini-conference in syllabus Appendix A ). These assignments brought to light the problems associated with integrating on-line resources into actual instructional design and delivery. It is one task to learn and master a skill, and quite another to apply those skills to a novel situation. In addition to actual assignments due by mid-semester, the instructor continued to discuss electronically with the class those assignments that would follow the second F-2-F meeting in an effort to show future application of the skills they were mastering (see on-line activities 1-5 in syllabus Appendix A ).

The data showed that designing the TE 365 course to be "front loaded" clearly influenced the students at the mid-point of the semester. At the second F-2-F meeting partnered students presented a demonstration and critique of a Winsock application (stand alone Internet application such as weather, global positioning, or stock programs) they planned to incorporate into their final instructional module. In addition, at this point in the course partners or small groups were to have identified sites for conducting an infrastructure inventory and a case study assessment of another course that was being delivered fully or partially on-line. The identification of the infrastructure and case study sites, along with learning a Winsock application, was preparation for completing assignments during the second half of the semester. It is understandable then, as reflected in the stages of concern data (see Table 4 ), that this mid-semester preparation would cause a significant rise in external (consequence/ collaborative/refocusing) concerns.

As students worked through the final assignments during the remainder of the semester, their understanding and level of comfort regarding the process of integrating on-line resources and activities into curricula significantly increased. This resulted in a significant decrease in external stages of concern as the course drew to an end, as indicated by post course data. The mitigation of external concerns by the end of the course is an indication that the format used to deliver TE 365 is appropriate for achieving the stated course objectives.

Prior Internet Knowledge, Learning Styles, and Stages of Concern: Simple Regression

Prior Internet Knowledge: Internal and External Concerns. Results from pre-, mid-, and post-data collection points showed no significant relationships between internal concerns and prior Internet knowledge. The majority of students performed adequately on the Internet Knowledge instrument at the start of the semester, which could account for their lack of these internal concerns throughout the course. The data supports the notion that students, prior to taking the course, had a general awareness of the Internet, were informed about it, understood it at a personal level, and had little concern for managing Internet-related activities.

Pre and post data for all external stages of concern showed no significant relationships to prior Internet knowledge. Data on prior Internet Knowledge did indicate positive significant relationships for Collaboration and Refocusing at the mid-treatment point and a positive trend toward significance at the mid-point for Consequence (see Table 4 ). Differences in student Internet knowledge levels at the start of the course provide a likely explanation for these results. The mid-semester trend toward significance points out that those individuals in the course with higher prior Internet Knowledge had elevated concerns regarding the consequences surrounding the integration of Internet activities into instruction. Furthermore, the significant relationships found at the mid-treatment collaboration and refocusing stages allude to concerns held by those with higher prior Internet Knowledge regarding the utilization of the Internet for instruction in novel or more innovative ways and the need for increased collaboration to deliver on-line instruction. Yet, by the end of the course, no significant differences were found among the participants. This can be interpreted to mean that course activities during the second half of the semester mitigated the concerns of those with lower prior Internet knowledge.

Individual Learning Style: Internal and External Concerns. The Group Embedded Figures Test (GEFT) resulted in scores ranging from 5 to 18, with an overall mean of 14.62. The GEFT score data found one student to be field-dependent (FD; GEFT 0-5), two were field-mixed (FM; GEFT 6-12), and ten were field-independent (FI; GEFT 13-18). These scores reveal that better than 75% of students enrolled in the course had learning styles that were highly field-independent (FI), and less than 1% exclusively field-dependent. GEFT scores determine field dependence preferences for each student (FD prefer less structured learning environments and FI prefer self-directed, structured environments) and were used in this study as indicators for effective design strategies and web-course structure. With the majority of TE 365 students being FI learners, their learning styles would be well suited for participating in the individualized web-based teaching/learning environment provided by the course.

Mid point data for Personal and Consequence concerns showed significant relationships with student learning styles, as did the pre data point for Refocusing. A positive trend was observed at the pre point for Personal and post point for Management. Very few relationships were found between learning style and all seven stages of concern, and those that did occur were predominantly at pre and mid points (see Tables 3 & 4 ). These data indicate that students with higher GEFT scores had high concerns at the beginning and middle of the course for only a few areas, all of which were lowered by the end of the semester.

The majority of TE 365 instruction was delivered in a non-traditional classroom format (CMC graphical user interface, asynchronous and synchronous modes), which removed many of the opportunities that would typically occur to vary instruction according to student learning styles. Yet the results of this study support the design used for the TE 365 on-line course (style, format, layout, and modes of interaction), and the effectiveness of that design to accommodate the learning styles and levels of prior Internet knowledge within the participant group.


The purpose of this investigation was to determine if delivery of instruction via CMC, prior computer experience, prior Internet knowledge, and individual learning styles had a significant effect on students' attitudes toward the Internet. The learning activities and instructional strategies used in TE 365 were effective in teaching the necessary enabling skills and allowing students to eventually focus on methods of integration. Prior computer experience was not an issue for graduate students taking the on-line course; basic computer skills do not need to be taught and advanced computer skills are not necessary for CMC participation. Increased concerns regarding ways to integrate CMC activities into instruction were attributable more to instructional design issues than the acquisition of enabling skills. Students struggled with the process of integration, and were not concerned with how to use the innovation, or that TE 365 was being delivered via CMC. Course requirements (assignments), not the method of delivery, consistently caused a rise in student internal and external concerns. These results suggest that assignments be adjusted to lessen student workload and management issues related to completing instructional tasks. This could be accomplished by combining assignments that address similar skills and tasks, thereby reducing student workload. In addition, because teaching via CMC requires different modes of collaboration and different levels of coordination between collaborators, the study indicates more attention be focused on how this type of activity is facilitated in the on-line classroom. CMC assignments, therefore, should be tailored in such a way that students are afforded more practice with a variety of integration strategies, which they can then draw from when designing their own instructional modules.

Course length and the breadth of content delivered influenced students' ability to acquire complex integration strategies. The time frame and activity structure for TE 365 were sufficient to develop Internet enabling skills and understand how CMC can be used to deliver/enhance instruction; design, and delivery modes are sound. However, additional time and practice are needed for students to develop a deeper understanding of strategies used to integrate CMC modes of instruction more extensively in their own classrooms. The study shows that basic CMC skills alone are not enough for full integration to occur. Additional advanced level courses that develop complex integration skills are needed to ensure students will make the leap toward integration.

Overall, the gradual increase in student CMC enabling skills, greater understanding of infrastructures necessary for on-line learning, and improved abilities to integrate CMC activities into the design of instruction had a positive effect on student attitudes toward the Internet. Little influence was found on overall student performance, indicating CMC delivery of instruction does accommodate a variety of learning styles without negative consequences for learning.

Rapid development of easy to use CMC technologies is promoting the proliferation of on-line courses. In technology education programs throughout the country, as with many disciplines, CMC is fast becoming an established medium for the delivery of on-line courses; a quick web search easily confirms this trend. On-line education is still in its infancy and will require a continuous examination of the design and effectiveness of this medium by those educators delivering instruction in CMC environments. Results of this study can provide some guidance for technology educators in the design of on-line courses, taking into consideration those factors that affect the attitudes of students receiving instruction in CMC teaching/learning environments.


Wells is an Associate Professor in the Technology Education Program at West Virginia Unviersity, Morgantown.


Beller , M., & Or, E. (1998). The crossroads between lifelong learning and information technology: A challenge facing leading universities. Journal of Computer Mediated Communication , 4 (2). [Online] Available: http://www.ascusc.org/jcmc/vol4/issue2/beller.html [June, 1999]

El-Tigi , M., & Branch, R. M. (1997). Designing for interaction, learner control, and feedback during web-based learning. Educational Technology , 37 (3), 23-29.

Foell , N. A., & Fritz, R. (1997). The influence of technology on vocational teacher education. Journal of Vocational Technical Education , 14 (2). [On Line] Available: http://borg.lib.vt.edu/ejournals/JVTE/v14n2/JVTE-4.html

Green , K. (1999). High tech vs. high touch: The potential promise and probably limits of technology-based education and training on campuses. In N. Stacey (Ed.), Competence without credentials, (pp. 29-50). Jessup, MD: U.S. Department of Education, Office of Educational Research and Improvement.

Hall , G. E., George, A. A., & Rutherford, W. L. (1977). Measuring the stages of concern about an innovation: A manual for use of the stages of concern questionnaire. Austin: The University of Texas, Research and Development Center for Teacher Education

Hassenplug , C. A., & Harnish, D. (1998). The nature and importance of interaction in distance education credit classes at technical institutes. Community College Journal of Research and Practice , 22 (6), 591-606. [Online] Available: WVU/Ebscohost/1342228 [This is no longer a valid link] [June, 1999]

Jones , M. G., & Farquhar, J. D. (1997). User interface design for web-based instruction. In B. H. Khan (Ed.), Web-based instruction, (pp. 239-244). Englewood Cliffs, NJ: Educational Technology Publications.

Moore , M. G. (1989). Three types of interaction. The American Journal of Distance Education , 3 (2). [Online], Available: http://www.ed.psu.edu/acsde/ajde/ed32.asp [July, 1999]

National Council for Accreditation of Teacher Education, Task Force on Technology and Teacher Education (1997). Technology and the new professional teacher: Preparing for the 21st century classroom. [On Line] http://ncate.org/projects/tech/TECH.HTM [This is no longer a valid link]

Reed , W. M. (1990). The effect of computer-and-writing instruction on prospective English teachers' attitudes toward and perceived uses of computers in writing instruction. Journal of Research on Computing in Education , 23(1), 3-27.

Romiszowski , R. J. (1997). Web-based distance learning and teaching: Revolutionary invention or reaction to necessity? In B. H. Khan (Ed.), Web-based instruction (pp. 25-37). Englewood Cliffs, NJ: Educational Technology Publications.

Schrum , L. (1995). Online courses: What have we learned? Paper presented to the World Conference of Computers in Education, Birmingham, UK. (ERIC Document Reproduction Service No. ED 385 245)

Schweizer , H. (1999). Designing and teaching and on-line course: Spinning your web classroom. Needham Heights, MA: Allyn & Bacon.

Starr , R. M. (1997). Delivering instruction on the world wide web: Overview and basic design principles. Educational Technology , 37(3), 7-15.

Takacs , J., Reed, W. M., Wells, J. G., & Dombrowski, L. (1999). The effects of on-line multimedia project development, learning style, and prior computer experiences on teachers' attitudes toward the Internet and hypermedia. Journal of Research on Computing in Education , 31(4), 341-355.

Wells , J., & Anderson, D. (1997). Learners in a telecommunications course: Adoption, diffusion, and stages of concern. Journal of Research On Computing In Education , 30(1), 83-105.

Witkin , H. A., & Goodenough, D. R. (1981). Cognitive styles: Essence and origins. New York, NY: International Universities.

Witkin , H. A., Oltman, P. K., Raskin, E., & Karp, S. A. (1971). A manual for the group embedded figures test. Palo Alto, CA: Consulting Psychologists Press.


- Course Syllabus -

Schedule of Course Activities

Date Topic Assignment
Week 1 Course








Netscape mail

Email instructor

Sign on to TE365-L


Email instructor (next day); Sign on to distance
education listserv

Week 2 Domains

Applications of
CMC in Education

Distribution List


Create Distribution list

Create Signature file

Practice CC/BCC

Readings: Schrum Chapters 1 & 2

Week 3 Netnews



File Attachments

Email review of Netnews to instructor with signature
file attached and CC to the TE 365 listserv
Week 4 Search Engines Search Engine Review - select, review, and critique
one World Wide Web search engine. Email critique
to instructor and CC your class distribution list.

Locate and download a document related to the
teaching/learning field of interest to you and
email a fully annotated bibliography of it to the
instructor (follow APA 4th Ed style).

Readings: Schrum Chapter 4

Week 5 File Transfer



Developer teams identify and select a winsock
application. Email the TE 365 listserv with
the name of the application, the URL where
it can be downloaded. Teams work together
to download the winsock application
and prepare a demonstration and
critique of the application, including
comments on its educational value. (To be
presented at the Mini-Conference.)

Readings: Schrum Chapter 5

Week 6 Infrastructure Analysis Identify sites for:
a) infrastructure inventory
b) case study

Infrastructure and Case Studies Conducted


(Rm 405 Allen Hall)
Developer teams present winsock application
demonstration and deliver critique on its
educational value.
Week 9 Integration of the Internet

Module Development Process

Curriculum Integration Plan (CIP)

Development of Instructional module

Readings: Schrum Chapter 6

Week 10 On-Line CHAT

Netscape Chat Software

Chat text files

Download Netscape Chat software

Download text files for Netscape Chat software

Turn in CIP ­ Review/discuss development procedures

Developer teams meet

Week 11 Chat sessions

Module Development


Case Study

1-hour chat session from 401 lab

Developer teams meet ­ continue module

Email Inventory to instructor

Present Case Study and email copy to instructor

Readings: Schrum Chapter 7

Week 12 Internet Trends research 1-hour chat session from 401 lab

Locate an Internet Trends article and critique

Continue module development

Week 13 Current CMC Research

National Trends

Skills Review

1-hour chat session in 401 lab Article

Review for final exam

Readings: Schrum Chapter 8

Week 14 CMC Instructional

(Rm 405 Allen Hall)

MINI CONFERENCE ­ Student presentations of
instructional units

Student Evaluations

Week 15 FINAL EXAM Submit Course Portfolios

Course Objectives -At the conclusion of the course, all students will be able to:

  1. Use Netscape browser software as a communication interface
  2. Log on to the Internet through SLIP/PPP/Ethernet and send/receive messages,
  3. Perform file transfers - retrieval and attachment,
  4. Identify and evaluate web winsock/Mac TCP-IP applications,
  5. Conduct a local community/information system infrastructure study,
  6. Project future hardware/software products/services that could be integrated into teaching/learning environments of the future,
  7. Design and develop instructional (teaching/learning) activities using computer-mediated communication (CMC) services,
  8. Integrate CMC activities into existing curricula,
  9. Identify and evaluate research conducted on the use of CMC in educational settings,
  10. Recognize ethical and legal uses of CMC in educational setting.

Individual Requirements -All students will:

  1. Attend all scheduled course sessions (virtual or otherwise), and participate in computer mediated communication applications and discussions.
  2. Complete all CMC applications & reading assignments when due.
  3. Critique, in writing, one (1) Winsock/MacTCP-IP application and prepare a summary for presentation to the class, along with a demonstration of the Web application.
  4. Conduct an Infrastructure study of an educational and/or business facility. Visit one educational/business site and: (a) Provide a written detailed summary to fully describe the existing infrastructure and its CMC use within the facility. (b) Observe the utilization of computer mediated communication for teaching/learning.
  5. Conduct a Case Study. Identify and locate a course that is being fully or partially delivered on-line (virtual or local) OR one that fully integrates the Internet as an essential component of the curriculum and instruction. Conduct a case study of the course, evaluating the pedagogical foundation and effectiveness of the instruction. Topics to be addressed in the written description should include: Infrastructure required to conduct the course; Classroom utilization; instructional materials developed/in use; Instructional or teaching/learning theory practiced; An evaluation matrix.
  6. Develop an instructional module based on an existing curriculum (preferably within your field of expertise) that integrates the Internet as a teaching/learning tool. Prepare these materials for presentation and demonstration to the class.
  7. Research Internet Instructional Trends. Identify an instructional trend occuring on the Internet. Research that trend and deliver a presentation summarizing your findings.
  8. Submit a course portfolio package (3-ring binder), containing all course materials, assignments, notes, reprints, papers, and other items relevant to the course. Include a title page, table of contents and cover sheets for each section. Portfolios must be reclaimed in 509-D Allen Hall within 2 weeks after evaluation by the course instructor. After that time materials will be discarded.
  9. Final Exam - Essay and practicum.


On-Line Activities Lecture Materials and/or Sessions
1. Activity # 1
  • Sign up to Distance Ed. Listserv
  • Sign on to CMC IN EDUCATION
  • Signature File
  • CC & BCC
  • File Attachments
  • Reading the News
2. Activity # 2
  • Gopher
  • Veronica
  • Web Search Engines
3. Activity # 3
  • File Transfer
  • Winsock Application
4. Field Work: Infrastructure
  • Instrastructure Inventory (group)
5. Activity # 4
  • Case Study
  • CHAT SOFTWARE ­ download!
6. Activity # 5
  • Developing the Instructional Module
7. Activity # 6
  • Curriculum integration Plan (CIP)
  • Internet Article
8. Instructional Modules
  • presentations of Modules
9. Final Exam
  • Portfolios
  • On-Line Exam!


Stages of Concern Instrument
Attitudes Toward Using The Internet

Directions: Answer as completely and truthfully as you possibly can when thinking how each of the following statements applies to your present attitude toward the Internet. Circle the number that best reflects your present attitude. The higher the number, the better the statement reflects your present attitude.

0 1 2 3 4 5 6 7
Not true of me now Somewhat true of me now Very true of me now

0 1 2 3 4 5 6 7 1. I am concerned about students' attitudes toward the Internet.
0 1 2 3 4 5 6 7 2. I now know of several approaches for how I might go about using the Internet.
0 1 2 3 4 5 6 7 3. I don't even know what the Internet is.
0 1 2 3 4 5 6 7 4. I am concerned about now having enough time to learn about the Internet so that I can use it effectively.
0 1 2 3 4 5 6 7 5. I would like to help other faculty use the Internet.
0 1 2 3 4 5 6 7 6. I have very limited knowledge about the Internet.
0 1 2 3 4 5 6 7 7. I would like to know how the Internet might affect me when I am trying to teach.
0 1 2 3 4 5 6 7 8. I am concerned about what my employer(s) might expect me to know about the Internet and how those expectations might be in conflict with what I like to do.
0 1 2 3 4 5 6 7 9. I am concerned about improving what I presently know about the Internet.
0 1 2 3 4 5 6 7 10. I would like to work with present fellow workers and others who are using the Internet.
0 1 2 3 4 5 6 7 11. I am concerned about how the Internet might affect my students.
0 1 2 3 4 5 6 7 12. I am not concerned about the Internet.
0 1 2 3 4 5 6 7 13. I would like to know who will make decisions about my using the Internet.
0 1 2 3 4 5 6 7 14. I would like to discuss the possibility of using the Internet.
0 1 2 3 4 5 6 7 15. I would like to know the resources available if the Internet is to be integrated into my job.
0 1 2 3 4 5 6 7 16. I am concerned about my inability to learn all there is to know about using the Internet effectively.
0 1 2 3 4 5 6 7 17. I would like to know how my teaching is supposed to change because of the Internet.
0 1 2 3 4 5 6 7 18. I would like to familiarize my fellow workers and my employers about the Internet as I learn about it and work with it More.
0 1 2 3 4 5 6 7 19. I am concerned about evaluating my impact on students.
0 1 2 3 4 5 6 7 20. I would like to change how the Internet might be used as I learn more about it.
0 1 2 3 4 5 6 7 21. I do not care much about the Internet; my schedule prevents me from caring too much.
0 1 2 3 4 5 6 7 22. I would like to modify the use of the Internet in my job based on students' experiences.
0 1 2 3 4 5 6 7 23. Although I don't care much about the Internet, I am concerned about it.
0 1 2 3 4 5 6 7 24. I would like to excite my students or fellow teachers about the uses of the Internet.
0 1 2 3 4 5 6 7 25. I am concerned about the time needed to learn about the Internet that will keep me away from doing what I am supposed to be doing as part of my job.
0 1 2 3 4 5 6 7 26. I would like to know what using the Internet will require in the immediate future.
0 1 2 3 4 5 6 7 27. I would like to coordinate my efforts in learning about the Internet with fellow workers.
0 1 2 3 4 5 6 7 28. I would like to have more information on the time required to learn about the Internet.
0 1 2 3 4 5 6 7 29. I would like to know what other people are doing in relation to the Internet.
0 1 2 3 4 5 6 7 30. At this time, I am not interested in learning about the Internet.
0 1 2 3 4 5 6 7 31. I would like to determine how to supplement and enhance the use of the Internet.
0 1 2 3 4 5 6 7 32. I would like to use feedback from my students to change the use of the Internet.
0 1 2 3 4 5 6 7 33. I would like to know how my job will change when I am using the Internet.
0 1 2 3 4 5 6 7 34. My present schedule is preventing me from learning too much about the Internet.
0 1 2 3 4 5 6 7 35. I would like to know how the Internet is better than the methods I presently use or plan to employ when I do my job.