JIAHR Issue 10 - Effective Placement of Reflective Feedback in Computer-aided Instruction for Maintaining Dining Room Standards

August 1998 Issue 10 ISSN 1052-6099

Editor: Eliza C. Tse, Ph.D.
Department of Hospitality and Tourism Management
Virginia Polytechnic Institute and State University
tel.: 540-231-8424
fax: 540-231-8313
tseet@vt.edu
Effective Placement of Reflective Feedback in Computer-aided Instruction for Maintaining Dining Room Standards

Annette Graham, PhD
Assistant Professor
School of Human and Consumer Sciences
Ohio University

Carolyn U. Lambert, PhD
Associate Professor
School of Hotel, Restaurant and Recreation Management
Pennsylvania State University

Table of Contents

ABSTRACT

This research utilized structural communication to teach service quality standards. Five computer-based, multi-media scenarios illustrated discrepancies, or errors, in either the American style of meal service or basic food safety principles found in restaurant dining rooms. One hundred and twenty undergraduate restaurant management students were asked to identify the discrepancies. The inclusion of reflective feedback and the placement of that feedback were examined. Reflective feedback allowed students to select the reasons for their responses. The students were divided into three groups to test the impact of immediate feedback (IFT), delayed feedback (DFT), and no feedback (NFT). Results indicated that the mean scores for each group improved as they progressed through the five scenarios. The participants who received the IFT treatment had the highest mean scores for the five scenarios. The NFT treatment scores were significantly lower than the other two treatments. These results indicate that incorporating the reflective feedback process in instructional lessons is important, not the placement of the reflective feedback.

Keywords: structural communication, service, reflective feedback

INTRODUCTION

Quality service is essential to the success of the foodservice industry. Surveys conducted for the National Restaurant Association (NRA) have found that an insufficient amount of time, money, or effort is devoted to the development of skills necessary for managers and front-of-the-house employees to maintain adequate levels of service standards. The 1992 Tableservice Operator Survey of customers reported that the most common (39%) category of complaints was service. The two most common complaints in this category were the speed of the service experience and the lack of attentiveness of the staff ( Restaurants USA, 1992 ). In a 1994 National Restaurant Association survey, 25 percent of tableservice operators surveyed believed that service was the most important factor of the dining experience for their customers ( National Restaurant Association, 1994 ). In 1996, when operators were asked which component was the most important to customers, food and service received equal ratings ( National Restaurant Association, 1996a ). A 1996 NRA consumer survey ( National Restaurant Association, 1996b ) showed that service was the second most important component of a pleasant dining experience.

With this significance placed on service by consumers, foodservice managers need to be aware of the service standards necessary to meet the expectations of customers. While customers define quality based on expectations, once the customers' expectations have been identified, service standards can be developed ( Heskett, Sasser, & Hart, 1990). If foodservice managers understand the importance of service quality, they can stress the importance of maintaining quality service standards to employees. While service standards can be taught in traditional, classroom-based training programs, annual employee turnover rates for full-service restaurants with average checks of $10 or more averaged 75 percent ( Restaurants USA, 1998 ). Given this high turnover of employees and the need to continually reinforce standards, computer assisted training is a viable option because it can be delivered to individuals at their convenience. The use of media (computers, videotape, and CD-Rom) in training can save a significant amount of instructional development time, learning and delivery time, and money ( Jost, 1990 ).

One advantage of utilizing computers for instruction or training is that the learner can be given immediate feedback. This feedback allows the learner and the instructor to assess progress and determine if, and what additional training should follow. Additionally, reflective feedback, a type of feedback that allows the learner to contemplate on why a response is correct or not, can be incorporated into computerized instruction. If computers are to be used as a delivery system in education and training applications, additional research is necessary to determine which types of feedback are important, where that feedback should be placed in the lesson, and the frequency of the feedback.

PURPOSE OF STUDY

This study was designed to determine if the placement and frequency of reflective feedback affects the ability of the participants to identify discrepancies, or errors, in either the American style of meal service or basic food safety principles found in restaurant dining rooms. Reflective feedback allows the learner to contemplate on why a response is correct or incorrect, not just whether the response is correct or not ( Egan, 1973 ). Structural communication is a self-instructional technique that promotes higher level learning or thinking through reflective feedback. The placement and frequency of the reflective feedback was varied to determine where the feedback maximized learning ( Egan, 1976 ).

LITERATURE REVIEW

Quality service has become an important issue in the United States. Service industries account for approximately 75 percent of the gross national product and nine out of ten new jobs the economy creates ( Zeithaml, Parasuraman, & Berry, 1990 ). Improving customer satisfaction through quality service has been identified as one of the most important challenges facing businesses of the 1990's ( Bitner, 1990 ) Zemke and Schaaf ( 1989 ) offered four reasons why there is a shortfall in delivering quality service. First, companies fail to stay in touch with their customer needs. Second, society's perception of service sector employees is poor. Third, management lacks responsibility for maintaining quality service, and the fourth reason for poor service is that the employees responsible for customer contact are usually the lowest paid, least educated, and most poorly trained employees of the company. Service managers must have a clear understanding of customers' expectations and perceptions so that the service concept can be well defined, communicated and produced. Managers in the hospitality industry are faced with the dilemma of assuring that products and services are delivered by employees according to customer expectations ( Glover, 1988 ). Training programs that utilize the most appropriate instructional design and pedagogical techniques are needed to ensure that customers receive the service expected.

Measuring customers' perceptions of quality has been researched extensively by A. Parasuraman, V. Zeithaml, and L. Berry ( 1988 ). The instrument they developed, SERVQUAL, has been used by many researchers in a variety of industries to measure customers' perceptions of service quality. Stevens, Knutson, and Patton ( 1995 ) adapted the SERVQUAL instrument for use in restaurants to measure customers' perceptions of service quality. Measuring perceptions can be a challenge as most customers who respond have had either an excellent or a bad experience. Obtaining an accurate measure of service quality is difficult because the output of a service is intangible and the perception of its quality is subjective. Therefore, managers have devised programs that relate perceptions to facts through the use of service standards. For example, airlines and supermarkets can monitor the length of lines, the duration of waiting times, and the service times. Restaurant managers can identify standards for sanitation, table service items, and service times. These standards can be quantified, and then tracked by either management or employees to determine compliance and improvement ( Heskett, Sasser, & Hart, 1990 ).

Instructional Design and Structural Communication

Instructional design is a process that has been found to aid learner performance by assisting in the development of efficient and effective instruction. Formal instructional design encourages the setting of design objectives and provides a way to know when the objectives have been met ( Gagné, Briggs, & Wagner, 1988 ). The two major viewpoints on how instruction should be designed are behavioral theory and cognitive theory. Behavioral theory focuses on learning processes that are observable while cognitive theory is learner-centered and examines how the learning organizes information for longer retention and application to other materials. Structural Communication is based on cognitive theory.

Structural Communication is a self-instructional design strategy that aids the learner in organizing the concepts and facts in a study unit. This teaching method was developed using aspects of cognitive psychology, field psychology and cybernetics ( Romiszowski , 1986). Structural Communication is aimed at furthering higher level learning, thinking, and creativity ( Baath, 1979 ).

The basic element of Structural Communication is the study unit, which includes material that can be effectively communicated or learned in a single session of work (Egan, 1973 ; 1976 ). The six steps used in a study unit are: intention, presentation, investigation, response, discussion, and viewpoint ( Figure 1 ). Each step plays a different role in communicating the subject matter. The intention step outlines the theme of the study unit. The presentation step contains a carefully structured presentation of the subject matter and theme. The investigation step presents the learner with three to five challenges about the subject matter. Each challenge consists of a complex question, or problem, related to the subject. The learner is then given a response matrix, comprised of facts, theories, and principles about the subject, arranged in a random order. Learners select the concepts and facts from the matrix that they think are relevant to the presentation ( Egan, 1976 ). The learner is required to consider each of the basic concepts and facts of the presentation, as contained in the response matrix, to solve the problems posed in the investigation step. Then the learner determines which facts are appropriate answers to the challenge and selects those facts from the response matrix. This allows the student to reflect on the reasons for their response.

[figure 1]

The discussion step consists of the discussion guide, a means by which the learner processes his or her responses to the problem; and the discussion comments, which are organized so the learner can receive an explanation for each of the author's statements in the response matrix. If the learner does not select certain statements that the author has included, the learner finds an explanation for why the author included them. If the learner selects concepts and facts for the answer that the author did not, the learner can read the author's justifications. Sometimes the comments are used to expand the theme and even though the learner responded correctly he or she is directed to a comment that further elaborates the point being made.

Viewpoint is the final step in which the author states his or her opinions explicitly and may direct the learner to other resources that offer opinions that are different from the author's ( Egan, 1976 ). To test the different placements of reflective feedback in the study unit, three treatments were developed: an immediate feedback treatment(IFT), which provided reflective feedback immediately after a correct selection; a delayed feedback treatment (DFT), which delivered reflective feedback after the participant selected all discrepancies in a scenario; and, no feedback treatment (NFT), which provided no feedback.

METHODOLOGY

One hundred and twenty junior and senior level undergraduate university students enrolled in a hotel, restaurant, and institutional management program volunteered to participate in this study. One category included sixty students that had not yet completed the course "Introduction to Quantity Food Production Management," and the second category included sixty students that had completed this course. This segmenting was completed to determine if students that had completed the course would score better than the students who had not completed the course. While service quality was a topic in the course, different instructors presented the material in a variety of ways. Each category was further divided into three groups, based on which type of reflective treatment they received.

Five computer-based multi-media scenarios were presented to each participant. Participants were asked to identify the discrepancies in each scenario. Scores were assigned for each scenario based on the total number of correct and incorrect selections. The major research questions were:

1. Are the mean scores for the participants who have completed the course, "Introduction to Quantity Food Production Management" equal to the mean scores for the participants who have not completed the course?

2. Are the mean scores for the participants who completed the immediate feedback treatment, the delayed feedback treatment, and the no feedback treatment equal?

Additional research questions were posed to determine if there was interaction or effects related to the order of presentation of each of the five scenarios and if the level of difficulty of the five scenarios was equal. Specific questions were:

3. Are the mean scores for the five order positions of viewing equal?

4. Are the mean scores for all five individual scenarios equal?

Development of the Study Unit

The goal of the computer based study unit was to improve the participants' ability to select meal service discrepancies in a dining room and to identify why that item is a discrepancy. The intention step outlined the theme of the lesson for the participants. Students were given instructions on how to access the computer based unit and then viewed the study unit objectives.

For the presentation step, five service scenarios were developed using Asymetrix's Toolbook 3.0 ( Asymetrix Corporation, 1994 ) software package. Students scheduled a 2-hour session to complete the study unit in a university computer lab. One researcher was present during the sessions to answer any questions. Each scenario consisted of a series of still photographs showing scenes of a dining room prior to service. The pictures were taken from the viewpoint of a manager walking through the dining room prior to opening for service. In each series of three to five photographs, five table service items were out of place or incorrect in the dining room, such as missing silverware, incorrect table setting, or soiled glassware. The participants were not informed of the number of incorrect items, or discrepancies, in each scenario. The order in which each student viewed the five scenarios was randomly based on the ten possible combinations. The student used the mouse to click on any part of the dining room scene that they believed to be incorrect. Programmed hidden buttons, located under areas of pictures on the computer screen, were activated by the mouse 'click'. These hidden buttons acted as navigational devices to direct the student through the lesson.

During the development of the scenarios, a panel of three university foodservice management instructors evaluated each scenario and rated the level of difficulty of each discrepancy. Point values for each level of difficulty were determined from this information. For each scenario, participants' scores were determined by adding points to the score for each correct selection of a discrepancy made and deducting points when the participant failed to select a discrepancy in the scenario.

The response matrix for this study unit was comprised of nine facts, theories, and principles that must be followed to successfully maintain the expected standards of a fine dining room prior to opening for service, shown in Figure 2. These principles were obtained from a standard dining service text ( Axler & Litrides, 1990 ) and from professional industry experiences of the researchers to test the value of reflective feedback. Participants receiving the immediate feedback treatment (IFT) were shown the response matrix immediately after selecting each discrepancy. After making selections from the response matrix the students were told if their original selection of the discrepancy was accurate and if they selected the correct concepts and facts from the response matrix that supported that selection. The participants receiving the delayed feedback treatment (DFT) selected all discrepancies from each scenario before viewing the response matrix. The participants were given the option on each screen of the scenario to conclude that they had identified all of the discrepancies. When that decision was made, they selected the option to view the response matrix. The participants were reminded of the discrepancies they had chosen and were asked to select the concepts or facts from the response matrix that explained why that discrepancy was chosen. Selection of anything in the scenario that was not a discrepancy followed the same procedure as in the IFT. The participants in the control group did not receive any feedback or the response matrix. They selected any discrepancies in the scenario and continued on to the next scenario.

Using the response matrix, students in the immediate and delayed feedback groups selected the relevant concepts and facts that they thought supported their selection of a particular discrepancy. For this study unit, the discussion guide and the discussion comments were presented to the participants at the same time. After making this comparison an explanation for each of the author's answers from the response matrix was displayed. If the participants did not select certain facts or concepts from the response matrix that the author had, the participants were given an explanation for why the author included them. If the participants selected concepts and facts for the answer that the author did not, the participants were given the author's justifications for why they were left out.

Figure 2. Structural communication response matrix for dining room standards
It is important to ensure that waiters maintain efficient time management during the service encounter. The customer expects and deserves a sanitary dining environment. Customer safety is an important priority.
Service personnel safety is an important priority. It is important for the service personnel to maintain consistency of the serving setting. It is important for the customers that the consistency of the service setting is maintained.
It is important to maintain the physical atmosphere of the dining room for the customer. The quality of the dining experience effects the customers' impression of the restaurant. Maintaining the physical service components of the dining room leads to cost efficiency.

Analysis Methods

Data collected for each participant included the treatment type, the scenario pattern, if the participant had completed "Introduction to Quantity Food Production Management," total time to complete all scenarios, the discrepancies selected, and the number of null or incorrect selections of discrepancies. A model was developed for a repeated measures study using the Latin square double cross-over design ( Neter, Wasserman, & Kutner, 1990 ). The design contains two squares that organized the scenarios by order position and pattern. There were five order positions (which scenario was presented first, second, third, etc.) and ten possible order patterns (A, B, C, D, E, vs. B, C, D, E, A, etc.) for delivery of the five scenarios. Each square contained the five order positions and five of the ten possible order patterns. With this model each order position was shown the same number of times to the participants and the ten possible order patterns of the five scenarios were also delivered the same number of times. The Latin square design was selected to eliminate the possibility of an order effect, the order in which the participants were shown the scenarios. The design also controls for a pattern effect, the sequence of the scenario presentations. If these effects were not controlled by the design, it would be impossible to conclude if the instructional lesson had an impact on the participants' ability to select dining room discrepancies or if it were the order and pattern sequence in which the scenarios were delivered that had an impact. Data were analyzed to determine if assignment to one of the two Latin squares effected the participants' scores. Data were also analyzed to determine how the combination of prior instruction on maintaining quality standards and receiving a particular treatment effected the participant's score. Comparisons of students within each treatment group, IFT, DFT, NFT, classified by whether they had received prior instruction or not, were not determined since each treatment group was not selected based on level of prior knowledge of quality standards. Analysis of variance and Scheffes' multiple comparison procedure for testing pairwise comparisons were used for analyzing the data using the statistical analysis system package SAS ( 1990 ). An alpha level of .05 was used for all statistical tests.

RESULTS AND DISCUSSION

Data analysis of the first research question indicated that the mean scores for the participants who had completed "Introduction to Quantity Food Production Management" were statistically different (F=16.84, p<.0001) from the mean scores for the participants who had not completed "Introduction to Quantity Food Production Management." The group of students who had completed "Introduction to Quantity Food Production Management" had a higher mean score than the group who had not completed "Introduction to Quantity Food Production Management." This finding indicated that prior instruction on maintaining quality standards improved the students' ability to select discrepancies in this instructional lesson. The effect of a combination of prior instruction on quality service and receiving a particular treatment was not statistically significant.

Analysis of the second research question showed that the participants who received the NFT treatment had significantly different scores than the participants who received the IFT or DFT treatments (F=8.17, p <.0008). Since the NFT group mean score was lower, this finding seemed to indicate that reflective feedback was fundamental for the participants in selecting discrepancies during the instructional lesson. The Scheffes' multiple comparison procedure showed no differences between the mean scores of the participants who completed IFT or DFT treatments. These results indicated the importance of reflective feedback but failed to indicate whether the IFT or DFT is the best position for presentation of the reflective feedback in a study unit.

The third research question examined whether the mean scores for the five order positions of viewing were equal. Analysis indicated that the mean scores for the five order positions of viewing, were not equal (F=2.15, p<.0001). The experimental design was developed to avoid this possible effect. The Scheffes' multiple comparison results indicated that order position one was significantly different from order positions four and five. The mean scores for the scenarios positioned at the end of the instructional lesson were higher than the mean scores for the scenario positioned at the beginning of the lesson. This result indicated that there was a possible learning effect, separate from other effects, as the participants progressed from one scenario to the next. This finding suggested that reflective feedback increased the participants' ability to select discrepancies as they progressed through the study unit.

The final research question looked at whether the mean scores for all of the five individual scenarios were equal. Analysis of data indicated that the mean scores for the five scenarios were statistically different (F=7.74, p<.0001). The expert panel used in the design process evaluated each scenario and determined them to be of equal difficulty. However, the results from the data analysis seemed to indicate that the scenarios had different levels of difficulty.

The mean scores for students assigned to Latin square one were not statistically different from the mean scores for Latin square two. This result was expected, as the experimental design was developed to prevent any possible interaction effects between the two squares.

Limitations

The study was not controlled for the course material students received on service standards. Several instructors were employed to teach laboratory sections, and some emphasized service standards more than others. Since the goal was to separate the students who had been exposed to service standards in the course and those who hadn't taken the course, this limitation was not critical.

The use of still photographs may have limited the participants' ability to identify discrepancies. The study unit did not allow the participant to view the setting from various positions or angles. Additionally, the images were two dimensional representations of three dimensional settings. As video technology improves these limitations will not occur, however, the quality of full motion video was not acceptable when the study unit was developed.

CONCLUSIONS, IMPLICATIONS AND RECOMMENDATIONS

The design and development of an effective and efficient instructional lesson for the maintenance of standards in a dining room was a major goal of this research. The challenge was to develop an instructional lesson that would improve the participants' ability to identify consistency of operation principles, and safety and sanitation issues in a dining room, using the most effective instructional system design principles. An instructional lesson was developed based on the instructional system design theory, structural communication. The study unit was delivered to participants and data were collected and analyzed.

Application of Structural Communication with reflective feedback appeared to improve participants' ability to learn lesson material. Results indicated that the mean scores for participants who received reflective feedback were higher than the mean scores for participants who did not receive reflective feedback. Reflective feedback appeared to contribute to the participants' success in selecting discrepancies from the scenarios contained in the instructional lesson. Mean scores for the participants who received the study units containing either the IFT or DFT were not significantly different. These data indicated that the immediate placement of reflective feedback in the study unit was not more effective than the delayed placement of the reflective feedback, for increasing the ability of a participant to select discrepancies in the instructional lesson. The researchers concluded that the placement of feedback within the lesson was not as critical as the inclusion of feedback. These results have important implications for both hospitality management faculty and hospitality professionals. It is imperative for them to realize how important reflective feedback is and to use it effectively in any instructional materials they develop.

Reflective feedback appears to be a successful strategy to improve participants' ability to learn specified material. The participants were randomly assigned to one of the three treatments, and order pattern of the five scenarios so any bias was eliminated. Additionally, the research design was structured so the five scenarios were viewed in one of the ten order patterns by the same number of participants. Participants' mean scores increased as they proceeded through the five scenarios. The research analysis model was developed to control any possibility of an interaction effect in terms of the order in which the scenarios were presented. Since there was no outside influence or instruction during the session, it was concluded that reflective feedback was the primary factor that provided students with the knowledge to improve their scores. The reflection process increased the participants' ability to select discrepancies as they progressed from one scenario to the next in the study unit.

An NRA expert panel ( NRA, 1994 ) projected that virtual reality, multimedia, and interactivity would be the hallmarks of future foodservice training as these technologies allow individuals to study lesson materials, interact with other learners and teachers, and access current resources on the Web according to their time schedules. Pedagogical techniques will include modeling, simulation, tutorials, and intelligent computer assisted instruction. Well-designed software will provide feedback to students to help them solve problems and make decisions effectively. If hospitality management educators wish to participate in the design and development of interactive training programs, they will need to become cognizant of instructional theory and how to effectively incorporate the theory in the design of course materials.

Two major areas of additional research are indicated. Further research is needed to investigate the long term effects of structural communication and reflective feedback to determine if participants are able to apply the knowledge attained in the study unit. Participants would complete the instructional study unit and then be tested at specified time intervals in actual dining rooms. Additionally, research is warranted to determine the number of scenarios that optimize the learning curve. If reflective feedback allows instructional objectives to be attained more efficiently, more subject matter can be addressed. As instructional technology continues to evolve, more research will be needed to determine which technology is appropriate for the various hospitality education settings.

BIBLIOGRAPHY

Asymetrix Toolbook 3.0 (1994). Asymetrix Corporation. Bellevue, Washington.

Axler, B, & Litrides, C. (1990). Food and Beverage Service. New York, NY.: John Wiley and Sons, Inc.

Baath, J. A. (1979). Correspondence education in the light of a number of contemporary teaching models. (Report No. IR 010 508). Lund, Sweden: Department of Education, University of Lund. (ERIC Document Reproduction Service No. ED 224 465).

Bitner, M.J. (1990). Evaluating service encounters: The effects of physical surrounding and employee responses. Journal of Marketing, 54(2), 69-82.

Egan, K. (1973). Measuring the ability to organize knowledge. Educational Technology, XIII (5), 53-56.

Egan, K. (1976). Structural communication Belmont, CA.: Fearon Publishers, Inc.

Gagneacute;, R.M., Briggs, L.J., & Wagner, W.W. (1988). Principles of instructional design, Orlando,FL:

Holt, Rinehart, and Winston, Inc.

Glover, G. (1988). Managing quality in the hospitality industry. FIU Hospitality Review, 6 (1), 1-14.

Heskett, J.L., Sasser, W. E. & Hart, C. W. L. (1990). Service Breakthroughs: Changing the Rules of the Game. New York, NY,: The Free Press

Jost, K.L. (1990). Computer-based interactive video: The potential for effective instructional environments. Instructional Developments, 1 (3), 16-21.

National Restaurant Association (1994). Foodservice 2050: A Glimpse at the Future of Foodservice. National Restaurant Association.

National Restaurant Association (1996a). 1996 Tableservice Operator Survey. National Restaurant Association.

National Restaurant Association (1996b). Dinner Decision Making, 1996. National Restaurant Association.

Neter, J., Wasserman, W., & Kutner, M. (1990). Applied Linear Statistical Models (3rd ed.). Homewood, IL.: Irwin.

Parasuraman, A, Zeithaml, V. & Berry, L. (1988). SERVQUAL: A multiple-Item Scale for Measuring Customer Perceptions of Service quality. Journal of Retailing. 64(Spring), 12-40.

Restaurants USA. (1992, December). Forecast 1993. Restaurants USA, pp. 12-32.

Restaurants USA. (1993, December). Forecast 1994. Restaurants USA, pp. 15-38.

Restaurants USA. (1998, February). Turnover takes a turn for the better. Restaurants USA, pp. 39-41.

Romiszowski, A.J. (1986). Developing auto-instructional materials: From programmed texts to CAI & interactive video. London: Kogan Press.

Statistical Analysis Software, for UNIX Systems, Release 6.07 (1990). Cary North, Carolina.

Stevens, P., Knutson, B., & Patton, M. (1995). DINESERV: A Tool for Measuring Service Quality in Restaurants. Cornell Hotel and Restaurant Administration Quarterly. 36(2): 56-60

Zeithaml, V.A., Parasuraman, A., & Berry, L.L. (1990). Delivering quality service: Balancing customers perceptions and expectations. New York: The Free Press.

Zemke, R., & Schaaf, D. (1989). The service edge. New York: Penguin Books.