JITE v40n2 - Evaluation of a Community College Technical Program by Local Industry
Evaluation of a Community College Technical Program by Local Industry
Richard W. Zinser
Western Michigan UniversityDuring the past two decades, postsecondary occupational education has become a central issue in workforce development because of the complexities of new technology and global business. Community colleges are a logical source for employee skill development, and many colleges are making a significant business impact ( Bailey & Kienzl, 1999 ). Although community colleges have always provided vocational education, there is an increasing sense of urgency to work with individual industries to offer content-specific courses. This was a new direction for the community college, "from an institution focused on educating students to one centered on meeting the needs of business and the local economy" ( Dougherty & Bakia, 2000 , p. 1). Some colleges even operate advanced technology centers that help companies track and try out new technologies, and then introduce them into the workplace.
Industrial employers are trying to create a more effective system of workforce development. According to Glenn (2001) , "the number one business problem for members of the National Association of Manufacturers is hiring, retaining, and maintaining a skilled workforce" (p. 8). On the local level, companies are working with their community college and other providers for education and training. In southern Texas, for example, an alliance of companies and colleges has developed an associate's degree in process technology for the petrochemical industry. The partners collaborated to design a curriculum based on the skills and tasks performed by technicians, including applied modules developed by chemistry, mathematics, and physics faculty ( Imel, 2001 ).
In their article on American workforce development, Hamm and Mundhenk (1995) posed the question: Are community colleges "prepared to become the central player in a comprehensive education and training system?" (p. 1). If this ideal were realized, there would be exclusive connections between community college programs and businesses ( Bragg & Hamm, 1996 ). Accordingly, the college program would hold the major share of the training market and be directly involved in preparing employees for the local industry. Like their counterparts in the private sector, community colleges should offer market-driven products that are high in quality and demonstrate continuous improvement.
To establish this position as the preferred supplier, the community college must evaluate its programs on specific outcome variables agreed upon with their business partners. Traditionally, colleges have conducted evaluation on career outcomes that focused on the extent to which graduates are working and progressing in the field for which they were prepared ( Conklin, 1990 ). For contract training, evaluation models are often used ( Jacobs & Bragg, 1994 ) to collect information on students' reactions and amount of learning, and ideally on the extent to which they are using the skills on the job and the impact it is having on the business.
However, according to Brinkerhoff (1988) , evaluation on the latter two levels is often omitted for practical reasons (time and money); thus these important sources of information are lost. There is very little research on evaluating individual programs from the perspective of employers. Research is probably being conducted informally, by advisory committees, for example; but because the issues and decision making are largely a local affair, the results may not be published. So the evaluation project which is the subject of the current article may serve as a case study for the process of making connections between a specific technical curriculum and the local industry that employs the graduates.
Statement of the Problem
The problem was clear: the local plastics companies were growing, but enrollments in the plastics program were falling. Based on the companies' apparent need for education and training, the companies should have been sending numerous employees to take plastics courses at the college and encouraging employees to complete either the one-year certificate or the two- year associate's degree in plastics technology. Instead, some courses were canceled due to low enrollment; and there were only a few program completers in the past five years.
The plastics industry was recognized as a major player in the local economy through two recent initiatives. In 1998 an economic development group established clusters of various industries in the county. The plastics cluster was one of the first to be formed for the purpose of meeting with executives across companies to discuss common issues and develop industry-wide strategies. Secondly, in 1999 the county applied for and was awarded a major state grant to build an advanced technical education facility; and, again, the plastics industry was put near the top of the list for training needs. These two developments helped bring to light the fact that the college's plastics program was being underutilized. Since there were many opinions as to the cause of the problem, from both industry representatives and the educational institution, a systematic evaluation of the program was necessary.
The objectives of the study were to involve the plastics companies and current students to evaluate why the program was not being utilized to its potential and what could be done to improve it. Following Brinkerhoff's (1998) evaluation model, there are six stages or levels of evaluation.
- Evaluate needs and goals.
- Evaluate program design.
- Evaluate implementation and operation.
- Evaluate immediate outcomes.
- Evaluate endurance of outcomes.
- Evaluate organizational benefits (p. 27).
The last stage was conducted for the purpose of deciding whether to continue the program and to account to top management. The managers in this case were the company executives who sponsor employee education and the college dean who supervises the program. Questions from this evaluation included the following.
- How do graduates perceive the quality and value of the program?
- How do graduates' supervisors perceive the value of the program?
- Who has used the training and how?
- What value has derived from its use? (p. 208)
Answers to these questions would provide a sound basis for making decisions about the program.
Another criterion for the evaluation was to insure that an objective, valid process was used because of the high-profile nature of the plastics industry in the immediate area. To this end, the evaluation process outlined by Knox (1998) was followed. Knox included eight guidelines to conduct a specific program evaluation project: "purpose, stakeholders, planning, coordination, sources, collection, analysis, and utilization" (p. 9-10). It was thought that following the guidelines for the plastics evaluation would help make the study credible for the diverse audience reading the report and discussing the results.
Research Design and Methodology
The project's purpose was to evaluate the status of the plastics program, specifically, to find out if there were important differences in the views of the major stakeholders. The major stakeholders were plastics company managers, current students, and college staff. Planning began with reviewing individuals to contact (managers and students) and identifying the questions to be asked.
A structured interview with 21 questions was designed to study the perceptions of the company managers. The topics included company growth and hiring trends, training activities, college enrollments, an evaluation of the plastics curriculum, and other elements of the program. The questions were developed and refined by the committee based on their experience in working with both the college and the industry.
A list of 17 plastics companies was compiled from a chamber of commerce database and targeted for the study. All of the companies were contacted by telephone by the coordinator and invited to participate in the evaluation. Three of the companies did not participate because none of their employees had ever taken plastics courses, so they were not familiar with the program; three other companies did not respond to the telephone inquiry. The 11 consenting companies were representative of local industry: two companies were classified as very large, three large, four medium-sized, and two small. The interviews were scheduled for 45 minutes over a two-week period. Each company representative was given a copy of the questions and an outline of the college's plastics curriculum.
For the students, it was decided to use structured questions in a focus group format. The 20 questions covered topics on their program attendance, employment situation, and satisfaction with the courses and other elements of the program. Open discussion time was also planned to encourage their suggestions. Utilizing a college database, 37 current and recent students were contacted through a letter from the dean requesting that they attend a meeting to discuss the program. After only six students responded to the letter, follow-up calls and announcements in classes were made to encourage attendance. According to Grubb (1999) , response rates from students were typically very low. In retrospect, there may have been a better response with a different method, such as telephone interviews. A total of 10 students attended two different meetings. They were given a copy of the questions; then each question was discussed as a group, with the facilitator making notes on significant comments.
Results and Analysis
Industry Survey
As a result of the structured company interviews, it was documented that the plastics industry does make a significant impact on the local economy: The 11 companies in the sample population employ 3470 people, with 77% in production/technical positions and the balance in managerial/professional positions. All companies have experienced steady growth, ranging from 8% to 50% per year, with an average of 21% growth per year. All of the companies have current unfilled openings in virtually all positions and levels in the companies, with the majority in production/technical. A tuition assistance program for employees for related courses or degree programs is offered by 10 of the companies. They expressed satisfaction with the cost of the plastics program (11/11) and the location of the college (8/11); they expressed dissatisfaction (2/11) with the course schedule, facilities and equipment, and the level of expertise.
Of particular interest was the companies' perception of the adequacy of specific courses to prepare employees for work. Table 1 lists the courses in the curriculum and the number of companies satisfied with the content. The data represent a relatively low level of satisfaction with the plastics courses. During the interviews, some of the company representatives commented that most of these topics were crucial for their employees, so the courses should be helpful but were not helpful as currently presented.
Table 1
Company Satisfaction with Plastics Courses
Individual Course Percent of Companies Satisfied N = 11
Introduction to Plastics 27.3 Injection Molding 54.5 Introduction to Polymers 18.2 Advanced Injection Molding 45.5 Extrusion Processes 0.09 Plastics Processes 0.09 Quality Control for Plastics 54.5 Molding Machine Maintenance 36.4 Injection Machine Controls 36.4
Companies were also asked what other topics should be included in the curriculum. Without hesitation, they mentioned numerous topics for which they were currently providing training for their employees, either by in-house trainers or private contractors. The list included, for example, training on QS9000 procedures, computer controls, and technical problem solving, all of which they felt could be covered in the college courses. The main focus of the companies' dissatisfaction with the college program was that the curriculum and instructional methods were out of date; and it was therefore more effective to train their employees themselves, since they have the latest technology.
Student Survey
The results of the two focus group meetings with students were similar to the industry survey, although in some cases the responses were more detailed. All of the students were currently attending classes; one student was working toward a certificate, and the other nine said they were planning to complete the associate's degree. Half of the students were working part-time; the other half were working full-time at plastics companies.
As in the company interviews, discussions with students revealed that they were very satisfied with the cost of the courses and the location of the college. Some students commented that, because they work on the afternoon or night shifts, it was difficult to take courses during the normal college schedule of late afternoon and early evening classes. They even suggested that they would consider weekend and summer classes. In response to the question on facilities and equipment, the students agreed that the machines in the lab were outdated and did not resemble the equipment they use at work. Finally, they felt that there was insufficient assistance from the college with job placement. Perhaps this means that students currently working in plastics companies felt that they should hold higher positions in their company compared to other employees because of their college course work.
Discussion and Recommendations
This evaluation project collected both quantitative and qualitative data to help decide on the direction of the community college plastics technology program. There was no doubt that the local industry is growing, as evidenced by the hiring and training of many new employees. However, the companies are not utilizing the college program to any great extent, mainly because of the outdated equipment, and to a lesser degree because of the traditional college schedule and course structure. There seemed to be a disconnect between the priorities of the industry and the college: the companies need focused, high-quality training for job preparation and employee development, while the college must consider the requirements of a 16-week course for credit along with fulfilling the requirements of an associate's degree program and possible transfer to a four-year program. Clearly, a lot of work must be done to reach the ideal of customer-driven quality for education that has been discussed in many circles lately and described succinctly in the National Alliance of Business (1999) report Baldrige in Education .
The first recommendation for the plastics program was that in order to thrive and make a significant contribution, the status quo cannot be maintained. To improve the program, an action plan that includes the more specific recommendations, starting with updating the equipment, should be developed and implemented. The college, the students, and the companies agree that the plastics lab must have current model machines, the kind actually used in industry. This would obviously improve the credibility of the program; but, because of the cost, this would not be easily accomplished. One company manager suggested that perhaps a consortium of businesses could help fund the project; another suggested that the equipment suppliers might be willing to loan or sell demonstration models of their machines. Certainly this needs further investigation, as other technical programs are successfully dealing with the same challenge.
A second recommendation was to further clarify the difference between objectives for training and education. In-depth training on specific topics identified by industry may be provided more effectively through a contract training arrangement with the college. The companies are already doing this with independent contractors, and the colleges are already doing contract training with other companies for other topics. It would still require updated equipment or training conducted on-site with the companies' equipment, and college credit may not be awarded for such training.
Another training program needed by industry is pre-employment training because of the large number of new hires due to growth and turnover. This is currently being done by community colleges in areas in which a certain industry has a large local presence. A recent study by a large plastics company ( Zinser, 2001 ) showed that training new employees on career opportunities and what to expect from the job significantly reduced turnover for 90-day employees. Other general topics such as safety and quality that were applicable to all companies could be included, which would reduce the amount of training that businesses are currently providing their employees. In other words, the college would be in a position to save companies money.
It was recommended that the college faculty work more closely with the companies to improve credibility and relevance in the classroom. This was supported by a study by Brewer and Gray (1997) in which it was found that faculty linkages to local labor markets were "generally ad hoc and informal in nature" (p. 6). Most institutions do not have systematic plans or strategies to develop these links; the teachers are generally on their own and do not receive any encouragement from their institutions to pursue it. Many vocational programs have advisory committees and occasionally take students on tours; but only about half have ever asked a company to review a course syllabus, and very few instructors use business case studies or assignments involving companies in their classrooms (p. 26).
A broader issue in postsecondary education identified by this study was the nontraditional pathways of students and the impact on course scheduling. Grubb (1999) described the "more fluid environment" (p. 15) of occupational education, in which many students do not take a direct, linear path for their education. Students do not enroll, attend full time, progress through course requirements in general education and their major, and achieve an associate's degree in two years. Although community colleges have made many adaptations to accommodate these nontraditional students, still more is necessary. Most of the students in the current study work the late shifts for companies with 24-hour operations, which makes the conventional college schedule difficult. They suggested offering classes in the morning, a Saturday section as an alternative to weekdays, and summer courses to provide continuity throughout the year. Obviously the companies would have to commit to filling these sections to make it practical for the college to offer them.
For other workshops or seminars, a one-day format or several half-days may work the best for the companies; and these should be offered both at the college and on-site. The instructor would have to be willing to work some unconventional hours, such as 5:00 a.m. to 9:00 a.m. to overlap the last two hours of the night shift and the first two hours of the day shift, or 9:00 p.m. to 1:00 a.m. to overlap the afternoon and night shifts. Other possibilities include several days of training during the Christmas and July 4th holidays that were shutdown periods for some companies. Somehow the curriculum should be modularized to "create shorter credentials and early-exit options" ( Grubb, 1999 , p. 14).
Concerning the individual courses in the plastics technology program, the employers had strong opinions about needing appropriate updates. The introductory courses in molding and quality provide good foundations for employees, but the advanced courses were not as helpful. The machine maintenance and machine controls courses were badly needed by industry; but, again, because of the obsolete equipment, they were not very effective. Several of the courses were useful only for employees working toward a degree. In general, the curriculum should be updated to include more of the topics that the employers need, as evidenced by their current training programs for employees.
A final recommendation was that all of the college's technical programs should be evaluated by the corresponding industries, using a similar process. One of the main lessons learned from the plastics project was that the gulf between the college and the employers had existed for some time before any action was taken. Therefore, colleges should be proactive by conducting regular evaluations of each program, perhaps every second or third year; and this evaluation should become a permanent process in the institutional system.
Implications for Technical Teachers
The plastics industry is growing in the geographical area of this study, and there is a high demand for education and training. The plastics programs at the colleges are well positioned to help provide these services, but their technology and their partnership with industry in general have declined to the point where credibility with the employers may have been lost. The potential, however, is great. The companies have many job openings for process technicians, with relatively high wages and opportunities for individual advancement; but they must support and be more involved with the college.
How can community colleges meet industries' needs? How can this disconnect be prevented at other colleges, and what can technical teachers do to improve the situation? First, many of the recommendations outlined in this study were applicable to technical programs in general. For example, technical teachers often take the initiative to keep their equipment up to date for technology-conscious students, which requires substantial support from businesses. Teachers must be constantly campaigning in the offices of administrators and executives for funding. Secondly, faculty must work closely with technicians in industry to insure that currently needed specific topics are included in their courses. In this way they can make more explicit connections to the workplace for their students. Another recommendation that teachers can implement is to respond to business's need for nontraditional education by providing different formats, schedules, and even locations for their courses.
There were also several specific implications from the study that technical teachers might consider. It should be acknowledged that in many cases teachers are already doing some of these things, at least to some extent. First, there should be a program advisory committee, with the majority of members from business, which meets a minimum of twice per year. The cost of the meetings should be funded jointly by business and education, and the members should be accountable for conducting meetings and achieving their objectives. The location of the meetings could be rotated to different members' sites; but all meetings should be managed in a business-like manner with an agenda, action items, and assigned responsibilities. This will help maintain a healthy relationship among the partners because it serves as an on-going program evaluation.
Secondly, there must be continuous curriculum review by the committee, which was described as a "critical issue" by Anderson and Kosarek (1997) . As gaps between the program and the industry are identified, solutions can be discussed and implemented. Perhaps industry will recognize the importance of academic requirements; and, likewise, the college will understand the rationale for industry's suggestions. The curriculum can then be fine-tuned by making additions and deletions of content where appropriate, and by constantly renegotiating who will do what. When needs for the program surface, such as material and equipment, industry may be more supportive because of their involvement in the process. In addition, both partners may become more willing to consider non-traditional solutions to problems, such as increased flexibility of class schedules and locations, and more customized on-site training.
To accomplish this, perhaps what is needed is a more strategic alliance, described by Holton and Trott (1996) as a blurring of the boundaries between the corporate human resource development function and public vocational education. In other words, the individual training managers and college instructors need to meet frequently to keep up to date on their respective needs. Ideally, the college would then become part of the corporate planning process and serve as a preferred supplier of technical training
Finally, it may be helpful for technical teacher educators and technical deans to emphasize how important it is for technical faculty to be involved in the business community. It is often left to the individual teacher to use his/her initiative in making contacts and building partnerships. However, when this is done systematically as part of the teaching process, improvements in the program and thus enrollments are inevitable.
Conclusion
This project was an opportunity to conduct an evaluation of a specific technical program for a community college, as perceived by students and local industry. It was useful to discuss the study in the larger context of trends in occupational education and business partnerships for which the recommendations can be generalized. Several important lessons were learned from this project, such as the need to be proactive in conducting evaluations. Certainly more evaluation research needs to be done so that colleges can keep their programs up to date and satisfy their business customers.
References
Anderson, A., & Kosarek, D. (1997) . Critical issues in the evolving relationship between business, industry, and post-secondary education . Waco, TX: Center for Occupational Research and Development.
Bailey, M., & Kienzl, G. (1999) . What can we learn about post-secondary vocational education from existing data? National Assessment of Vocational Education. From http://www.ed.gov/offices/OUS/PES/NAVE/bailey.html .
Bragg, D.D., & Hamm, R. (1996) . Linking college and work: Exemplary policies and practices of two-year college work-based learning programs (MDS-795). Berkeley: National Center for Research in Vocational Education, University of California at Berkeley.
Brewer, D., & Gray, M. (1997) . Connecting college and community in the new economy? An analysis of community college faculty-labor market linkages (MDS-1084). Berkeley: National Center for Research in Vocational Education, University of California at Berkeley.
Brinkerhoff, R. (1988) . Achieving results from training . San Francisco: Jossey-Bass.
Conklin, K. (1990) . Assessment of institutional effectiveness: Career student outcomes. Community/Junior College Quarterly of Research and Practice , 14 (4), 349-357.
Dougherty, K., & Bakia, M. (2000) . The new economic development role of the community college (CCRC Brief No. 6). Community College Research Center, Columbia University.
Glenn, J. (2001) . The giving and the taking: Business-education partnerships come of age. Business Education Forum , 55(3), 6-9. National Business Education Association.
Grubb, N. (1999) . Edging toward effectiveness: Examining postsecondary occupational education . National Assessment of Vocational Education. Retrieved from http://www.ed.gov/offices/OUS/PES/NAVE/GrubbI.html .
Hamm, R., & Mundhenk, R. (1995) . American workforce development: Community and technical colleges prepare to meet the challenges . Columbus, OH: National Council for Occupational Education.
Holton, E., & Trott, J. (1996) . Trends toward a closer integration of vocational education and human resource development. Journal of Vocational and Technical Education , 12(2), 49-57.
Imel, S. (2001) . Business-industry relationships and CTE. In Brief No. 12 , National Dissemination Center for Career and Technical Education.
Jacobs, J., & Bragg, D. (1994) . The evaluation of customized training. New Directions for Community Colleges . 22(1), 13-24.
Knox, A. (1998) . Evaluating adult and continuing education . Columbus, OH: Center on Education and Training.
National Alliance of Business (1999) . Baldrige in education: Improving student performance . Washington, DC: Author.
Zinser, R. (2001) . Developing a career matrix for technology students. The Technology Teacher , 60(4), 9-12.
Zinser is Assistant Professor in the Department of Family and Consumer Sciences at Western Michigan University in Kalamazoo, Michigan. Zinser can be reached at richard.zinser@wmich.edu .