Journal of Career and Technical Education
Animal Science Experts’ Opinions on the Non-Technical Skills Secondary Agricultural Education Graduates Need for Employment in the Animal Science Industry: A Delphi Study
Wendy L. Slusher
J. Shane Robinson
M. Craig Edwards
Oklahoma State University
Non-technical, employability skills are in high demand for entry-level job-seekers. As such, this study sought to describe the perceptions of Oklahoma’s animal science industry leaders as it related to the employability skills needed for entry-level employment of high school graduates who had completed coursework in Oklahoma’s Agricultural, Food and Natural Resources, animal systems pathway. In all, 33 non-technical statements describing skills which students need to be employable were generated from the panel of experts. All 33 reached consensus in Round Two of the study. Panelists agreed unanimously that being honest was the most important non-technical skill for a potential employee to possess. Because all 33 skills reached agreement, it is recommended that secondary agricultural education instructors integrate all 33 into their existing curriculum whenever appropriate to increase human capital holding value related to the entry-level employment of high school graduates.
Workers in the 21st century need skills such as problem-solving and analytic, decision-making, organization and time management, risk-taking, and communication (Evers, Rush, & Berdrow, 1998; Lynch, 2000; Robinson, Garton, & Vaughn, 2007) to be employable in the workforce. Lynch (2000) posited ". . . there is a tremendous demand for educated people with general employability and specialized technical skills . . ." (p. 4). Therefore, a need exists to determine what types of skill sets are demanded of secondary high school graduates by industry because there is a "general consensus that occupational preparation . . . should begin sometime in high school" (p. 7).
Several studies have examined career and technical education (CTE) and its role in the employment of U.S. citizens (Conroy, 1998; Lynch, 2000; Martinez Jr., 2007; Rojewski, 2002). Conroy's (1998) study related to how adolescent males and females chose to enroll in CTE programs according to their future career aspirations. Forty-one percent of participants intended to pursue professional careers after high school graduation. However, the participants did not intend to pursue post-secondary education and assumed they would receive lower job positions versus individuals who had obtained some post-secondary education. Based on these findings, Conroy (1998) suggested that CTE courses be "integrated into the total education program, and provide a broad-based exposure to the world of work through experiential learning" (p. 9). Lynch (2000) reaffirmed the concepts of integrating CTE into the overall education curriculum by stating that "career and technical education is integral to whole school, comprehensive reform; it is not separate from it" (p. 8).
Researchers have suggested that the future of CTE should be flexible enough to prepare students for both industry and college simultaneously (Lynch; Roberts & Ball, 2009). This suggestion exists because employability skills, such as work ethics and decision making, are important for students who plan to attend post-secondary education institutions (Conroy) as well as those who have a desire to enter the workforce immediately on graduation (Roberts & Ball).
One area of CTE is agricultural education. Specifically, Oklahoma's agricultural education programs served over 26,000 students in 2007 (Oklahoma Agricultural Education Division, 2007). Approximately, 55% of these students were projected to enroll in college after graduating from high school (Oklahoma Agricultural Education Division). The remaining 45% of Oklahoma's agricultural education high school graduates were projected to enlist into the military service, enter the workforce, or be unemployed (Oklahoma Agricultural Education Division). Therefore, obtaining valuable skill sets needed to be employable and for successful citizenship is imperative for these graduates as well as society.
Czaja (2006) reported that as soon as "2010, the number of workers aged 55+ will be at about 26 million . . ." (p. 283). This increase in retirees will create areas of employment in top level positions among all industries (Carnevale, 2003), thus, requiring highly skilled employees to emerge in the workplace. Although employment opportunities will be available, the question remains whether viable employees will be available to fill these positions. Czaja found "at the same time that the work force is aging there is also a slowed growth in the number of younger workers and slowing in the growth of the labor supply" (p. 283). So, it could be hypothesized that this will create an opportunity for willing and able high school graduates to fill needed positions of employment if they possess the desired skill sets employers need.
Robinson (2000) found that "most discussions concerning today's workforce eventually turn to employability skills," which are seen as "basic skills" that aid individuals in "getting, keeping, and doing well on a job" (p. 1). These skills can be, but are not limited to, reliability, responsibility, problem-solving, and social competencies (Robinson). Overtoom (2000) defined employability skills as "transferable core skill groups that represent essential functional and enabling knowledge, skills, and attitudes required by the 21st century" (p. 1). Overtoom suggested that employability skills are necessary for success in the job market regardless of the employee's chosen career path, employment level, or educational background.
Czaja and Ascher (1988) asserted that employers will focus more on previous work experience than academic accomplishments. Additionally, employees need to be capable of learning on the job and using a cognitive style of skills, which will include handling success and failure on the job and recovering from the resulting outcomes (Carnevale, 2003). Employers will demand future employees have problem-solving and behavioral skills (Ascher, 1988; Carnevale) as well as the ability to work in teams and perform under increased human interaction scenarios between employees and consumers (Carnevale).
In a study of 1000 job advertisements by Bennett (2002), it was revealed that the entry-level skills needed for marketing positions focused on IT and presentation. Management positions required skills involving motivation and communication. Finance positions required analytical, numeracy skills. And, Human Resource Management positions sought communication skills. Yet, employers have been most hesitant to hire young, inexperienced workers (Krahn, Lowe, & Lehmann, 2002) because of factors such as poor grades, attitudes, motivation levels, preparation, and basic skills, such as oral and written communication, to name a few (Charner, 1988). Therefore, the question remains, where will employees be able to acquire such skills for future employment? Public school systems could be a viable source for meeting employers' demands.
The need to improve the employability skills of the workforce has been an issue across all phases of education. Steps have been taken to define and address key skill areas needed for improvement, specifically at the secondary level. In 1991, the Secretary's Commission on Achieving Necessary Skills (SCANS) report was initiated to define the skills needed by high school graduates in the workforce. The focus of the report was to determine how high schools could best mirror industry in an effort to make the transition from school-to-work less difficult for secondary school graduates. In addition to aiding an easier transition for graduates, unemployment was also a concern. The report highlighted the need for students to learn basic skills such as math, oral communication, and written communication; thinking skills such as problem solving, decision making, and lifelong learning; and foundational skills such as responsibility, integrity, and confidence. A related document from SCANS (1992), titled "Learning a Living," suggested the best way to ensure K-12 students have acquired said skills is for teachers to teach "in context" by linking the skills needed in industry to specific objectives within the curriculum (para. Reinventing K-12 Education).
The theoretical framework for this study was based on the human capital theory. In its simplest form, human capital can be viewed as an investment in education and training (Becker, 1964). When analyzed, "it is human because it is embodied in man, and it is capital because it is a source of future satisfactions, or of future earnings, or both" (Shultz, 1971, p. 48). It is often argued whether human capital is limited to reading and writing or specialization within a given industry. Nevertheless, investing in an individual's knowledge of any given field will result in that person becoming a more productive asset in society.
Purpose of the Study
The purpose of this study was to describe the perceptions of Oklahoma's animal science industry leaders as it related to the employability skills needed for entry-level employment of high school graduates who had completed coursework in Oklahoma‘s Agricultural, Food and Natural Resources, animal systems pathway. Specifically, the animal science career pathway is one of seven offered in Oklahoma secondary agricultural education programs. The purpose of the pathways is to provide secondary students an opportunity to specialize in one area of agriculture and develop skill sets needed for employment in that industry. Courses in the animal systems career pathway include units and topics on basic introductory animal science, nutrition, reproduction, and equine science.
The research objective of this study was to identify the non-technical competencies deemed necessary for entry-level employment of high school graduates in the animal science industry after completing coursework successfully in the animal systems career pathway, as determined by consensus of an expert panel of animal science industry leaders in Oklahoma.
Population and Sample
The population for this study was Oklahoma's animal science industry experts from nine areas of specialization: beef cattle; dairy; equine; goat; implements/miscellaneous; poultry; sheep; swine; and veterinarians. Stitt-Gohdes and Crews (2002) stated that "careful selection of the panel of experts is the keystone to a successful Delphi study" (p. 60). Moreover, Wicklein (1993) asserted that it is important to have a strong set of criteria when selecting a panel "because the success of the Delphi Technique relies upon the use of informed opinion" (p. 57). Therefore, panel members were selected using a purposive sampling technique, which "is the process of selecting a sample that is believed to be representative of a given population" (Gay, Mills, & Airasian, 2006, p. 113).
A sample (n = 42) was obtained for the study. The criterion used for selecting individuals was based on their prior experience and knowledge of the industry as it pertained to employment, including the hiring of entry-level employees. Only those individuals who had previously hired or would consider hiring high school graduates were considered for this study. Additionally, all experts who served on the panel were affiliated with the animal science industry in Oklahoma. Finally, due to the nature of the data collection process, only those individuals who had access to the Internet and could respond to the questionnaire via electronic mail (e-mail) were considered as panel members.
An animal science professor at Oklahoma State University consulted with departmental colleagues to determine possible experts who should be included in the study. Following the initial collection of potential panel members, remaining committee members reviewed the list and made further additions. Panel members were then contacted via e-mail or telephone to determine their willingness to participate in the study.
Turoff and Linstone (2000) stated that a computer version of the Delphi technique "has the advantage of eliminating the delay caused in summarizing each round of Delphi . . ." (p. 5). Also, the heterogeneity of the panel members is protected and higher validity of the data results when utilizing an electronic questionnaire (Turoff & Linstone). Panel members who expressed a desire to participate were sent a consent form via e-mail. Their replies to the e-mail were considered as panelists' consent to participate in the study.
Dalkey (1969) stated that when a Delphi group is larger than 13 members, a reliability of at least .80 can be achieved. Of the 42 participants selected to participate, 32 responded in Round One. This produced a response rate of 76.2%. In Round Two, 26 participants responded for a 61.9% response rate. The data collection period terminated after Round Two due to panelists reaching consensus of agreement on all the items generated in Round One. Based on Dalkey (1969), within both rounds of the Delphi, reliability was maintained due to the level of responses surpassing the minimum of 13.
Round One consisted of collecting data related to the following open-ended stem:
Please list all the non-technical, employability skills (e.g., strong work ethic, punctuality, ability to communicate, etc.) you believe a high school agricultural education graduate should have to obtain entry-level employment in your area of the animal science sector.
The responses generated from Round One were transcribed into a Microsoft Word document and categorized by the researchers into common themes. Three researchers independently coded the statements into themes and then negotiated consensus regarding the appropriate categorization i.e., the emergent themes (Montgomery & Crittenden, 1977). Categories presented by Evers et al. (1998) were used as a framework to assist in creating and categorizing competencies originated by the panel. As such, statements generated from the panel members were grouped into six themes: Communication; Decision Making; Lifelong Learning; Personal Organization and Time Management; Personal Strengths; and Problem Solving.
Following the categorization of Round One responses, Round Two questions were developed in Microsoft WORD and transcribed into the WS FTP 95 software program. A total of 33 non-technical statements were generated for the Round Two questionnaire. The Round Two questionnaire asked participants to rank statements using a 4-point summated-rating scale: ("1" = "Strongly Disagree"; "2" = "Disagree"; "3" = "Agree"; "4" = "Strongly Agree").
Because panelists reached agreement in Round Two on all items generated in Round One, the data collection period was terminated thereafter. So, no items were re-submitted to the panelists for further review.
The statement "be honest" (M = 4.0; SD = 0.00) was the sole competency that received unanimous agreement from the panel (Table 1). Moreover, the following statements received a mean score above 3.70: "demonstrate a strong work ethic" (M = 3.96; SD = .20), "maintain a positive attitude" (M = 3.77; SD = 0.43), "gain trust of employer" (M = 3.77; SD = 0.43), "display a desire to learn new skills/information" (M = 3.77; SD = 0.43), "function as a team member" (M = 3.77; SD = 0.43), "adhere to deadlines" (M = 3.73; SD = 0.45), "acquire new knowledge on-the-job efficiently" (M = 3.73; SD = 0.45), and "respond verbally to customers, co-workers, and supervisors appropriately" (M = 3.73; SD = 0.45). Twenty non-technical competency statements received mean ratings ranging from 3.69 to 3.50 (Table 1). The top four skills provided by the panelists were "foundational" SCANS skills and represented the decision making and problem solving theme areas.
In addition, four statements received mean scores ranging from 3.42 to 3.38. Those statements were "apply knowledge gained through past work experience(s) to future task(s)" (M = 3.42; SD = 0.50), "recognize the needs of co-workers and supervisors" (M = 3.42; SD = 0.50), "practice strong public relation skills" (M = 3.38; SD = 0.57), and "correspond in writing to customers, co-workers, and supervisors appropriately" (M = 3.38; SD = 0.57).
Agreement Levels for Entry-level Non-technical Skills Needed in the Animal Science Sector According to Animal Science Experts per Round Two of the Delphi Procedure (N = 26)
Statement Topic Themea SCANS Skillb M SD % of Agreement
(3 or 4)
Note. 1 = Strongly Disagree; 2 = Disagree; 3 = Agree; 4 = Strongly Agree
aDM = Decision Making; Comm = Communication; LL = Lifelong Learning; POTM = Personal Organization and Time Management; Pr Solv = Problem Solving; PS = Personal Strengths;
bB = Basic; F = Foundational; T = Thinking
1. Be honest DM F 4.00 0.00 100.00 2. Demonstrate a strong work ethic (e.g., responsible, accountable, dependable) PS F 3.96 0.20 100.00 3. Maintain a positive attitude PS F 3.77 0.43 100.00 4. Gain trust of employer DM F 3.77 0.43 100.00 5. Display a desire to learn new skills/information LL T 3.77 0.43 100.00 6. Function as a team member Comm B 3.77 0.43 100.00 7. Adhere to deadlines (e.g., accomplish tasks on time) POTM F 3.73 0.45 100.00 8. Acquire new knowledge on-the-job efficiently LL T 3.73 0.45 100.00 9. Respond verbally to customers, co-workers, and supervisors appropriately Comm B 3.73 0.45 100.00 10. Motivated to perform at an optimal level PS F 3.69 0.47 100.00 11. Realize the success of the enterprise DM T 3.69 0.47 100.00 12. Follow instructions Comm B 3.69 0.47 100.00 13. Be attentive Comm B 3.69 0.47 100.00 14. Demonstrate punctuality (e.g., arrive on time for work/meetings) POTM F 3.69 0.84 100.00 15. Perform tasks independently PS F 3.65 0.49 100.00 16. Display passion for the job being performed PS F 3.65 0.49 100.00 17. Demonstrate a desire to progress forward in his or her career LL T 3.65 0.49 100.00 18. Understand ethical implications of decisions made DM T 3.62 0.50 100.00 19. Respond positively to constructive criticism PS F 3.58 0.50 100.00 20. Identify problems Pr Solv T 3.58 0.50 100.00 21. Recognize possible solutions to problems Pr Solv T 3.58 0.50 100.00 22. Apply best solutions Pr Solv T 3.58 0.50 100.00 23. Maintain attention to detail POTM F 3.58 0.86 100.00 24. Utilize resources to overcome personal weaknesses PS F 3.54 0.51 100.00 25. Listen to the views of fellow co-workers and supervisors Comm B 3.54 0.51 100.00 26. Practice efficiency while on the job POTM F 3.54 0.86 100.00 27. Demonstrate leadership attributes PS F 3.50 0.51 100.00 28. Exhibit a professional appearance (e.g., appropriate attire, physical appearance, personal hygiene) PS F 3.50 0.51 100.00 29. Maintain organization of task/work responsibilities POTM F 3.50 0.86 100.00 30. Apply knowledge gained through past work experience(s) to future task(s) LL T 3.42 0.50 100.00 31. Recognize the needs of co-workers and supervisors Comm B 3.42 0.50 100.00 32. Practice strong public relations skills Comm B 3.38 0.57 96.15 33. Correspond in writing to customers, co-workers, and supervisors appropriately Comm B 3.38 0.57 96.15
The purpose of this study was to describe the perceptions of Oklahoma‘s animal science industry leaders as it related to the employability skills needed for entry-level employment of high school graduates who had completed coursework in Agricultural, Food and Natural Resources, animal systems pathway. In all, 33 non-technical skills were generated from the panel of animal science industry experts.
Because 100% consensus on the employability skills was reached in Round Two of this Delphi study, i.e., all experts indicated "agree" or "strongly agree," it can be concluded that the expert panelists were in agreement with the non-technical competencies they had generated in Round One of the study. Moreover, panelists "strongly agreed" with 29 of the 33 non-technical competencies that emerged in Round One reinforcing further the importance of non-technical competencies in the workforce, which resonates with the findings of previous researchers (Evers et al., 1998; Lynch, 2000; Overtoom, 2000; Robinson, 2000; Robinson et al., 2007).
In all, when using the SCANS (1991) report as a framework, this study yielded 16 foundational skills, nine thinking skills, and eight basic skills students need to be employable. The theme area "Personal Strengths" contained the most accepted statements as identified by the panelists. Statements in this category included skills such as "strong work ethic," "maintain a positive attitude," and "motivated to perform at an optimal level." The importance of these skills is supported by Robinson (2000) who verified that employees will need skills such as reliability, responsibility, problem-solving, and social skills to perform efficiently in a job setting. Panel members unanimously agreed that being honest was the most important non-technical skill for a potential employee to possess. This finding is in concert with research on college graduates by Graham (2001) and Alston, Cromartie, Wakefield, and English (2009) who found that the character trait "honesty" was extremely important to employers when hiring graduates for the workforce. This finding serves as one of the foundational skills (i.e., responsibility, integrity, and confidence) noted in the SCANS (1991) report.
Recommendations for Practice
The study's expert panelists were in unanimous agreement about the importance of 31 of 33 non-technical skills (Table 1) vis-à-vis entry-level employment of a high school graduate. So, it is important that secondary agricultural education instructors integrate these competencies into their curriculum where appropriate to increase the human capital (Becker, 1964; Schultz, 1971) of their program‘s graduates on entering the workforce. Specifically, secondary agricultural education instructors should model related behaviors to students enrolled in animal systems pathway courses. Honesty, in particular, should be modeled because it received unanimous agreement (i.e., "strongly agree") among the experts. Moreover, employers seek out employees with general employability skills, including honesty and integrity (Lynch, 2000). Curriculum materials, such as case studies involving animal science issues, should be integrated in which students must employ honesty and integrity when making decisions and solving problems related to class activities and assignments.
Recommendations for Future Research
Although this study was limited to a panel of experts in animal science in Oklahoma, it would be important to include students and teachers in future studies. This inclusion would allow for a form of "triangulation" as perceptions of necessary employability skills from both groups could be telling. Future studies should enable agricultural education teachers to rank animal systems pathway graduates on their performance of the skills generated from this study and employers to rank graduates on their level of performance after completing their first year in the animal industry successfully. Subsequently, "gaps" or incongruencies that emerge could serve as rationale for new curriculum resources to use in courses comprising the animal systems pathway, as well as inservice and professional development topics for agricultural education instructors.
Implications and Discussion
As defined by this Oklahoma's Agricultural Education Division (2007), clustering similar occupations, such as animal systems, with outlined curriculum increases the likelihood that students learn competencies necessary for success in college and the workplace. This implication is supported by the human capital theory. As stated by Shultz (1961), one of the best ways to improve human capital is through "formally organized education at the elementary, secondary and higher levels . . ." (p. 9). Improving the non-technical competencies acquired in the animal systems pathway in Oklahoma stands to enhance students' probability of gaining entry-level employment and their job success thereafter.
Because every non-technical competency statement generated by the panel of experts reached consensus of agreement, it can be implied that these skills are held in great demand by employers in the animal industry. The findings of this study seem to indicate that employers desire to hire graduates who have adequate non-technical competencies before entering the workforce. This implication is consistent with research by other scholars (Evers et al., 1998; Lynch, 2000; Robinson et al., 2007), who concluded that workers in the 21st century will need to possess non-technical skills to enter and succeed in the workforce. Based on the non-technical competency statements supported by panelists in this Delphi study, it could be surmised that honesty is an issue regarding the job performance of their employees. The statement "be honest" received the highest mean rating (M = 4.00) from panelists per Round Two. Therefore, secondary agricultural education instructors should teach and model honesty at all times and in all aspects of the secondary agricultural education program.
Alston, A. J., Cromartie, D., Wakefield, D., & English, C. W. (2009). The importance of employability skills as perceived by the employers of United States‘ land-grant college and university graduates [Electronic version]. Journal of Southern Agricultural Education Research, 59, 56–69.
Ascher, C. (1988). High school graduates in entry level jobs: What do employers want [Electronic version]. New York: Eric Clearinghouse on Urban education.
Becker, G. (1964). Human Capital: A theoretical and empirical analysis with special reference to education. Chicago: The University of Chicago Press.
Bennett, R. (2002). Employers‘ demands for personal transferable skills in graduates: A content analysis of 1000 job advertisements and an associated empirical study [Electronic version]. Journal of Vocational Education and Training, 54(4), 457–475.
Carnevale, A. P. (2003). Preparing students for the knowledge economy: What school counselors need to know — special issue: Career development and the changing workplace [Electronic version]. Professional School Counseling, 6(4), 1–13.
Charner, I. (1988). Employability credentials: A key to successful youth transition to work [Electronic version]. Journal of Career Development, 15(30), 30–40.
Conroy, C. A. (1998). Influence of gender and program of enrollment on adolescents‘ and teens' occupational and educational aspirations [Electronic version]. Journal of Vocational Education Research, 14(2), 1–12.
Czaja, S. J. (2006). Employment and the baby boomers: What can we expect in the future? In S. K. Whitbourne & S. L. Willis (Eds.), The baby boomers grow up — Contemporary perspectives on midlife. Mahwah, NJ: Lawrence Erlbaum Associates.
Dalkey, N. C. (1969). The Delphi method: An experimental study of group opinion. Santa Monica, CA: The Rand Corporation.
Evers, F. T., Rush, J. C., & Berdrow, I. (1998). The bases of competence — Skills for lifelong learning and employability. San Francisco: Jossey-Bass Inc.
Gay, L. R., Mills, G. E., & Airasian, P. (2006). Education research — Competencies for analysis and applications (8th ed.). Columbus, OH: Pearson Education, Inc.
Graham, D. L. (2001). Employer perception of the preparation of agricultural and extension education graduates. Journal of Southern Agricultural Education Research, 51(1), 89–101.
Krahn, H., Lowe, G. S., & Lehmann, W. (2002). Acquisition of employability skills be high school students [Electronic version]. Canadian Public Policy, 28(2), 275–296.
Lynch, R. L. (2000). High school career and technical education for the first decade of the 21st century [Electronic version]. Journal of Vocational Education Research, 25(2), 1–20.
Martinez Jr., R. L. (2007). An evolving set of value-based principles for career and technical education [Electronic version]. Journal of Career and Technical Education, 23(1), 72–84.
Montgomery, A. C., & Crittenden, K. S. (1977, Summer). Improving coding for open-ended questions. The Public Opinion Quarterly, 41(2), 235–243.
Oklahoma Agricultural Education Division. (2007). Untitled document [Electronic version]. Oklahoma Department of Career and Technology Education. Accessed at http://www.okcareertech.org/AgEd/Word/2008%20Courses%20for%20blue%20book.doc
Overtoom, C. (2000). Employability skills: An update. (Report No. EDO-CE-00-220). Columbus, OH: Center on Education and Training for Employment. (ERIC Document Reproduction Service No. 220).
Roberts, T. G., & Ball, A. L. (2009). Secondary agricultural science as content and context for teaching. Journal of Agricultural Education, 50(1), 81–91.
Robinson, J. P. (2000). What are employability skills [Electronic version]. Alabama Cooperative Extension System — The Workplace, 1(3), 1–3.
Robinson, J. S., Garton, B. L., & Vaughn, P. R. (2007). Becoming employable: A look at graduates' and supervisors' perceptions of the skills needed for employability. North American Colleges and Teachers of Agriculture (NACTA) Journal, 51(2), 19–26.
Rojewski, J. W. (2002). Preparing the workforce of tomorrow: A conceptual framework for career and technical education [Electronic version]. Journal of Vocational Education Research, 27(1), 1–17.
Secretary‘s Commission on Achieving Necessary Skills (1991). What work requires of schools: A SCANS report for America 2000. Washington, DC: U.S. Department of Labor, 1991.
Secretary‘s Commission on Achieving Necessary Skills (1992). Learning a living: A blueprint for high performance. A SCANS report for America 2000. Washington, DC: U.S. Department of Labor, 1992. Accessed at http://www.ttrc.doleta.gov/SCANS/lal/LAL.HTM
Shultz, T. W. (1971). Investment in human capital: The role of education and of research. New York: The Free Press.
Stitt-Gohdes, W. L., & Crews, T. B. (2002). The Delphi technique: A research strategy for career and technical education [Electronic version]. Journal of Career and Technical Education, 20(2), 55–67.
Turoff, M., & Linstone, H. (2000). The Delphi method: Techniques and applications. Accessed at http://www.is.njit.edu/pubs/delphibook/
Wicklein, R. C. (1993). Identifying critical issues and problems in technology education using a modified Delphi technique [Electronic version]. Journal of Technology Education, 5(1), 54–71.