JVTE v15n1 - A Delphi Study of Research Priorities in Tech Prep
A Delphi Study of Research Priorities in Tech Prep
Edgar I. Farmer
The Pennsylvania State University
Abstract
The purpose of this study was to determine research priorities in Tech Prep. The research issues facing Tech Prep-based education over the next ten years were identified, categorized, and ranked based on results from questionnaires distributed to subject matter experts (SMEs). A three-round Delphi was used to generate responses and achieve consensus from Tech Prep SMEs. The findings from the round one questionnaire were used to identify and categorize results into 10 tentative major research focus areas such as marketing strategies, partnerships and linkages with business, staff-development and professional training, curriculum criteria and performance standards, and evaluation methodologies and program assessment. Responses to the second-and third-round questionnaires were analyzed with descriptive statistics. The results of the study produced 11 major research categories and 98 research questions and problem statements. Twenty-six of the 98 research priority items had a mean score of four or more points, based on the assigned five-point Likert-type scale.
Background
Technical Preparation, commonly referred to as Tech Prep , is a significant aspect of vocational education literature and was authorized by Congress with the passage of the Carl D. Perkins Vocational Applied Technology Education Act Amendments of 1990, which specified Tech Prep initiatives under Titles II and III of Public Law 101-392. The concept was initially discussed during an American Vocational Association (AVA) Workshop Symposium in 1983 (Bottom, 1994) . However, the phrase was coined by Dale Parnell and appears in his book, The Neglected Majority (1985), where he introduced the 2 + 2 Tech Prep/Associate Degree program (TPAD). The TPAD concept, according to Parnell (1985) , is essentially a new approach to vocational education and is designed to integrate academic subjects (i.e., mathematics and science) with vocational-technical education subjects (i.e., engineering technology, applied science, and mechanical, industrial, or vocational subjects, such as agriculture, health, and business).
The Tech Prep concept is further described by Bragg (1995) as combining "academic and occupationally oriented education, using applied academics or other approaches to curriculum integration. Tech Prep also requires formal articulation between secondary and post-secondary institutions, ensuring that the last two years of high school are connected programmatically to two years of college leading to an associate degree" (p. 191). Other researchers and scholars have expressed similar thoughts about the Tech Prep concept, including their beliefs that many students need a vocational education that is integrated with an academic college preparatory curriculum and designed to prepare them to be successful in two-year technical education (Gray, Wang, & Malizia, 1995) .
Purpose of the Study
The purpose of this study was to identify, categorize, and rank research needs for Tech Prep. A three-round Delphi survey approach was used to generate responses and achieve consensus from a select sample of nationally recognized subject matter experts (SMEs) in Tech Prep. The results may contribute to scholarly efforts toward establishing a research agenda for Tech Prep educators in four-year college and university teacher education programs. Three research questions were developed to identify and prioritize the goals and directions.
- What are the proposed research needs for Tech Prep educators over the next ten years?
- What major research categories are identified for the next ten years by the subject matter experts (SMEs) in Tech Prep?
- What should be the research priorities for Tech Prep over the next ten years?
Methodology
A Delphi procedure was selected as the most appropriate method for attaining consensus in a national panel of SMEs. The Delphi approach was devised by Olaf Helmer and his colleagues at the Rand Corporation in the early 1950s as a means to gather expert views, and to determine the extent of expert consensus regarding the future (Cope, 1981 ; Cunico, 1974 ; Taylor, 1992) . Scholars in vocational education, such as Griggs (1990); Rojewski & Meers (1992) ; Silva-Guerrero & Sutphin (1990) ; and Connors (1998) also used the Delphi approach to gather consensus from a panel of experts. Originally, Helmer (1983) used the Delphi approach to obtain opinions about urgent military defense problems. The procedure allows the search for consensus to proceed in an orderly, systematic fashion without some of the problems inherent in bringing a group together for meetings (Starling, 1988) . The intention of the Delphi approach is to ensure that changes in opinions or estimates reflect rational judgment, not the influence of certain opinion leaders. The Delphi approach prevents strong personalities from forcing judgments in certain directions as frequently occurs in meetings of SME panels. After all, "good, perhaps more accurate, response may come from persons on the panel with only modest credentials" (Nickell, 1988, p. 225) . Although the Delphi approach was originally used to predict future defense needs, it can be used, according to Borg and Gall (1983) , whenever a consensus is needed from persons who are knowledgeable about a particular subject.
In this study, the vast majority of SMEs were nationally recognized teacher educators and administrators at four-year colleges and universities that play a pivotal role in successfully promoting Tech Prep. In addition to their national reputations, the SMEs were selected because of their involvement with the National Tech Prep Network, which collaborates and articulates with the Center for Occupational Research and Development (CORD). A select panel of respondents was given a series of questionnaires. Through controlled feedback with each round, carefully considered group opinions were formed. Initially, 56 Tech Prep panel members were contacted and 37 agreed to participate. However, only 33 respondents completed the first round, 30 completed round two, and 27 completed the three-round process.
Round One
The process began with a one-page, open-ended questionnaire (mailed in March 1994), that asked the panel to generate or identify research needs (priorities) in Tech Prep over the next 10 years. Specifically, respondents were directed to develop five research questions or problems that they felt should be addressed. Upon the return of the round one questionnaire, a four-member panel coded and categorized 187 identified research needs and priorities into major research categories using an empirically grounded coding scheme. Members of the panel were selected based on their leadership positions and knowledge of Tech Prep. The major research categories emerged from an examination of the common themes among the initial list of 187 research needs and priorities. Efforts to eliminate duplication by editing and modifying some of these reduced the number to 98 research items. Some of the responses were written as questions and some as statements of action because the directions on the round one questionnaire permitted respondents to create either research questions or problems.
Round Two
The second questionnaire, mailed in April 1994, contained the 98 research items generated from the 33 respondents in round one. The respondents were asked to rate each item as listed in Part A by placing a (X) in the appropriate space. Ratings were based on a 5-point Likert scale (1 = least need, 5 = highest need). In Part B , the respondents were asked to rank the 10 major research categories by order of importance -- the first would be the most important and the tenth, the least important.
Round Three
In the third and final questionnaire, respondents were asked to review their round two responses and contrast them using group consensus data. The respondents' round two responses were tabulated to include the following information: (a) interquartile range, middle 50% of all responses; (b) median; and (c) mean. The respondents' initial ratings on the round two questionnaire were identified with a blue dot. Based on the information provided, the respondents could keep their initial rating or change it by placing a (X) in the space for the new rating. They were asked to explain a new rating for any research item that was outside the group consensus (interquartile range).
Results
The results of the three-round Delphi survey produced 11 major research categories and 98 research questions and problem statements. Twenty-six of the 98 research priority items had a mean score of four or more points, based on the assigned five-point Likert-type scale. As shown in Table 1, the research priority item with the highest rating mean score (M = 4.0, SD =.9) was Institutionalize Tech Prep into the higher education delivery system and focus instruction on data from learning styles/cognitive sciences research, and non-traditional teaching methods . The remaining 25 research priority items had a range of mean scores from 4.4 to 4.0, as shown in Tables 1.
Table 1
Research Priority Items with High Rating *
Research Priority Items with a Mean Score of 4.48 to 4.00
Mean
IQR
Mdn
Institutionalize Tech Prep into the higher education delivery system and focus instruction on data from learning styles/cognitive science research, and non-teaching methods (item 11). 4.48
4-5
5
What type of staff development will help teachers and schools to create practices that result in higher performances by all students? (item 35) 4.44
4-5
5
What techniques or models can be used to create or enhance relationships and curriculum development between and among: (a) academic and vocational/technical instructors, (b) secondary and post-secondary staff and (c ) educators, and business/industry representatives (item 21) 4.37
4-5
5
How important is articulation between programs and what are the most useful models for articulation with local high schools, community colleges and four-year universities? (item 4) 4.37
4-5
4
To shift the paradigm of all teaching from content-oriented learning to applied work-based contextual learning (item 43). 4.33 4-5 4 4.33
4-5
4
What will business/industry personnel need to know and be able to do to participate in Tech Prep programs? (item 7) 4.30
4-5
4
How can teacher education programs be redesigned to ensure that teachers are competent in the use of active and diverse teaching methodologies? (item 15) 4.30
4-5
4
Identify or develop counselor education pre-service programs which include an appropriate emphasis on career development, career assessment, career guidance, leadership of assessment and guidance teams, and awareness of a full range of post-secondary educational options (item 8) 4.26
4-5
4
How can teacher education programs be structured to provide future teachers with an understanding or an appreciation of technical career fields? (item 17) 4.26
3.25-5
4
How can we develop effective teams of educators, counselors, and industry mentors to ensure the quality and consistency of the work-based education component of the school-to-work transition programs? (item 48) 4.26
3-5
4
What school and classroom conditions will enable 90 percent of high school youth to master more advanced academic and technical content? (item 55) 4.26
4-5
4
Identify or develop effective means for improving the instruction in colleges of education resulting in modeling of instructional strategies and techniques appropriate for Tech Prep (item 12) 4.22
3.25-5
4
To prepare all future teachers to recognize the need for business/industry/education, linkages and partnerships (item 28) 4.22
4-5
4
How can teacher education programs prepare teachers to continuously and effectively modify curricula in collaboration with persons who are not educators in the traditional sense? (item 9) 4.19
4-5
5
What are the curriculum requirements and needs for Tech Prep Teacher Education, and what are the most valuable philosophical and pedagogical foundations for Tech Prep Teacher Education programs? (item 10) 4.19
4-5
5
How can university professors best be trained in the use of applied methodologies, and how should the delivery of teacher education courses be modified so that professors can appropriately model applied methodologies? (item 34) 4.19
3.25
4
How well prepared are teachers to advise students, mentor them, and help them to see how each Tech Prep component fits with each component in the overall program? (item 36) 4.19
4-5
4
Expand student teaching to include an industry-based internship. Require student-teacher to write and teach a unit of study module that demonstrates the practical application of the academic discipline (item 75) 4.19
4-5
4
The impact of using an applied or contextual approach to teach mathematics and communication skills or the impact of student achievement on the applied academics (item 32) 4.15
4-5
4
Is there a difference in student achievement between students taught in traditional instructional settings versus applied settings and if so, among what student populations (item 77) 4.07
3.25-5
4
Tech Prep education involves the integration of academic and occupational curricula which can be accomplished in a variety of ways. Which approach, or combination of approaches, is the most effective in terms of student achievement and faculty satisfaction? (item 16) 4.04
4-5
4
Involve teacher training colleges/universities in the process of preparing new teachers with the tools necessary to implement competency-based education and skills for successful employment (item 29) 4.04
4-5
4
How well are we, as teacher educators adapting our traditional content and delivery methods to pre-service teachers to prepare them to participate successfully in Tech Prep programs? (item 31) 4.04
3-5
4
What effect has the integration of Tech Prep systems had on schools, educators, and employers? (item 33) 4.04
3-5
4
Can we identify some best practices models that can serve as benchmarks for schools to utilize when they begin to implement school-to-work transition programs? (item 90) 4.04
3-5
4
Should teacher educators be required to spend some time each year working in the public schools as a volunteer, or as an intern in business and industry (item13) 4.00
3-5
4
1 Research Priority Items were based on the assigned five-point Likert-type scale (1=least need, 5=highest need).
On the other hand, Table 2 shows that 16 of the 98 research priority items had a mean score of less than 3 based on the assigned scale. Moreover, research priority item 40, in Table 2, had the lowest mean score (M=2.4, SD = 1.1) of the 98 items; its focus was on identifying the best Tech Prep journals (if any exist).
Table 2
Research Priority Items with Low Rating 1
Research Priority Items with a Mean Score of less than 3.00
Mean
IQR
Mdn
What are the best Tech Prep journals/periodicals and are there any specific for Tech Prep? (item 40) 2.37
1-3
3
Some states are administering funds for Tech Prep through community colleges, and some community colleges are writing the proposals and administering the funds. However, many community colleges do not receive any monetary incentive for this activity. Does this act as a barrier to the implementation of Tech Prep? (item 53) 2.44
1-3
2
Who determines total track content? Which school decides which courses are included in the total track? (item 24) 2.56
1-4
3
The state department of public instruction should provide sufficient information to faculty and counselors about Tech Prep and applied academics, (item 38) 2.56
1.25-3
3
What comparisons are there between what the latest NAVE report says about Tech Prep's effect and what exists during the last year of this Perkins reauthorization, since the NAVE study was based on 90-'91 and 91-'92 data that is highly preliminary, considering that the Tech Prep Act was authorized in September 1990? (item 93) 2.63
1.25-3.75
3
What effect on institutional governance has occurred in the stakeholder group approach to Tech Prep? (item 94) 2.63
2-3
3
What is the effect of Tech Prep on AVTS enrollment/programs? (item 52) 2.67
2-3
3
What are the problems caused by incorrectly certifying accomplishment of competencies? Do teachers or administrators set standards for Tech Prep? (item 96) 2.70
2-3
3
What is the percentage of Tech Prep students in paid work experience programs? What is the comparison of those students to general Tech Prep students? (item 71) 2.74
2-3
3
How does the negative connotation of occupational/vocational education impact on Tech Prep? (item 63) 2.78
2-3
3
How can Tech Prep serve the under-achieving student who may not be capable of success in the Tech Prep or college prep track? (item 80) 2.78
1.25-4
3
What consortia management strategy works best, and what organizational structure works best? (item 89) 2.78
2-4
3
Develop a Tech Prep model for general education courses such as social studies, English, etc. (item 14) 2.82
2-4
3
What is the rate of apprenticeship (formal) agreements in Tech Prep? And what are the effects of this plan on students? (item 44) 2.85
2-4
3
To prepare future teachers with the mission of education as the foundation for students' further learning, citizenship, and productive employment (item 27) 2.93
2-4
3
Learning new methodologies—contextual, co-operative integration of academic and vocational education. Learn the theory first and then practice (item 56) 2.93
2-4
3
1 Research Priority Items were based on the assigned five-point Likert-type scale
(1=least need, 5=highest need).The respondents also were asked to rank the 11 major research categories in order of importance from 1 to 11, with the first being the most important and the eleventh being the least important. Initially, there were ten major research categories; however, during the second round, another was added and the final tabulations were based on 11 categories. Research in instruction and curriculum development received the highest rank order of importance with a mean score of 3.1. The remaining major categories were ranked from 2 to 11:
Research in staff-development and professional training, M= 3.7
Research in alternative teaching and learning models, M= 4.1
Research in partnerships and linkages with business, industry, and education, M= 4.6
Research in work-based education, M= 4.9
Research in perceptions and attitudes of Tech Prep, M= 6.5
Research in evaluation methodologies and program assessment, M= 7.1
Research in student-related issues, M= 7.4
Research on defining curriculum criteria and performance standards for Tech Prep programs, M= 7.9
Research in marketing strategies and funding resources, M= 8.1.
Research in policy-related issues, M= 8.4.
Group consensus was determined using the interquartile range of each research priority item. This type of analysis was used because the participants' responses indicated a wide variance of opinion in rating the research items. The interquartile range is defined as the difference between the upper and lower quartiles. In other words, the middle one-half (50%) of the scores is within the upper and lower quartile range (Agresti & Agresti, 1979) . Moreover, the interquartile range for the 98 research priority items was from 3.0 to 5.0 on a five-point Likert-type scale. Based on responses from the SMEs, 72 percent (71 of 98) of the research priority items had a round three interquartile range of 3.0 or more. The remaining research priority items (27 of 98) had a low degree of consensus and were identified as not being very important.
Discussion
The major research categories developed from survey responses in this study may guide the design of a national research agenda for Tech Prep educators. However, the study results should be interpreted cautiously since the data were collected four year ago. Nevertheless since there is currently no such agenda, the findings from this study should act as a preliminary guide in its development. In other words, the research priorities should provide a starting point for young scholars and other vocational educators interested in developing a researchable problem or topic in Tech Prep. Although Tech Prep initiatives have had a considerable impact on reform of vocational-technical education, further research is still needed. For example, in the area of curriculum development, the first-ranked research category as identified in this study reveals a need for more Tech Prep research on curriculum models that would provide opportunities for collaborative activities with academic and vocational/technical instructors, secondary and post-secondary staff, and educators and business/industry representatives.
Further research also is needed in the area of staff development and professional training; respondents identified this as the second most important research category. Most educational leaders would agree that teachers need continuous professional training to create and enhance instructional practices that result in higher student performance. Moreover, the Perkins legislation (Public Law 101-392) mandates in-service training for teachers' professional growth and development. Another area of importance (identified by respondents as the third highest concern) was more research in alternative teaching and learning models. In other words, Tech Prep leaders should focus more research on learning styles, cognitive sciences, and nontraditional teaching methods. The opportunity exists for real changes to take place in education by breaking down the barriers that prevent change in the traditional processes of teaching (Johnson & Thomas, 1992) . Professional educators generally know that all students do not have the same learning style nor the same cognitive ability. Therefore, nontraditional teaching methods should be implemented into the higher education curriculum because many of the traditional show and tell teaching methods will not prepare students for high-tech and high-touch employability (Naisbett, 1982) . Moreover, the high-tech demands of the next century dictate a need for technical teachers with high-tech employability skills themselves; traditional vocational education teachers are currently insufficiently prepared (Farmer, 1993) . Curriculum development, staff development, and instructional strategies are required of practitioners and instructional leaders in Tech Prep initiatives and other school-to-work transition programs.
Finally, more research is needed in areas of partnerships and linkages with business, industry, and education. They were noted by respondents as their fourth highest concern.
Policymakers, business leaders, and educators need to give more attention to prioritizing research in Tech Prep for the next century. Such research priorities were identified and categorized in this study. The challenge now is to take action. For example, based on the results of this study, researchers could conduct a follow-up national study every five years or state and regional levels studies to track trends on local needs in their respective communities.
References
Agresti, A. & Agresti, B. (1979) . Statistical Methods for the Social Sciences . San Francisco, CA. Dellen Publishing Company.
Borg, W.R., & Gall, M.D. (1983). Educational research: An introduction (4 th ed.). New York, NY: Longman, Inc.
Bottoms, G. (1994) . Personal communication, Southern Regional Education Board. Atlanta.
Bragg, D.D. (1995) . Linking high schools to post-secondary institutions: The role of tech prep. In W. Norton Grubb (Ed.), Education through occupations in American high schools: The challenges of implementing curriculum integration, Vol. 2. New York: Columbia University, Teachers College.
Cope, R.W. (1981) . Education 1990: A Delphi study of possible future problems or issues for public education in Missouri (Doctoral dissertation, University of Missouri). Dissertation Abstracts International , 43 (06), 1761A.
Connors, J.J., (1998). A Regional Delphi study of the perceptions of NVATA, NASAE, and AAAE Members on critical issues facing secondary agricultural education programs. Journal of Agricultural Education, 39(01), 37-47.
Cunico, G.E. (1974). A Delphi approach to the future of industrial education (Doctoral dissertation, Utah State University). Dissertation Abstracts International , 34 (12), 7565A.
Farmer, E.I. (1993) . What will tomorrow bring? Preparing a new type of technical teacher for the twenty-first century: Implementing Tech Prep Teacher Education in four-year colleges and universities. Journal of Industrial Teacher Education, 30 (2), 100-104 .
Gray, K., Wang, W., & Malizia, S. (1995). Is vocational education still necessary? Investigating the educational effectiveness of the college prep curriculum. Journal of Industrial Teacher Education. 32 (2) p. 28.
Griggs, M. B., (1990) . Research priorities and goals for vocational education personnel development. National Center for Research in Vocational Education, Berkeley, CA: University of California at Berkeley.
Helmer, O. (1983) . The Delphi Technique. Looking Forward: A guide to future research . Beverly Hills, CA: Sage Publications, Inc.
Johnson, S.D. & Thomas, R. (1992). Technology education and the cognitive revolution. The Technology Teacher 52 (4), 7-12.
Naisbitt, J. (1982) . Megatrends: Ten new directions transforming our lives . New York: Warner Books, Inc.
Nickell, D.B. (1988) . Technological forecasting. Forecasting on your microcomputer, second e dition . Blue Ridge Summit, PA: Tab Books, Inc
Parnell, D. (1985) . The neglected majority . Washington, DC: The Community College Press, A division of the American Association of Community Colleges.
Public Law 101-392. Carl D. Perkins Vocational and Applied Technology Education Act Amendments of 1990.
Silva-Guerrero, L. & Sutphin D. (1990). Priorities for research in agricultural education. Journal of Agricultural Education 31 (4).
Rojewski, J.W. & Meers, G.D. (1991). Research priorities in vocational special needs education: A Delphi approach. Journal for Vocational-Special Needs Education. 13 (2), 33-38.
Starling, G. (1988) . Strategies for policy making . Chicago, IL: The Dorsey Press.
Taylor, R.E. (1992). Pros and cons of the Delphi technique and suggested relationship to management science. Decision Science in the Public Sector . North Whitefield, ME: Felicity Press, Publishers.
Appreciation is extended to Li-Shyung (Lily) Hwang and Chang-Jung (Victor) Lin for their technical assistance in analyzing data for the Delphi Study. Their input made a valuable contribution to this study's success.