JVER v28n1 - What Do We Know About the Effects of School-to-Work? A Case Study of Michigan
What Do We Know About the Effects of School-to-Work? A Case Study of Michigan
David Neumark
Public Policy Institute of California
Michigan State University
National Bureau of Economic ResearchAnn Allen
Michigan State UniversityAbstract
As states continue to develop school-to-work programs following the federal Schoolto-Work Opportunities Act (STWOA), a critical question regards the causal effects of school-to-work programs in achieving their goals. Rather than relying on published studies that would appear only with long lags, and which would be unlikely to focus on particular local programs that might nonetheless be effective, this paper adopts a case study approach to assembling evidence on the effectiveness of school-to-work in a particular state (Michigan) that has been at the forefront of school-to-work efforts. We set out to obtain all available evidence and information of the effectiveness of school-to-work in Michigan, based on a review of available studies often at a very local level, as well as interviews with academic and career educators and state representatives involved in school-to-work. This evidence and information is analyzed from the perspective of whether it establishes beneficial causal effects of school-to-work on student outcomes. Our case study establishes that Michigan has developed a comprehensive career preparation system that reaches many students. Nonetheless, a critical analysis of the evaluation efforts to date indicates a serious shortfall in these efforts, and an absence of solid evidence regarding causal effects of school-to-work in the handful of evaluations that have been carried out.The 1994 Federal School-to-Work Opportunities Act (STWOA) provided more than $1.5 billion to support increased career preparation activities in the country's public schools. Congress passed the STWOA in response to research by scholars and educators suggesting that schools were inadequately preparing students for an increasingly complex world of work, both in terms of teaching skills and behaviors inschool that would be valued in the labor market, and providing the information necessary to assist students in making a successful transition from school into a productive and remunerative career. The overall perception of the problem was wellcharacterized in a report by the U.S. Office of Technology Assessment describing the existing system as producing youths who were " … unmotivated in school and spend years bouncing from one low-paying job to another" ( 1995 , p. 3). The goals of the STWOA were therefore to help young people develop the skills needed in the workforce and make better connections to careers through school-to-work transition systems. Spurred in part by the STWOA, many states have implemented extensive school-to-work systems, generally entailing close cooperation with schools in the implementation of curriculum changes, and partnerships with private business and government bodies.
A natural and indeed compelling question to ask at this juncture is whether school-to-work programs have been effective in achieving their goals. In particular, the critical question is whether school-to-work programs have had causal effects on participants that point to success in achieving the goals of the STWOA. This question has come to the fore in some recent work. In particular, Mathematica Research, Inc., completed a national school-to-work study in 1999 ( Hershey, Silverberg, & Haimson, 1999 ). However, the Mathematica study does not assess the causal relationship between school-to-work and student outcomes, but rather provides a national picture of school-to-work initiatives. And a recent survey of published academic research on school-to-work across the U.S., by the Institution on Education and the Economy, generally supports the claim that little progress has been made in estimating the causal effects of school-to-work programs ( Hughes, Hamilton, & Ivry, 2001 ).
This paper takes a different approach to assembling evidence on the effectiveness of school-to-work. In particular, because state and local governmental units are at the forefront of school-to-work efforts, and because much of the "action" with respect to school-to-work is new, the most up-to-date evidence on the effects of school-to-work is unlikely to have yet appeared in academic journals-if indeed it will ever do so-and is perhaps most likely to still be in the hands of practitioners rather than researchers. The strategy chosen, therefore, was to undertake a case study of one state that has been at the forefront of school-to-work efforts. Specifically, the study focuses on Michigan, which was one of the first eight states to receive federal money from the STWOA in 1994. STWOA funding was set to end in 1999, but Michigan applied for a grant extension that continued through September 2001. The current incarnation of school-to-work in Michigan is the Career Preparation System, the goals of which parallel those of the STWOA.
We set out to obtain all available evidence and information regarding the effectiveness of school-to-work in Michigan, to determine what can be established at this juncture concerning the impact of Michigan's school-to-work efforts on youth education and employment and early career decisions of youths. This evidence and information was gathered from a review of available studies and data, often from the local governmental level, and from interviews with academic and career educators and government representatives involved in school-to-work. In particular, we interviewed representatives of the state's Educational Advisory Groups, including Intermediate School District superintendents and staff, and state school-to-work administrators. After summarizing the development of school-to-work in Michigan, and describing the current institutional structure, we present an exhaustive summary and, most importantly, a critical analysis of the available evidence on the effectiveness of Michigan's school-to-work programs, from the perspective of whether it establishes beneficial causal effects of school-to-work on students.
Our case study of school-to-work in Michigan does establish that the STWOA has pushed Michigan forward toward the development of a comprehensive career preparation system, creating an impressive structure for the implementation of school-to-work, and one that is apparently reaching many students. Furthermore, state and local districts are starting to recognize and perhaps act on the need for ongoing assessment of program outcomes-that is, assessment of the extent to which school-to-work transitions are improved relative to what would have occurred in the absence of this system. Nonetheless, our current state of knowledge regarding the causal effects of school-to-work in Michigan is extremely limited, as we found only a smattering of studies estimating such effects, and virtually none employing standard social scientific methods of program evaluation. At this point, therefore, the combined evidence from this case study and the national surveys cited above does not provide a strong empirical case for concluding that school-to-work programs achieve their intended goals. Furthermore, given the absence of such an empirical case, we would argue that investment in research that endeavors to establish causal effects of school-to-work would play a highly constructive role in shaping the further development of school-to-work systems.
While the potential gains from our research strategy have already been noted, it also has an important limitation. Specifically, it is limited to a single state, although one that developed an extensive school-to-work system under the STWOA. As a consequence, the conclusion that this particular state has little evidence on the causal effects of its school-to-work programs does not generalize to other states. We believe that studies using a similar strategy to assess the evidence on the effectiveness of school-to-work in other states would be valuable, although it would be useful to use other sources to try to identify states in which more rigorous evaluations are purported to have been used or in which more suitable data have been collected, in order to narrow the set of states meriting such close scrutiny.
The Federal School-to-Work Opportunities Act and School-to-Work in Michigan
The Federal School-to-Work Opportunities Act
The 1994 Federal School-to-Work Opportunities Act (STWOA) provided more than $1.5 billion in grants to states and local partnerships to support increased career preparation activities in the country's public schools ( Hershey et al., 1999 ). According to a report by the U.S. Office of Technology Assessment (1995) , Congress passed the STWOA in response to research by scholars and educators that highlighted three areas of concern for public education. These were: (1) a lack of connection between school and work that led many youths to be unmotivated in school and to experience subsequent difficulty moving out of low-wage jobs; (2) youths completing school with insufficient skills needed for the labor market; and (3) increasing labor market demands for complex thinking, close teamwork, and the ability to learn on the job. The STWOA aimed to help young people develop the skills needed in the workforce and make better connections to careers through school-to-work transition systems, which fostered partnerships among schools, employers, and others (Office of Technology Assessment, 1995). Specifically, the STWOA set out to increase: (1) school-based initiatives such as career links to academic curriculum, and career awareness activities; (2) work-based activities such as job shadowing, internships, and apprenticeships; and (3) connecting activities, such as the development of partnerships with employers and post-secondary institutions. A more detailed summary of the STWOA is provided in Figure 1.
Figure 1.
Summary of the School-to-Work Opportunities Act
State and local school-to-work transition systems are to be planned and developed by partnerships of school staff, business leaders, labor representatives, and other interested parties. Governors are given considerable discretion in structuring and administering the partnerships for the state systems. At the local level, the lead entities may be schools, colleges, nonprofit organizations, and chambers of commerce.
STWOA encourages development of school-to-work transition systems that coordinate career orientation, academic and occupational education, high school and postsecondary schooling, work-based learning, and skill credentialing. The legislation specifically divides these elements into the following three components:
Source: Office of Technology Assessment (1995) .
- School-based Learning
- Academic instruction in high school that meets the state standards for all students and the applicable standards of the National Education Goals.
- Career exploration and counseling, beginning no later than 7 th grade for interested students.
- Initial selection by interested students of a career major beginning no later than the 11 th grade.
- Instruction that integrates academic and occupational learning.
- Arrangements to coordinate high school and postsecondary education and training.
- Regularly scheduled evaluations of students' personal goals, progress, and needed learning opportunities.
- Work-based Learning
- Job training and work experiences aimed at developing pre-employment skills and employment skills at progressively higher levels, and leading to the award of skill certificates.
- Broad instruction in all aspects of industry to the extent practical.
- Workplace mentoring.
- Connecting Activities
- Activities to encourage employers to participate and to aid them in doing so.
- Assistance in the integration of school-based and work-based learning, and of academic and occupational instruction.
- Matching of students with the work-based learning opportunities offered by employers.
- Liaison among the students, schools, employers, and parents.
- Assistance for graduates in finding appropriate jobs, getting additional job training, or pursuing further education.
- Monitoring of participants' progress after they complete the program.
- Linkage of these youth development activities with employer and industry strategies for upgrading the skills of incumbent workers.
School-to-Work in Michigan
Michigan was one of the first eight states to receive federal money from the STWOA in 1994. STWOA funding was set to end in 1999, but Michigan applied for a grant extension that continued through September 2001. From 1994 to 2001, Michigan received more than $50 million of federal STWOA funds to develop and support local and state-wide school-to-work initiatives ( Levin, 1999 ).
Figure 2 provides a time-line for the development of school-to-work in Michigan. Before the STWOA, Michigan schools were providing career and technical education through such programs as cooperative education and vocational education. But it was the state's tech prep initiative, supported through the Carl Perkins Amendments of 1990 ( Bailey, 1995 ; Urquiola et al., 1997 ), which attempted to move career and technical education from isolated programs to a school-reform initiative-improving educational outcomes for students through more relevant, contextual learning, and stronger connections to employers and careers (Jacobs & Teahen, 1998).
Figure 2
Time-line of Development of School-to-Work in Michigan
Coop/Vocational Perkins Amendment STWOA Career Education System Tech Prep Reform Preparation 1960s____________ 1990______________ 1994_________ 1997________ 2004 a
a A report by the Michigan Department of Career Development (2000) indicates a commitment by nearly 90 percent of state school districts to develop career pathway programs and establish Education Development Plans for secondary students.
Cooperative and Vocational Education and Tech Prep
Tech prep, as a federal program, aimed to encourage the development of programs from high school to college in specific occupational areas ( Bailey, 1995 ). In a report on Michigan's tech prep initiatives, Jacobs and Teahen (1998) quoted the 1988 Executive Summary of the Tech Prep Task Force to define "tech prep initiatives" as associate degree programs made up of partnerships among school districts, community colleges, and employers to prepare youths and adults for entry into technical career fields ( Tech Prep Task Force, 1988 , p. 1).
Jacobs and Teahen found that tech prep set the stage for career preparation as educational reform. They pointed to four contributions tech prep made to educational reform in Michigan including: (1) encouraging the development of courses that connected critical thinking skills to workplace skills; (2) the creation of tech prep consortia, made up of secondary and postsecondary educators and business and industry representatives, which became a vehicle through which business and labor representatives could more directly affect secondary and postsecondary education; (3) bringing together representatives of secondary and postsecondary institutions for meaningful dialogue; and (4) developing new personnel roles in the education system, including work as liaisons between schools and employers. Jacobs and Teahen cited reports from the Michigan Department of Education indicating that tech prep programs in Michigan were engaging 50 percent of all 12 th graders in the state, and local consortia were developing activities and programs that reached not only high school, community college, and business representatives, but K-12 educators and administrators as well. Tech prep, then, began to make some inroads in changing the educational system to attempt to better connect students to careers.
Yet one of the issues tech prep coordinators faced was gaining parental support. Jacobs and Teahen (1998) contended there was a sense among vocational educators that parents believe all children should go to college, thereby perpetuating the belief that tech prep is not college preparatory curriculum. Stemming from this, presumably, their study found little evidence of parental support for tech prep initiatives. (In partial contrast, a 1995 survey of community response to the schoolto- work initiative in Michigan ( EPIC-MRA, 1995 ) indicated broad-based community support for a school-to-work system in the state.) In particular, Jacobs and Teahen found that parents and students were represented on 58 percent and 50 percent of consortia boards, respectively. However, activities involving parents were found to be mostly marketing programs intended to convince parents of the value of tech prep initiatives, and the authors found no state or local parent organizations that played a significant role in the tech prep initiative. This led them to conclude that tech prep could not be institutionalized without parental support. In addition, state and national studies of tech prep indicate that little data exist exploring how the initiatives affect student outcomes ( Jacobs & Teahen, 1998 ; Urquiola, et al., 1997 ).
One of the challenges to tech prep noted by Jacobs and Teahen was that the STWOA came on the scene as the tech prep initiatives were getting underway, causing some confusion about the goals and roles of the two systems. However, they found that the majority of the state's tech prep directors aligned the tech prep programs with the STWOA-even though school-to-work was funded and governed from a different state agency than tech prep-resulting in more than 80 percent of the state's tech prep programs receiving funding from the STWOA ( Jacobs & Teahen, 1998 ). (While the tech prep initiative was initially administered out of the Michigan Department of Education, the STWOA was administered by the Michigan Works! agencies and overseen by workforce development boards-decision-making boards with a majority of citizens from the private sector (i.e., employers) that are responsible for the state's workforce development programs.) In many cases, the people involved in the tech prep consortia were the same people leading the STWOA initiatives. Yet the STWOA was seen to have broader implications than tech prep programs ( Urquiola et al., 1997 ; Bailey, 1995 ), in part because it encompassed career development that went beyond the two-year college.
The STWOA in Michigan
The STWOA called for schools to establish links with two-year and four-year postsecondary institutions to better help students prepare for careers and the postsecondary education that leads to those careers. The 1994 school-to-work initiatives in Michigan focused on building a K-12 system that better supported students' preparation for careers through the three types of activities articulated by the federal initiative: school-based learning; work-based learning; and connecting activities ( Bailey, 1995 ; Office of Technology Assessment, 1995 ; Urquiola et al., 1997 ). According to the state's original vision for a school-to-work system, schoolbased learning included career awareness, career exploration and counseling, challenging academic standards, skill standards, a coherent multi-year sequence of instruction, and regularly-scheduled evaluations of progress. Work-based learning included a planned program of job training and experiences relevant to a student's career and leading to the award of a skill certificate, paid work experience, workplace mentoring, and instruction in general workplace competencies. Connecting activities included school courses that taught knowledge and skills used at work, helped parents become knowledgeable about school and work, and matched students with employers' work-based learning opportunities (personal communication with J. Mahoney, coordinator of tech prep and career education for the state in the Office of Career and Technical Preparation, Michigan Department of Career Development, October 2001 ).
Like tech prep, school-to-work in Michigan struggled to find broad-based support. J. Mahoney (personal communication, October 2001), reported that one struggle was communicating the mission of the initiative to parents and to some educators who feared that school-to-work was an occupational program and did not promote or support college-bound programs, high academic achievement, or professional careers. In addition, funds for school-to-work in Michigan were funneled through the local Workforce Development Boards-former Private Industry Councils reorganized by the state in 1996 to oversee the planning and delivery of service for the state's workforce development programs ( Michigan Department of Career Development, 2000 ). The state's 25 Workforce Development Boards were made up primarily of business and community representatives, and educators were accountable to those boards for the use of school-to-work funds. In areas of the state where school and business relationships were strong (such as Macomb and Kalamazoo counties), these initiatives had more support. In other areas, according to J. Williams, Director of the Michigan Department of Career Development's Office of Career and Technical Preparation (personal communication, October 2001), educators and employers struggled to find common ground, and educators saw the initiative as a work-based program.
Another challenge in implementing systemic reform initiatives in Michigan was related to the state's decentralized system of local control ( Jacobs & Teahen, 1998 ). Implementing systemic reforms in a system of local control is more likely to result in variable than standard practice ( Elmore, 1980 ). On the other hand, it was the variability of practices, or the various best practices seen throughout the state, which helped the state determine the structure for its current Career Preparation System (discussed in the next section). (See McLaughlin (1987) for elaboration of this general point.) In particular, at conferences hosted by the state each year around school-to-work, there were a multitude of sessions where people shared their best practices. J. Mahoney (personal communication, October 2001) asserted that this allowed the state to learn about the "smorgasbord" of activities and take the best of them to design guidelines for all districts to follow.
The Career Preparation System
In 1997, Governor John Engler articulated the design of a state-wide Career Preparation System, and the legislature supported the system with a revision to the School Public Aid Act, which currently supports the system with $24 million a year ( Michigan Department of Career Development, 2000 ). The Career Preparation System incorporated the aims of the school-to-work initiative, but broadened the scope of activities, participants, and goals. Similar to the STWOA, the Career Preparation System aims to "…ensure that each graduate will receive a quality education to prepare for higher education and their first job in today's competitive market" ( Michigan Department of Career Development, 2001 , p. 1), and that "…[a]ll students completing the Michigan education system will have the necessary academic, technical, and work behavior skills for success in a career of their choice and lifelong learning" ( Michigan Department of Career Development, 2001 , p. 3). Figure 3 provides a schematic description or "map" of Michigan's Career Preparation System ( Michigan Department of Career Development, 2001 ). The diagram represents the flow of resources and processes that serve customers of the Career Preparation System, resulting in student achievement in academics, workplace readiness, career competency, college and career placement, and employer satisfaction.
Figure 3.
Michigan's Career Preparation System Diagram
Michigan's Career Preparation System calls for schools to provide curriculum that emphasizes application of academics, opportunities to provide all students with career exploration and guidance, and general employability and technology skills. The system also calls for the majority of high school programs to coordinate with postsecondary programs at community colleges and four-year institutions across the state. The voluntary system sets out guidelines for schools to follow in order to have access to state Career Preparation funds. There are three goals embedded in the Michigan Career Preparation System. They are: (1) to ensure that career preparation is fully integrated into the Michigan education system; (2) to ensure that all students, with their parents, are prepared to make informed choices about their careers; and (3) to ensure that all students have the types and levels of skills, knowledge, and performance valued and required in their education and career choices.
Turning to Figure 3, the Career Preparation System is composed of seven "components" and 17 "activity" categories. The seven components or broad focus areas of the system are academic preparation, career development, workplace readiness, professional and technical education, work-based learning, accountability, and school improvement. On the Career Preparation System map (Figure 3), the state places the components and their accompanying activities under the category of "processes." The map also defines customers of the system-including learners, parents, and business and industry-and customer needs.
Other aspects of the Career Preparation System that appear on the map are the resources that flow into the processes, results of the processes, and indicators of results. Listed as resources of the Career Preparation System are: collaboration with employers, community groups, educational institutions, parents, and government offices; financial resources from local, state, federal, and private sources; physical resources including facilities, equipment, and materials; systems for the delivery of services including school districts, career centers, trade academies, colleges and universities, private schools, dual enrollment, and distance learning; and data and information resources such as labor market data, standards, assessment data, placement data, curriculum materials, and educational research. Intended results of the system, as indicated in the map, are academic achievement, workplace readiness, career competency, college/career placement, and employer satisfaction. Indicators of results include academic endorsements, certificates, licensures, and college degrees.
Two of the major elements in the state's Career Preparation System that developed from elements of the school-to-work initiative are Career Pathway Programs and Education Development Plans for secondary students. The Career Preparation System defines Career Pathways as "…broad groupings of careers that share similar characteristics and whose employment requirements call for many common interests, strengths, and competencies" ( Michigan Department of Career Development, 2001 , p. 5). Each Career Pathway curriculum area covers state academic standards, but does so within the context of career areas, in an effort to increase the relevance of the material to individual student interests. The state has defined six career pathways in the Career Preparation System. They include: Arts and Communication; Business Management, Marketing, and Technology; Engineering/Manufacturing and Industrial Technology; Health Sciences; Human Services; and Natural Resources and Agriculture. The Career Preparation System also calls for Education Development Plans (EDPs) for every secondary student in the district. These plans must include: personal information; career pathway goals; educational/training goals; career assessment results; plan of action; and parent/family consultation and endorsement for students under the age of 18.
As the development of the Career Preparation System was underway, there was an effort by the state to address the concerns of educators who saw past and current school-to-work efforts as work-based. In 1998, the state redesigned its system of Workforce Development Boards to include Education Advisory Groups (EAGs). Education Advisory Groups are advisory committees made up of academic and career technical educators from intermediate and local school districts as well as representatives from business and industry. Money for career preparation activities is now funneled through the state's 25 EAGs, which require a plan from school districts as to how money will be used and how the local efforts will support the regional vision for career preparation. In addition, the EAGs designate public educational agencies to serve as the fiscal agency for their particular region, and these agencies provide planning/coordination/oversight for Career Preparation funds that flow into the region ( Michigan Department of Career Development, 2001 , p. 18). Figure 4 details the flow of funds in the Career Preparation System.
Figure 4.
Funding Stream to Support Career Preparation Activities
Allocations for Michigan's Career Preparation System
Michigan Legislature
⇓
Michigan Department of Career Development
⇓
Workforce Development Boards
⇓
Education Advisory Groups
(25 in the state)
(Educators, local and intermediate school district superintendents, business leaders)
⇓
Intermediate School Districts and/or Individual School Districts
Another shift the state made to offset the reticence regarding work-based programs, and hence to increase participation in state career preparation efforts, was to change the name of the program from "school-to-work" to "career preparation." State and local career preparation representatives interviewed reported that the state changed the terminology because "school-to-work" was perceived by too many parents, educators, and community members to mean vocational or technical education.
Interviews with representatives from all 25 of the state's EAGs and with several representatives from the Michigan Office of Career and Technical Preparation, including the coordinator of career and technical preparation, were used to learn more about the implementation of the Career Preparation System (and to learn about the scope of evaluation activities that have taken place to measure the impact of school-to-work activities on youth employment, discussed below). In talking with EAG representatives, it became evident that some EAGs require more accountability than others. In Muskegon and Oceana counties, for example, districts must provide a signed contract that specifies how quality criteria will be met and measured. The contract is a performance-based model in which districts are paid according to the number of students served, the number of work-based learning placements achieved, the number of teachers participating in teacher-in-industry training, or other activities outlined in the contract by the EAG. According to the area's EAG chair and Muskegon Area Intermediate School District Superintendent, it is the system of accountability that is helping to spread the Career Preparation System throughout the school districts (personal communication, M. Bozym, November 2001).
In terms of participation, the state's efforts to build a Career Preparation System are succeeding, based on figures reported in the 1999-2000 Michigan Department of Career Development Progress report ( Michigan Department of Career Development, 2000 ). In 1998-99, 90 percent of Michigan school districts participated in voluntary Career Preparation programs. In addition, more than 60 of the state's high schools during the 2000-2001 school year were in the process of implementing Career Pathway programs. Finally, according to the district educational plans reported to the State of Michigan, 88 percent of the state's school districts have committed to implementing Education Development Plans for each secondary student and developing a Career Pathways curriculum by 2004 (personal communication, J. Williams, Michigan Department of Career Development, October 2001 ).
Evaluating School-to-Work in Michigan
This section turns to the central goal of our research, which was to assemble and assess all available information that could be used in evaluating whether school-towork as implemented in Michigan is achieving the goal of helping students better prepare for success in the labor market. We used our phone interviews of representatives of the state's 25 EAGs, and discussions with the Director and several other representatives from the Michigan Office of Career and Technical Preparation, to identify and then gather any available measures or existing studies regarding how effective these activities are for youths in Michigan.
The interviews were conducted mostly in the fall of 2001, with some follow-ups in the spring of 2002. There was not a formal interview protocol, because we were not so much interested in qualitative or quantitative data obtained directly from the interviewees, but rather in information they could provide on evaluation efforts in the geographic areas they covered, or statewide. That is, the interviews were used to identify any evaluation efforts or studies that had been done or were under way, but to a large extent the core information we present and the critical analysis we offer is based on the evaluation efforts or studies themselves, rather than the direct content of the interviews. Nonetheless, we took great care to explain to the interviewees the kinds of information or studies for which we were searching, and often had to have numerous conversations to clarify the nature of our inquiry and to follow up on leads they had provided concerning evaluation studies. We did, though, also use the interviews to flesh out our understanding of school-to-work institutions and of evaluation efforts in Michigan, in cases where printed sources were inadequate.
Like the tech prep initiative ( Jacobs & Teahen, 1998 ), little data have been gathered in the state to show how school-to-work initiatives in general, or the state's Career Preparation System in particular, are affecting students' decisions about careers or their career outcomes. However, we did find some information from the state's annual follow-up survey of career and technical education students, and from five other studies of school-to-work at the local level.
To effectively determine whether the state's Career Preparation System (or earlier school-to-work programs) produce better prepared students who are making better career decisions and experiencing better career outcomes, evaluation studies should compare outcomes for students who have participated in these activities-the treatment group-with outcomes for comparable students who have not participated in these activities-the control group. The control group is essential to eliminate spurious inferences of positive (or negative) effects attributable to other variables associated with both school-to-work participation and outcomes. For example, if youth labor markets are improving at the same time that school-to-work is expanding, we have to be careful about attributing greater labor market success to school-to-work. Such an inference would only be valid if students participating in school-to-work fared relatively better than other (comparable) students. (See Porter (1997) for a discussion of this point with regard to education research, Neumark and Joyce (2001) for the particular context of school-to-work, and Heckman, Lalonde, & Smith (1999) for a thorough discussion of evaluation studies.) Ideally, students should be placed in control and treatment groups based on random assignment, although as Heckman et al. (1999) point out, this is not necessarily a panacea, and is often not practical. In practice, statistical methods of adjusting for differences between non-randomly selected treatment and control groups are likely to be necessary.
In general, our search for evaluations of school-to-work activities in Michigan revealed a handful of such studies, and most have some serious limitations. With that said, though, there are some valid reasons for the lack of such studies, so our main message is not to criticize the current lack of evaluation studies, but rather to encourage such studies to be undertaken in the future. First, the complexity of social sciences, in which multiple human factors affect outcomes, leads many researchers to believe that mixed methodologies such as surveying, interviewing, and observation are required to truly understand the root of the outcome ( Dunn, 1994 ). J. Mahony (personal communication, November 2001), of the Office of Career and Technical Preparation, suggested that the complexity of analysis that might be needed to fully evaluate school-to-work may have inhibited the state or the majority of the state's practitioners from engaging in significant evaluation. Second, the state's tech prep efforts indicate there are other challenges to evaluating student outcomes early in the process of systems building ( Jacobs & Teahen, 1998 ). Some of these include confusion over what constitutes success, and a lack of students involved in the program long enough to measure the impact of their participation. Jacobs and Teahen also reported that the many changes occurring within the state's decentralized education system made it difficult to determine whether changes were the result of tech prep or some other initiative. The same is likely true of the evolving Career Preparation System. In the remainder of this section we review the available evaluations, both summarizing their findings and, most importantly, assessing their ability to establish causal effects of school-to-work.
State-wide Data
Mathematica study . Representatives from the EAGs and the Office of Career and Technical Preparation reported that most of the activity regarding measurement and assessment of the STWOA in Michigan occurred through participation in a national school-to-work study by Mathematica Research, Inc. ( Hershey et al., 1999 ). According to J. Mahony (personal communication, November 2001), "There was a sense that if people were using their resources for evaluation, they wouldn't have anything left for program implementation." The Mathematica report indicates that 90 percent of Michigan's schools participated in some school-to-work activity during the STWOA funding period. During that time, more than 80 percent of the participating schools provided data to Mathematica. As one of Mathematica's eight study states, Michigan schools participated in surveys, observations, and interviews with Mathematica researchers during 1996, 1997, and 1998.
While the Mathematica study provides a national picture of school-to-work initiatives, little information was disaggregated to isolate findings from specific states. Therefore, while Michigan schools actively participated in the research efforts, little evaluation specific to Michigan's school-to-work initiatives was provided. In addition, the Mathematica study did not constitute an evaluation per se, as it did not attempt to assess the causal relationship between student outcomes and school-to-work. As the study argues:
The evaluation can help us understand the extent to which a STW system is being created and how students' experiences are changing. It cannot, however, provide evidence of whether STW activities cause changes in student outcomes. STW implementation generally involves broad and diverse initiatives that in varied ways touch most or all students, so it is impossible to distinguish between participants and an unaffected comparison group. ( Hershey et al., 1997 , p. xviii)In other words, the authors contend, it is not even possible to define treatment and control groups. While the Mathematica study offers data showing that students in school-to-work programs receive more training and are employed in a broader range of industries than are other students in paid positions, Neumark and Joyce (2001) counter that the Mathematica report lacks evidence that the cause of these outcomes was the school-to-work experience, suggesting instead that "Students most likely (to find these jobs) may simply have sorted into school-to-work programs" (p. 668). This problem with comparing non-randomly selected treatment and control groups is referred to as the "selection problem" in evaluation studies. In addition, Neumark and Joyce argue that the Mathematica study could, in principle, have carried out something more akin to an evaluation, exploiting the variation across school districts and states in the incidence of school-to-work partnerships supported by grants under the STWOA.
A study commissioned by the state by Detroit-based Moore & Associates, Inc., provided to the state in November 2000, appears to be based in part on a review of data submitted for this national evaluation, as well as interviews with school-to-work coordinators or directors at the Michigan Works! offices. (This study was not publicly available, but was described in a personal communication with J. Mahoney, October 2001.) It provides some participation data and a summary of interview responses, indicating that most respondents believed school-to-work initiatives were making a difference to students by increasing opportunities for career awareness. However, the study includes no evaluation of the impact of these opportunities on students' career decisions.
Career and Technical Education Survey . The limitations of the Mathematica study also appear in the majority of studies on school-to-work programs in Michigan. At the state level, the Michigan Office of Career and Technical Preparation, through their Follow-up Survey of Completers in Career and Technical Education, collects data on career and technical education students who recently graduated, to determine how their high school experiences relate to their current job or schooling. However, the surveys are required to be given only to students in career and technical education programs, providing no comparison with students not in such programs, let alone attempting to control for selection of students into these programs.
The state's 2001 Follow-up Survey of 2000 Completers in Career and Technical Education shows a breakdown of how participants perceive the relationship between their current job and education and their secondary career preparation experiences ( Michigan Department of Career Development, Office of Career and Technical Preparation, 2001 ). The survey results show the following: 18.1 percent of the 4,966 respondents report current work and education as related to their secondary career preparation programs; 7.2 percent report that neither their work nor their current education are related; 3.2 percent report that their job is related, but their current education is not; and 8.6 percent report that their current education is related, but their job is not. Yet it is unknown if these percentages have a direct relationship with career and technical education. It could be, for example, that the same percentage of all students, regardless of their high school experience, find their postsecondary work and school relevant to their high school education. Another shortcoming of this evidence is that by limiting surveys to students in career and technical programs, no data are collected to assess broader career preparation activities-such as Career Pathways curricula, job shadowing, internships, career visits, or career fairs-for students who have not enrolled in career and technical education programs.
A few EAG representatives report using the state surveys as an assessment tool for all graduating seniors. In particular, representatives from EAGs in Macomb, Gratiot, Ionia, Isabella, and Montcalm counties reported distributing the state survey to all students, and extending the survey to collect data that connects students' high school career preparation activities to their current education and employment. This is a promising development, but our interviews yielded no indication that EAG areas have made much progress in their evaluation efforts. There is, though, some indication that the state is taking steps to move evaluation of the Career Preparation System forward, as J. Mahoney (personal communication, November 2001), of the Office of Career and Technical Preparation, reported that a career preparation accountability committee reportedly has been established at the state level to look at issues of assessing the effectiveness of career preparation activities in better preparing students for careers.
Local Data
Some practitioners in the state are also making efforts to evaluate the effects of career preparation for their graduates. A report by one of the Michigan Works! agencies stated that all Berrien county high schools have adopted the Career Pathways model and are in the process of measuring the impact of the results for every graduating student ( Michigan Works! Agency, 2001 ). Other local efforts we found that attempt to evaluate the effectiveness of school-to-work or career preparation activities for students include annual reports on Kalamazoo County's Education For Employment (EFE) program, Marquette-Alger Intermediate School District's (MAISD) school-to-work initiative, and follow-up studies of students in Macomb County.
Berrien County . Data collected so far in Berrien County suggest that the district's adoption of the Career Pathways model led to large increases in student enrollments in math and science courses and in career and technical education ( Rudy, 2001 ). These data indicate a dramatic increase in career and technical education enrollment, from 895 students in 1997 to 5,554 students in 2001. Likewise, enrollment in 3 rd year math programs increased from 686 students in 1997 to 2,080 students in 2001, and enrollment in 3 rd year science programs increased from 769 students in 1997 to 2,389 students in 2001. The Berrien County report also shows large increases in the number of students enrolling in postsecondary education either as high school students through dual enrollment or as graduates. In 1997, 62 high school students were enrolled in college level courses, increasing to 299 in 2001. Similarly, in 1997 61 percent of Berrien County graduates attended postsecondary institutions, rising to 69 percent in 2001.
These data, which coincide with the implementation of Career Pathways and Education Development Plans, suggest that these activities have had a positive effect on students' postsecondary enrollments and career preparation. Of course, in line with our general point regarding evaluation studies, it is difficult to conclude that these impacts are attributable to school-to-work programs, without a comparison showing relative gains with respect to students in other districts where these efforts were not implemented.
MAISD school-to-work initiative evaluation report . The Education and Human Services Committee of the Lake Superior Community Partnership commissioned a study to assess the effectiveness of school-to-work initiatives in schools served by the Marquette-Alger Intermediate School District (Dubow & Mourand, 1999). The local plan for the MAISD school-to-work initiative was to engage the community in the preparation of young people for work. The local efforts focused on high school students' work-based learning experiences, with career planning beginning as early as elementary school ( Dubow & Mourand, 1999 ). The local plan intended to accomplish the following: promote the use of local businesses for the development of work-based learning programs; restructure the role of counselors and teachers to provide students with access to school-to-work activities and better information to make career decisions; promote two high school pathways-Tech Prep and University Prep; form partnerships through four local school-to-work focus groups with more than 50 percent membership from businesses; and hire local school-to-work coordinators in participating counties.
The intent of the study was not to address how well each of the above goals were met, but rather to "… paint a picture, based on the information and perceptions of key stakeholders of Marquette and Alger Counties, regarding the school-to-work initiative" ( Dubow & Mourand, 1999 , p. 7). Researchers gathered quantitative data on all of the area's high school students to measure progress on the state's five school-to-work goals. The researchers reported that MAISD met the state's graduation rate goal of 90 percent, and that 98 percent of the area's graduates were employed or attending a postsecondary school one year following graduation, which they characterized as falling short of state goals (although it would seem unreasonable to expect 100 percent success). (We have not been able to identify the source of the state standards cited in this study.) While approximately 38 percent of the area's high school seniors completed a career and technical course in 1997, the authors reported that there was no state-certificate program available at the time the data were collected, making it difficult to achieve the state goal of certification for 35 percent of seniors. Data also indicate that 75 percent of the area's students participated in a work-based learning experience, versus the state goal of 100 percent participation, and about 86 percent of students in the area achieved an endorsed diploma in communication arts, mathematics, and science, versus the state standard of 90 percent.
To attempt to establish progress due to school-to-work, data on some goals such as academic achievement, employment, and graduation over three years are provided. But there is in fact little evidence that establishes this progress. For example, follow-up surveys of graduates at all of the MAISD high schools for 1995, 1996, and 1997 indicate a consistent rate of 25 percent of graduates employed fulltime. While the number of students participating in school and work increased each of the three years, the number of students enrolled in school alone decreased in each of the three years. The result is a slight increase in the share of students either enrolled in school alone or in school and working, from 64 percent in 1995 to 66 percent in 1996, and remaining at 66 percent in 1997. The share of unemployed graduates was consistent at 2 percent for all three years. Other data presented are simply for one year, providing no comparison to gauge progress on state goals, and- echoing our earlier criticism-providing no indication that the outcomes represented in the data are a direct result of school-to-work activities.
The report's qualitative data provide some evidence that school-to-work activities help students better connect to career interests. For example, responses from educators as to the strengths of school-to-work included statements such as "students find out they don't like certain types of work" ( Dubow & Mourand, 1999 , p. 11). However, most of the questions and responses in the report reflect the local plan's efforts to build partnerships among businesses and schools and to establish priorities for school-to-work within schools and communities. Based on the qualitative data, the authors contend that key issues regarding the effectiveness of the school-to-work efforts are the need for a school-to-work coordinator to help build partnerships between educators and employers and to coordinate school-to-work opportunities for students, and the priority of school-to-work activities within the school community. Much of the MAISD study underscores the systems issues discussed earlier ( Jacobs & Teahen, 1998 ), such as the need for educators and parents to see value in school-to-work activities for all students. Overall, there is little evidence in the MAISD report that directly ties school-to-work activities to youth employment (or education) outcomes.
Kalamazoo County's Education for Employment (EFE) . Kalamazoo County provides career and technical education for a broad range of careers, offering both classroom-based and work-based programs through its EFE program. The classroombased programs are conducted in schools, post-secondary institutions, or work sites, and cover 30 different occupations. Work-based programs supplement classroombased programs by including cooperative learning (co-op), business/industry worksite training, and apprenticeships.
Kalamazoo County's EFE program commissioned the Upjohn Institute to conduct a study of its programs through annual follow-up surveys with program participants. In addition, Upjohn conducted focus groups with students engaged in EFE programs in 1996. Surveys of participating students and completers were conducted covering the period 1996-2001, to garner an understanding of the impact of the EFE program on career and postsecondary decisions, opinions about EFE programs, and trends in EFE experiences and employment.
Approximately 2,500 students in Kalamazoo County were enrolled in EFE programs at the time of the surveys conducted in 2001 ( Hollenbeck & DeBurman, 2001 ). Results from the focus groups and surveys suggest that the school-to-work experiences of students in EFE have an impact on the students' career decisions. "The programs provided students with information that helped them to select specific institutions and to narrow their fields of study. Articulation agreements allowed students to acquire college credits for courses they were taking, and work experiences allowed students to gain hours toward occupational certification" ( Hollenbeck, 1996 , p. 27). Even though the study surveyed only students involved in the EFE programs, the Upjohn Institute study does attempt to offer some causal evidence based on a comparison of student groups. For example, students enrolled in EFE programs may participate in programs with work-based learning components or programs without work-based learning components. By surveying students in both of these EFE tracks, the data indicate some differences between students who have experienced work-based learning and those who have not.
The study indicates, for example, that work-based learning experiences affected students' choices of postsecondary programs. "… [S]tudents who were not in a work-based program in high school were much more likely to report that their EFE training was not at all related to their field than the students who were in a workbased program" ( Hollenbeck, 1996 , p. 46). Specifically, survey data from students in 2001 indicate that students' participation in EFE programs influenced student decisions to attend a postsecondary institution for about 40 percent of the students. In addition, about two-thirds of the survey respondents who were in postsecondary programs and who have selected a field of study reported that it was related to the EFE class either "a lot" or "somewhat."
But because EFE students self-select into particular EFE programs, determining for themselves whether they will enroll in a work-based program, it is possible that there are characteristics of students who self-select into work-based programs that are associated with better career decisions, or greater focus on career decisions at this point in their lives. However, the 1996 focus groups with EFE students suggested that students who were exposed to work-based learning were better able to make career decisions based on their experiences. The study cites three specific types of benefits based on the student interviews: exposure to all aspects of an industry; identification of specific careers; and development of personal contacts with employers.
Employment data on EFE graduates indicate that two-fifths of the students who were working reported using skills and training received in their EFE classes. The other three-fifths, however, reported using "hardly any" of their EFE skills and training. These data are not broken down according to students who were in workbased or non-work-based programs, so again it is difficult to determine if work-based experiences make a difference to whether students are using their training on the job. And data from a comparison group of students not enrolled in EFE would provide a basis to determine whether the programs had any effects at all.
Finally, some of the most recent employment figures are, at least on the surface, discouraging. Employment data for the 2001 report show a decrease in the number of participating students who were currently employed in jobs other than those affiliated with the work-based EFE programs, which could suggest failure to move students into jobs outside of the school-to-work system. The percentage dropped from 60 percent in 1996 to 50 percent in 2001, although the drop may have been caused by increased enrollment of 9 th and 10 th grade students ( Hollenbeck & DeBurman, 2001 ). In addition, the 2001 employment rate of EFE completers was lower than in any of the previous years. In 2001, 75 percent of completers surveyed were employed compared with 82 and 88 percent in the previous two years. However, the 2001 survey was conducted earlier in the year than the previous surveys, which might account for the discrepancy in employment figures ( Hollenbeck & DeBurman, 2001 , p. 49). Finally, some of the 2001 results may also have been influenced by a slowing economy, yet again emphasizing the need for a comparison group.
Macomb County . A series of five-year follow-up telephone surveys of Macomb County graduates was initiated by the Macomb County School-to-Work Partnership in 1997. The fourth poll by the group, now called the Macomb County Career Preparation System, was conducted in 2000 and covered students who graduated in 1995 ( Macomb County Career Preparation System, 2000 ). This survey-the Macomb County Five-year Follow-up Survey of the Class of 1995-provides data on the educational programs attended by 1995 graduates, the kinds of careers chosen by graduates, and their employment and expected and actual income. None of these data, however, are tied to students' experiences in high school school-to-work programs. The survey does, though, ask graduates about the effectiveness of high school counseling in finding postsecondary education, training, and careers, and there is a brief discussion of the impact high school employment had on graduates.
Whereas a 1995 survey of Macomb County seniors indicated that 97 percent of the respondents planned to continue their education following graduation, data from the five-year follow-up survey indicate that 90 percent of the respondents attended some type of "formal education" or education leading to a diploma, degree, or certification upon completion. Eighteen percent of the respondents who reported attending an educational program also reported not completing the program. Without knowledge of the school-to-work initiatives that may have influenced students' postsecondary education and career decisions, or a comparison group, data from the follow-up survey provide no indication about the relationship between the county's school-to-work or career preparation system and student career and education outcomes.
A review of survey results from the previous three years is also presented in the report. A comparison shows that college enrollment rose 10 percentage points from the class of 1994 (77 percent) to the class of 1995 (87 percent). However, approximately the same difference exists between the class of 1993 (84 percent) and the class of 1992 (75 percent)-classes graduating prior to the STWOA-indicating that it is difficult to draw any inference of an effect of the STWOA. There was a significant decrease, however, in the number of students who reported not attending any further education from the class of 1994 (17 percent) to the class of 1995 (10 percent), possibly indicating that school-to-work initiatives help more students see the need for further education. But without more information about school-to-work activities and students' participation in them, it is impossible to determine if schoolto- work helps explain the drop.
The Macomb County Five-year Follow-up Survey of the Class of 1995 indicates that about half of the 1995 graduates believe high school counselors could do a better job preparing students for college, postsecondary training, or careers. Students who engaged in training and education programs other than college gave a more favorable assessment of high school counseling than other students, with only 48 percent reporting that high school counseling could be improved. In contrast, of the graduates who went on to college and responded to the survey, 59 percent indicated that high school counseling for helping students explore what various colleges had to offer could be improved, and 57 percent indicated that improvements could be made in helping students find ways to finance a college education. Among all of the survey respondents, 55 percent reported high school counselors could have provided more help in exploring and choosing a career.
Employment rates of Macomb County graduates have remained consistent over the past four years, with unemployment at 2 percent for the classes of 1995, 1993, and 1992, and unemployment slightly lower at 1 percent for the class of 1994. The class of 1995 reported a lower number of respondents employed full-time (71 percent) than in the other three years (78 percent in 1994, and 73 percent in 1992 and 1993), and a higher number of respondents employed part-time (19 percent) than in the other three years (14 percent in 1994, 17 percent in 1993, and 16 percent in 1992). Both the class of 1995 and the class of 1992 reported 7 percent of graduates attending school five years after graduation, higher than the classes of 1994 (4 percent) and 1993 (5 percent). Again, the employment data offer little evidence on the effectiveness of school-to-work or the career preparation system without any tie to students' high school career preparation activities, and aside from that, indicate no clear trends over time that might at least informally be linked to school-to-work.
The report also discusses the impact of employment during high school. Eightyeight percent of the respondents of the class of 1995 reported having a job during their senior year in high school. Of these students, 76 percent reported that their high school jobs had no effect on the grades they earned in high school and 67 percent said that their high school job did not influence them at all when it came to choosing a career. Sixty-six percent, however, said their high school job helped prepare them for the world of work. Of course, because of selection into high school employment, and because these data are self-reported perceptions of preparedness rather than objective measures of outcomes, such results are not too informative about the impact of work in high school.
Discussion
Building on the state's tech prep initiative, school-to-work in Michigan has resulted in a comprehensive Career Preparation System, which is being adopted by most of the state's school districts. This fact alone provides evidence that the goal of the STWOA to build a system of career preparation for all students has been successful in Michigan. But what has been the impact of these activities on Michigan students? State-level evaluation work has focused on students in career and technical education programs. In general, outcome-based assessment activities other than the collection and reporting of data on career and technical education for the state have not been universal or even widespread. We did, though, uncover some local studies of school-to-work at the level of counties or school districts. Overall, the strength of the evidence in this limited set of studies in support of positive effects of school-towork or career preparation activities on student career education, training, and employment, is weak. But for the most part, at this point there is simply insufficient information to provide either a constructive assessment of school-to-work in Michigan, or much guidance for improvement. At best, there are a couple of instances of suggestive evidence that career preparation activities may assist with students' career and employment decisions, and that work-based learning experiences, in particular, may help students make better career decisions.
In contrast, there is somewhat more information available on the progress local school districts and the state have made toward building a career preparation system. Assessing this progress can provide some useful information with regard to schoolto- work, including: the implementation of school-to-work; the number of students being served by the system and characteristics of those students; the number of students earning employable certification; and the extent of employer participation. However, these "progress reports" generally do not provide any information on the causal impact of the system on student outcomes such as attendance, discipline, academic achievement, graduation rates, and enrollment in postsecondary education, or on labor market outcomes.
The goal of this paper was to ask whether by engaging in an exhaustive case study of school-to-work initiatives in one state a more compelling picture of beneficial effects of school-to-work programs would emerge than has been established by the work done on this question at the national level. Unfortunately, the answer appears to be in the negative.
But we hope that this case study helps to clarify some of the imperatives for future research on school-to-work. To gauge the impact of school-to-work programs on students, evaluators need to look at the outcomes of students in relation to their career preparation activity. Equally important, efforts should be made to rule out other causes for changes or differences in student education, employment, and career outcomes. Finally, efforts must be directed to assessing the effects of school-to-work on all students, not just those enrolled in particular programs (such as, in Michigan, career and technical education). The difficulty in engaging in evaluation activities during program implementation is apparent, but waiting until full implementation is achieved can lead to important missed opportunities. In particular, beginning to collect data on student career outcomes before all students are participating in a school-to-work system can provide at least some data that can be helpful in constructing comparison groups that can be used to help evaluate the effects of career preparation on students. Efforts such as those in Michigan to develop a career preparation accountability committee are on the right track.
But it is most important that government bodies responsible for school-to-work not delay efforts to formally evaluate school-to-work programs, including collection of data while implementation remains incomplete, and consideration of whether some components of implementation of the system-not necessarily restricted to ex post data collection, but perhaps also expanded to include, for example, differential program treatments across geographic regions-may be utilized to construct more rigorous evaluations of school-to-work programs. Indeed, we would go so far as to suggest that serious thought be given to including mandates for rigorous evaluation in any future legislation allocating funds to school-to-work. A related policy was pursued in the run-up to welfare reform in the early 1990s, as states requesting waivers from federal welfare regulations in order to try implementing their own welfare reforms were required to engage in experimental evaluations of their waiver programs. Blank (2002) describes the results of these evaluations as contributing importantly to the federal reforms adopted in 1996.
References
Bailey, T. R. (1995) . Learning to work: Employer involvement in school-to-work transition programs . Washington, DC: The Brookings Institution.
Blank, R.M. (2002) . Evaluating welfare reform in the United States. Journal ofEconomic Literature , 60, 1105-1166.
Dunn, W. (1994) . Public policy analysis . New Jersey: Prentice Hall.
Dubow, K., & Mourand, D. L. (1999) . School-to-work evaluation report . Marquette, Michigan: Lake Superior Community Partnership.
Elmore, R. (1980) . Complexity and control: What legislators and administrators can do about implementing public policy . Washington, DC: National Institute of Education, Department of Health, Education and Welfare.
EPIC-MRA. (1997) . Statewide survey on school-to-work . Lansing, MI.
Heckman, J. J., Lalonde, R. J., & Smith, J. A. (1999) . The economics and econometrics of active labor market programs. In O. Ashenfelter & D. Card (Eds.), Handbook of labor economics, volume 3 (pp. 1865-2097). Amsterdam: Elsevier Science B.V.
Hershey, A. M., Silverberg, M. K., & Haimson, J. (1999) . Expanding options for students: report to Congress on the National Evaluation of School-to-Work Implementation . Princeton, New Jersey: Mathematica Policy Research, Inc.
Hollenbeck, K. (1996) . In their own words: Student perspectives on school-to-work opportunities . Washington, DC: National Institute for Work and Learning, Academy for Educational Development.
Hollenbeck, K., & DeBurman, N. (2001) . Assessment of Kalamazoo County's Education for Employment (EFE) programs, using 2001 survey data . Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.
Hughes, K. L., Hamilton, S. F., & Ivry, R. J. (2001) . School-to-work: Making a difference in education . New York: Institute on Education and the Economy, Teachers College, Columbia University.
Jacobs, J., & Teahen, R. C. (1998) . Michigan tech prep in the midst of systemic reform. Journal of Vocational Education Research , 23, 115-132.
Levin, C., Sen. (September 30, 1999) . Levin announces school-to-work awards for Michigan. Washington, DC: Press Release.
Macomb County Career Preparation System. (2000) . The Macomb County five-year follow-up survey of the class of 1995 . Macomb County, MI.
McLaughlin, M. (1987) . Learning from experience: lessons from policy implementation. Educational Evaluation and Policy Analysis , 9, 171-178.
Michigan Department of Career Development (2000) . 1999-2000 progress report . Lansing, MI.
Michigan Department of Career Development (2001) . Career preparation system overview . Lansing, MI.
Michigan Department of Career Development, Office of Career and Technical Preparation. (2001). Respondent status profiles 2001 follow-up survey of 2000 completers . Lansing, MI.
Michigan Works! Agency. (2001) . Report card strategic assets and comprehensive plan: Goals for the future . [On-line.] Available: http://www.miworks.org/Comprehensive%20Plan.htm .
Neumark, D., & Joyce, M. (2001) . Evaluating school-to-work programs using the new NLSY. Journal of Human Resources , 4, 666-702.
Office of Technology Assessment, United States Congress (1995) . Learning to work: Making the transition from school to work , Washington, DC: U.S. Government Printing Office.
Porter, A.C. (1997) . Comparative experiments in education research. In R.M Jaeger (Ed.), Complementary methods for research in education, 2 nd ed . (pp. 523-544).Washington, DC: American Educational Research Association.
Rudy, D. W. (2001) . Career Pathways, a reform model for K-12 education . New Buffalo, MI: Lakehouse Evaluation.
Tech Prep Task Force (1988) . Tech prep partnerships: Executive summary of the tech prep task force . Lansing, MI: Michigan Department of Education.
Urquiola, M., Stearn, D., Horn, I., Dornsife, C., Chi, B., Williams, L., Merritt, D., Hughes, K., & Bailey, T. (1997) . School to work, college and career: A review of policy, practice, and results, 1993-1997 . National Center for Research in Vocational Education. Berkeley, California: University of California.
Acknowledgments
DAVID NEUMARK is Senior Fellow in Economics at the Public Policy Institute of California, Professor of Economics at Michigan State University, and a Research Associate of the National Bureau of Economic Research. [e-mail: neumark@ppic.org ] His research interests include school-to-work, the returns to education, discrimination, affirmative action, minimum wages, living wages, and the economics of aging. ANN ALLEN is a graduate student in the College of Education at Michigan State University. [e-mail: AnnAllen@aol.com ] We thank David Plank and anonymous referees for helpful comments. This research was supported by the Education Policy Center at Michigan State University. Any opinions expressed are the authors' own, and do not necessarily express the views of the Education Policy Center or the Public Policy Institute of California. Corresponding author: David Neumark, Public Policy Institute of California, 500 Washington St., Suite 800, San Francisco, CA 94127, 415-291-4476, neumark@ppic.org .