Impact of an Engineering Mentorship Program on African-American Male High School Students’ Perceptions and Self-Efficacy

Cameron D. Denson
Utah State University

Roger B. Hill
University of Georgia

Abstract

The purpose of this study was to examine the impact of an engineering mentorship program on African-American male high school students’ perceptions of engineering as a viable career choice. In this study, indicators included students’ perceptions of engineering, their self-efficacy in the area of mathematics, and their self-efficacy in the area of science. Using an independent t-test to determine a difference of statistical significance, inferential statistics were provided to answer the following research questions: (a) Is there a significant difference in perceptions of engineering for students who participated in the NCETE/NSBE mentorship program when compared with non-mentored students?, (b) Is there a significant difference in self-efficacy in the area of mathematics for students who participated in the NCETE/NSBE mentorship when compared with non-mentored students?, and (c) Is there a significant difference in self-efficacy in the area of science for students who participated in the NCETE/NSBE mentorship when compared with non-mentored students?

Introduction

If the United States (U.S.) is to meet its need for world class talent in science, technology, engineering, and mathematics (STEM), it is essential that a diverse population be attracted to engineering and other technical fields (Chubin, May, & Babco, 2005). Culturally, the preclusion of minorities from technical fields has significant ramifications. According to Jenkins (1999) , for minorities to be able to skillfully adapt to an ever changing economy in a capitalist society it is pertinent that they become technologically proficient in the coming years. Technological proficiency not only speaks to the understanding and manipulation of technological devices but it also speaks to increased representation in fields that require technological literacy, particularly engineering, computer science, and technology education. Technological proficiency is not only vital to the socioeconomic and educational growth of minorities; it also has implications for the nation as a whole as the U.S. strives to maintain a competitive workforce.

To effectively begin to diversify the fields of engineering and other technical fields, several challenges need to be addressed including (a) a current technical workforce that is undiversified in relation to the total workforce ( Wheeler, 1996 ), (b) ineffective plans of action currently in use for recruitment and retention of minority students and faculty ( Jeria & Gene, 1992 ), and (c) a pedagogical approach to STEM instruction that is culturally unresponsive ( Carter, 2005 ). A review of literature on diversity within technical fields shows that mentorship programs have provided some answers to these puzzling challenges. Within organizations, formal mentoring programs have benefited the growth of women and minorities in the workplace by helping with assimilation to the workplace ( Hansman, 2002 ). For the U.S. to adequately address the disadvantages of an undiversified technical workforce ( Wheeler, 1996 ), a promising strategy is the use of interventions such as mentorship programs as a means to recruit minorities to engineering and other technical fields.

As a grass-roots initiative, mentorship programs act as a vehicle for change, satisfying the need for connections with family and community as exemplified in the following quote, “The structural and attitudinal changes required for instituting changes that transcend single professional field and agency auspices cannot occur without rooted connections with families and the community” ( Oates, Weishew, & Flores, 1998 , p. 53). Formal mentorship programs offer a viable approach for recruiting minorities to engineering disciplines and other technical fields by serving as extensions of these communities. As a tool of affirmative action, mentorship programs have been utilized since the 1970s and 1980s ( Van Collie, 1998 ). Research shows that formal mentoring programs have become effective recruitment tools for many organizations seeking to recruit and retain minorities in the workplace ( Allen & O’Brien, 2006 ). Further illustrating the feasibility of mentoring as a tool to promote diversity in technical fields, Maughan (2006) stated that mentoring has repeatedly been shown to enrich the process of learning. This enrichment of learning may in itself positively impact retention, recruiting, and knowledge management of organizational members.

Although there is research available that documents the effectiveness of mentorship programs on a student’s academic success, especially for at-risk students ( Campbell-Whatley, Algozzine & Obiakor, 1997 ; Hall 2006 ), there has not been much research dedicated to the examination of mentorship programs in relation to minorities’ perceptions towards career choices related to engineering. Using a specialized group, this study examined the impact of a formal mentorship program on African-American male high school students’ perceptions of engineering as a viable career choice. Findings from this research may provide a basis for future initiatives seeking to introduce effective strategies for recruitment and retention of underrepresented populations.

Purpose of the Study

The purpose of this study was to examine the impact of an engineering mentorship program on African-American male high school students' perception of engineering as a viable career choice. In this study, indicators included students' perceptions of engineering their self-efficacy in the area of mathematics, and then self-efficacy in the area of science. This study used a two-group, posttest only, experimental design with randomly selected participants. After participation in the National Center for Engineering and Technology Education (NCETE)/National Society for Black Engineers (NSBE) mentorship program, the treatment for this study, a survey was used to collect data to answer the following research questions:

Research Question

1. Is there a significant difference in perceptions of engineering for students who practiced in the NCETE?NSBE mentorship program when compared with non-mentored students?
2. Is there a significant difference in self-efficacy in the area of mathematics for students who participated in the NCETE/NSBE mentorship program when compared with non-mentored students?
3. Is there a significant difference in self-efficacy in the area of science for students who participated in the NCETE?NSBE mentorship program when compared with non-mentored students?

The primary construct for this study was students’ perceptions of engineering as a viable career choice. Students’ perceptions were understood by measuring three different variables to include; students’ conceptual perception of the engineering field, students’ self efficacy in the area of math, and students’ self efficacy in the area of science. Students’ perceptions of engineering was derived by examining students self-reporting on their understanding of engineering concepts and their confidence to perform requisite skills associated with the profession. Measures of students’ perceptions included their self-efficacy in math and science due to the importance of these subject areas within the engineering profession. As noted by Wicklein (2006) , an integral part of the engineering experience is the application of mathematics and science. In addition, studies have shown that a child’s perception of an occupation and their self-efficacy greatly influence the decision of a child to pursue the occupation ( Bandura, Barbaranelli, Vittorio, & Pastorelli, 2001 ). Using Bandura, Barbaranelli, Vittori and Pastorelli’s (2001) previous work as a template, the researcher attempted to measure the impact of mentorship programs on students’ perceptions of engineering and their perceived self-efficacy to perform tasks associated with that profession.

Rationale and Theoretical Framework

In stating a rationale for the intervention of mentorship programs, it must be reiterated that federal legislation distinctly mentions that one purpose for mentoring is to "encourage students from underrepresented groups to pursue scientific and technical careers" ( U.S. Energy Policy Act. Sec 1102, p.10, line 16, 2006 ). As organizations and institutions look to meet the demanding needs of the nation's workforce, more research is needed that clearly delineates the benefits of formal mentorship programs. With respect to engineering and other technical fields, this study was particularly focused on the characteristics of mentoring and its functions in an academic setting. In this role the mentor usually acts as a sponsor who will provide his/her prospective protégé with exposure, coaching, and awareness of potential career opportunities ( Allen & Day, 2002 ). Within the scope of the mentoring relationship, these mentoring activities are categorized by the term career functions ( Allen & Day, 2002 ).

This investigative study utilized Kram’s (1983) theory of mentoring in an effort to gain insight into how mentorship programs influence students’ perceptions and self-efficacy. According to Kram (1983) , mentoring is a relationship between an experienced member of an organization and an understudy where the experienced employee acts as a role model and provides support and direction to the protégé. Due to the dynamic characteristics of the mentoring relationship (including social interactions), social learning theory was used to extend the understanding of this relationship. Merriam and Carafarella (1999) helped identify the relevance of social learning theories in reference to mentoring by stating “Social learning theories contribute to adult learning by highlighting the importance of social context and explicating the process of modeling and mentoring” (p. 139). The inclusion of social learning theories (inclusive of social cognitive theory) as a part of the theoretical constructs relevant to mentoring is the result of social learning theory’s emphasis on how social context and the environment reinforce behavior ( Ormund, 1999 ). This theory states that people learn from one another and it includes the concepts of observational learning, imitation, and modeling.

Methodology

This study used a two-group, posttest only research design model. This research design is useful in studies where the administration of a pretest may influence the participants' behavior during the experiment or on the posttest ( Gall, Gall, & Borg, 1996 ), The effects of the treatment administered can be measured by comparing the posttest scores of two populations. This particular research design is appropriate when trying to influence a stable characteristic such as students' perceptions and self-efficacy.

The dependent variables for the study were students' perceptions and self-efficacy, which included students perceptions of engineering and their self-efficacy in the areas of mathematics and science after participating in the NCETE/NSBE mentorship program. The mentorship program that the students participated in represented the treatment for the study. This research study was carefully designed and yielded useful information that could be generalized within margins of error to the target population of male high school students attending comparable alternative high schools that cater to "at-risk" male minority students.

Random assignment was used in this study to select participants, thus allowing all African-American male students attending the alternative high school an equal opportunity to be selected for the study. Factors of internal invalidity that were of particular concern were differences in the individual's history, maturity level and individual attrition rates as it relates to test taking. Random assignment among the participants was employed in an effort to spread the measurement error across the sample population.

Participants

The participants in this experiment were drawn from an alternative high school in North Carolina, which began in 2003 as an initiative designed to offer young men a new chance at success. The alternative high school is a single gender high school in North Carolina that provides smaller classes and a nurturing environment with the goal of boosting self-esteem and providing opportunities for a promising future to at-risk male students. In the literature, the term “at-risk” represents a construct used to designate a high probability of poor development and low academic achievement ( Werner, 1986 ). At-risk students also suffer from a sense of alienation from the culture of schools ( Fine, 1986 ). Research has shown that perceptions of a caring relationship with a teacher and a positive environment were related to school satisfaction ( Baker, 1999 ). A review of literature indicated that more research is needed to examine alternative interventions that can effectively impact the educational environment of at-risk students ( National Center for Educational Statistics, NCES, 2001 ).

To facilitate the mentorship program, the researcher recruited active members of the National Society for Black Engineers (NSBE). NSBE is the largest student-managed organization in the country. With over 2000 elected leadership positions, 12 regional conferences and an annual convention, NSBE provides opportunities for involvement that rivals that of any other organization (http://www.nsbe.org/). With its established name and reputation, NSBE serves as an exemplar student-based organization in the area of engineering and engineering education. Mentors were purposefully assigned to their respective participants based on adequate time schedules, similar backgrounds, and other salient information gleaned from a student information sheet each prospective NSBE mentor completed.

A simple random sample was used to select study participants. This sample was selected from the population of eighty-three students attending the alternative high school by a process that provided every member an equal opportunity of being selected. The main advantage of randomly selected samples is that it yields information that can be generalized to a larger population within margins of error which can be determined by statistical formulas ( Gay & Airasian, 2000 ). A list was generated that numbered all students from the alternative high school from 1 to 83. To provide a treatment group for this study, a computer software program was used to generate a random list of which the first twenty-one students of African descent generated in the random sorting were chosen as the treatment group for the mentorship program. To provide a control group for the study, the next twenty-one students of African descent in the random sorting were chosen as the control group for the study in descending order. The control group did not receive any mentoring during the program. Of the twenty-one students selected to be in the treatment group for the mentorship program, only fifteen provided parental consent and minor assent allowing them to participate in the program. The control group was reduced to this number to match the number of students participating in the mentorship program. It is suggested that equal group size is required to account for mean variances among groups ( Weinberg & Goldberg, 1990 ). Student participants were allowed to be a part of the study only after securing parental consent from a parent or legal guardian and providing minor assent.

Instrumentation

The survey instrument used in the study was designed using information based on literature related to perceptions of engineering disciplines and self efficacy in the areas of mathematics and science. A review of literature revealed a lack of existing instruments that could sufficiently answer the research questions framing this study. Articles and numerous publications from peer-reviewed journals describing the use and development of various instruments were reviewed. Instruments developed by the New Traditions Project (http://newtraditions.chem.wisc.edu/) and Marat’s (2005) study entitled Assessing Mathematics Self-efficacy of Diverse Students from Secondary Schools in Auckland provided the basis for an instrument that could effectively measure perceptions and self-efficacy related to science and math. The New Traditions Project is one of five systemic chemistry curricular reform projects funded by the National Science Foundation (NSF). The mission of this project is to “optimize” opportunities for all students to learn chemistry (http://newtraditions.chem.wisc.edu/). The format of the instrument used in this study closely resembles the evaluation survey created by The New Traditions Project. Marat (2005) developed an instrument that measured mathematics self-efficacy for students learning in a multicultural environment of which the results are provided in Assessing Mathematics Self-efficacy of Diverse Students from Secondary Schools in Auckland. Using existing questionnaires and literature that examined the intended constructs, an instrument was drafted. This instrument, according to face validation, measured the desired constructs that framed this particular study.

To ensure validity and reliability of the scale items, a panel of five experienced engineer and technology educators from Purdue University, North Carolina A&T State University, Duke University, Southern Illinois University, and Robert Morris University, reviewed the scale used in the study and provided feedback regarding clarity of questions and their relevance to the constructs being examined. To test the validity of the instrument and ensure that the instrument was measuring the desired constructs, the researcher had the survey reviewed for validity and after careful consideration of the feedback provided from the panel of experienced engineer and technology educators, the scale was revised and reviewed again. The final form only achieved approval after the researcher’s panel of experts was satisfied with the revisions and consensus had been reached.

The reliability of the test was evaluated using Cronbach’s alpha statistic. Stability, based on test-retest, indicates the degree to which scores on the same instrument are consistent over time. To evaluate the reliability coefficient the scores of the pilot test were correlated. To achieve test-retest form reliability the researcher sought to achieve a coefficient of r = .80 or better ( Crocker & Algina, 1986 ). The reliability of the instrument was verified through a pilot test. As recommended by Borg and Gall (1989), the results of the pilot test were used in order to determine Cronbach’s alpha for inter-item reliability. For the purpose of this study a coefficient rate of r = .80 was deemed adequate to establish inter-item reliability. Preliminary analysis of the results revealed that Cronbach’s alpha had not reached the desired degree of r = .80. Three particular items were determined to be problematic and their “alpha if item removed” produced scores within the desired rating of r = .80. The exclusion of three items from the instrument (item 2, item 7 and item 16) produced a rating of r = .81. These items were not highly correlated within their intended construct and further examination revealed problems with the items which could potentially impact the reliability of score-based inferences.

The final instrument consisted of 43 closed-ended questions, using a four-point Likert-type scale response with a range of Strongly disagree=1, Disagree=2, Agree=3, Strongly agree= 4. Participants were not asked to put their name on the surveys, thus protecting confidentiality. At the time of the test, participants were notified of their rights related to human subjects’ research guidelines. Demographic information of the participants was collected at the beginning of the survey, only identifying the participant’s age (at last birthday), grade level, and respective mentor. The dependent variables were represented by data collected from the posttest survey that students completed after the mentorship program ceased. The survey scores were interpreted to represent students’ perceptions of engineering disciplines and self-efficacy in the areas of mathematics and science. The independent variable was set by participation or non-participation in the experimental treatment of the NCETE/NSBE mentorship program.

Instrument Details

Section one of the instrument collected the background information of the participants including, (a) grade level, (b) gender, (c) race, (d) highest level of formal education of participants' parents, and (e) GPA. This section of the instrument contained ten items.

Section two of the respective instrument pertained to participants of the NCETE/NSBE mentorship program. This section collected feedback on the participants’ experience in the mentorship program, the program’s characteristics, and activities encompassing the mentorship program. This section of the instrument contained twelve items addressing the participants’ mentorship experience. The control group, students not participating in the mentorship program, was asked to skip this particular section.

Section three of the instrument dealt with students’ perceptions and self-efficacy as it related to engineering. This portion of the survey asked students about their conceptual knowledge of engineering as a field and career. Students were also questioned on their confidence and self-belief to do design and other related tasks of an engineer. This section of the instrument contained seventeen items addressing the desired construct.

Section four of the instrument asked about students' confidence and self-belief to use math to solve technological problems and engineering problems. This section in the instrument contained eight items addressing the desired construct.

Section five of the instrument pertained to students' confidence and self-belief to use their understanding of science to solve technological and engineering problems. This section of the instrument contained nine items addressing the desired construct.

Procedure

Unique to this formal mentorship program was the career function which, notwithstanding the psychological support that mentors provide, focused the mentor relationship on influencing individual student’s perception of a particular field or career ( Allen & Day, 2002 ). A four-point protocol was developed as a general guide for the mentors to use in conducting their sessions. The four-point protocol included (a) a film presentation that was representative of some aspect of engineering as a field and/or profession, (b) a field experience selected by the mentor that offered the protégés some exposure to engineering as a field and/or profession, (c) a design challenge that was culturally relevant to the protégés and offered practical applications of science and mathematics principles, and (d) one-on-one counseling that offered the protégés psychological support in the way of a role model and/or counselor.

A recent review of “best practices” for mentorship programs revealed some overarching themes that framed the structure and facilitation of the mentorship program. Best practices for good mentorship described that good mentoring is determined by the selection of mentors, how mentors and protégés are assigned or matched to each other, how formal or informal the relationship should be, how mentors should be rewarded for the contribution, and where and when mentoring can be found ( Hargreaves, & Fullan, 2002 ). Other factors considered included a nonschool setting for mentoring activities, parent support, and structured activities. It is also recommended facilitation of the mentoring program should include supervision and provision of structured activities and mentors with a background in a helping role ( Dubois, Holloway, Valentine, & Cooper, 2002 ). To address these criteria, mentors were chosen from the NSBE organization; mentors were matched to mentees based on similar interests, future aspirations and availability. Although it was a formal mentorship program, the researcher was careful to incorporate practices of informal mentor relationships into the program. The mentorship program was not able to provide a nonschool setting due to the fact that mentorship was performed during regular school hours. However, parental support was achieved through parental consent and the researcher spoke personally with parents and guardians to answer questions and alleviate any concerns about the mentorship program.

In an effort to inculcate the four-point protocol and “best practices” into the mentors’ sessions, two separate dates were scheduled for mentor training as provided by the researcher. The two training sessions lasted one hour and encompassed delineating the roles, responsibilities, and duties of each mentor participating in the mentorship program. Potential mentors who were not able to be present at the first training session on October 29, 2007 were subsequently given an opportunity to complete training on November 20, 2007. Mentors participated in a presentation on current educational practice as it pertains to engineering education and the under-representation of minorities in STEM fields. Mentors were informed that the mentorship program was to address the following concerns; (a) lack of exposure at younger ages, (b) absence of role models, and (c) difference in learning styles. The mentorship program solicited the services of nine mentors to facilitate the program.

Prior to engaging in any activities with the students, mentors were asked to complete extensive training and background checks. In order to receive approval from the mentors’ respective university allowing the mentors to work with the students, mentors had to complete the Collaborative Institutional Training Initiative (CITI). The mentors were registered as social behavior researchers for the purpose of this study. Those who successfully completed CITI training visited with the principal at the alternative high school and were given background check forms to be completed. The respective high school conducted background checks on all potential mentors seeking to participate in the mentorship program. Institutional Review Board (IRB) approval was secured from The University of Georgia allowing the researcher to conduct research involving a vulnerable population. The researcher also had to secure IRB approval from the Guilford County School District allowing the mentors to work with the students.

Following completion of mentor training, CITI training, and successful background checks, five mentors were available to participate in the study. Four other potential mentors were not able to participate in the program due to either (a) failing to complete mentor training, (b) failure to complete CITI training, (c) unsatisfactory reports on their background checks, or (d) truancy. The five mentors selected to participate in this study were all students and were active members in NSBE. There were four male mentors and one female mentor. The mentor group was comprised of one graduate student, one senior, one junior, and two sophomores. The mentors’ ages ranged from 18 to 23 years of age. Two of the mentors majored in electrical engineering, one in chemical engineering, one mentor was a computer science major while another double majored in electrical engineering and chemical engineering. Based on data provided from a Student Information Form, mentors were assigned three students each from the randomly selected treatment group.

Mentors were responsible for securing a space where their sessions could appropriately be facilitated. Mentors provided the researcher with their availability schedule and this was forwarded to the principal and administrative assistant at the alternative high school involved in the study. Mentors were asked to sign-out students when working with the students for the session and the mentors were responsible for signing students back in at the end of the session. The mentors were allotted no more than an hour to conduct their mentorship sessions and were scheduled to meet students the second and fourth week of each month. The mentorship program was initiated in February and lasted through May.

At the conclusion of the mentoring program, a posttest survey was administered in the form of a pencil and paper written assessment, which the researcher distributed in person. All respective participants attending the alternative high school were instructed to complete the posttest survey with the researcher providing incentive to ensure full participation from the students. To maintain the reliability of the results the researcher asked that all students take the posttest exam in the same classroom and within three hours of the first administered exam. To ensure confidentiality, identification numbers rather than names were used to distinguish the mentored students from the non-mentored students. Using a binary system, random four digit numbers were provided at the top of the survey ending in either a one or zero. Students who were participants in the mentorship program where given surveys that ended in one and students who were not part of the survey were given surveys that ended in zero. Students were asked to identify their age and grade level in addition to the identification number that they were given.

Results

Descriptive Statistics

Out of the fifteen students selected to participate in the mentorship program only twelve students completed the program. One mentor reported that two of his participants transferred to other high schools during the program. Another mentor reported that one of his participants declined to finish the program after agreeing to participate. At the conclusion of the mentorship program, twelve students had participated in the treatment for this study. The fifteen students generated for the control group produced from the random sorting of the alternative high school students was reduced to the first twelve in the list in descending order to represent the control group. A total of twenty-four male students out of the eighty-three alternative high school students were randomly selected to participate in the study.

Twenty-four students participated in the study, however only twenty-one surveys yielded useable data. One student was considered an outlier due to the fact that his ethnicity was determined to be White or Caucasian. Another student did not complete the survey, bringing the total number to twenty-two. Upon further analysis, one participant's responses were deemed invalid and unreliable. The markings on the paper and pencil test clearly demonstrated that the participant did not complete the survey to the validity and reliability of the results. Throughout the survey the participant marked the first response on the Likert-scale even if this answer contradicted the previous one. The participant simply marked: Strongly disagree=1 for the entire survey, which in the eyes of the researcher was not indicative of answering the survey to the best of his knowledge. With twenty-one valid entries to compare, the researcher randomly eliminated one participant to ensure an even amount of participants for the control and experiment groups. Again, it is suggested that equal group size is required to account for mean variances among groups ( Weinberg & Goldberg, 1990 ). The total number of useable data resulted in twenty participants (N=20).

The treatment group consisted of ten (n=10) Black/African-American male students. The control group consisted of ten (n=10) Black/African-American male students as well. The grade level breakdown is provided in Table 1 below:

Data Analysis

Results of the posttest survey were represented by three separate univariate, single-scale data reports. The constructs being examined for each variable were distinct so the data analysis consisted of analyzing the dependent variables independent of each other. Conclusions were drawn based on these computations, and the researcher used a medium effect size set at 0.5, alpha level set at p=0.05, and a statistical power of 0.7. According to Olejnik (1984) effect size is the “specific minimal relationship or minimal difference in populations means that the investigator believes would be important to detect a practical perspective.” In studies that require a hypothesis testing of sample means, Cohen suggested differences of .2 (small), .5 (medium), and .8 (large) standard deviation (Olejnik, 1984). Due to the relatively small and unique population that the sample was derived from, a medium effect size was deemed appropriate. A sample size of twenty-seven students was needed to achieve a statistical power of .7. However due to the loss of participants, which reduced the total number of participants to twenty (N=20), a post-hoc analysis revealed a final power analysis of .56.

For the purpose of this study, independent t-tests were used to determine whether differences between group means were statistically significant. In determining significance, the t-test makes adjustments for the fact that the distribution scores for small samples become increasingly different from the normal distribution as sample size becomes smaller ( Gay & Airasian, 2003 ). T-tests strategy entails comparing the actual mean difference observed with the difference expected by chance. It reports very little else about the nature of that relationship, however it does reveal whether a significant difference exists between groups.

Inferential Statistics

The first research question sought to determine if there was a significant difference in participants’ conceptual perceptions of engineering for students who participated in the NCETE/NSBE mentorship program when compared with non-mentored students. An independent sample t-test was used to compare the means for responses on items related to this question and determine whether they were statistically significant. For perceptions of engineering, the mean score for the treatment group equaled M= 40.30 and for the control group it was M= 38.40. Standard deviations were SD= 5.72 for the control group and SD= 3.95 for the experimental group. Although the experimental group produced a higher raw mean score than the control group, these results were not statistically significant at an alpha level of .05, t (18, .05) =.399.

The second research question sought to determine if there was a significant difference in self-efficacy in the area of mathematics for students who participated in the NCETE/NSBE mentorship program when compared with non-mentored students. Using the same analysis techniques as described above, results were provided for participants’ self-efficacy in the area of mathematics as it related to engineering. For self-efficacy in mathematics the control group yielded a mean score of M= 23.30 and the treatment group had a mean of M= 22.60. The standard deviation for responses on self-efficacy in mathematics was SD= 3.75 for the control group and SD=3.62 for the treatment group. Though there is a slight difference in the mean scores of the control and treatment group these results failed to reach significance, t (18, .05) = .676.

Research question three sought to determine if there was a significant difference in self-efficacy in the area of science for students who participated in the NCETE/NSBE mentorship program when compared with non-mentored students . In a comparison of mean scores for students' self-efficacy in science as it related to engineering, an independent sample t (18, .05)=220. The treatment group produced a mean score of M=25.80. The standard deviation for each group equaled SD=4.12 and SD=3.96 respectively.

Conclusions

The research findings pertaining to research question one did not produce a significant difference for students’ perceptions of engineering. Analyses of the exit interviews conducted with the mentors helped provide answers to many questions that arose regarding the mentorship experience. It was evident that more time may be needed in order to significantly impact students’ perceptions and self-efficacy. The relatively short duration of the program and time allotted for each mentoring session appeared to have been inadequate and greatly impacted the ability of the mentorship program to affect change. This result was consistent with the work of Garet et al. (2001) and their recommendation that at least 100 hours were required for reform activities to have an effect.

Findings from the research pertaining to research question two did not detect a difference in group mean scores that reached a level of significance. Upon further investigation into exit interview comments, in addition to time constraints, the lack of set activities posed a problem for the mentors and participants alike. The four-point protocol called for mentors and participants to develop challenges that were deemed “culturally relevant.” However, this strategy backfired for many mentors because of some participants’ reticence to become more involved in the learning process. The time lost and uncertainty of activities may have contributed to the lack of significant difference found between groups.

Research question three sought to identify if there was a significant difference in self-efficacy in the area of science for students who participated in the NCETE/NSBE mentorship program when compared with non-mentored students. The study did not reveal a significant difference in group mean scores for findings pertaining to research question three. As identified earlier, issues of time constraints and the lack of set activities may have contributed to not finding significant differences on this indicator.

Implications for the Field

Findings from this study provided several implications specifically for African-American males with regard to engineering and other related technical fields. It raised questions about activities designed to diversify technical fields, specifically engineering, and could inform organizations looking to implement formal mentorship programs as a way to impact perceptions and self-efficacy of students. The NCETE/NSBE mentorship program was unique in its structure, facilitation, and unprecedented in the field. The mentorship program developed, including data collection instruments, provides a basis for further research on mentoring and its potential to impact underrepresented populations. The mentorship program developed was unique in that it had a career function and a psychological function. While the implementation in this study did not produce significant differences in results, the techniques used and the mentoring procedures could be modified to address areas identified as problematic and additional data collected to determine impact.

Additional findings answered some questions regarding the ineffectiveness of the mentorship program and could be used to inform modifications prior to future research. The qualitative interviews conducted with the mentors provided insight into some of the barriers that likely prevented significant differences. The first area identified where changes should be considered is that of duration of the mentoring experience. In assessing the structural features of the mentorship program, the researcher relied on best practices for "reform" activities. Almost all literature on mentoring and professional development calls for programs that are sustained over time ( Garet et al, 2001 ; Penuel, Barry, Ryoko, & Lawrence, 2007 ). Practical constraints limited the amount of time available for the treatment in the study, but a longer mentoring experience should be examined to determine potential impact on student perceptions and self-efficacy.

Issues that took away from the mentoring time included lack of involvement by the alternative high school staff, and difficulties with gathering the students together in a timely fashion. The omission of set activities also had major implications for this study. The time involved to create culturally relevant activities with the students may have affected the overall impact of the mentoring sessions. Feedback from study participants suggested that providing the mentors with set activities that they could embellish on, would have had a positive impact on the overall mentoring experience. This is consistent with literature on best practices that recommended structured activities be provided to mentees ( Hargreave, & Fullan, 2002 ).

As a researcher it is important to examine all variables that may impact the results of a study. In relation to this research study, the disproportionate amount of upperclassmen in the control group may help explain the lack of statistical significance. Furthermore, the precision of the instrument used in this study must come under scrutiny, When trying to measure sensitive constructs such as perceptions and self-efficacy it is important to ensure that the instrument used is measuring what it is intended to measure. Further examination of the instrument may be in order to ensure its reliability and that the score-based inferences made from the data collected are valid. It is also worthy to note the small sample size for this study. Of the eighty-three students attending the alternative high school, this study of which only 20 provided useable data. Although the sample represented twenty-four percent of the population, sample sizes this small are hard to generalize or make inferences to a larger population or determine differences that are statistically significant.

Future research in this area should allow more time for the mentorship program to properly develop. It was expressed several times by the mentors involved and validated by the research that the three months allotted for this study was inadequate to produce real change. Mentors also suggested extending the time for each session. These two factors are critical to the success of the mentorship program and future research should seek to make needed adjustments in these areas. Furthermore, a similar study should provide further analysis regarding between group differences and within group differences. The final results revealed a disproportionate amount of upperclassmen in the control group, which potentially could have implications for total group mean score. Chi-squared analyses could be utilized to discern if students’ grade levels have any correlation with students’ perceptions and self efficacy. Multiple-regression is another statistical approach that could be utilized to provide further analysis of the results. This procedure could be utilized to determine if the completion or lack thereof of each point on the protocol has any impact on the outcomes. This would help reveal if a particular point in the protocol is effective or ineffective. If procedures were repeated with larger control and treatment groups, these types of analyses would be feasible.

The most vital contribution of this research was the formal mentorship model developed including techniques for training mentors, identifying mentor requirements, and developing and testing measurement instruments to evaluate mentoring outcomes. This study was instrumental in providing an example which could serve as a model for the evaluation of formal mentorship programs to positively influence perceptions and self-efficacy of students. Although the survey failed to reveal in mean score that was statistically significant, the study made inroads by establishing a model for comparing the self-efficacy of students participating in a formal mentorship program against those not participating. This data is pertinent to the implications of this research study and those wishing to examine the impact of mentorship programs. The qualitative data provided by the mentors allowed recommendations to be formulated for future research.

Cameron D. Denson is a Postdoctoral Research Associate at the National Center for Engineering and Technology Education at Utah State University. He can be reached at cameron.denson@aggiemail.usu.edu.

Roger B. Hill is a Professor and Department Head of the Workforce Education, Leadership and Social Foundations at the University of Georgia. He can be reached at rbhill@uga.edu.

References