Retaining At-Risk Students in School Using a Cognitive-based Instructional System: Teachers' Reflection in Action
University of Manitoba
Christian A. Chinien
University of Manitoba
Too many young Canadians are dropping out of school without the critical skills necessary to obtain good jobs being generated by a super-symbolic and brain-based economy which is characterized by ever-increasing, accessible, and manipulable sources of information (Human Resources & Labour Canada, 1993). These young people are being marginalised by economic dislocation. According to Human Resources and Labour Canada (1994) there were approximately 420,000 young people in any given month who were unemployed and actively looking for work in 1993. That's an unemployment rate of almost 18% for those 15 to 24 years of age. In 1992, youth made up 16.6% of Unemployment Insurance recipients and received approximately $2.5 billion in benefits. Canadian youth also received more than $2 billion in social assistance in 1992. Youth currently represents 21% of all social assistance cases. Further analysis of youth unemployment data indicates that single women with children represent almost half of the youths not in school or the labor force. Poverty among young Canadians remains a serious social problem as over half of single people under the age of 25 lived in low-income circumstances in 1992. While 420,000 young Canadians are unemployed, there are approximately 600,000 jobs that are not filled because employers cannot find employees with adequate skills.
Boutin and Chinien are both Professors, University of Manitoba, Canada and share the authorship of this article equally.
The Canadian Youth Foundation (1995) indicated that youth unemployment is a hidden deficit that has become a chronic condition in Canada and that structural unemployment has had a major impact on Canadian youth. The Organization for Economic Cooperation and Development (1994) defines structural unemployment as pressures on economies to adapt to change and their inability to do so. Owen (1995) noted that "the changing labor market is hardest on those at the bottom of the skills distribution hierarchy and that good jobs for 'strong backs and weak minds' are becoming scarce" (p. 7). According to Toffler and Toffler (1995) the only effective strategy for reducing joblessness in a super-symbolic economy must depend less on the allocation of wealth and more on the allocation of knowledge.
The Steering Group on Prosperity (1992) notes that the alarmingly high dropout rate makes Canadians wonder whether we are being adequately prepared "for a world in which we must rely more on our brains than on brawn to innovate and compete" (p. 35). Letteri (1992) argued that students must become symbol analysts in order to cope with the emerging super-symbolic economy. He described a symbol analyst as a person having the ability to solve, identify, and break problems by manipulating symbols (p. 69).
This research project focused on the empowerment of students who were at risk of dropping out of school because they could not cope with the information processing demand imposed upon them by school learning. They were empowered to succeed in school learning through cognitive skills augmentation and transfer training.
Background and Conceptual Framework
Research indicates that students drop out of school for a variety of reasons (Hargroves, 1986). In this project the primary interest was in students who were at risk of dropping out of school due to problems in coping with the information processing demands prevalent in typical school situations. Individual differences among students regarding their preference for various modes of gaining, storing, processing, and using information constitute souroes of considerable variation in learning (Witkin, Moore, Goodenough, & Cox, 1977). Some students are unable to accomplish tasks simply because they lack the necessary information processing skills (Regan, Back, Stansell, Ausburn, Ausburn, Butter, Huckabay, & Burkett, 1979).
Cognitive-based research over the last 15 years has demonstrated that one of the most important factors contributing to achievement differences is the profile of cognitive skills that a student brings to academic tasks (Letteri, 1992). Letteri further argued that in order to succeed a student, "must possess a repertoire of thinking skills that meet the cognitive demands of learning and performance tasks. Without appropriate cognitive skills students can never be self-directed and independent in academic tasks" (p. 59). Cognitive-based research has validated the use of the Cognitive Profile Assessment Instrument (CPAI) to measure students' cognitive skills and to identify those who are at risk of failure in school learning (Letteri, 1992). The CPAI assembled by Letteri consists of seven bipolar measures of basic cognitive skills. These include:
Analytical/Global: marks a tendency of a student to either experience items as part of a background (global) or to overcome the influence of an embedded context and view items as separate from the background (analytic).
Focus/Non-Focus: describes an individual's extent and intensity of attention-deployment to a given task.
Reflective/Impulsive: marks the degree of consistency in the speed and accuracy with which an individual selects hypotheses and processes information relative to a comparative analysis task.
Narrow /Broad: marks a student's consistent preference for the degree of inclusiveness in establishing the acceptable and appropriate range of specific category parameters.
Complex/Simple: describesindividual differences in the variety of highly organized, distinct, and highly specific categories by which information is structured in memory, as well as the ability to use this information to examine new information from a variety of relevant perspectives.
Sharpener/Leveller: describes reliable individual variations in the assimilation of information in memory.
Tolerant/Intolerant: represents the skills required to engage and examine apparently ambiguous information for the purpose of modifying existing structures of information and of accommodating new information within these related structures.
These seven cognitive skills dimensions have been found to determine end predict with high accuracy (p< .05 or better) students' level of success in academic learning and performance tasks. Each dimension can be assessed separately and the results used to generate the student cognitive profile. The CPAI sorts the student population into three large categories (Type I, II and III) which are called Cognitive Profile Types (Letteri, 1985) as described below:
Type I Profile students show evidence of strength in a majority (four or more) of the seven cognitive skills. They are typically in the top 15-18% of the population in academic achievement (Letteri, 1992).
Type II Profile students do not demonstrate particular strengths or weaknesses in the controls included in the Cognitive Profile. They tend to be highly inconsistent and are usually average (mediocre) in academic achievement. Type Two students comprise 60-70% of the population (Letteri, 1992).
Type III Profile students demonstrate a major deficit (4 or more) in terms of the cognitive skills as indicated by their Cognitive Profiles. These students typically present severe learning problems and are several grade levels below their placement grade in all areas of standardized testing. These students usually have long histories of failure and, as research indicates, no amount of assistance has been able to rectify the situation. Type III students represent 15-18% of the school population (Letteri, 1992).
The learning problems of Type III students, as well as those of Type II students, do not tend to be related to the difficulty or complexity of the subject matter. Rather, these problems relate to their lack of specific cognitive skills required by their assigned academic learning and performance tasks (Chinien, 1990; Letteri, 1988). The Cognitive Based Instructional System (CBIS) can be used to help Type II and Type III students enhance their cognitive skills through augmentation and transfer training. Research conducted by Letteri (1985) indicates that placing Type II and III students in a Cognitive Augmentation program for a minimum of 20 hours is likely to have significant and long-lasting impact on grade level achievement. Although the cognitive empowerment of these students through the CBIS is manifested in improved academic performance, enhanced sense of competence, improved self images, and more positive attitudes toward school (Letteri, 1980, 1985), little is known regarding its potential contribution in preventing at-risk students from dropping out of school. Consequently, the following framing question provided the guiding theoretical orientation for the study: Can the CBIS assist at-risk students to stay in school by helping them to enhance their cognitive skills, self-esteem, academic performance, and attitudes toward school?
Methodology and Procedures
The target population for this study was junior high school students who have been identified as potential dropouts. Four junior high school sites were included in this demonstration project. In consultation with the school principals, a purposive sample of eight teachers (two from each site) and all the students in one of each teacher's intact classes were selected using the following criteria: (a) grade level, (b) significant number of at-risk students, (c) teacher willingness to participate, and (d) native language (English or French). Purposive sampling was used to ensure the participation of a sufficient number of at-risk students in the project and to meet the requirements of the funding agency. Three resource teachers from three different sites were included in the project at the request of their school principals.
These project activities spanned the entire school year. All participating teachers received seven full days of training in the Cognitive Based Instructional System (CBIS). Teachers were briefed on project goals, objectives, procedures, and anticipated outcomes. They were introduced to CBIS and trained to administer the CPAI and to analyse the results. They were also trained to use some basic augmentation and transfer strategies to modify students' cognitive profiles using the cognitive apprenticeship model (Collins, Brown & Newman, 1987). The CBIS teachers also received on-site individualized training and coaching while they were working on the augmentation strategies with their students.
The CPAI was used to assess the cognitive profile of all 175 students. The CPAI was administered to students on an individual basis. The average administration time is approximately one hour per student. The CPAI consists of seven different sections, each designed to measure one of the seven cognitive skills.
A specifically assembled battery of standardized tests was also administered to all students as a pretest. The following tests were included in this battery: (a) the Canadian Test of Basic Skills - form 7 and 8 from Nelson Canada, (b) the Culture Free Self-Esteem test from Special Child Publications, (c) the School Attitude Measure from American Testronics, and (d) the Dropout Prediction Scale from The Ohio State University.
Students were classified as Type I, II, or III according to their performance on the CPAI. All Type III and extreme Type II students (N=45) were selected for the CBIS cognitive augmentation and transfer training. The 45 students selected for the CBIS training were assigned to the classroom teachers who were involved in the project. The average teacher-student ratio for the CBIS training was 1:4. Twenty-seven students were selected to form a non-equivalent control group. The same battery of tests was also administered to all treatment and control group students as posttests.
Augmentation training: The treatment was administered to students on a one-to-one basis. Each student received an average of 20 hours of CBIS training during the school year. The teachers used CBIS augmentation strategies to modify the cognitive profile oftheir Type II and III students. The objective was to enable Type II and III students to perform as Type I on cognitive tasks. Students were given training and practice in the skills of monitoring, directing and controlling their information processing system sequentially for each of the seven cognitive skills (analytical/global, focus/non-focus, reflective/impulsive, narrow/broad, complex/simple, sharpener/leveller, and tolerant/intolerant) using a workbook designed by Letteri. During the augmentation session, teachers used a think aloud technique to monitor students' ability to manage and manipulate information, analyze content, identify problems, and determine what was required for successful problem solving or task completion. The teachers also modelled each of the cognitive skills and coached the students to use these skills effectively.
Transfer training: Once the students acquired a degree of comfort with specific cognitive skills these guiding principles were used to assist them in transferring the newly acquired cognitive skill to a variety of subject matter content. This was achieved by showing the students the relationship between the augmentation exercises and the cognitive skill requirement for academic tasks. Students also were coached to apply the augmented cognitive skills to complete academic tasks. Authentic materials such as home work and other assignments were used to increase the meaningfulness and effectiveness of the transfer process.
Quantitative results showed that 38 percent of the CBIS students experienced important improvement in the overall cognitive profile type. Analysis of the overall cognitive profile change revealed a 20 percent movement from Type II to I, 2 percent from Type III to I and 16 percent from Type III to II. Additionally, there were also important gains on the seven cognitive controls, as many students moved from Type III to I, Type II to I and Type III to II. No significant differences were observed from pretest to posttest on mean scores in math, reading comprehension, attitudes toward school, and self-esteem. A follow-up conducted two-and-one-half years after the initial project implementation indicated that the great majority of the students were performing well academically and that only 9.9 percent had dropped out. A complete discussion of the quantitative results ofthe study has already been previously reported in the JITE (see Chinien, Boutin & Letteri, Volume 34, No. 4, 1997).
The purpose of this article is to report the results generated from data collected from the teachers' journals. All CBIS teachers were requested to keep a reflective journal to document their daily actions and thoughts about these actions as they work with the CBIS and students. Suggestions provided by Connelly and Clandinin (1988) for journal writing were given to the CBIS teachers as guidelines in order to ensure some uniformity in journal entries:
Write as much as you can in your journal. Be descriptive of actions, children, events, and reactions that you have both while teaching and after you have finished teaching. Records the feelings you have about your practices and the various events that happened. Be alert to past experiences that come to mind as you react emotionally and morally Write regularly. Make an entry every few days or more frequently if you have time or if something particular occurs to you It is helpful to reread your journal entries from time to time and to try and make sense of the kind of things that are important to you. (pp. 36-37)
Each CBIS teacher was given a notebook for making their journal entries. Eight of the twelve CBIS teachers handed in their journal for analysis. Debriefing interviews indicated that four CBIS teachers were simply too busy to make their journal entries. There were some variations regarding the volume of information generated by the teachers. While some teachers provided long narrative descriptions of actions and thoughts, others were more precise and to the point.
The qualitative data analysis was structured using the framework of constant comparative analysis (Lincoln & Guba, 1985). All journal entries were first coded and categorized independently by a team of three researchers. The team then met as a group to review the three independent classifications using the operational guidelines of Lincoln and Guba (1985). The three independent classifications were analysed to ensure agreements in establishing categories. This process helped to build consensus in categorizing the data, review of the categories, and examination of the category sets for relationships. Finally, the categorization and interpretation of the data were taken to the CBIS teachers for confirmation using the member check strategy. Four major themes emerged from this analysis: (1) Teachers' Expectations, (2) Addressing Students' Needs, (3) Changes in Teachers' Beliefs and Practices, and (4) Conditions for the Successful Implementation of CBIS Program. Given that Teachers' Expectations is only a small category, it was combined with Addressing Students' Needs for reporting purposes. The analyses of leachers' journal entries for each of these themes are presented in this section. In presenting the results, journal entries helped to exemplify various critical issues and concerns and contributed to provide vivid and thick descriptions of events and teachers' constructions are cited verbatim.
Addressing Students Needs
Only three of the teachers expressed initial feelings about the CBIS project and their comments reflect high hope and great expectations. On a very personal level one teacher felt: "completely revitalized and at the same time drained and I find this pilot almost all consuming in my thoughts." This teacher strongly believed in the potential of the CBIS program: "I really think this program could open doors for the kids." The high expectation from the CBIS intervention is exemplified in this teacher's comments: "When the lights go on in (the students') eyes because they suddenly made sense out of something that was previously nonsense it makes all the effort worthwhile. I am convinced, and pray with all my heart that stats prove that this program can work, and maybe give kids especially tike B and J - the resources to succeed in school."
As the CBIS teachers worked with their students, some began to see things "through a different window." They seemed to become more aware of the discrepancy of students' background knowledge and the prerequisite knowledge necessary to understand directions and to perform academic tasks. Teachers' comments indicated that CBIS has helped them to appreciate the need to perform tasks analyses in order to identify the cognitive requirements of the tasks. This awareness is reflected in comments such as these.
I am struck by how much assumption there is in teaching, how global we are in our content, and how little time or consideration is given to the student and what we are asking them to do.
Upon now realizing the variation in students' ability to analyse and the importance of this cognitive skill in learning, I find myself rethinking how I issue notes and model example questions. I realize the need for specificity and clear illustration.
This seems to be hard to explain unless you've done it, but it really helps to see things: (a) from a student's perspective and (b) not to take for granted that students understand everything they read - even though they may say they do.
In acknowledging the importance of developing and enhancing students' learning skills a teacher wrote:
Students experiencing difficulty in MathlScience tend to lack reflectiveness. I understand the truth of this statement to a much fuller extent now. Also, I believe, we as teachers often engineer out the reflectiveness that students may possess. In Mathematics the repetitive approach used to develop and reinforce computation skills is often void of the need for reflectiveness If this in fact is the case then computational should frequently be presented using problem solving as a vehicle.
Teachers also realized that vocabulary used in providing directions may represent a severe stumbling block for students as demonstrated by the following example.
I asked P to explain the first question. P did not accurately understand the vocabulary and therefore obviously would not be able to do the question. The first question began with the word 'calculate'. P thought that means 'add'. We talked about mathematical relationships and eventually we brought P around to understand that calculate really means figure out.
Another teacher discovered that a student didn't know the word "partial".
We drew a diagram of the words and discovered relationship between them. I asked him to now draw the diagram from memory (which he couldn't do before). He drew it perfectly. The definitions made sense to him as well.
These types of gross misconceptions were only uncovered through the think-aloud processes used during the augmentation and transfer training sessions.
Results revealed that some teachers learned to adapt their teaching to the individual needs of their students. When teachers focussed on the students' perspectives and analysed where the discrepancies exist between students' needs and the instruction, they were able to make the necessary adjustments. This change is evidenced by this journal entry.
They got 5 out of 6 problems wrong the first time through. I pretty well had to lead them through it. Basically the activity was awful. I feel frustrated! I tried a pictorial approach with M and C. What a difference! The boys got 5 out of 6 correct! They also seemed to enjoy the activity when illustrated with pictures.
Changes in Teachers' Beliefs and Practices
Some teachers reported definite changes in beliefs that would have tremendous impact on their teaching. In commenting on the implications of CBIS for teaching a teacher noted.
This week in Math Club I had students explain their solutions to problems to the rest of the group. I emphasized to them to articulate their thought processes I was reminded at this time that very often thinking is a hidden tool Also I was amazed at how I was able to follow their moving from one cognitive skill to another as they explained their solutions. I plan to use this approach much more in class for modelling purposes and my own diagnostic purposes.
As teachers began to look at the CBIS as a foundation for information processing, they were eager to adopt it as a framework for guiding them to make complex decisions about teaching and learning.
I will never be able to teach the same way again never before have I felt so sure that some of the decisions I am making are sound and valid, and with all my educational training I was never, before this time in my career, able to say that with as much confidence.
The profound impact of the CBIS on teaching is also demonstrated by attempts made by some teachers to integrate CBIS principles in their regular teaching duties. The comment which follows describes a teacher's experience.
In my regular classes I have begun implementing idea trees (networks). I constructed the framework for the idea tree and then had students read a unit, then fill in the empty cells. Two days later, without having asked the students to study the tree or refer back to it I informally quizzed them on reconstructing it. It was significant, the amount of detail put back into the tree, and their ease for recalling this information was impressive.
The majority of the CBIS teachers indicated that they will never be able to revert back to their previous beliefs about teaching and learning. This shift in belief is probably due to the gratification derived from the CBIS: "This was the first time I've felt that I helped out my group. Great Feeling!".
Comments made by teachers also indicated that when students were made aware of the benefits of using a particular mental process, a positive impact on student willingness to work at developing these skills was witnessed. The most revealing comments included the following:
simply making students aware of the benefits of sub vocalizing is an initial step in helping them become aware and develop this skill.
making students aware of the benefits of representing information externally in working memory has a positive impact on their willingness to show and present their work.
A is becoming more engaged in the sessions asking questions as to why we are doing certain exercises. We discussed the importance of focussing, of being aware of and discriminating between relevant and irrelevant data.
Conditions for the Successful Implementation of CBIS Program
Analysis of leachers' journal entries revealed five key ingredients that seem to be necessary for the successful implementation of the CBIS: (1) belief and commitment, (2) hard work, (3) support of colleagues, (4) flexibility, and (5) openness to changes in beliefs about teaching. First, a strong belief that CBIS was worthwhile and that it would present new learning opportunities for themselves and their students appeared to be a necessary condition. This belief formed the basis for the commitment required to spend a significant amount of mental energy and carry through a great amount of work. Second, a lot of hard work had to be done. Teachers had to prepare for the one-to-one sessions with students by reading books and articles, attending Saturday workshops, and meeting the requirements for a regular teaching job and family life. Time was a big factor for those teachers who had many other pressures to deal with. For some there wasn't enough time for the program; for others there wasn't enough time for all the other things. Lack of time was also the reason given by the four teachers who did not complete their journal entries. These teachers also indicated that previous commitment in extra curricular activities did not allow suffficient time to complete the cognitive augmentation and transfer training in all seven cognitive controls. Third, support, especially within the school seemed to be closely linked to a teacher's ability to clarify ideas and generate opportunity to articulate ideas and work together with colleagues on methods of implementation. Having the backing of an interested staff seemed to have the most positive outcome. Fourth, successful teachers seemed to be flexible within the structure of the program. For example, the teacher who worked with a student who was overwhelmed by the "specific content approach" found an alternative in a "pictorial approach." Instead of persisting in a futile direction, she found a solution that works. Lastly, something that very quickly puts teachers on the defensive is the threatening prospect that one's own teaching holds inadequacies. Those teachers who expressed a change in belief seemed to be prepared to attribute students' failure on current teaching practices and were eager to make changes. These five ingredients were present to a larger degree in journals of teachers who indicated a feeling of success in the CBIS project.
Results of this study showed that some teachers felt that they were not empowered by their professional preparation to face the challenges of teaching students with cognitive skills deficit. A teacher believed that quite often teachers engineer out reflectiveness from learning. Johnson (1992) indicated that schools are not helping students develop these important skills because schools are still too entrenched in the behavioural psychology foundations. The CBIS appears to provide teachers with a framework to make critical and complex decisions regarding teaching and learning. In their own terms teachers felt that the CBIS training empowered them to "engineer" instruction. This finding has important implications for industrial teachers given their mandate in assisting students in developing essential employability skills, such as critical thinking, decision making, and problem solving. Industrial teachers should make a deliberate effort to engineer reflectiveness and other higher order skills in their teaching and learning.
Results indicated that CBIS teachers" reflection in action helped them to change their beliefs about teaching and learning. That shift in belief had an immediate and long-lasting impact on classroom practice, as some teachers indicated that they will never be able to teach the same way again. The CBIS methodology not only helped teachers to enhance learning, it also enabled them to identify and correct teaching deficiencies. This finding supports the need for helping teachers to become reflective practitioners.
As the CBIS teachers understood the principles of cognitive-based teaching and learning, they began to see themselves less as subject matter specialists whose primary objective is to teach content and more like educators who are responsible for assisting students in developing learning how to learn skills. They also saw a need to design instructions that will facilitate learning and help students to develop higher order skills. Given that "learning how to learn" is fast becoming a workplace basic skill, more emphasis should be placed on cognitive-based learning. The CBIS teachers were able to identify gross misconceptions in information processing due to the help of the "think aloud process." This finding points to the need for integrating principles of cognitive psychology in the teaching and learning process. Teacher education institutions should ensure that future industrial teachers are introduced to cognitive-based learning therapy. Additionally, learning how to learn should form an integral part of undergraduate teacher education programs. This cognitive empowerment may assist teachers perform learning skills enhancement intervention.
Technological and socio-technical changes have dramatically altered job content and skill requirements at the workplace. Toffler and Tofffler (1995) argued that in a knowledge intensive economy the proletariat is being replaced by a cognitariat where work is increasingly shifting away from manual tasks toward symbolic processing. The concept of the "mind workers" or knowledge workers has emerged from this transformation of work. A study conducted in the province of Nova Scotia (Nova Scotia Department of Finance, 1993) indicated that knowledge workers outnumbered production workers 3 to 1. There is a growing concern that the bulk of the population (75% in the case of Nova Scotia) will be marginalized, will never become knowledge workers, and will have to rely on other talents for their livelihood. CBIS could provide a useful framework for industrial educators to enable this segment of the population to acquire the critical workplace readiness skills for this knowledge intensive economy.
The Canadian national high school dropout rate is 18 percent. Given that the population for this study consisted of at-risk students, the anticipated dropout rate was significantly higher than the national average. However, a follow-up conducted two-and-one-half years after the initial project implementation indicated that the great majority of the students were performing well academically and that only 9.9 percent had dropped out. According to Toffler and Toffler (1995) the allocation of knowledge is the most effective way to reduce poverty and joblesness in a brain-based economy. Teacher journal entries indicated that CBIS was instrumental in empowering students to succeed in learning. CBIS may, therefore, provide industrial educators with a tool for retaining at-risk students in school and at the same time with a weapon to fight youth unemployment and underemployment.
Canadian Youth Foundation (1995). Youth unemployment: Canada's hidden deficit. Ottawa: Ontario. Canadian Youth Foundation.
Chinien, C., Boutin, F., & Letteri, C. (1997). Empowering at-risk students to stay in school using a cognitive-based instructional system. Journal of Industrial Teacher Education, 34 (4), 43-64.
Collins, A., Brown, J. S., & Newman, S. E. (1987). Cognitive apprenticeship: Teaching the craft of reading. writing and mathematics (Technical Report No. 403). Cambridge, MA: Bolt, University of Illinois at Urbana, Champaign, Center for the Study of Reading. (ERIC Document Reproduction Service No. ED 284 181).
Connelly, F. M., & Clandinin, D. J, (1988). Teachers as curriculum planners: Narrative of experience. Tororto: OISE Press, Ontario Institute for Studies in Education.
Hargroves, J. S. (1986). The Boston compact: A community response to school dropouts. Urban Review, 18 (3), 207-217.
Human Resources and Labour Canada. (1993). Leaving school: Results from a national survey comparing school leaversand high school graduates 18 to 20 vears of age. Ottawa-Hull, Ontario: Public Enquiry Centre, Employment and Immigration Canada.
Human Resources and Labour Canada. (1994). Social security in Canada: Background facts. Ottawa-Hull, Ontario: Author.
Letteri, C. A. (1980). Cognitive profile: Basic determinant of academic achievement. Journal of Education Research, 7 (4), 195-199.
Letteri, C. A. (1985). Teaching students how to learn. Theory Into Practice, 14 (2), 112-122.
Letteri, C. A. (1988). The NASSP learning style profile and cognitive processing. In J. W. Keef (Ed.), Profiling and utilizing learning style. Reston, VA: NASSP.
Letteri, C. A. (1992). Diagnosing and augmenting basic cognitive skills. In I. W. Keef & H. J. Walberg (Eds.), Teaching for thinking (pp. 59-71). Reston, VA.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills: Sage.
Organization of Economic Cooperation and Development. (1994). The OECD jobs study: Facts. analysis, strategies. Paris: OECD.
Owen, J. D. (1995). Why kids don't study. Baltimore: John Hopkins University Press.
Regan, T. J., Back, K. T., Stansell, V., Ausburn, L. J., Ausburn, F. B., Butter, P. A., Huckabay, K., & Burkett, J. R. (1979). Cognitive styles: A review of literature. Interim Report, Brooks, TX: Lory Airforce Base, Colorado. (ERIC Document Reproduction Service No. ED 174 655).
Steering Group on Prosperity. (1992). Inventing our future: An action plan for Canada's prosperity. Ottawa, Ontario: Prosperity Secretariat.
Toffler, A., & Toffler, H. (1995). Creating a new civilization. Atlanta: Turner.
Witkin, H. A., Moore, C. A., Goodenough, D. R. & Cox, P. W. (1977). Field-dependent and field-independent cognitive styles and their educational implications. Review of Educational Research. 47 (1), 1-64.