A Critical Meta-Analysis of Mobile Learning Research in Higher Education

ABSTRACT

This paper is a critical meta-analysis of mobile learning research based on qualitative descriptions of meta-analyses of m-learning research studies published between 2009 and 2013. The study covered in this paper looks at the conceptual frameworks and theories underpinning mobile learning research studies, the global experiences of using mobile digital devices for learning, and the factors enhancing or hindering the acceptance and use of mobile digital devices for learning in higher education. The study concludes that without reference to theoretical and pedagogical issues, including the design of m-learning, studies of m-learning will not necessarily further our understanding of how m-learning can contribute to successful learning outcomes globally.

Keywords: Mobile learning, meta-analysis, mobile devices in learning

CONTEXT OF STUDY

Mobile devices are “popular everyday tools and services that are also potential or de facto resources for education” ( Kukulska-Hulme, 2012 , p. 247). The increased availability of personal mobile devices is taking education and learning to a new level. It has been predicted that mobile devices will be the primary connection tool to the internet for most people in the world in 2020 ( Anderson & Rainie, 2008 ). Mobile learning (m-learning) can be defined as examining how mobile technology can be used for educational purposes such as “the process of using a mobile device to access and study learning materials [and] to communicate with fellow students, instructors, or institution” ( Ally, 2009 ). Mobility, informality, and personal ownership are all characteristics which make learning with mobile devices different from other forms of e-learning ( Naismith, Lonsdale, Vavoula, & Sharples, 2004 ). Mobile devices allow e-learning to be delivered virtually anywhere at any time ( Rouse, 2005 ). Mobile learning has been identified as a new stage of distance and e-learning ( Georgiev, Georgieva, & Smrikarov, 2004 ). However, m-learning is more than using mobile technologies to access information and a potential solution to global demands for more access to education, m-learning represents a challenge to conventional education practices. Some of these challenges are explored in this critical metaanalysis of mobile learning research on the acceptance, readiness, and use of mobile digital devices for learning in higher education.

According to the United Nations Educational Scientific and Cultural Organization (UNESCO) there are currently over six billion mobile phone subscriptions worldwide, and for every one person who accesses the internet from a computer s/he does so from a mobile device. The emergence of the digital native generation, that is children who grow up using and relating to modern ICTs, provides a strong motive for research of learning with mobile devices in order to understand and achieve the potential educational benefits of m-learning. Although the focus of this paper is m-learning in higher education contexts, m-learning also has implications for employers and industries as an educational resource and training in other than educational contexts ( Mungania 2003 ).

This study attempted to discover what benefit m-learning brings to education and to identify what questions remain unanswered by research in the field of m-learning. This study also examines current findings concerning social attitudes towards m-learning (acceptance), individual and institutional preparedness for pedagogical uses of mobile devices (readiness), and the impact of mobile learning on students’ educational success (outcome). This review is guided by the following questions:

Research Questions

• What are the conceptual frameworks and theories underpinning mobile learning research studies?
• What have been the global experiences of using mobile digital devices for learning in the context of higher education?
• What factors are indicated by research as enhancing or hindering the acceptance and use of mobile digital devices for learning?

RATIONALE FOR A CRITICAL META-ANALYSIS

Gilliam and Zigler (2001) used the term ‘critical meta-analysis’ to describe their review of available impact evaluations from 13 state-funded preschools in the United States from 1977 to 1998. The concept of meta-analysis originates in the fields of psychiatry and medicine where it is “a standardized approach for examining the existing literature on a specific, possibly controversial, issue to determine whether a conclusion can be reached regarding the effect of a treatment or exposure” ( Russo, 2007 , p. 637). In the field of education “instead of students or participants being the unit of analysis, the primary studies themselves become the unit of analysis” (Denson & Seltzer, 2011, p. 216). The term critical is used in many different educational contexts. Critical thinking has been defined as:

[A] way of deciding whether a claim is true, partially true, or false. Critical thinking is a process that leads to skills that can be learned, mastered and used. Critical thinking is a tool by which one can come about reasoned conclusions based on a reasoned process. This process incorporates passion and creativity, but guides it with discipline, practicality and common sense. [ http://en.wikipedia.org/wiki/Critical_thinking ].

In the context of this study, the term critical is used to identify trends in recent m-learning research and generate recommendations for future researchers in the field of m-learning.

The majority of meta-analyses used statistical techniques to combine the findings from independent studies into a single finding, in order to provide greater validity and reliability for those findings by producing an ‘effect size’ of the activity studied. Variation between studies “provides an opportunity to ask additional questions, and to investigate more closely the reasons for the observed differences in effect size across studies” ( Denson & Seltzer, 2011 , p. 216). However, because a statistical review does not always provide the degree of insight provided by interpretive approaches, some researchers argue for more qualitative descriptions. Qualitative descriptive studies have as their goal “a comprehensive summary of events in the everyday terms of those events” ( Sandelowski, 2004 , p. 334). This is the approach taken by Coursaris and Kim (2011) in their meta-analysis of empirical mobile usability studies. This study combines the basic meta-analysis procedure of systematic review with qualitative descriptions such as that taken by Coursaris and Kim (2011) . This study also incorporates elements of the constant comparative method which compares recently collected data with data that was collected in earlier studies. This helps to establish the trustworthiness of a study ( Lincoln & Guba, 1985 ). Lincoln and Guba (1985) argue that the trustworthiness of any research study is important to evaluating its worth. They identify four standards by which trustworthiness may be established:

• Credibility which establishes confidence in the ‘truth’ of the findings.
• Transferability which shows that the findings have applicability in other contexts.
• Dependability which shows that the findings are consistent and could be repeated by other researchers.
• Confirmability which indicates the extent to which the findings are shaped by the respondents and not researcher bias, motivation, or interest.

These four standards are used to guide this research. This study also aims to follow the frequently stated principle that the goal of a review is to “summarize the accumulated state of knowledge concerning the relation(s) of interest and to highlight important issues that research has left unresolved” ( Creswell, 1994 , p. 22).

SEARCH PROCESS

A combination of keyword and snowball searching was conducted to identify relevant literature and studies. Five different databases: ERIC, Google Scholar, ProQuest, A+Education and PsycINFO were searched for articles in peer reviewed journals using the keywords: ‘mobile learning’ and ‘higher education.’ In addition to peer reviewed journals a search of conference proceedings, papers and reports was conducted using the same keywords. The bibliographies of relevant articles and reports were also used to identify relevant material. Additional search terms: impact, acceptance, readiness, outcome, and achievement, were added to the original keywords to refine the search and filter the results. Theory and model were added as supplemental search terms part way through the search process. The search was limited to material published in English after 2005.

INCLUSION AND EXCLUSION CRITERIA

In addition to excluding studies published before 2005* and articles that were not based on original research, studies which could be identified primarily as product evaluations were excluded. Studies that examined m-learning in other than higher educational contexts, or only in K-12 educational settings were also excluded. An exception to this was made when the study included m-learning in higher educational settings. For example, although the meta-analysis of m-learning research trends by Wu, Wu, Chen, Kao, Lin & Huang (2012) includes studies of mobile devices used in K-12 and formal and informal education settings; the majority (95) of studies were concerned with formal education in higher education institutions. This criterion was also applied to the meta-analysis study of the use of audio podcasts by Hew (2009) . Initially Hew’s (2009) study was considered to be excluded because of the finding that the majority of the study participants used home based desktop computers rather than mobile devices to listen to audio podcasts. However, because of the current trend of replacement of static desktop computers with laptops and other mobile devices and the focus of the study being on the use of podcasts as an educational aid, Hew’s study was considered relevant to be included. Mbarek and El Gharbi’s (2013) meta-analysis is included because it covers the period from establishment of e-learning as an educational tool in higher education to the introduction of m-learning.

Of the eight meta-analysis studies identified in the search process (see Table 1) and covered in this paper, three focused on research trends in m-learning studies. Another also looked at the research methodologies used. The remaining studies focused on some aspect of the acceptance, readiness or outcome of m-learning (see Table 2). Close and repeated readings of the identified studies were conducted to produce the qualitative descriptions of these meta-analyses as presented in the following sections.

Table 1. Meta-Analyses of m-Learning

Author(s)/Year	Title
Cheung & Hew * (2009)	A review of research methodologies used in studies on mobile handheld devices in K-12 and higher education settings
Hew (2009)	Use of audio podcast in K-12 and higher education: a review of research topics and methodologies
Coursaris & Kim (2011)	A meta analytical review of empirical mobile usability studies
Hwang & Tsai (2011)	Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010
Pollara & Broussard (2011)	Student perceptions of mobile learning: a review of current research
Hung & Zhang (2012)	Examining mobile learning trends 2003–2008: a categorical meta-trend analysis using text mining techniques
Wu, Wu, Chen, Kao, Lin & Huang (2012)	Review of trends from mobile learning studies: a meta-analysis
Mbarek & El-Gharbi (2013)	A meta-analysis of e-learning effectiveness antecedents

Note. ^* In addition to being the co-author of the meta-analysis on the research methodologies, Hew is the author of the review of research topics and methodologies in studies of the use of audio podcast in K-12 and higher education ( Hew, 2009 ).
^* It should be noted that although all of the meta-analysis of m-learning are published after 2009 they include studies carried out before 2005.

CODING PROCEDURE

The next stage of the research process was the coding of the selected studies. Once a study had met the inclusion criteria, it was assigned a unique code based on the author initial and dates of publication, then a brief description of the study and the study findings were entered into a table. The participant numbers, gender and underlying theory or models were identified when supplied by the original researchers. Additional alphabetical coding was then conducted using a letter or letters to identify the main subject of the study: student (S), teacher (T) or institution (I); the focus of the study, based on the previously identified search terms: acceptance (A), readiness (R) and outcome (O); the type of research: survey (SU), experimental (EX) or action research (AR); and, the research methods: quantitative (QN), qualitative (QL) and mixed method (MX). The initial coding took place before it was decided to focus on meta-analysis of m-learning studies and was applied to the meta-analyses. The coded data was then transferred to an Excel spreadsheet and Excel was used to filter and sort the studies according to the assigned codes.

DATA ANALYSIS METHOD

A combination of manual counting and computer assisted tabulation using Excel filters was used to analyse the data. Because the emphasis of this study is on qualitative description rather than statistical analysis, numerical counts and not percentage figures are used in the meta-analysis of m-learning reviews. Both percentage figures and numerical counts are used in the original meta-analysis m-learning studies conducted between 2005 and 2013.

LIMITATIONS OF THIS STUDY

It is recognised that this study is necessarily limited in scope and there are aspects of m-learning that are not covered in depth. Further, although a comprehensive search of databases was conducted, because of the rapid spread of m-learning and the growing ongoing nature of research it is not possible to say that this study includes all relevant studies. In addition, it should be recognized that this review is based only on published reports that may present an incomplete picture of the actual research done and this could be subject to misinterpretation by the readers.

OVERVIEW OF META-ANALYSES STUDIED

As noted earlier, four of the meta-analyses identified in the search process focused on research trends or research methodologies while the remaining meta-analyses focus on some aspects of the acceptance, readiness, or outcomes of m-learning (see Table 2). Most of the meta-analyses examined used some form of aggregation to establish a degree of statistical credibility for their findings. The exception is the meta-analysis conducted by Coursair and Kim (2011) , which includes quantitative information from the studies examined but places more importance on the interpretation of those findings.

Table 2. Focus of meta–analysis studies

Focus of the study	Count	Author(s)/Year
Research trends	3	Hwang & Tsai (2011) ; Hung & Zhang (2012) ; Wu, Wu, Chen, Kao, Lin & Huang, (2012)
Research methodologies	1	Cheung & Hew (2009)
Acceptance (perception)	1	Pollara & Broussard (2011)
Readiness (usability)	2	Coursaris & Kim (2011) ; Hew, (2009)
Outcome	1	Mbarek & El-Gharbi (2013)

RESEARCH TRENDS AND METHODOLOGIES

As noted in Table 2, three of the meta-analyses ( Hwang & Tsai, 2011 ; Hung & Zhang, 2012 ; Wu, Wu, Chen, Kao, Lin & Huang, 2012 ) examine research trends in m-learning. These are considered by date of publication. It should be noted that the studies of Hwang and Tsai (2011) and Hung and Zhang (2012) were conducted concurrently and used the same database (SSCI). Although there is a small difference in the time periods covered, there are many similarities in their findings. The fourth meta-analysis considered in this section ( Cheung & Hew, 2009 ), which reviews the research methodologies used in studies of mobile handheld devices in K-12 and higher education settings also considers many of the same studies.

Research Trends in Mobile and Ubiquitous Learning from 2001 to 2010 ( Hwang & Tsai, 2011 )
Hwang and Tsai (2011) used the Social Science Citation Index (SSCI) database to identify items concerning mobile and ubiquitous learning published from 2001 to 2010. Hwang and Tsai specified the research sample group, the number of articles published, the contributing countries, and the research learning domains. They found that the number of papers published in the second 5 years of their study period was nearly four times (122) that of the first 5 years (32). The majority of these studies were concerned with student users of mobile devices for learning. Though a greater number of studies in the second period dealt with the experiences of teachers and working adults, student users from higher education and elementary schools remained the major samples of mobile and ubiquitous learning research. Few of the studies in the earlier period identified any specific learning domain and were “mainly focused on the investigation of motivations, perceptions and attitudes of students toward mobile and ubiquitous learning” (p.67). This contrasts with studies in the second period (2006- 2010) which are more specific about the learning domains in which m-learning is implemented. Science, Languages, Arts, and Social Science are the main learning domains for studies of m-learning with a relatively few studies being carried out in Mathematics. Hwang and Tsai (2011) recognized the need for more longitudinal studies, but consider the availability of the large number of existing studies to be “good references for educators and researchers” (p. 69).

An interesting finding of Hwang and Tsai (2011) is the broadening distribution of countries contributing to studies in mobile learning. The majority of studies conducted in the first five years of the research period originated from the United States and the United Kingdom, followed by contributions from Taiwan. However, in the following five-year period there were increased contributions from other countries and Taiwan became the dominant contributor. The dominance of Taiwan is not explained.

Mobile Learning Trends 2003–2008 ( Hung & Zhang, 2012 )
Hung & Zhang (2012) used text mining techniques to conduct their meta-analysis of mobile learning trends between 2003 and 2008. They used the Web of Science database which includes the Science Citation Index (SCI) and the Social Science Citation Index (SSCI) to identify relevant studies using ‘mobile learning’ and ‘m-learning’ as their primary search terms. A total of 115 usable articles were retrieved. Hung and Zhang acknowledged that their decision to limit their study to the SCI/SSCI database and reliance on abstracts to identify relevant articles has limitations regarding the selection of material and the information provided – nevertheless it still provides a good overview of m-learning research trends. Documents were grouped based on the similarity of abstracts using the following four categories:

• Strategies and frameworks (22)
• Acceptance and issues (18)
• Effectiveness, evaluation and personalized systems (50)
• M-learning case studies (25)

Each category is divided into related topics, also referred to as clusters, which are then used as the basis of their analysis -- these include trends in frequency of each topic over time, the predominance in each topic by country, and preferences for each topic by journal. The top three research clusters that were identified were: m-learning tool development, effectiveness of m-learning and content protection, transmission, and management. Hung and Zhang identify two major growth phases of m-learning publications: 2003-2005 and 2007-2008. Studies of the effectiveness of m-learning contributed to the growth in both phases. In the second phase, studies concerning the acceptance of m-learning, adaptive evaluation or intelligent tutoring systems on mobile devices, m-learning tool development and m-learning projects in engineering education, language learning, and music education all contributed to the growth of m-learning publications. Research into collaborative m-learning was the only topic to decrease in publication. There were no significant fluctuations in the quantity of publications for topics such as strategies or frameworks for m-learning, interactivity of m-learning, acceptance of m-learning, personalized m-learning systems, m-learning in K-12 and other environments and issues such as content protection, transmission, and management throughout the research period. Hung & Zhang (2012) identified the period from 2003 to 2007 as the ‘innovators stage’ of m-learning studies” with the ‘early adopters stage’ beginning after 2007.

Hung & Zhang (2012) also included analysis of contributions of m-learning studies by the different countries. They found that both the USA and Taiwan were dominant in publications on interactivity of m-learning. Researchers in the United States were also concerned with instructional aspects of m-learning including collaborative m-learning and m-learning in K-12 environments. Taiwan dominated in 5 out of the 12 clusters: strategies or frameworks for m-learning, acceptance of m-learning, adaptive evaluation or intelligent tutoring systems on mobile devices, personalized m-learning systems and m-learning tool development. The dominance of Taiwan as a publisher of m-learning research was explained by pointing to the Taiwan’s government’s “strong and aggressive e-learning initiatives” and “financial, managerial and legislative support to promote e-learning development.” It was suggested that China’s lack of clear m-learning focus, despite the large quantity base of m-learning research “might be attributed to the lack of a strong mobile technology infrastructure there” (p. 13). No countries were dominant in the topics of effectiveness of m-learning, m-learning applications in training or m-learning projects in engineering education, language learning, and music education, although Turkey was the highest publisher of research concerning m-learning projects in engineering education, language learning, and music education.

Trends from Mobile Learning Studies ( Wu, Wu, Chen, Kao, Lin & Huang, 2012 )
Wu et al. (2012) analysed 164 studies from 2003 to 2010. Although they used Hwang and Tsai’s (2011) and Hung and Zhang’s (2012) meta-analyses as the starting points for their study, they however suggested that the two previous literature reviews “failed to examine or categorize research trends from the standpoint of research purposes, methodologies, and outcomes.” Further, the earlier studies “failed to examine or analyse the mobile devices from the standpoint of teaching- and learning-assistance, and their critical role in ubiquitous learning.” These authors did not consider that “the development and usage patterns of technology are changing quickly, requiring an up-to-date analysis of trends in mobile device types and functionality, along with learner types and the use of mobile devices in various disciplines and courses” (p.817-818). To overcome these failings Wu et al. included the purposes of the m-learning studies, the methodologies used and outcomes of m-learning use in their meta-analysis. From this they identified seven major findings (see Table 3).

Wu et al. note that it is particularly significant that most studies of m-learning focused on effectiveness and identify this as a significant new finding.

Table 3 Trends in mobile learning research ( Wu, Wu, Chen, Kao, Lin & Huang, 2012 )

1	The research purpose of most mobile learning studies focused on effectiveness, followed by mobile learning system design.
2	Surveys and experimental methods were the preferred research methods, regardless of whether the research purpose focused on evaluation or design.
3	Research outcomes in mobile learning studies were significantly positive.
4	Mobile phones and PDAs were the most commonly used devices for mobile learning, but these may be replaced in the future by new emerging technologies.
5	Mobile learning is most prevalent at higher education institutions, followed by elementary schools.
6	Mobile learning most frequently supports students in the professions and applied sciences, followed by the humanities and formal sciences
7	The most highly cited articles fell into the categories of mobile learning system design followed by effectiveness.

Wu et al. attribute the more frequent citing of studies concerning mobile learning system design (7) to their earlier dates of publication because they have been available to researchers for a longer period of time. The finding that most m-learning studies feature positive outcomes (3) is an important one when it is known that m-learning is increasingly being adopted. It is no surprise that m-learning learning most frequently supports students in the professions and applied sciences (6) although Wu et al. suggest that m-learning can be applied to any course or subject matter.

Research Methodologies used in Studies on Mobile Handheld Devices ( Cheung & Hew, 2009 )
Cheung and Hew (2009) used the ‘snowball’ method to search for journal articles concerning the use of mobile handheld devices in K-12 and higher education settings published before December 2008, After excluding 92 articles of a total of 136 from their study because they were opinion papers, conceptual articles or non-empirical descriptions, Cheung and Hew examined 44 articles to identify how mobile handheld devices such as PDAs, palmtops, and mobile phones are used by students and teachers. They also look at the types of research carried out, the data collection methods used and the research focus of the different studies.

Cheung and Hew (2009) categorized the research methods used to study mobile device use into eight types: descriptive research, true experiment, experiment (weak), quasi-experiment, ex-post facto, single-subject, design-based research, and mixed method. They found that overwhelmingly the most used method was descriptive research (65.9%), the next most common research method used was weak experimental method (11.4%) where a single group is measured both before and after device use and there is no comparison or control group. Ex-post facto studies (where mobile device use had occurred prior to the study), single-subject and design-based research were the least used (one study each). One third (31.4%) of all data collection methods used questionnaire, 22.5% used test or quiz items, 20.6% used content analysis, 18.6% used interview or focus group, and 6.9% used observation. Cheung and Hew (2009) suggested that questionnaires and test or quiz items are the most frequently used methods of data collection because they can collect data from a large sample and are relatively easy to use.

Cheung and Hew (2009) identified seven major categories of use for mobile devices in education: (a) multimedia access tool, (b) communication tool, (c) capture tool, (d) representational tool, (e) analytical tool, (f) assessment tool, and (g) task managing tool. They noted that the first five categories originated from a framework put forward by Churchill and Churchill (2007) from a case study of a teacher who explored “the educational affordances of PDA technology” over a period of six months (p.1439). The remaining two uses of mobile devices as assessment tools and task managing tools emerged inductively from the data.

The four main research topics found were usage profile (26.3%), the viability of mobile devices as an assessment tool (7%), learning outcomes after using mobile devices (17.5%), and user attitudes (49.1%). The studies indicated that mobile handheld devices are most commonly used by students and teachers as communication and multimedia access tools. Cheung and Hew (2009) attributed this to the convenience and portability of handheld devices. However, they suggest that the dominant use of mobile devices as multimedia access tools indicated that mobile devices function primarily as replacements for other means of multimedia access rather than functioning in uniquely transformative ways. Acceptance of mobile devices is linked closely to physical features such as portability and ease of use while resistance is linked to unfamiliarity with devices as well as physical limitations such as limited battery life, limited memory, small screen size and difficulties in making inputs using the stylus or phone keypads. Limited Internet browsing through server capabilities or formats on small screens were also considered barriers.

Cheung and Hew (2009) found that although the research regarding learner outcomes appears to suggest that students’ learning is enhanced through the use of mobile handheld devices, they note that this finding should be viewed with caution because of the lack of rigorous research designs used in these studies. They also advise that before universities and schools invest in mobile devices “it may be more crucial to investigate the cost effectiveness of using a mobile handheld device” for learning (p.167). In the few studies concerning the use of mobile devices as an assessment tool, there was no significant difference in terms of test scores achieved by students using the PDA and paper and pencil based assessment methods.

Acceptance, Readiness and Outcomes of m-Learning
The second category of meta-analyses was that which covered studies concerned with the acceptance, readiness, and outcomes of m-learning ( Coursaris & Kim, 2011 ; Hew, 2009 ; Pollara & Broussard, 2011 ; Mbarek & El Gharbi, 2013 ) . Although the specific focus of each study varied from the use of a specific m-learning application such as Hew’s (2009) review of the use of audio podcast in educational contexts to more general studies of usability such as Coursaris and Kim’s (2011) meta-analysis, they all dealt with aspects related to the acceptance, readiness and outcomes of m-learning.

The use of audio podcast in educational contexts ( Hew, 2009 )
Hew (2009) reviewed 30 studies concerning the use of audio podcast in K-12 and higher education settings. The majority of the studies reviewed were descriptive. Although K-12 education settings are included, most podcasts were used in higher education and traditional course settings. Using a constant comparative method, Hew identified three major research categories: how participants use podcasts including barriers to using podcasts; the cognitive and affective outcomes of podcast use; and, the institutional aspects of podcast use, specifically: the impact of podcast on learner attendance in class/lectures and the costs of producing podcast.

The study found that although podcasts may be played on a variety of mobile devices in the majority of studies “students both in the traditional and distance education settings tend to listen to the podcasts at home using desktop computers, rather than on the move” (p. 348). There was also a preference for shorter (5-10 minutes) podcasts that longer (over ten minutes). Students generally enjoy using podcasts — however they did not always consider them to be relevant to their learning. The availability of podcasts did not appear to encourage students to skip classes. Barriers to using podcasts could be categorized as either student-encountered or instructor-encountered. The barriers to student use of podcasts were unfamiliarity with podcasts, technical problems in accessing and downloading podcasts, and students not seeing the relevance of podcasts for their learning. Instructor-encountered barriers also included unfamiliarity with podcasts and not seeing the relevance of podcasts for their subject areas. Another barrier was lack of time to prepare podcasts. The review found that the most common pedagogical use of podcasting was limited to instructors distributing podcast recordings of lectures or supplementary materials to enable students to review subject material at their own time and place.

To help mitigate the possible novelty effects of using audio podcasts, Hew suggested that future research needs to be carried out over a longer period of time, to examine the impact of using podcast on students’ learning and affective domains.

Student Perceptions of Mobile Learning: a Review of Current Research ( Pollara & Broussard, 2011 )
Pollara and Broussard (2011) reviewed 18 reports published between 2005 and 2010 of studies of learning with mobile devices in a variety of different contexts. They focused specifically on student perceptions of mobile learning, although they also considered the types of technology used, the learning tasks involved, the kinds of interactions technology was used to support, the outcomes measured, and the design and methodology of the study -- excluding studies that did not provide adequate information about research design or methodology. Their review was limited to experimental and non-experimental studies which examined mobile devices that were personally owned and used by students, such as PDAs, mobile phones, and mp3 players and were used for either formal or informal learning or as part of practical work experience or practicum. Although case studies were excluded as “too restrictive” (p. 1645) because they were case specific, no restriction was placed on sample size.

Six (6) of the studies were experimental and twelve (12) were survey driven. Fourteen (14) involved the use of mobile phones or PDAs and two studies concerned mp3 players. No device is specified in the remaining two studies. The two studies involving mp3s specifically were concerned with podcasts for educational purposes. Pollara and Broussard (2011) noted that technological convergence means that many mobile devices may now be used for multiple purposes. Learning tasks varied between studies with some studies looking at more than one task. These tasks were identified as the following way: tasks facilitating the individual learning of content (5), group projects/discussion (6), assessment (6), and teacher-directed lecture through the use of the m-learning technology tool (2). The most commonly used types of interactions with mobile device were between student and content (14) followed by interactions between student and instructor (11) and interactions amongst students (8). All of the studies except for one reported overall positive results for student perceptions of m-learning. The remaining study reported mixed results with respect to student attitudes on m-learning.

In the positive reports Pollara and Broussard (2011) found that:

– Three (3) studies reported that m-learning generated a strong interest amongst the students
– Eight (8) studies reported a strong, positive reaction to integrating m-learning into the classroom
– Three (3) studies reported that learners found that learning with mobile devices was enjoyable
– Five (5) studies reported that students recognized the potential of m-learning
– In four (4) studies “participants found that using mobile devices was convenient and enabled learning to be flexible and portable because of the portability and perceived convenience associated with mobile applications and tools” (p.1646)
– Three (3) studies found that students reported competence and ease in using the devices and performing the learning tasks.

Student participants with previous experiences of mobile devices were more likely to encounter fewer problems with m-learning. Students already aware of m-learning in two (2) studies reported little or no change in their perceptions, either positively or negatively of m-learning. Expense is the only deterrent to m-learning identified in one study. Pollara and Broussard (2011) note that only four studies looked at both student attitudes to m-learning and student achievement gains from m-learning; thirteen (13) studies focused exclusively on student attitude to m-learning and one study was designed specifically to focus on student achievement gains. Pollara and Broussard (2011) also noted that researchers did not appear to be specifically interested in how a particular m-learning strategy can influence achievement because their concern was whether participants are amenable to the use of m-learning as an educational tool.

A review of mobile usability studies ( Coursaris & Kim, 2011 )
Coursaris and Kim’s (2011) carried out a qualitative review of more than 100 studies of user perceptions of mobile learning devices and studies of using mobile devices for learning, published between 2000 and 2010. To analyze the finding of the studies they developed a framework of contextual usability for mobile computing based on key usability dimensions, contextual factors and consequences. Usability is defined as the degree “that people can employ a particular technology artifact with relative ease in order to achieve a particular goal within a specified context of use” (p. 118). Key usability dimensions include effectiveness, efficiency, satisfaction, errors, attitude, learnability, accessibility, operability, accuracy, acceptability, flexibility, memorability, ease of use, usefulness, utility, and playfulness. Four contextual factors affecting usability were identified as the following: user, technology, task/ activity, and environment. Task characteristics were identified as open and closed tasks. Consequences refer to the result or end purpose of using the mobile device, for example, improving systems integration, increasing adoption, retention, loyalty, and trust. The study was carried out in two phases; as a result Coursaris and Kim were able to compare studies carried out before 2006 with studies conducted after this time. Coursaris and Kim (2011) found that overall:

– empirical mobile usability studies focused on investigating task characteristics (47%), followed by technology (46%), environment (14%), and user characteristics (14%). Distribution exceeds 100% as multiple areas may have been studied in a single study. This contrasts with the distribution of research emphasis in the earlier phase which showed research on task (56%), user (26%), technology (22%), and environmental characteristics (7%).

There is a lack of empirical research on the relevance of user characteristics and the impact of the environment on mobile usability. Coursaris and Kim (2011) pointed out that “because on-screen keyboards are now a standard of smartphone technology, it would be important to understand the optimal design of on-screen smartphone/ mobile device keyboards according to target user groups and their characteristics” (p.122). Additional findings of the Coursaris and Kim (2011) meta-analysis are that:

– Open and unstructured tasks, interactivity and complexity are understudied.
– User characteristics: A narrow focus on studied user dimensions is prevalent
– Technology characteristics: Enabling technology beyond the interface is overlooked in mobile studies
– Environmental characteristics: Area with greatest potential for future mobile usability research

Coursaris and Kim (2011) found “efficiency, errors, ease of use, effectiveness, satisfaction, and learnability are most commonly measured in empirical mobile usability studies” (p. 128). After reviewing the frequency with which the different measures appeared in the reviewed literature, Coursaris and Kim (2011) identified three core constructs for the measurement of usability of m-learning. These are:

– Efficiency: Degree to which the product is enabling the tasks to be performed in a quick, effective, and economical manner, or is hindering performance.
– Effectiveness: Accuracy and completeness with which specified users achieved specified goals in a particular environment.
– Satisfaction: The degree to which a product is giving contentment or making the user satisfied. (p. 128)

Coursaris and Kim (2011) pointed out that these findings could easily be predicted and don’t introduce anything new to the field, which they suggest raises questions about the continued use of these measures for usability studies on the implementation of m-learning in the future.

Antecedent of e-learning effectiveness ( Mbarek & El Gharbi, 2013 )
Mbarek and El Gharbi (2013) reviewed 60 research reports of studies concerning employees or students in learning programmes designed to prepare them to reproduce and generalize knowledge and skills for class or job tasks. Included is research that reported gain scores, learning achievement, and training performance between 1984 to 2009. This period covers the establishment of e-learning as an educational tool in workplaces and formal educational contexts and the introduction of m-learning. Mbarek and El Gharbi (2013) examined the variables identified by researchers as contributing to or limiting positive learning outcomes and learning transfer using the concept of nomological networks.

Trochim, (2006) explained the nomological network as a means of linking the conceptual/theoretical realm with the observable one by identifying the concepts of interest in any study and the interrelations among and between them.

Mbarek and El Gharbi (2013) presented their findings: they first identifyied the focus of m-learning studies regarding factors contributing to the affectiveness of e-learning (see Table 6). These variables are divided into trainee characteristics: motivation to learn, self-efficacy, and anxiety. And, contextual characteristics: feedback (positive and negative), training method, learning delivery.

Table 4. Focus of m-learning studies (from Mbarek & El Gharbi 2013 )

Factors contributing to e–learning effectiveness	Number of studies
motivation to learn	22
self–efficacy	28
anxiety	6
learning delivery	21
training method	10
feedback	6
learning performance	4

Based on their meta-analysis Mbarek and El Gharbi concluded that motivation to learn has a moderate relationship with learning outcomes; self-efficacy has a small relationship with learning outcomes. Anxiety has a significant relationship with learning outcomes. Training method has an important relationship with learning outcomes. Learning outcomes are positively related to learning transfer. Learning delivery has a small relationship with learning outcomes. Feedback has a strong relationship with learning outcomes.

Table 5. Impact of variables on learning outcomes and learning transfer (from Mbarek & El Gharbi 2013 )

Small	Medium	Large
self–efficacy	motivation	Anxiety
learning delviery		training method
		Feedback

Mbarek and El Gharbi (2013) recognized that some of studies in their review were based on small sample sizes. Further, their review focuses on the analysis of the variables that have been directly correlated to learning performance -- other possible moderators such as self-efficacy and learning outcomes are not examined. Mbarek and El Gharbi concluded that despite efforts to gain a greater understanding of the factors which lead to e-learning effectiveness, researchers have not reached a consensus on the interrelations among and between these factors or their impact on learning outcomes.

CONCEPTUAL FRAMEWORKS AND THEORIES

This study shows that a number of authors have made efforts to identify conceptual frameworks and theories specifically appropriate to the use of mobile devices for learning (for example, Sharples, 2000 ; Sharples et al., 2005 ; Berking et al., 2012 ). Although there is reference to these efforts in the met a-analyses and studies reviewed, there is little evidence that theory plays a significant role. The exception to this is the use of variations of TAM and other models related to predictions of technology acceptance. The design of m-learning is identified as an important aspect of m-learning in some of the studies – however, there is little examination of how design or lack of it affects learner outcomes. There is some evidence that the design of m-learning is becoming more important ( Wu et al., 2012 ). Wu et al. (2012) suggested that the increase in the number of studies related to m-learning design is related to rapid development of potential m-learning technologies combined with the willingness of researchers to trial those new technologies in developing mobile learning systems. But in general there is a noticeable lack of emphasis on pedagogical issues. Cheung and Hew (2009) found that the majority of the studies “tended to place greater emphasis on the features of the mobile devices and procedures for using them, rather than on the theoretical rationale or justification for using them” (p.166). Hwang and Tsai (2011) noted that there is greater attention to learning domains made in the later period of their meta-analysis (2006-2010) but do not give any indication of the extent to which studies focus on the pedagogical role of m-learning or learning outcomes using mobile devices. Like Cheung and Hew (2009) , Hwang and Tsai (2011) also recommend that future researchers consider this aspect of m-learning more deeply.

It should be noted that Wu et al. (2012) identified the evaluation of the effectiveness of m-learning to be emerging as a dominant area of research. This can be attributed to the increased use of mobile devices in education and concerns about the effectiveness of m-learning. A second interesting point concerns the preferred research methods used by m-learning researchers. Wu et al. found that regardless of whether the research purpose focused on evaluation or design, the preferred research method was usually surveys. Wu et al. do not provide any rationale for why this is so, nor do they examine potential weaknesses of surveys as a research method. Pedagogical preparedness by teachers and institutions appears to be only covered incidentally in m-learning research. While this may be attributed in part to a lack of research in the area, it may also be attributed to an underlying lack of attention to pedagogical concerns. This is a serious oversight. Mishra and Koehler (2009) pointed out that “if you’re not going to change pedagogy, then technology uses make no significant difference.” To be effective, m-learning must be supported by good pedagogic practices.

GLOBAL EXPERIENCES

Overwhelmingly the experience of using mobile digital devices for learning is presented as a positive experience regardless of the application or type of mobile device. Much of the research about m-learning concerns the impact of using mobile technology and the effect of using mobile devices on learners’ motivation to learn. It has been suggested that it is the feelings of ownership associated with the use of mobile devices and the informality of many m-learning applications which help motivate students by engaging them in activities that they like and give them a sense of control over their learning ( Pollara & Kee Broussard, 2011 ). The feelings of ownership and motivation to learn with mobile devices help promote good habits of learning and are the reason that m-learning is considered to contribute to developing life-long learning skills. This may also explain why acceptance studies dominate m-learning research. It should also be noted that although there is evidence of an increase in studies of teachers and working adults’ experience of m-learning ( Hwang & Tsai, 2011 ) students remain the major subject of mobile learning research studies. Neither gender nor age appear to be significant factors in the acceptance, readiness to use or experience of m-learning ( Uzunboylu, Cavus, & Ercag, 2008 ; Wang, Wu, & Wang, 2009 ) although there is evidence that familiarity with technology is an important factor.

In their review of research trends Hung & Zhang (2012) note that they could not find any longitudinal studies focused on m-learning and suggest this is due to the relatively short history of using m-learning. The lack of research concerning learning outcomes may also be attributed to the relatively short history of m-learning; however, it does not explain the lack of attention to institutional and teacher acceptance and readiness or how learning with mobile devices may be successfully integrated into pedagogic practice. Pollara and Broussard (2011) observed that researchers do not appear to be specifically interested in how a particular m-learning strategy can influence achievement and this reveals two important things about m-learning research to date. First, because the focus of researchers is on whether participants are agreeable to the use of m-learning as an educational tool, they fail to investigate how and in what ways m-learning can contribute to positive learning outcomes. However, acceptance of the possibility of learning through the use of mobile devices does not indicate individual and institutional readiness for m-learning or evaluate the impact of teaching and learning using mobile devices.

Hung & Zhang (2012) present a more detailed analysis of research trends in m-learning than Hwang and Tsai (2011) . Through their text mining techniques and identification of topics, Hung and Zhang show how the concerns of researchers have changed from 2003 to 2008. They link this directly to developments in m-learning technology and suggest that this both “created new possibilities for research” (2012, p.10) and contributed to the frequency of m-learning articles in journals. The increased contribution from researchers from countries other than the United States and the United Kingdom ( Hwang & Tsai, 2011 ; Hung & Zhang, 2012 ) undoubtedly reflects the growth of m-learning globally.

From this global research we can expect to learn more about the impact of m-learning in different cultures. As noted earlier the role of culture in education is frequently controversial, however it is impossible to avoid this factor when looking at educational issues globally. Cheung and Hew (2009) identified a relationship between culture and m-learning as an area for further study but do not themselves address culture in relation to m-learning. However, there is some evidence that culture influences attitudes toward m-learning. In their discussion of the factors influencing the adoption of e-learning at University of Bahrain, Al-Ammari and Hamad (2008) included culture as a significant factor. Based on the results of a study that asked participants from various cultural backgrounds to perform a number of e-learning tasks, Adeoye and Wentling (2007) found that suitable awareness of cultural diversities and the effects this has on the individual user is vital to the success of e-learning systems.

RELEVANT FACTORS

Abachi and Muhammad (2013) noted that although a growing number of academics accept that today’s mobile devices are tomorrow’s textbooks, there are still issues related to the use of a mobile device for learning that must be resolved before m-learning achieves its fullest potential. As discussed previously, the most frequently quoted factors affecting the successful implementation of m-learning in educational contexts are: technology, accessibility, affordability, acceptance, readiness and support. To this list, the understanding of m-learning can be added. This study suggests that the limited knowledge and understanding of m-learning by teachers represents significant barriers to the successful implementation of m-learning. Trifonova, Georgieva, and Ronchetti’s (2006) study revealed that there can be a lack of understanding of m-learning even when students are accustomed to using mobile devices. Hew’s (2009) review of the use of audio podcast in K-12 and higher education settings found that a lack of familiarity with podcasting technology limited how both teachers and students used podcasts.

The successful implementation of m-learning in educational contexts may be affected by teachers’ attitudes toward learning with mobile device -- the ways teachers create opportunities for learning with ICT are “highly dependent” on the “pedagogical orientation” that teachers adopt ( Law, Pelgrum, & Plomp, 2008 , p. 275). Research suggests that for m-learning to be used effectively both students and faculty must be ready and open to the potential benefits of a change in the teaching and learning environment.

RECOMMENDATIONS FOR FUTURE RESEARCH

A recurring claim by researchers in any field is that more research is necessary. A review of the literature related to m-learning shows that there is already a large body of research about m-learning, but the majority of studies continue to focus on the attitudes and perceptions of the users rather than the impact of using mobile devices for learning or the problems of integrating m-learning with other pedagogic practices. The imbalance of the research focus of m-learning studies and the use of weak experimental methods and concentration on self-reported data identified by Cheung and Hew (2009) support the argument that there has not been enough research around learning outcomes related to m-learning or the implications of m-learning for assessment. To gain a true picture of the impact of m-learning in higher education more longitudinal observational studies should be conducted.

One of the difficulties experienced by Mbarek and El Gharbi (2013) in their meta-analysis of m-learning research was the wide variety of variables used by researchers to measure the effectiveness of learning outcomes. This suggests that more effort is required by researchers to be consistent in their approach to evaluations of m-learning. Although there are weaknesses to be found in the analysis of m-learning research trends of Hung & Zhang (2012) and Hwang and Tsai (2011) , together they form a strong foundation for future analyses of research trends in m-learning particularly when combined with the insights of Wu et al. (2012) . These studies offer many useful suggestions for future research. For example, note the suggestion from Hung & Zhang (2012) that future researchers “should pay more attention to interdisciplinary approaches to research and development of ML in order to synthesize knowledge from both disciplines” (p. 13).

There are clear indications that the use of mobile technologies will continue to increase globally, and it is the challenge for educators to ensure that m-learning will be part of their use. Although more empirical evidence of the benefit of m-learning to learning outcomes is needed ( Laxman, 2012 ), research indicated that m-learning is beneficial particularly with respect to the development of lifelong learning habits and skills. To gain from the potential benefits of learning with mobile devices, schools and teachers need to be well informed about all aspects of m-learning -- not only on how to use the technology but what they need to do to support m-learning through pedagogical changes. For this reason more attention should be given to pedagogical preparedness of teachers and institutions for m-learning. It has been argued that in order to be effective, learning via mobile devices must follow good instructional design ( Berking et al. 2012 ). This suggests that more research with an emphasis on m-learning instructional design should be conducted. Research shows that in general technology acceptance is not a problem, m-learning research needs to move beyond issues of acceptance of technology to how that technology may best be used.

CONCLUSION

As digital technology advances and the use of PDAs, such as the iPhone and the iPad make online learning more accessible, it is likely there will be more use of mobile technologies in education both in the United States and globally. However, as Basta (2009) warned us, “[E]xcessive confidence in information and communication technologies in the learning discipline may lead to a situation similar to the dot-com [bubble] burst that happened in the late 2000” (p. 1). This study shows there is a significant absence of attention being paid to pedagogical details in conceptuaizing m-learning research. Although, as many studies indicate, m-learning is attractive to learners, the use of mobile technology does not guarantee that effective learning will occur. Without reference to theoretical and pedagogical issues, studies of m-learning will not necessarily further our understanding of how m-learning can contribute to successful learning outcomes globally. As Laxman (2012) pertinently pointed out: “Gratuitous use of technology for the sake of technology will not necessarily improve teaching and learning processes” (p. 48). M-learning can only bring about an improvement in learner outcomes when it is matched by the application of pedagogical practices that take into account the characteristics and opportunities presented by m-learning and recognize the demands of the differentiated educational and cultural contexts it will be used in.

Hasan Ahmad Al-Zahrani is an Educational supervisor at the Ministry of Education, in Saudi Arabia.

Dr Kumar Laxman is an associate professor of education with the Faculty of Education, University of Auckland.

REFERENCES

Abachi, H. R., & Muhammad, G. (2013). The impact of m-learning technology on students and educators. Computers in Human Behavior

Adeoye, B., & Wentling, R. M. (2007). The relationship between national culture and the usability of an e-learning system. International Journal on E-Learning , 6(1), 119-146.

Al-Ammari, J., & Hamad, S. (2008) Factors influencing the adoption of e-learning at UOB (University of Bahrain) . In the proceedings of the Second International Conference and Exhibition for the Zain E-Learning Center, Manama, Bahrain, 28- 30 April. Retrieved: https://uqu.edu.sa/files2/tiny_mce/plugins/filemanager/files/30/papers/f82.pdf

Ally, M. (2004). Foundations of educational theory for online learning. Chapter 1. In Anderson, T. & Elloumi, F. (2004) (Eds). Theory and practice of online learning . Athabasca University (e-book) Retrieved: http://cde.athabascau.ca/online_book/

Anderson, J., & Rainie, L. (2008) The future of the Internet III, Pew Internet & American life project . Retrieved: http://www.pewinternet.org/~/media/Files/Reports/2008/PIP_FutureInternet3.pdf.pdf

Basta, T. (2009). Assessment impact on online learning credibility . Paper presented at the American Canadian Conference for Academic Disciplines, Toronto, Canada, May 25-28, 2009.

Berking, P., Haag, J., Archibald, T., & Birtwhistle, M. (2012) Mobile learning: Not just another delivery method . Paper presented at the Inter-service/Industry Training, Simulation, and Education Conference (I/ITSEC 2012).

Cheung, W. S., & Hew, K. F. (2009). A review of research methodologies used in studies on mobile handheld devices in K-12 and higher education settings. Australasian Journal of Educational Technology , 25(2), 153-183.

Churchill, D., & Churchill, N. (2007). Educational affordances of PDAs: A study of a teacher’s exploration of this technology. Computers & Education , 50, 1439-1450.

Coursaris, C. K., & Kim, D. J. (2011). A meta analytical review of empirical mobile usability studies. Journal of Usability Studies , 6(3), 117-171.

Creswell, J. (1994). Research design: Qualitative and quantitative approaches . London: Sage.

Denson, N., & Seltzer, M. H. (2011). Meta-analysis in higher education: An illustrative example using hierarchical linear modeling. Research in Higher Education , 52(3), 215-244.

Georgiev, T., Georgieva, E., & Smrikarov, A. (2004). M-learning -- a new stage of Е-learning . Paper presented at the International Conference on Computer Systems and Technologies - CompSysTech’2004. IV.28-1-5

Gilliam, W. S., & Zigler, E. F. (2001). A critical meta-analysis of all evaluations of state-funded preschool from 1977 to 1998: Implications for policy, service delivery, and program implementation. Early Childhood Research Quarterly , 15(4) 441-473.

Hew, K. F. (2009). Use of audio podcast in K-12 and higher education: A review of research topics and methodologies. Educational Technology Research and Development , 57(3), 333-357.

Hung, J. - L., & Zhang K. (2012). Examining mobile learning trends 2003–2008: A categorical meta-trend analysis using text mining techniques. Journal of Computing in Higher Education . 24, 1-17.

Hwang, G. J., & Tsai, C. C. (2011). Research trend in mobile and ubiquitous learning: A review of publications in selected journal from 2001 to 2010. British Journal of Education Technology , 42(4), E65–E70.

Kukulska-Hulme, A. (2012). How should the higher education workforce adapt to advancements in technology for teaching and learning? Internet and Higher Education , 15, 247-254.

Laxman, K. (2013). Making a difference for the future in mobile learning: Examining the educational affordances of portable computing devices within a New Zealand context. In Stephen. M. & East, M. (Eds). Making a difference in education and social policy (pp. 103-119). New Zealand: Pearson.

Law, N., Pelgrum, W. J., & Plomp, T. (2008). Pedagogy and ICT use. Retrieved from http://dx.doi.org/10.1007/978-1-4020-8928-2 Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry . Newbury Park, CA: Sage Publications.

Mbarek, R. & El Gharbi, J.E. (2013). A Meta-analysis of e-learning effectiveness antecedent. International Journal of Innovation and Applied Studies , 3(1), 48-58.

Mishra, P., & Koehler, M. (2009). Too cool for school? No way! Using the TPACK framework: You can have your hot tools and teach with them, too. Learning & Leading with Technology , 36(7), 14-18. Retrieved: http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=EJ839143

Mungania, P. (2003). The Seven E-learning barriers facing employees . Final Report. Louisvilee, KY: University of Louisville.

Naismith, L., Lonsdale, P., Vavoula, G., & Sharples, M. (2004). Literature Review in Mobile Technologies and Learning , NESTA Future lab Series, Report 11, United Kingdom.

Pollara, P., & Broussard, K. (2011). Student Perceptions of Mobile Learning: A Review of Current Research . In Proceedings of Society for Information Technology & Teacher Education International conference 2011, Chesapeake, VA: AACE.

Rouse, M . (2005). Distance learning (e-learning) . Retrieved: http://searchcio-midmarket.techtarget.com/definition/distance-learning .

Russo, M. W. (2007) How to review a meta-analysis, Gastroenterology & Hepatology , 3(8): 637-642. Retrieved: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3099299/

Sandelowski, M. (2004). Using qualitative research. Qualitative Health Research , 14(10), 1366-1386

Sharples, M. (2000). The design of personal mobile technologies for lifelong learning. Computers and Education , 34, 177–193.

Sharples, M., Taylor, J., & Vavoula, G. (2005). Towards a theory of mobile learning, Proceedings of mLearn , 1 (1), 1-9.

Trifonova, A., Georgieva, E., & Ronchetti, M. (2006). Has the time for University’s mobile learning come? Determining Students’ readiness. WSEAS Transactions on Advances in Engineering Education, 3(9), 1790–1979.

Trochim, W. M. K. (2006). The nomological network. The Research Methods Knowledge Base. http://www.socialresearchmethods.net/kb/nomonet.php .

U.S. Department of Education (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies . Report for the U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, Washington, D.C.

UNESCO (2013). Mobile Learning Publications, Retrieved from http://www.unesco.org/new/en/unesco/themes/icts/m4ed/mobile-learning-resources/unescomobilelearningseries/

Uzunboylu, H., Cavus, N., & Ercag, E. (2008). Using mobile learning to increase environmental awareness. Computers & Education , 52, 381–389.

Wu, W., Wu, Y., Chen, C., Kao, H., Lin, C., & Huang, S. (2012). Review of trends from mobile learning studies: A meta-analysis. Computers & Education , 59, 817–827.