This study addressed how usability improvement and motivational design affect learners’ motivation and learning performance in a self-paced, online learning environment. The study also investigated the interrelationships between commonly-used usability measures and the motivation measures based on Keller’s ARCS model. A two-phase study approach was used. In Phase I, an existing self-paced, online safety training tutorial was used as the baseline. Two alternative designs were developed with improved usability and motivational design based on the ARCS model. In Phase II, the effects of the three interface designs were evaluated through a three-group, generalized randomized block covariate design experiment. A total of seventy-two college students (48 males and 24 females) participated in the usability testing and the online training session using one of the interface designs. Both quantitative and qualitative data were collected and analyzed.
Results suggested a significant interface design effect on learner motivation. Learners who used the interface design with both usability improvement and motivational design applied (the UM group) showed the highest level of motivation. In particular, the attention level of the learners in the UM group was significantly higher than the baseline group. Results also indicated motivation differences between genders. Females showed higher scores than males in overall motivation score and in each of the four subscales of attention, relevance, confidence, and satisfaction. No significant difference in learning performance was found among the three treatment groups using different interface designs. None of the usability or motivation measures was a significant predictor of learning performance.
Small to medium positive correlations were found between usability satisfaction and three motivation measures, i.e., attention, relevance and satisfaction. Content analysis identified a number of interface design components to be relevant to learners’ motivation components: overall appearance, graphics/multimedia, text appearance, page layout, navigation, and paging/scrolling. Implications and design recommendations for online tutorial interface design were discussed. Additional discussion was provided regarding the online learning environment and the integration of usability, motivation, and instructional design and technology.
