Much traditional user interface evaluation is conducted in usability laboratories, where a small number of selected
users is directly observed by trained evaluators. However, as the network itself and the remote work setting have
become intrinsic parts of usage patterns, evaluators often have limited access to representative users for usability
evaluation in the laboratory and the users' work context is difficult or impossible to reproduce in a laboratory
setting. These barriers to usability evaluation led to extending the concept of usability evaluation beyond the
laboratory, typically using the network itself as a bridge to take interface evaluation to a broad range of users
in their natural work settings. The over-arching goal of this work is to develop and evaluate a cost-effective
remote usability evaluation method for real-world applications used by real users doing real tasks in real work
environments. This thesis reports the development of such a method, and the results of a study to:
- investigate feasibility and effectiveness of involving users with to identify and report critical incidents
in usage,
- investigate feasibility and effectiveness of transforming remotely-gathered critical incidents into usability
problem descriptions, and
- gain insight into various parameters associated with the method.
