Cross cultural usability testing: The relationship between evaluator and test user

In this paper, we present the results of a pilot study in Denmark of cross cultural effects on Think Aloud usability testing. We provide an overview of previous research on cross cultural usability evaluation with a special focus on the relationship between the evaluator and the test user. This relation was studied in an experiment with usability testing of a localized clipart application in which eight participants from Denmark and India formed pairs of evaluator-test user. The test users were asked to think aloud and the evaluators' role were to facilitate the test users thinking aloud and hereby identify usability problems with the clipart application. Data on the evaluators' and test users' behaviour were recorded and analyzed by coding and summarizing statistics on these behavioural events. The results show that Think Aloud Usability Test of a localized application is most effectively performed, in terms of number of think aloud events and number of usability problems found, when both the evaluators and the test users are local. These results are however limited to the Danish context and need to be investigated in other cultural settings.