Designing Higher Performing e-Commerce Search Engines: Design Objectives From an Evaluation of Existing E-commerce Search Engines with Users

Abstract

With the aim of formulating design science objectives for increasing performance of e-commerce search engines, the present research used a mixed methods approach to perform an extensive evaluation of current e-commerce search engine solutions, with actual e-commerce users. Divided into two phases, the study started, by the help of users following a think aloud protocol with different e-commerce related search tasks, with exploring and identifying e-commerce search engines, user behaviours and problems. By combining theory from the e-commerce and information retrieval domains, and the findings from the first phase, a framework for performance evaluation was then developed. In the second phase, more users evaluated the performance of three e-commerce search engines, using the produced framework. A quantitative analysis of the results found that the search engines Google.se and Prisjakt.se performed significantly better in four out of seven of the performance measures, than the search engine Elgiganten.se. Using Google.se, Prisjakt.se, user behaviours and the identified problems as a foundation, an analysis of the findings from both phases, combined with theory of what is feasible in search, allowed the researcher to develop recommendations, in the form of ten different design objectives for increasing the performance of e-commerce search engines. The design objectives propose e-commerce search engines to (1) choose what products to index and make sure their information is complete, (2) allow users to filter and sort the categorized products already at the results listing, (3) allow users to use common natural words (e.g. “budget” or “lightweight”) by helping in translating such words to product attributes, (4) carefully consider how much influence advertisers should have on the ads, as ads easily could cause relevance problems, (5) clearly show the destination of ads, in order not to confuse users, (6) implement optional personalization that shows its benefits for the users, (7) use the historical interests of users to make the search results more relevant, (8) use referral queries of other search engines, to understand and adjust the content to the users’ information need even before they type a query, (9) implement a browser add-on that complements existing search engines with e-commerce related information and (10) implement a browser add-on that tracks the users’ information needs between search engines, and adjusts the content thereafter. Given these research objectives, developers of e-commerce search engines should be able to increase the performance of their search engines.