The Effect of Search Engine Ranking Elements on the User Experience (UX): Uncovering How Google's Algorithmic Choices Affect the Users Behind the Screens

Lærke Saura Birk

Student thesis: Master thesis


Purpose - Algorithms define how users look for information online. Search engines, notably Google, have marketed their services by fiercely defending an algorithmic ideology that sustains their business model. A consequence of this underlying algorithmic logic is the unquestioned everyday use of search engine rankings. These have become a convenient tool for users to unproblematically look for information on the web as they reduce complexity in a cluttered online sphere. Experts have focused on investigating how these rankings appear regarding content and form; and come with suggestions on how they can be improved through evaluative research. Despite that, the effect of these rankings on the user experience (UX) is underinvestigated. In this work, stance bias and featured snippets are explored as ranking elements that potentially affect the measured UX on Google. Methodology - On the researcher’s own initiative, data on different subjects’ UX was collected with a surveybased 2x2 full factorial vignette experiment (N=136) to assess three hypotheses through a quantitative data analysis. Participants were presented 3 hypothetical search situations and synthetically created Google rankings. These were assessed through a measurement instrument that calculated a participant’s overall UX mean score with the use of a 5-point Likert scale. The experimental data was analyzed with the Python programming language performing a multivariate parametric test (two-way ANOVA) to compare experimental group means. The adjusted R2 is reported to show the model fit, and the partial η 2 quantifies effect sizes. Findings - The difference among the mean UX scores in the different experimental groups showed that participants treated with stance bias on rankings had a significantly lower UX than participants who did not experience any stance bias (p = 0.000, coef. = -0.6216), reporting a significant simple effect. No simple effect was reported for participants exposed to featured snippets (p = 0.506), and interaction effects were neither identified for participants facing featured snippets in combination with stance bias (p = 0.415). Implication - Users register a decrease in their UX when exposed to stances on rankings, whereas seeing featured snippets does not affect the UX. The study outcome suggests that users are willing to engage in research tasks when performing informational queries instead of seeing results on rankings that offer a particular editorial positioning on a topic. This finding encourages managers to reassess their search engine optimization (SEO) strategy to enhance the UX. SEO strategists can attract users to their content on SE rankings by removing stances from their pages’ snippets. This research also challenges some pre-established beliefs in academia, such as claiming that users rely blindly on rankings or their idealized view towards Google. Future research can contextualize these findings with qualitative data and build on the regression model presented in this work.

EducationsMSc in Business Administration and E-business, (Graduate Programme) Final Thesis
Publication date2021
Number of pages91
SupervisorsTina Kretschel