Thesis
Beyond ranking: integrating user interaction and implicit feedback in optimising search engine result pages
- Creator
- Rights statement
- Awarding institution
- University of Strathclyde
- Date of award
- 2024
- Thesis identifier
- T17062
- Person Identifier (Local)
- 202088679
- Qualification Level
- Qualification Name
- Department, School or Faculty
- Abstract
- The process of searching for information through an information retrieval (IR) system is intrinsically interactive, involving users in a series of actions such as formulating queries and evaluating numerous result snippets and their corresponding documents to ascertain their relevance. These interactions, which can be regarded as costs, underscore a notable deficiency in conventional IR systems. The Probability Ranking Principle (PRP) states that ranking documents in decreasing order of relevance with respect to the user’s query is the optimal way to maximise their expected utility from the results. However, the PRP fails to account for the nuances of result presentation and the inherent costs associated with interacting with the search engine results page (SERP). Acknowledging the diversity in users’ preferences, which can range from closely aligned to significantly divergent, poses a challenge in optimising the display of relevant results on a page to accommodate these varied inclinations. Different users will prefer distinct types of result presentations, and the layout of the result pages significantly influences their ability to interact with the system, discover relevant information, and, consequently, their overall satisfaction. This variability necessitates a nuanced approach to designing IR systems, one that goes beyond traditional ranking methods to consider the individualised ways users engage with and perceive the utility of search results. To address these challenges, the interactive probability ranking principle (iPRP), implemented via the Card Model offers a robust theoretical framework within the interactive IR space, enabling us to model the user interaction process while accounting for constraints such as presentation and screen space. By incorporating users’ implicit feedback, it is possible to assess these costs and preferences towards certain result items. This assessment can then inform the calculation of the Expected Perceived Utility (EPU), thus offering a more nuanced understanding of user interaction with IR systems. This thesis builds upon the card model, by expanding it to estimate the costs associated with user interactions in terms of time and to re-rank search engine results pages (SERPs), thereby raising three compelling questions; (1) how does the operationalisation of EPU affect result ranking, (2) what is the relationship between system-sided and user-side search costs and lastly (2) how do we optimise the presentation and what is its impact on user satisfaction. Each of these questions is explored through user studies centred on a news search task, designed to understand user preferences, presentation effects, and optimisation strategies. Our first study operationalises the notion of EPU and examines the impact of different result presentation formats, revealing how presentation significantly influences user perception through metrics such as time spent and clicks, and how we can re-rank results beyond traditional ranking paradigms like the PRP. We find that changing the presentation of results significantly impacts system-side metrics such as DCG, RBO and TBG. In our second study, we uncover the dynamics of interaction costs, result presentation, and user satisfaction at both the query and session levels. Our findings indicate that while at the query level, user satisfaction is predominantly influenced by performance metrics such as nDCG rather than presentation, at the session level, satisfaction emerges from a complex interplay of factors, delineating a non-linear relationship with the presentation. In the concluding study, we propose and evaluate a novel optimisation technique that synchronises ranking with presentation, tailored to individual user preferences. Although presentation optimisations lead to several behavioural changes in user interactions, they do not consistently align with user-reported satisfaction metrics, highlighting a subtle yet crucial gap between objective system enhancements and subjective user experience. In this thesis, we operationalise and empirically validate the iPRP. The findings from this thesis advocate for a shift in the design of future interactive IR systems, emphasising the need to personalise and dynamically display search results to enhance user experience. This research not only lays the foundational work for further exploration but also paves the way for validating the universal applicability of EPU-based result ranking across various interactive IR platforms and user demographics, thereby setting the stage for future studies in this vital area.
- Advisor / supervisor
- Azzopardi, Leif
- Halvey, Martin
- Resource Type
- DOI
Relazioni
Articoli
| Thumbnail | Titolo | Data caricata | Visibilità | Azioni |
|---|---|---|---|---|
|
|
PDF of thesis T17062 | 2024-09-17 | Pubblico | Scaricare |