Ordering Matters: An Experimental Study Of Ranking Influence On Results Selection Behavior During Exploratory Search

Abstract

The design of Exploratory Search tools acquires more importance as the amount of information on the Web grows. Accordingly, informed design decisions concerning the users' behavior during search activities can be used in novel approaches for Exploratory Search tools. With regard to users' behavior on search result selection, literature indicates higher ranked results tend to attract more attention and, therefore, more hits. However, the cause of such behavior is not clear. We experimentally investigate the hypothesis of this behavior being due to ranking. A group of 72 participants was asked to select a search result from a randomly ordered results list. The experiment was carried out on paper to remove specific search engine and digital medium biases. We obtained evidence indicating there is a trend towards choosing higher ranked results even in a different medium and in the context of an Exploratory Search, corroborating to the hypothesis that ranking has influence on users' selection behavior.