Collaborative hierarchical clustering in the browser for scatter/gather on the web



Scatter/Gather is a powerful browsing model for exploratory information seeking. However, its potential on the web scale has not been demonstrated due to scalability challenges of interactive clustering. We have developed in previous research a two-stage method to support on-the-fly Scatter/Gather, in which an offline module pre-computes a hierarchical structure to support constant time on-line interaction. In this work, we focus on the offline hierarchy construction and develop a novel distributed approach to hierarchical agglomerative clustering (HAC). Relying on Javascript that is commonly supported by browsers, the distributed clustering method has the potential to scale with growing traffics of a site. We show in experiments that a moderate increase in the number of parallel processes (in visitors' browsers) leads to a dramatic decrease of clustering time. This demonstrates great potentials in supporting large-scale Scatter/Gather interactions on the web. We present preliminary analysis of clustering effectiveness and a related Scatter/Gather prototype for web search.