Integrating Behavior User Studies with Log Analysis

Method: Transaction Log Analysis, Log Visualization, Data-driven Task Design, Behavior Observation
Tools: Python, R, SAS, Morae

Challenge

The library catalog (or discovery tool) transaction logs are believed as one of the important sources of big data in libraries. Although researchers can mine detailed information about users’ search behavior from logs, one of the obstacles of log analysis is the lack of contextual information such as users’ motivations, information needs, and step-by-step actions. Analyzing logs alone also has the danger of reaching oversimplified conclusions about search behavior without appropriate understanding of the tasks’ contexts and users’ preferences.

Method

Through case studies, we have shown that integrating behavioral user studies with log analysis is an effective way to uncover the contextual information of search tasks and drive user interface improvement. We used behavior patterns from log analysis to develop test tasks for user tests so that we can target certan search behavior and task context for observation.

Results

Common findings and behavior patterns on search field usage, facet selections, and query formulations include these:

  1. Users predominantly use keyword search;
  2. Use of facets is low, and nested facet selections are rare;
  3. Most search sessions involve fewer than four queries;
  4. The average number of words per query is generally less than three;
  5. More than half of search sessions reformulate the search by adjusting the original keywords.

In addition, the content coverage of catalogs and discovery tools can affect users’ search behavior. For example, users of discovery tools tend to have a higher percentage of keyword searches and a lower percentage of title, author, subject, and call number searches.

Presentation

Integrating Behavior User Studies with Log Analysis from Tao Zhang

Niu, X., Zhang, T., & Chen, H. (2014). Study of user search activities with two discovery tools at an academic library. International Journal of Human-Computer Interaction, 30(5), 422-433.

Zhang, T., Niu, X., Zhu, L. & Chen, H. (2015). Search in One’s Hand: How Users Search a Mobile Library Catalog. Paper presented at the 17th International Conference on Human-Computer Interaction, Los Angeles, CA. August 2-7, 2015.

Updated on November 12, 2015