Method: Task Analysis
Tools: XMind

This project was the Second Runner-Up of LITA’s Contest: Great Library UX Ideas Under $100.

Challenge

Discovery layers (or discovery tools) are search interfaces that use pre-indexing to provide speedy discovery of relevant materials across millions of records of library collections, from books and articles, to databases and digital archives. Discovery tools also aggregate hundreds of millions of scholarly e- resources, including journal articles, e-books, reviews, legal documents and more that are harvested from primary and secondary publishers and aggregators, and from open-access repositories. Discovery tools are projected to help create the next generation of federated search engines that utilize a single search index of metadata to search the rising volume of resources available for libraries.

There have been a large body of literature on usability tests assessing discovery layers (or discovery tools) in the library context. Those tests were mostly focused on the search interface itself, and the testing tasks were often mismatches of users’ real search scenarios. Understanding how well a discovery tool supports users’ search goals and workflows remain a challenge.

Method

We conducted hierarchical task analysis (HTA) to evaluate how a discovery tool (Ex Libris Primo) supports 11 search cases. The search cases involved different formats (article, print book, and e-book) and availability (not available, available in print, available online, and available both in print and online), which present users with possible frustrations and obstacles.

The HTA is a workflow centered analysis method without testing participants. We used a desktop and a free mind mapping software (XMind) for our analysis, hence the budget is zero if not considering the time cost.

Findings

We broke the 11 search goals into subtasks and actions, allowing us to visualize the workflow and cognitive decision points. All 11 cases involved four sub-goal-process: 1) start search, 2) find relevant results, 3) view the desired item, and 4) retrieve, locate or request the item.

The first two cases offered nearly identical experiences. The third and fourth sub-goals, however, presented different workflow issues depending on the item searched and its availability. Article search was least guided in Primo’s interface and searching for an article not available in Primo involved a higher number of cognitive steps (13) than other availabilities (in print (9), online (4), or both in print and online (8)). For book searches, it was challenging to verify the right book when there were many similar results or same results in different locations; and the flow to place a request for a book from Primo to the interlibrary loan system (Illiad) was also a challenge.

Hierarchical Task Analysis Visualizations

Impact

Following our analysis, the web team at Purdue Libraries redesigned the Primo interface by eliminating confusing information in search results and unnecessary actions. For example, users no longer need to click on the obscure “multiple versions available” link in the brief item description area to see different versions of the same book; instead, they can consistently click on the book title in the search results to view the single or multiple versions. It is much easier now to specify the exact publication date facet on a timeline than previously inconsistent time ranges. For items that are not available, the redesigned interface now displays possible search and request options like interlibrary loan, UBorrow, WorldCat, and Google Scholar. Our latest user tests showed better search workflow and improved user satisfaction with the redesigned interface.

The HTA effectively helped us identify potential workflow issues not typically found in usability tests and new user requirements that discovery tools need to support. Our HTA analysis could also offer a comparable baseline and low-cost assessment for different discovery tools at other institutions.

Related publications

Promann, M., & Zhang, T. (2015). Applying Hierarchical Task Analysis Method to Discovery Layer Evaluation. Information Technology and Libraries, 34(1), 77-105.