QuickSearch + Drupal Usability Study

We conducted a usability test of how our custom search application shows results from our website, and interviewed participants to capture their thoughts on recently added QuickSearch (QS) features. Our results challenged several previously held assumptions about how our patrons search on the Libraries website.

Overview

QuickSearch is the Libraries’ custom search application, which presents users with a bento-box display of search results from our catalog, Summon, the Libraries website, and other sources (example search).

Drupal is the content management system for most of the Libraries website (www.lib.ncsu.edu), but it does not contain the catalog, Summon, and several other data sources. "Drupal" in this report refers to webpages about the Libraries' spaces, services, resources, staff, and more.

We wanted to observe users as they tried to find Drupal content through QuickSearch and see if any patterns of behavior emerged. We know most clicks on the QuickSearch results page are in the Articles or Books & Media sections, but how successful are users searching for other types of content, such as information about our services? The QuickSearch team also wanted to find out what users thought about some recently added features, such as the teal “Discover other services” bento box. We found that participants’ behavior challenged several of our assumptions about use of QuickSearch generally, and we recommended some specific changes that could be made to both the discovery of Drupal content in QS and features like the teal “Discover” box.

Assumptions, revisited

In past usability studies, we’ve observed that users tend to follow a few types of scrolling behavior on our webpages. Many users will do a quick scroll of the entire page as soon as it loads to see the length, headings, and scan for keywords. While we might expect QuickSearch users to follow a similar pattern, we instead observed that most participants treated the QuickSearch results page more like the Google results page: evaluating each search result from top to bottom individually and expecting their desired result to appear very close to the top of the page. Five out of six participants gave up if their desired result did not appear in the top half of the QuickSearch results page.

As librarians, we might assume that all users will try several different rewordings of their search query if their first formulation doesn’t succeed. However, we observed that participants gave up after their first search if the original search query did not return the desired result.

Additionally, analytics show the majority of QuickSearch users are looking for articles and books. However, participants in this study still expressed frustration when they searched for an explicitly non-article / non-book item and the "Articles" and "Books & Media" bento boxes showed results that seemed unrelated to their search.

Fixes to improve Drupal discovery

The Workshops bento box was highly utilized by participants, but it didn’t always show up when we expected it to. We recommended to the QuickSearch team that this bento box display not only when searching the name of a workshop but also when searching “[name of workshop] workshop.”

Best Bets made it very easy to find devices when users searched the exact name of the device (see screenshot in full report below). However, some users misspelled the device title and had to scroll all the way to the bottom of the results, to the Our Website section, in order to find the desired result. We recommended that these helpful Best Bets appear even when users misspell the device title in their search query.

Ideas for future exploration

During the interview portion of the usability study, participants had several suggestions for re-configuring the layout of the QuickSearch results page. We found some commonalities and recommended these ideas to the QuickSearch team for future exploration:

  • Move LinkedIn Learning, teal “Discover other Libraries services” bento box, and Get Help bento box further down the results page.
    • Participants told us these seemed “less important” than other types of search results.
  • Promote Journals, Databases, and Our Website results higher on the results page.
    • Several participants expressed confusion that they had to scroll so far down to see these results.
  • Make the teal Discover bento box into a “jewel box” at the bottom of the results page.
    • This is for users who have scrolled past all the functional content and might be interested in promotional content.

How We Did It

In June 2021, we sent a short survey about QuickSearch usage out to our user research participant pool via email and recruited six undergraduate and graduate students, who had each indicated they used QuickSearch regularly, for follow up interviews. We conducted a 30-minute usability test and an interview via Zoom with each participant. The usability test consisted of three to four search tasks, which we first observed participants completing with minimal input, and then asked participants to complete again using QuickSearch if they had used Google or another discovery method originally. The task goals were all to discover a different piece of Drupal content:

  • A workshop
  • A space
  • For grad students, a page about grad student study rooms
  • For undergrad students, a software page

We conducted a short interview after the usability test to ask participants about their experience using QuickSearch to find Drupal content. We also asked questions about specific QuickSearch features and asked for any general feedback on the layout of the search results page.

Results

Team