Summon Configuration Usability Test
In November, 2018, we conducted usability tests to see if a proposed change to our configuration of the Summon database would aid users searching for known-item articles without leading to confusion for those searching by topic.
Overview
In 2018 when our users got to our implementation of the Summon database -- usually by searching the All or Articles tabs on the Libraries’ homepage -- they arrived at a result set that included only online full-text articles. We chose this configuration to make the task of finding articles on a topic easier for novices. We assume novices can get confused by instructions to find print copies of journals or use interlibrary loan (interfaces like Figure 1 below).
The most important drawback to results sets in Summon with only online full-text articles is that they exclude known-item articles when those happen to be unavailable online (but available in print or via interlibrary loan). Known-item searchers saw results that did not acknowledge the existence of articles they knew were available somewhere.
RESEARCH QUESTIONS
Could we remove the full-text online filter in Summon for users searching the All and Articles tabs on the Libraries’ homepage, thus helping known-item searchers, without unduly confusing novices searching for articles on a topic? When topic searchers encounter more results that do not lead easily to online full-text, will those results turn into dead-ends that severely compromise their experience?
WHAT WE FOUND
After watching the four participants interact with dead-ends, we felt that turning off the "full-text online" filter would not greatly increase confusion, and, on the other hand, would improve known-item searching. The dead-ends did cause some confusion. One participant called the interface that shows options when online full-text is not available (the 360 Link interface; see Figure 1 below) an “error message.” But the kind of confusion the dead-ends caused was not severe; participants were able to either dig in and try to understand it, or, more often, just move on to other results that had online full-text. A key to their ability to recover is the way Summon opens individual results in new browser tabs. Participants who hit a dead-end would go back to the first tab with the results set and move on, untroubled.
When asked “is it odd that this database shows non-online results,” all the participants said no. We interpret this to mean that they either understood the existence of print-only journals, or the non-online results were not otherwise causing them confusion.
Three of the four participants said that when they find an article citation and have trouble finding the full-text, they use the Libraries’ Ask Us chat service, expressing confidence that library staff can help them find it.
RECOMMENDATIONS AND CHANGES
Remove the "full-text online" filter in Summon for users searching the All and Articles tabs on the Libraries’ homepage. This change was made soon after the usability tests in November. Our decision is an accordance with one of the 10 Usability Heuristics for User Interface Design defined by the Nielsen Norman Group: “flexibility and efficiency of use,” which endorses systems that “... cater to both inexperienced and experienced users.”
How We Did It
We conducted usability tests with four students, two undergraduates and two graduate students. The average duration of the tests was 18 minutes. We led participants to result sets in which online full-text was easily available and into the possible dead-ends where they read “NCSU does not appear to have online access to this item” e.g., Figure 1 below. We asked what they would do, and more importantly, watched how they reacted. We incentivized the students with boxed lunches.
