Controlled Digital Lending Application Diary Study

Students gave us feedback about a new textbook scanning service in this diary study. We also asked about their reading habits and scheduling preferences.

Overview

In Fall 2020, we launched the Controlled Digital Lending Application (CDLA) as a way to expand our textbook lending service. During CDLA’s first week in operation, a team with members from User Experience, Digital Library Initiatives, and Access Services conducted a diary study with five participants in order to test the usability of the new application and service.

Research Questions

  • Request and delivery of CDL scans
    • Does the request process set appropriate expectations about the timing and availability of scans? 
    • When the user sees the email, do they understand the access policy? (4-hour access window, 2-hour loan) 
    • How consistent are the CDLA access policies with students’ study habits and schedules? 
  • Using the PDF
    • What are the user’s expectations regarding being able to copy/paste from the PDF? What about downloading and printing?
    • Are the PDFs usable? 

What we found

  • All five participants remarked on how short the 2-hour loan period was, and most indicated the 4-hour access window was either too short or inconvenient/unworkable because of their schedules.
  • Having PDFs available for only a limited amount of time is inconsistent with students’ mental models of the PDF format, and prompts responses of confusion and incredulity. 
  • Participants were surprised by the 2-hour loan period upon receiving the email, and wished they had a “heads up” during the request process. 
  • Four participants took the time to say they appreciated this service was being offered by the library. 
Charts of five participants' user journeys, from request submitted to email received to opened file. Only one participant opened the file within 2 hours of receiving the email notification.

Timeline of our five participants' experiences with the Controlled Digital Lending Application, from request to opening the file. Of the three participants who successfully received and opened the scanned file within the week of our research, only one participant opened the file within 4 hours of receiving the email notification. (View full size.)

Recommendations and changes

Set user expectations from the start

  • Set user expectations during the request process by adding more information on the request form about the 2-hour loan period. 
  • Send a confirmation email after the request form is submitted.
  • Include a friendly description of why chapters can only be accessed for 2 hours on the CDLA landing page for chapter access, and anywhere else that it may be helpful.

Get rid of the 4-hour access window 

  • Create unmediated access to scans, so once a PDF is ready, users can access it at their convenience. Consider implementing queueing or availability notifications for scans that are in high demand by multiple users at once.

How We Did It

We chose to use the diary study model for this user research, since we were interested in participants’ feedback about a multi-step, multi-day process.

We recruited five participants from a pool of 47 respondents by selecting students who were required to use a textbook for a class that could only be accessed through CDLA. These students represented a broad variety of academic disciplines, and ranged from first-year undergraduate to second-year graduate student.

With each participant, we held an initial 25-minute Zoom call to orient them to the study. For each call, there was a facilitator (to ask questions) and a co-facilitator (to take notes and ask follow-up questions). In this first session, we asked the following questions:

  • Tell us a little about your major and what you study.
  • When do you prefer to do your class readings? 
  • What time of the day do you generally do your readings? When during the week? 
  • How far in advance do you normally read the materials you need for your work? 
  • Do you do all the reading at once, or break it up?

After asking questions, we explained the process of the diary study. We shared with each participant a diary document with prompts for which they would record their experiences as they submitted a CDLA request and interacted with the scanned document. Each participant was directed to fill out a CDLA request for a chapter from their textbook in the next few days, and then begin filling out their document. In the diary documents, participants were asked to record their experiences with:

  • Requesting the scan from CDLA (filling out a Google Form)
  • The email alerting them that their scan was ready
  • Opening/viewing the scanned chapter
  • Any frustrations/problems with the above tasks

After two days, we sent reminder emails to participants to fill out their diary documents. In these emails, we also included information about the second Zoom session.

One week later, we met with our study participants for another 25-minute Zoom call and discussed their experiences. In these follow-up meetings, we asked additional questions based on the thoughts they recorded in their diary documents. 

One of the participants did not receive a scan on time due to the timing of the CDLA request. Instead of completing the diary document, this participant provided live feedback as they interacted with a scan provided to them during the Zoom call.

We identified major usability issues based on participants’ diary documents and their feedback from the two meetings.  

Team