Building Your Usability Toolkit

Athena Hoeppner and Meg Scharf
University of Central Florida Libraries

Computers in Libraries 2006, Washington, D.C., 22 March 2006.

The content available here is far from identical to the presentation delivered live at CIL 2006.  Some additional material is included, such as instructions and handouts we gave to the participants of our usability studies.  Missing are the interactive, hands-on exercises and the text of our presentation.

We will share our experiences with 3 usability projects, two based on affinity mapping and one usability scenario.

Affinity Mapping Projects

Selected Definitions of Affinity Diagrams and Mapping: IAwiki | Roxanne O'connell | Usability Net

A note from Athena: A member of the audience asked why we called the following activities Affinity Mapping rather than a Card Sort.  I had to admit that the activity is similar to a card sort, however, this is an interactive group process. The lines between card sort, affinity diagrams, and affinity mapping seem inexact, and I admit that I use the term affinity map because that is what it was called by the person who introduced me to the idea. You can make your own choices about what to call the activity.

Affinity maps with library employees

Project Goals

• Develop an overall view of the web site content
• Find logical and intuitive organization for the content
• Discover which content is considered most important
• Generate understandable labels/link text for content

Process

• We invited library employees and selected outsiders to participate.
• We started with about 150 slips of paper each with a term from the library web site index.
• Groups of 3-5 participants took the slips and sorted them by putting similar slips together.
• Participants were asked to work silently, to create duplicate slips or add new slips as needed, to move slips placed by others if they felt it should go elsewhere.
• Participants then discussed their placements and created headers or labels for major groupings.
• Each participant had 5 dots to place by the slips they felt were most important.

Three Library Employee Affinity Maps

Analysis Process

• Entered all the terms into a spreadsheet, indicated number of dots for each term.
• Sorted by importance (dots), and terms to get an idea of which were most used and most important.

The top terms sorted by importance ranking

Outcome

• Created home page content mock ups and navigation mock ups based on the terms selected or marked most often.
• The mock ups were shared with the web implementation committee and the web policy committee. The WIC eventually used the input to help determine the content of the homepage and navigation.

Evaluation of the process

• We started with too many slips. Participants wondered if they needed to use them all.
• The process worked well for grouping similar ideas and creating labels or headings.
• The process did not point to any particular layout or visual design and left a lot open in terms of overall site navigation and depth.
• Did show participants priorities.
• Very low cost

Affinity maps in public

Goals

• What were most important links to students for homepage
• What language would be understandable for students?
• What groupings make sense to students?

Process

• Created slips - used fewer terms, about 50, based on the terms chosen in the previous affinity map project
• Put strips on magnets, used metal easel
• Created instructions and brief demographic survey
• Got candy bars.
• Set everything up in the library lobby
• Solicited passer-by
• Groups of participants took the slips and sorted them by putting similar slips together.
• Participants could create duplicate slips or add new slips as needed, to move slips placed by others if they felt it should go elsewhere.
• Participants were asked to circle groupings and to create headers or labels for major groupings.
• Each participant had 5 dots to place by the slips they felt were most important.
•  Resulting affinity maps were numbered then recorded with a digital camera.

Example Student Affinity Maps

Analysis of data

• Created a spreadsheet
• Recorded terms selected
• Recorded grouping and placement for each term

Outcome

• Generated more refined mock-up of homepage
• WIC interpreted the spreadsheets and focused on the most selected options. Applied understanding of our goals
• Resolved conflicting input

Evaluation of the process

• Mapping went fast!
• Needed 2 people could have used 3
• Candy bars were appreciated, but not the incentive for many students.
• Using a student for recruiting may have worked well.
• We still started with too many slips.
• The process worked well for grouping similar ideas but not for creating labels or headings.
• Students did not create new slips or suggest other wording, or rank with dots.
• The process did point to layouts.
• Only the homepage content was addressed, not deeper navigation or organization.
• Did show participants priorities.
• Participants had fun!
• We had a mix of different ages and groups.
• Very low cost

Usability Scenarios

Goals

• How students are using the home page
• How are they getting to services and content?
• Which databases and research pages are choosing and how are they using them?
• What do students do when they need to research and they aren't told where to start?

Process

• Wrote several scenarios
• Chose one to focus on - we couldn't do them all
• Collaborated with a usability faculty member
• Involved a graduate student to conduct beta testing with library student assistants
• Used Camtasia to record the students working through the scenario
• Evaluated the scenario and the technology, rewrote the scenarios
• Wrote a proposal to get a LEAD scholar, Jodi - an excellent UCF student
• Jodi reviewed the scenario, learned Camtasia, beta tested the scenario again
• Created web-based demographic survey and scenario instructions
• Jodi took a laptop into student areas and recruited participants and recorded each session.

Analysis Process

• Ongoing
• Each participant filled out a demographic survey and forms recording their satisfaction with each search process
• Jodi wrote "reflections" of her observations for each interaction.
• Review each recording, enter data for search options chosen, search terms used
• Correlate search behavior with demographics and satisfaction data.

Outcome

• Project is ongoing
• Reveal need for LI, for redesigns of database and subject guide pages, especially for MetaLib, and for more understandable SFX buttons.
• Shows strong tendencies of students to use any search box before browsing through subject or alphabetical database lists.
• Shows impatience of students - they act on what comes up first rather than waiting for results to fully load.
• Will share the clips with reference staff and web designers so they can see the difficulties that students have
• Reveals that students don't relate specific terms or topics with broader concepts.
• Security / login process can hinder searching.

Evaluation of the Usability Scenario Process

• As library departments heard about the Scenario testing, they asked to have their pages tested.
• We had to focus and could not write a single scenario for everything we want to know
• Beta testing was essential to work out unclear wording, awkward transitions, starting and stopping Camtasia, and other problems
• The interviewer may have to coax verbal comments out of participants
• The interviewer can be corrupted by the instinct to help
• Requires some method to thoroughly record the interactions and the screens
• Requires a skilled interviewer
• Takes lots of time per interaction and for review and evaluation
• Can provide all kinds of information about web design issues and much more
• Next steps will be to focus on pairs and expand our demographic

Athena Hoeppner, athena@mail.ucf.edu
Meg Scharf, mscharf@mail.ucf.edu
University of Central Florida Libraries
Orlando, FL 32816

03 Jun 2009

Need Help? Ask a Librarian | UCF Homepage | UCF Libraries Homepage | Orlando, FL 32816 | Contact Us ©2008