Usability Testing for websites and other applications

-focusing on tutorials and user guides

-started out as making sure web sites cohered to design standards, but has gradually morphed to include how users intact with the site—really have to think about how to include testing in implementation process

-ISO def’n: extent to which a product can be used by specified users…

-don’t forget the consent form! might have to talk w/ IRB, too

-Card sorting: use, perception, demand; have participants sort cards w/ headings, then have them come up w/ own headings, then interview (from soc and psych); can lead to navigation redesign; time consuming to test and analyze results (can take pictures of the cards post-sort to help)

-heuristic: best practice (Jakob Nielsen); need outside experts; evaluate for specific criteria (ie, match beween system and real world [library jargon]); fix now/soon/someday; hard to find experts (and $$); high learning curve; can be hard on the site designer due to negative focus

-Assessment testing: users complete tasks; objective or goal-oriented; review for duplication; arrange from easiest to hardest; best method for feedback on functionality and navigation; can be formal or informal; remember to debrief the participant

-choose right method: demographics of users, purpose of testing; need lots of user groups represented; use incentives to recruit (also, neutral location and timing) [do lib staff and librarians separately, since they use the site differently and staff might be more willing to be open w/out libs in the room—can be good to have outside moderators for some groups]

-testing 2.0 apps: Focus! assessment tests work best for this. specific audience; greater depth of test; user population may have no prior experience with the application, so have to account for that in the questions

-content testing: focus on info; tasks based on learning objectives; interfae independent

-software testing: focus on navigation; tasks based on finding info; interface dependent

-Be sure to be focusing on Content, not software (unless you’re doing OA, you can’t do anything about the software)

-pretest: use to refine questions; small sample user group; screen captures can really help; repeat until results are consistent; methods: interviews after, screen capture, filming

-designing test questions: be specific and task-oriented; pretest for validity and clarity; broad or narrow scope—keep to middle ground; longer is not better—don’t want to tire people out or have them get bored (on side of paper seems to be good; it looks do-able)

-samples:

-find book: Does the lib. own a copy of ____?

-access a db: Does the lib. have access to _________?

-find lib. hours: What time does the lib close on _____?

-find contact info: Where is liaison’s office?

-Use back button: How do you get to previous material?

-Nielsen says doing 5 should be plenty—diminishing returns after that. But they aren’t so sure.

-implications: highlights user interaction relative to design; focus on important content; indicates higher maintenance items; underscores tast complexity; potential redesign

-figured out that students were having different interactions w/ the info depending on the librarian who had created the libguide, so they’re going to write some standards for the guides based on their findings.

 

Q&A

-What type of 2.0? Tutorials and guides are the only ones they’ve done testing for.

-signifiant diff in user groups? Yes! Esp. btwn patrons and librarians, w/ libs not understanding what patrons want (user testing can debunk lib myths about what students want)

-institutional standards in the website redesign? yes. colors, header, and a few other things dictated by the school (Wartburg College) so they had to work around, but their head guy was ok with a little switcheroos

-how much time? card sorting: one afternoon (used magnetic board and handed out candy bars while at the ref desk);  assessment takes much longer, esp. w/ pretesting, plus fact that application are often new to the user (and sometimes the lib!)