Tag: user experience

Why don’t libraries share the results of UX work?

While conferencing at ELAG 2016, Simon made this suggestion for improving collaboration among libraries doing user experience (UX) work on their systems:

I’ve been thinking on-and-off about the question implied in Simon’s second tweet – the reasons libraries do not generally share results of user experience testing of their systems. Below I discuss some of these, which are drawn from examples from people with first-hand knowledge from academic libraries.

“Not all libraries…”

I know many libraries do share their results and analyses, whether more formally in articles, books or book chapters and conference presentations, or more informally in blog posts, conversations, and social media. I am extremely grateful for any and all sharing of this type.

What about libraries that are doing such work but not reporting it? When I know about such work, though it requires extra effort I have had good success asking colleagues about their findings. People are usually generous with sharing – sometimes surprised in the interest, but delighted to be asked about their work. In my experience the trick to doing this effectively is:

  • Knowing that others have been doing something in the first place, for which an active stance towards professional networking helps enormously.
  • Finding the best way to engage colleagues on their terms, and making it easy for them to help you. If you’re nearby offer to visit, or set up a video chat; ask for an hour of conversation rather than anything in writing or a presentation. Manage the time and conversation effectively: focus on what you are asking about; be explicit about asking for honest opinions; always ask about lessons learned.

Why we don’t share

Time and money

By this I primarily mean a lack of time and money to refine the work into something publishable by traditional routes, such as a peer-reviewed journal or presenting results at conferences. I think this is a huge barrier, and I have no simple answers. There is an huge qualitative difference between libraries that invest time and money in this way and those that do not, which is only really apparent with experience.

There are alternative routes. For me the actuality of dissemination and sharing always trumps the esteem in which a particular route of communication is held. Some of the most effective project communications I have seen are blog posts that summarize progress and demonstrate the momentum and trajectory of the work simply with clear and engaging text. This is the kind of work I or a team member would expect to do as part of internal project communication, so text can be reused with reworking for an outside audience.

For events, workarounds such as unconferences and similar events are a lower-cost approach, but these aren’t without issues. Events with no registration fee still favour those privileged in various ways. Even if one can attend unconferences these don’t always attract an audience with enough specialist knowledge, or are not promoted in a way that will attract such an audience. This may sound contrary to the Open Space Technology principle that “Whoever comes [are] the right people” (Owen, 2008), but in practice OST principles depend on a lot of work behind the scenes and careful management on the day to be effective.

Culture

By this I mean the overall stance of the library towards engagement with their sector and understanding of the value of sharing, as a cultural factor. Having a budget for conference attendance and other development activity is not enough if your workplace does not value this type of professional engagement in and of itself, or does not have confidence in staff ability or value the work. In practice this may manifest more subtly in general discouragement and ‘lack of permission’ from managers in the organization, or not sending staff operationally involved in project work to events to speak and network with peers.

Competitive edge

I have heard an argument that we should not give away findings that could help competitors elsewhere in our sector, sometimes scaffolded by belief that as contemporary universities exist in a competitive market they should behave more like private companies. I disagree with the ideological foundation of this argument, but it is logical in acknowledging the reality of a market that has been deliberately created and fostered by government. At Library Camp in 2012, Liz Jolly and I argued that:

Universities have a culture of sharing both internally and externally, and also between those working in the same disciplines across institutions. Furthermore, both within and without higher education, librarianship is a particularly collaborative profession.

(Preater, 2012)

We could remove some of the ideological focus, and simply ask if the investment of time and effort to communicate our work might be less worthwhile than the other things we could spend it on. Above I argue that communication strategies in projects or otherwise should provide you with reusable material, but looking at this strategically I think skipping communication is ultimately detrimental to your library.

To be sure, there are benefits to individual staff in building their professional profile and to the library in being seen as a place ‘where things happen’ and viewed as forward-thinking, including in recruiting and retaining staff. Additionally though, I see an advantage in shaping the speed and direction of thinking as a form of technological leadership in the sector, creating the ‘discursive formation’ (in the sense Foucault describes, for example in part II of The archaeology of knowledge, 1972) of user experience rather than waiting for others to do so.

In communicating our work and engaging our community in discourse we define the content of the discursive formation, of the body of knowledge, in what is still a relatively new and not yet fully-established area. Communication has power in and of itself in bringing in to existence this body of knowledge, and while the practice of user experience is contested, early movers are able to establish how the ‘truth’ of this practice is created and sustained, in our particular context.

For me this idea explains some of the meaning behind practitioners such as Andy Priestner stating, two years ago, that “UX in libraries is a thing now” (2014, emphasis mine), and from experience I would gauge this kind of engagement as putting you between two to three years ahead of libraries that are not doing so.

External validity or being ‘too special’

In this I include disbelief in the external validity of the work, or belief in the necessity of such validity as a precursor to sharing. ‘External validity’ means the extent to which the findings from particular research can be generalized beyond the specific context of the work. I first heard this argument when I was a participant in a library usability study, and naively asked the librarians how they were going to share their results – turns out they weren’t. Some libraries are indeed unusual in themselves, or attract an unusual user base, or both, but there is also a cultural aspect to this problem. Without deliberately maintaining wider awareness, we can lose perspective and end up believing it ourselves: thinking our service is ‘very complex’ or our situation ‘highly unusual’ when it is not particularly so.

My counter-argument is the commonalities between library services and membership mean ideas and concepts are often very transferable. Include some caveats or ‘health warnings’ on your results by all means, but let us weigh things up. Let us include you in our ongoing analysis. This is one reason I value making the theoretical underpinning of the approaches we use in our work explicit when describing what we are doing.

Fear of criticism, lack of confidence in the work

In this reason I include anything in the general space of a wish not to have one’s work critiqued, research methods problematized, or particular choices judged by others in the community. Perfectionism on our part can also play into and amplify this. All engagement in professional discourse includes some measure of risk-taking as there is implicit openness to criticism: speaking at a conference or using a platform or network where replying is easy invites replies. Criticism is tempting as it can be relatively easy for a clever person to say something high-impact. You could do as Ian suggests and start a blog and turn off the comments, but people can (and do) comment on your work elsewhere…

I experienced this recently (with my manager and her manager in the room) when a recording of a user from a piece of UX research was shown that could be interpreted as strongly critical of my project team’s work. This was extremely difficult to accept at the time, but on considered reflection seeing an interpreted piece of feedback was valuable, as it spurred ongoing development and provoked questioning of design choices that had seemed well-founded based on our research.

My recommendation is to trust your judgement if you are on top of your professional development, spend adequate time reflecting-on-action, and test your ideas by taking a critical stance toward your methods and work – including opinions and perspectives from peers. On this subject I recommend Elly’s post on ‘professional confidence’ (O’Brien, 2015). This is a great post that really bears re-reading as part of reflective practice in judging our own expertise and focusing where best to develop skills.

Concluding thoughts

Simon is right in that there is no central location for sharing results of our user experience work. I would love to see something like this created, with low or no barrier to entry so all practitioners could contribute. I think for academic libraries a space on the helibtech wiki could be a good starting point for collaboration. This is a slightly selfish request as in my workplace we maintain a list (a wiki page) of reports, presentations, industry white papers and so on about systems user experience, and used these in developing and shaping ideas for our ongoing research and development.

Absent such a location, I want to encourage practitioners to share their work. We want to hear what you have to say – share your methods, results, and conclusions where you can, as doing so contributes to and shapes discovery user experience as a professional practice.

References

Foucault, M. (1972) The archaeology of knowledge. New York, NY: Vintage.

O’Brien, E. (2015) ‘Professional confidence and “imposter syndrome”‘, Elly O’Brien, June 29. Available at: https://web.archive.org/web/20160921152614/https://ellyob.wordpress.com/2015/06/29/professional-confidence/

Owen, H. (2008) Open space technology: a user’s guide. 3rd edn. San Francisco, CA: Berrett-Koehler.

Preater, A. (2012) ‘Free and Open Source software and cultural change, at Library Camp 2012’, Ginformation Systems, 15 October. Available at: https://www.preater.com/2012/10/15/free-software-and-cultural-change-at-libcampuk12/

Priestner, A. (2014) ‘Why UX in libraries is a thing now’, Business Librarians Association conference, Leicester, 11 July. Available at: http://www.slideshare.net/AndyPriestner1/why-ux-in-libraries-is-a-thing-now-36899649

Discovery at Senate House Libraries: staff focus groups

Introduction

At Senate House Libraries, University of London we’re part way though our project to migrate our library management system (LMS) to Kuali OLE, a Free Software / Open Source library services platform. As a deliberate design choice OLE does not come with an online catalogue for end users, so we are approaching discovery as a work package in our LMS project. To this end I recently ran focus groups for staff to start to specify what goes into a functional specification for our discovery system.

Part way into writing my summary of the high-level requirements I realised I was writing something like a manifesto. I stopped and split this into a separate page in our Confluence wiki, and this is what I wanted to share with you. If you want to see something more like a specification for a library discovery / vertical search system, take a look at Ken Chad’s libtechrfp site on Vertical Search as a starting point.

Running focus groups

From my point of view it’s liberating to start with a clean slate for discovery and not with the limitations of existing library vendor solutions. There are a whole host of implications here, primary for me are not being tied to vendor development roadmaps and technology choices, and not having limitations from the LMS carry through to the discovery layer. So, I wanted to start with an open mind and not presume too much about staff received opinion or staff use of our existing discovery systems.

I asked participants to think about some questions as a prelude to a mix of small- and big-group discussion in a workshop context.

  • What’s valued in our current discovery systems, and what are outstanding problems?
  • What’s missing that should be included in the next discovery system?
  • What’s most important to researchers?
  • What would a good discovery system look like, and how would it behave?

I have to apologize for management-speak of asking what does good look like, but it’s a serious point. It is incredibly hard to describe what a successful new system would be like to use, but these are the things we need to be thinking about rather than say, a list of missing features in Innovative’s Encore Discovery versus their older WebPAC Pro catalogue.

Just to add that yes, testing with library members is to follow. This will include the usual usability testing that accompanies and informs any sensible, well-designed web project and also a more in-depth investigation into user behaviour using ethnographic methods.

“[We want] the moon on a stick”

These group discussions were incredibly productive and featured a good deal of imaginative and daring thinking about what a library catalogue should be and how it should behave. My favourite headline from these small group discussions was a page titled, “the moon on a stick”. I think this is a good starting point: we should think big and aspirational, not small and limited.

Our goal should be full discovery of everything including searching across books, journals, archives, images, and so on in a way that is clear about what you’re searching and provides options to include and exclude different content. In this context the local bibliographic database becomes the biggest of several databases that discovery draws from alongside the archives catalogue, eprints repository, and digital asset management system.

Unfolding complexity

So, how do we provide breadth and depth of discovery without overwhelming the reader with a firehose-like experience of masses of information; and impossible complexity that requires a LIS masters degree to understand?

The key point for us is discovery needs the ability to be as simple or complex as you want to at any given time. We need a way of providing a range of levels of complexity in the same system rather than hiding all the complexity behind an ‘advanced search’ link.

This doesn’t just mean copying from web search, as a single search box is very difficult to get right in the library context. Even if many libraries are going this way nowadays it is very hard to do it well and impossible to please everyone. On the one hand, old school OPACs rely too much on specialist knowledge of how the catalogue works and the structure of the underlying metadata that powers them. On the other, library attempts at single search boxes use keyword indexes that fail to make best use of the complexity and richness of that underlying metadata.

Instead we need an approach that respects the intellectual ability of our readership and the status of our institution, and respects the reader’s conceptual understanding of the library. Our approach should reflect the pride we have in being a library and our professional abilities as librarians, without attempting to turn readers into mini-librarians.

Discovery must include ways of bringing in complexity from a simple starting point, something web search engines can do quite well.

Readers shouldn’t feel they’re starting from a position where they’re telling the library, “I’m stupid”, or “I’m intelligent”. Discovery needs to reconcile providing a simple starting point with surfacing information that may be relevant, but is deep and complex. We know that buried somewhere in the clutter are results that are useful to the reader. The underlying technology may be very complicated but the experience of the system should be the opposite.

We need to meet the needs of different groups of users, or the differing needs of the same user. We know there are primary and secondary uses of our collections for the same person, and a reader may have a different approach in a different context depending on what they are using us for. Staff are users of the catalogue and discovery tools should be easier for staff, too.

User experience

To inform this ‘unfolding complexity’, discovery must bring in user experience concepts and best practise from elsewhere in and outside libraries. The starting point in our thinking here is discovery should be navigable like a modern web site is, and to achieve this it will be designed in a similar way to how our our website was designed. That is, similar approaches and techniques given a library spin that respects our role and the reasons readers choose to sign up for membership.

User experience is key to our web presence but it must be a theme throughout the services we offer: it can’t stop at the library website. There is a gulf between library websites and library catalogues and discovery that we need to bridge. Libraries spend time and money building good websites, but you’ll still find terrible usability when you move over to their catalogues. John Blyberg sums this up as:

The problem lies with inflexible and outdated systems rather than no-one bothering with usability testing or not caring about their readers. Our next discovery system can’t be another weird product from Libraryland that is disconnected from the approach we take when building our websites.

Objecting to my own methodology

I’d like to end on a little reflection about methodology and our subjective views of catalogues based on our experience and familiarity.

Many staff expressed a wish for a “simple”, “uncluttered”, “user friendly”, or “intuitive” interface as contrasted with a “busy”, “cluttered”, or “clunky” interface. I understand these wishes, and I think there is a certain know it when I see it gut feeling about overall user experience that makes something “simple” or “clunky”, but intellectually I know it’s difficult to unpack what these terms mean as they’re so subjective. You might guess a concern here is a term like “user friendly” being used as a proxy for personal preferences or familiarity, and there is a contrast between familiarity of staff traditional information retrieval interfaces versus familiarity of readers with modern websites that I think is important too.

So we do need to dig in! At this point the workshop format breaks down because it’s difficult to employ methods such as close questioning or laddering in a group work situation, you really need a one-to-one interview. However, I tried to unpick this as much as possible in the focus groups without anyone feeling too interrogated. For example, if the Encore feel is “cluttered” what is it that would improve it? What is it about the classic WebPAC Pro or another catalogue that is uncluttered?

I can see a danger here in acting as an interpretive layer or a translator between what someone says and what I think they really mean, and then how I think that should be implemented in a new discovery layer. In hindsight I wish I could’ve sat everyone down one-to-one and ran through some repertory grids to allow for comparisons between different catalogue interfaces based on those constructs such as “clutteredness”.

I had done this in my masters dissertation on library catalogue user experience and found it works really well, once you get over it being an “out-there method” (I smiled in agreement when I read that in Lauren Smith’s recent blog post on fieldwork).

This is something I may try as part of testing options for discovery interfaces such as VuFind and Blacklight.

Acknowledgements

Thanks to Andrea Meyer-Ludowisy (Research Librarian, Western European Languages) and Joe Honywill (Associate Director, Digital Futures) at Senate House Libraries for helpful discussion on the subject.