Discovery at Senate House Libraries: staff focus groups

Introduction

At Senate House Libraries, University of London we’re part way though our project to migrate our library management system (LMS) to Kuali OLE, a Free Software / Open Source library services platform. As a deliberate design choice OLE does not come with an online catalogue for end users, so we are approaching discovery as a work package in our LMS project. To this end I recently ran focus groups for staff to start to specify what goes into a functional specification for our discovery system.

Part way into writing my summary of the high-level requirements I realised I was writing something like a manifesto. I stopped and split this into a separate page in our Confluence wiki, and this is what I wanted to share with you. If you want to see something more like a specification for a library discovery / vertical search system, take a look at Ken Chad’s libtechrfp site on Vertical Search as a starting point.

Running focus groups

From my point of view it’s liberating to start with a clean slate for discovery and not with the limitations of existing library vendor solutions. There are a whole host of implications here, primary for me are not being tied to vendor development roadmaps and technology choices, and not having limitations from the LMS carry through to the discovery layer. So, I wanted to start with an open mind and not presume too much about staff received opinion or staff use of our existing discovery systems.

I asked participants to think about some questions as a prelude to a mix of small- and big-group discussion in a workshop context.

  • What’s valued in our current discovery systems, and what are outstanding problems?
  • What’s missing that should be included in the next discovery system?
  • What’s most important to researchers?
  • What would a good discovery system look like, and how would it behave?

I have to apologize for management-speak of asking what does good look like, but it’s a serious point. It is incredibly hard to describe what a successful new system would be like to use, but these are the things we need to be thinking about rather than say, a list of missing features in Innovative’s Encore Discovery versus their older WebPAC Pro catalogue.

Just to add that yes, testing with library members is to follow. This will include the usual usability testing that accompanies and informs any sensible, well-designed web project and also a more in-depth investigation into user behaviour using ethnographic methods.

“[We want] the moon on a stick”

These group discussions were incredibly productive and featured a good deal of imaginative and daring thinking about what a library catalogue should be and how it should behave. My favourite headline from these small group discussions was a page titled, “the moon on a stick”. I think this is a good starting point: we should think big and aspirational, not small and limited.

Our goal should be full discovery of everything including searching across books, journals, archives, images, and so on in a way that is clear about what you’re searching and provides options to include and exclude different content. In this context the local bibliographic database becomes the biggest of several databases that discovery draws from alongside the archives catalogue, eprints repository, and digital asset management system.

Unfolding complexity

So, how do we provide breadth and depth of discovery without overwhelming the reader with a firehose-like experience of masses of information; and impossible complexity that requires a LIS masters degree to understand?

The key point for us is discovery needs the ability to be as simple or complex as you want to at any given time. We need a way of providing a range of levels of complexity in the same system rather than hiding all the complexity behind an ‘advanced search’ link.

This doesn’t just mean copying from web search, as a single search box is very difficult to get right in the library context. Even if many libraries are going this way nowadays it is very hard to do it well and impossible to please everyone. On the one hand, old school OPACs rely too much on specialist knowledge of how the catalogue works and the structure of the underlying metadata that powers them. On the other, library attempts at single search boxes use keyword indexes that fail to make best use of the complexity and richness of that underlying metadata.

Instead we need an approach that respects the intellectual ability of our readership and the status of our institution, and respects the reader’s conceptual understanding of the library. Our approach should reflect the pride we have in being a library and our professional abilities as librarians, without attempting to turn readers into mini-librarians.

Discovery must include ways of bringing in complexity from a simple starting point, something web search engines can do quite well.

Readers shouldn’t feel they’re starting from a position where they’re telling the library, “I’m stupid”, or “I’m intelligent”. Discovery needs to reconcile providing a simple starting point with surfacing information that may be relevant, but is deep and complex. We know that buried somewhere in the clutter are results that are useful to the reader. The underlying technology may be very complicated but the experience of the system should be the opposite.

We need to meet the needs of different groups of users, or the differing needs of the same user. We know there are primary and secondary uses of our collections for the same person, and a reader may have a different approach in a different context depending on what they are using us for. Staff are users of the catalogue and discovery tools should be easier for staff, too.

User experience

To inform this ‘unfolding complexity’, discovery must bring in user experience concepts and best practise from elsewhere in and outside libraries. The starting point in our thinking here is discovery should be navigable like a modern web site is, and to achieve this it will be designed in a similar way to how our our website was designed. That is, similar approaches and techniques given a library spin that respects our role and the reasons readers choose to sign up for membership.

User experience is key to our web presence but it must be a theme throughout the services we offer: it can’t stop at the library website. There is a gulf between library websites and library catalogues and discovery that we need to bridge. Libraries spend time and money building good websites, but you’ll still find terrible usability when you move over to their catalogues. John Blyberg sums this up as:

The problem lies with inflexible and outdated systems rather than no-one bothering with usability testing or not caring about their readers. Our next discovery system can’t be another weird product from Libraryland that is disconnected from the approach we take when building our websites.

Objecting to my own methodology

I’d like to end on a little reflection about methodology and our subjective views of catalogues based on our experience and familiarity.

Many staff expressed a wish for a “simple”, “uncluttered”, “user friendly”, or “intuitive” interface as contrasted with a “busy”, “cluttered”, or “clunky” interface. I understand these wishes, and I think there is a certain know it when I see it gut feeling about overall user experience that makes something “simple” or “clunky”, but intellectually I know it’s difficult to unpack what these terms mean as they’re so subjective. You might guess a concern here is a term like “user friendly” being used as a proxy for personal preferences or familiarity, and there is a contrast between familiarity of staff traditional information retrieval interfaces versus familiarity of readers with modern websites that I think is important too.

So we do need to dig in! At this point the workshop format breaks down because it’s difficult to employ methods such as close questioning or laddering in a group work situation, you really need a one-to-one interview. However, I tried to unpick this as much as possible in the focus groups without anyone feeling too interrogated. For example, if the Encore feel is “cluttered” what is it that would improve it? What is it about the classic WebPAC Pro or another catalogue that is uncluttered?

I can see a danger here in acting as an interpretive layer or a translator between what someone says and what I think they really mean, and then how I think that should be implemented in a new discovery layer. In hindsight I wish I could’ve sat everyone down one-to-one and ran through some repertory grids to allow for comparisons between different catalogue interfaces based on those constructs such as “clutteredness”.

I had done this in my masters dissertation on library catalogue user experience and found it works really well, once you get over it being an “out-there method” (I smiled in agreement when I read that in Lauren Smith’s recent blog post on fieldwork).

This is something I may try as part of testing options for discovery interfaces such as VuFind and Blacklight.

Acknowledgements

Thanks to Andrea Meyer-Ludowisy (Research Librarian, Western European Languages) and Joe Honywill (Associate Director, Digital Futures) at Senate House Libraries for helpful discussion on the subject.

Towards ethnographies of the next-gen catalogue user

This is the third post in a series exploring user understanding of next-generation catalogues:

Talk

This is posted to coincide with the ChrisMash Mashed Library event organised by Gary Green in London on December 3rd. I spoke about the outcomes of an investigation into user experience and understanding of the next-gen catalogue and next steps we’re taking at Senate House Library. Not very Christmassy I admit…

@preater's presentation on flickr
‘@preater’s presentation’ on Flickr by Paul Stainthorp, license CC-BY-SA.

Slides from this talk are now available:

My slides were kept deliberately simple – it was presented in a pub on a flat screen TV! Notes are included to explain things further. Please get in touch if you want to ask anything about this.

Starting point

We implemented Encore from Innovative Interfaces in June to run alongside and partly replace the older WebPAC Pro catalogue, also from Innovative. Our Encore instance is here; the search I used in my talk was ‘industrial workers of the world‘.

Ahead of implementing we didn’t have much idea about how library users would understand this type of catalogue, so for my masters dissertation I had a look at this using various qualitative methods:

  • Usability-test style cognitive walk-throughs, done almost as a warm-up but providing lots of interesting data. As an aside I think every library should be doing this with their catalogue – it is so quick and easy to do.
  • A semi-structured interview using repertory grid technique. This was very good for comparing what my participants really thought of each type of catalogue.

Key findings

To summarise very briefly:

A Web-like catalogue encourages Web-like behaviour

Putting readers in front of a catalogue interface that looks and behaves like a Web search engine results in behaviours closer to a Web search engine than traditional information retrieval.

By this I mean:

  • A tendency to scan and skim-read Web pages quickly, concentrate on titles.
  • A process of iterative searching based on using a few keywords and then reworking the search over again based on what’s found on the results page.
  • Trust in the relevancy ranking of the catalogue; an expectation that the catalogue should be tolerant of small errors or typos via ‘did you mean…?’ suggestions.
  • The tendency to ‘satisfice’, meaning making do with results that seem good enough for the purpose rather than searching exhaustively.
  • The view that a search queries are an ongoing process, not something that should produce a single perfect set of results.

Caution: this is based on coding qualitative data from nine people and is not intended to be absolute or apply to every user. I found strongly contrasting opinions of the catalogue with an overall tendency for younger readers to take to the new interface much more easily.

The method I used was inductive, that is developed from analysis of what I observed. I really did not expect this ahead of time.

Using our catalogue is an affective experience

I found there was a strongly affective or emotional response to use of our catalogue beyond what you’d think you might get from using a mere lookup tool. The response was about more than just the catalogue being pleasant to use or familiar from other sites.

This was very interesting because I do not see why a library catalogue should not be a joy to use. Why should library catalogues be a painful experience where you have to “pay your dues”? Even if we changed nothing else behind the scenes and made the catalogue more attractive, you could argue this would improve things because we tend to believe more attractive things work better because they’re more enjoyable. Here I am paraphrasing from Don Norman (2004).

Next steps

Usability testing gets us so far, but as I’ve said previously in an artificial “lab” setting it does not produce natural behaviour. That’s a problem because we don’t get to see the reader’s true understanding emerge. We don’t get to see how they really behave in the library when using the catalogue.

I went fairly far in comparing systems – WebPAC Pro versus Encore – but what anchored that testing was the old catalogue. Having implemented the new catalogue and positioned it fairly aggressively as the default interface I wanted to dig deeper and better understand how the catalogue fits in to the reader’s experience of doing research at Senate House Library.

Think about the experience of library use: the reader comes in and experiences an entire “ecology”: the physical building; print book and journal collections; e-resources; the library staff; our catalogues and Web sites. I wanted to better understand how readers experience the catalogue in this context rather than just thinking about it in systems terms as a tool for looking items up that is used with a particular rate of error or success.

Towards ethnographies of the next-gen catalogue user

What we’re going to do is borrow techniques from anthropology to do ethnography in the library. This means studying and observing readers in their habitat: as they work in the library and do their research.

The outcomes I want from this are fairly “soft”, based around our staff knowing the readers better. What I want to know is: how can the library better support our readers’ use of the catalogue and improve their experience of Senate House Library? This is fundamental: I think without better understanding our readers use of our catalogues, we can’t start to improve what we do and provide a better service.

Properly speaking this is more a case of “borrowing ethnographic methods” than “doing ethnography”. This is OK as the methods aren’t owned by one field of social science, as Harry Wolcott (2008) says they “belong to all of us”.

Practically, what want to do is use a battery of techniques including semi-structured interviews, observation, and close questioning to generate data that will allow development of theory from that data as it is analysed qualitatively. This is a grounded theory approach. The actual work will likely be small “micro ethnographies” done over a period of some months in the library.

Examples

In my talk I mentioned some examples of ethnographic research done in libraries, these are:

  • Investigating user understanding of the library Web site – University of North Carolina at Charlotte (Wu and Lanclos, 2011)
  • Looking at how the physical library space is used – Loughborough University (Bryant, 2009)
  • Ethnographies of subject librarian’s reference work – Hewlett Packard Library and Apple Research Library (Nardi and O’Day, 1999)
  • The ERIAL (Ethnographic Research in Illinois Academic Libraries) project which has produced various outputs and has an excellent toolkit telling you how to do it (Asher and Miller, 2011)

References

Asher, A. and Miller, S. (2011) ‘So you want to do anthropology in your library?’ Available at: http://www.erialproject.org/wp-content/uploads/2011/03/Toolkit-3.22.11.pdf

Bryant, J. (2009) ‘What are students doing in our library? Ethnography as a method of exploring library user behaviour’, Library and Information Research, 33 (103), pp. 3-9.

Nardi, B.A. and O’Day, V.L. (1999) Information ecologies. London: MIT Press.

Norman, D.A. (2004) Emotional design. New York, NY: Basic Books.

Wolcott, H.F. (2008) Ethnography: a way of seeing. 2nd edn. Plymouth: AltaMira.

Wu, S.K. and Lanclos, D. (2011) ‘Re-imagining the users’ experience: an ethnographic approach to web usability and space design’, Reference Services Review, 39 (3), pp. 369-389.