Towards ethnographies of the next-gen catalogue user

This is the third post in a series exploring user understanding of next-generation catalogues:

Talk

This is posted to coincide with the ChrisMash Mashed Library event organised by Gary Green in London on December 3rd. I spoke about the outcomes of an investigation into user experience and understanding of the next-gen catalogue and next steps we’re taking at Senate House Library. Not very Christmassy I admit…

@preater's presentation on flickr — ‘@preater’s presentation’ on Flickr by Paul Stainthorp, license CC-BY-SA.

Slides from this talk are now available:

User experience of the next-gen catalogue from Andrew Preater

My slides were kept deliberately simple – it was presented in a pub on a flat screen TV! Notes are included to explain things further. Please get in touch if you want to ask anything about this.

Starting point

We implemented Encore from Innovative Interfaces in June to run alongside and partly replace the older WebPAC Pro catalogue, also from Innovative. Our Encore instance is here; the search I used in my talk was ‘industrial workers of the world‘.

Ahead of implementing we didn’t have much idea about how library users would understand this type of catalogue, so for my masters dissertation I had a look at this using various qualitative methods:

Usability-test style cognitive walk-throughs, done almost as a warm-up but providing lots of interesting data. As an aside I think every library should be doing this with their catalogue – it is so quick and easy to do.
A semi-structured interview using repertory grid technique. This was very good for comparing what my participants really thought of each type of catalogue.

Key findings

To summarise very briefly:

A Web-like catalogue encourages Web-like behaviour

Putting readers in front of a catalogue interface that looks and behaves like a Web search engine results in behaviours closer to a Web search engine than traditional information retrieval.

By this I mean:

A tendency to scan and skim-read Web pages quickly, concentrate on titles.
A process of iterative searching based on using a few keywords and then reworking the search over again based on what’s found on the results page.
Trust in the relevancy ranking of the catalogue; an expectation that the catalogue should be tolerant of small errors or typos via ‘did you mean…?’ suggestions.
The tendency to ‘satisfice’, meaning making do with results that seem good enough for the purpose rather than searching exhaustively.
The view that a search queries are an ongoing process, not something that should produce a single perfect set of results.

Caution: this is based on coding qualitative data from nine people and is not intended to be absolute or apply to every user. I found strongly contrasting opinions of the catalogue with an overall tendency for younger readers to take to the new interface much more easily.

The method I used was inductive, that is developed from analysis of what I observed. I really did not expect this ahead of time.

Using our catalogue is an affective experience

I found there was a strongly affective or emotional response to use of our catalogue beyond what you’d think you might get from using a mere lookup tool. The response was about more than just the catalogue being pleasant to use or familiar from other sites.

This was very interesting because I do not see why a library catalogue should not be a joy to use. Why should library catalogues be a painful experience where you have to “pay your dues”? Even if we changed nothing else behind the scenes and made the catalogue more attractive, you could argue this would improve things because we tend to believe more attractive things work better because they’re more enjoyable. Here I am paraphrasing from Don Norman (2004).

Next steps

Usability testing gets us so far, but as I’ve said previously in an artificial “lab” setting it does not produce natural behaviour. That’s a problem because we don’t get to see the reader’s true understanding emerge. We don’t get to see how they really behave in the library when using the catalogue.

I went fairly far in comparing systems – WebPAC Pro versus Encore – but what anchored that testing was the old catalogue. Having implemented the new catalogue and positioned it fairly aggressively as the default interface I wanted to dig deeper and better understand how the catalogue fits in to the reader’s experience of doing research at Senate House Library.

Think about the experience of library use: the reader comes in and experiences an entire “ecology”: the physical building; print book and journal collections; e-resources; the library staff; our catalogues and Web sites. I wanted to better understand how readers experience the catalogue in this context rather than just thinking about it in systems terms as a tool for looking items up that is used with a particular rate of error or success.

Towards ethnographies of the next-gen catalogue user

What we’re going to do is borrow techniques from anthropology to do ethnography in the library. This means studying and observing readers in their habitat: as they work in the library and do their research.

The outcomes I want from this are fairly “soft”, based around our staff knowing the readers better. What I want to know is: how can the library better support our readers’ use of the catalogue and improve their experience of Senate House Library? This is fundamental: I think without better understanding our readers use of our catalogues, we can’t start to improve what we do and provide a better service.

Properly speaking this is more a case of “borrowing ethnographic methods” than “doing ethnography”. This is OK as the methods aren’t owned by one field of social science, as Harry Wolcott (2008) says they “belong to all of us”.

Practically, what want to do is use a battery of techniques including semi-structured interviews, observation, and close questioning to generate data that will allow development of theory from that data as it is analysed qualitatively. This is a grounded theory approach. The actual work will likely be small “micro ethnographies” done over a period of some months in the library.

Examples

In my talk I mentioned some examples of ethnographic research done in libraries, these are:

Investigating user understanding of the library Web site – University of North Carolina at Charlotte (Wu and Lanclos, 2011)
Looking at how the physical library space is used – Loughborough University (Bryant, 2009)
Ethnographies of subject librarian’s reference work – Hewlett Packard Library and Apple Research Library (Nardi and O’Day, 1999)
The ERIAL (Ethnographic Research in Illinois Academic Libraries) project which has produced various outputs and has an excellent toolkit telling you how to do it (Asher and Miller, 2011)

References

Asher, A. and Miller, S. (2011) ‘So you want to do anthropology in your library?’ Available at: http://www.erialproject.org/wp-content/uploads/2011/03/Toolkit-3.22.11.pdf

Bryant, J. (2009) ‘What are students doing in our library? Ethnography as a method of exploring library user behaviour’, Library and Information Research, 33 (103), pp. 3-9.

Nardi, B.A. and O’Day, V.L. (1999) Information ecologies. London: MIT Press.

Norman, D.A. (2004) Emotional design. New York, NY: Basic Books.

Wolcott, H.F. (2008) Ethnography: a way of seeing. 2nd edn. Plymouth: AltaMira.

Wu, S.K. and Lanclos, D. (2011) ‘Re-imagining the users’ experience: an ethnographic approach to web usability and space design’, Reference Services Review, 39 (3), pp. 369-389.

Tracking usage of QR codes by smartphone users

Recently I added QR codes to the Senate House Library catalogue hoping to improve user experience for smartphone users. In true “dogfooding” style I have made a lot of use of it myself, but I need to see more data. One thing was missing was any analytics tracking for smartphone users recording these codes and following them into the mobile catalogue.

Tracking QR code use

I realised I could do this by adding parameters to the QR code URLs that would be picked up by Google Analytics: Analytics Help – How do I tag my links?

I tweaked the Javascript generating my markup to insert the required parameters utm_source, utm_medium, and utm_campaign:

Campaign Source (utm_source): webpac
Campaign Medium (utm_medium): qr
Campaign Name (utm_campaign): mobile

Values used can be whatever you want, I’ve tried to keep them short but meaningful. You can then track visitors under Traffic Sources – Sources – Campaigns in Google Analytics.

Adding complexity

Being able to track use of this service is very helpful, but providing more information in the QR code increases the complexity of the code and makes it more “dense” and “busy”. Though I’ve had no problems with this on my phone this could cause problems for older smartphones with lower-resolution cameras. My quick solution is just to bump up the size a bit as this makes the QR code easier for the smartphone to read.

For comparison including a longer URL takes you from this:

http://m.ulrls.lon.ac.uk/record=b2941947~S24

To this:

http://m.ulrls.lon.ac.uk/record=b2941947~S24?utm_source=webpac&utm_medium=qr&utm_campaign=mobile

As these QR codes are meant for a mobile phone camera to direct a Web browser to a page, I thought the URL itself need not be “cool”, bookmarkable, or even very human readable. One option is to shorten the URL as it is generated and encode that. Here is the result of shortening with our own shortening service. (Your library does have its own URL shortening service, right?)

http://senatehou.se/b2941947

That is much nicer! Better than the original link to the mobile catalogue, even.

How to do it

Actually achieving this result in the Millennium ‘classic catalog’ / WebPAC is more difficult. To do the extra step of shortening the URL you will probably need to use the API from your shortening service to first shorten the URL, then you can generate your QR code image. On the WebPAC you’re going to need to do this in Javascript.

In the WebPAC I knew I will run into problems with insecure scripts because our shortener doesn’t have an SSL certificate yet, so this will be just an example. I was able to do it using jQuery and this jQuery plugin jquery-urlshortener.js by James Robert combined with bit.ly as a shortener.

First add jQuery and jquery-urlshortener.js to your INSERTTAG_INHEAD wwwoption. I put a local copy of jQuery on our server for testing:

<script language="JavaScript" type="text/javascript" src="/screens/qrcode.js"></script><script type='text/javascript' src='/screens/jquery.js'></script><script type='text/javascript' src='/screens/jquery.urlshortener.js'></script>

Add your bit.ly API key and username to jquery-urlshortener.js.

Update the qrcode.js to request a short URL from bit.ly using jQuery and use that to generate a QR code using the Google Chart API:

function linkto_catalog_qr() {
    var qrairpacstub = "http://m.ulrls.lon.ac.uk/record=";
    var qrrecordlink = document.getElementById("recordnum").getAttribute("href");
    var str = qrrecordlink.indexOf("=");
    var qrrecordid = qrrecordlink.substr(str + 1);
    var longurl = '' + qrairpacstub + qrrecordid + '?utm_source=webpac%26utm_medium=qr%26utm_campaign=mobile';
    $.shortenUrl(longurl, function (short_url) {
        document.getElementById('qrcode').innerHTML = '<img src="http://chart.apis.google.com/chart?cht=qr&chs=130x130&chl=' + short_url + '" alt="QR code for this record" title="QR code for this record" /><br><a href="http://www.senatehouselibrary.ac.uk/library/helpandsupport/qrcodes.shtml">What's this?</a>';
    });
}

This works as a proof of concept and is enabled on our test / staging port, for example:

I am not so keen on sending thousands of requests to bit.ly every day and would prefer to use our own shortening service so I’m not making this live just yet.

Thoughts on usability testing the next-gen catalogue

This is the second post in a series exploring user understanding of next-generation catalogues:

What I like about usability testing

I have always found usability testing library systems enjoyable – as a participant, facilitator, and manager – and gotten useful things out of it. My preferred style is Steve Krug’s “Lost our lease, going-out-of-business-sale usability testing” from Don’t make me think (2006) with about five subjects and a very focused idea about what I wanted to get out of the process. By that I mean specific problems that needed user feedback to inform our judgments.

What I like best about this method is it represents effective action you can take quickly on a shoestring. You can short-circuit the endless librarians-around-a-table discussions you can get into about Web stuff: let’s test this out rather than just talking about it! I have defended using this method with small groups, as even testing a few users tells you something about what your users are actually doing whereas testing no-one tells you nothing at all. In writing that I realised I was paraphrasing Jakob Nielsen, “Zero users give zero insights”.

We’ll likely employ this method when we rework the Senate House Library Web site next year.

What I don’t

I think there are some problems with this style of testing as a methodology so have been looking into other methods for investigating Encore.

My main problem is the artificial nature of the test. Putting a person in your usability “lab” with a camera recording and asking them to do various tasks does not produce a natural experience of using your library catalogue. Your methods of observing the test will alter the users behaviour: these are observer effects you cannot hope to control for. In my dissertation interviews I tried to temper this by focusing on subject searching, browsing, and exploration of the next-generation catalogue interface rather than asking for subjects to complete tasks. I used a form of close questioning to explore participants’ understanding of Encore. This relies on asking probing questions along the lines of:

How?
Why?
What?
What if?

Ultimately this is based on a contextual inquiry approach described by Beyer and Holtzblatt in Contextual design (1998), but done with the understanding that it was taking place in an artificial environment not “the field”.

In truth the usability testing-style part of the investigation was meant as a warm-up towards comparisons between two or more catalogues using the repertory grid technique. I thought this worked reasonably well. The usability test section yielded up a good deal of qualitative data and certainly worked to get participants to the right frame of mind for grid construction.

It also produced useful results about for tweaks we could make to improve Encore as a better fit to readers’ expectations of a library catalogue. That is, it worked as usability testing.

However as I did the work I was aware of the artificial nature of the process affecting how my subjects behaved and their problems engaging with the process in anything like a natural way. The cognitive walkthrough style is difficult on two levels: it feels odd and a bit embarrassing to do it as a subject, but also it makes you think about what you are doing and how you should express yourself which affects your behaviour. Several participants picked up on this during their interviews and criticised it.

I’ve found our readers experience of the catalogue is deeply affective, and think we need to dig deeper into that affective layer to understand the user experience. I think ethnographic methods like the contextual inquiry approach is the way to go here, and will return to this in my next post.

Final point. I know our vendor has done their own usability testing on the Encore interface including informing changes to the current look and feel, in use on our catalogue. I have no reason to doubt its effectiveness or rigour. We could do usability testing of Encore, but I doubt we would add much beyond what the vendor already knows.

References

Beyer, H. and Holtzblatt, K. (1998) Contextual design. London: Morgan Kaufmann.

Krug, S. (2006) Don’t make me think. 2nd edn. Berkeley, CA: New Riders.