Tag: culture

Why don’t libraries share the results of UX work?

While conferencing at ELAG 2016, Simon made this suggestion for improving collaboration among libraries doing user experience (UX) work on their systems:

I’ve been thinking on-and-off about the question implied in Simon’s second tweet – the reasons libraries do not generally share results of user experience testing of their systems. Below I discuss some of these, which are drawn from examples from people with first-hand knowledge from academic libraries.

“Not all libraries…”

I know many libraries do share their results and analyses, whether more formally in articles, books or book chapters and conference presentations, or more informally in blog posts, conversations, and social media. I am extremely grateful for any and all sharing of this type.

What about libraries that are doing such work but not reporting it? When I know about such work, though it requires extra effort I have had good success asking colleagues about their findings. People are usually generous with sharing – sometimes surprised in the interest, but delighted to be asked about their work. In my experience the trick to doing this effectively is:

  • Knowing that others have been doing something in the first place, for which an active stance towards professional networking helps enormously.
  • Finding the best way to engage colleagues on their terms, and making it easy for them to help you. If you’re nearby offer to visit, or set up a video chat; ask for an hour of conversation rather than anything in writing or a presentation. Manage the time and conversation effectively: focus on what you are asking about; be explicit about asking for honest opinions; always ask about lessons learned.

Why we don’t share

Time and money

By this I primarily mean a lack of time and money to refine the work into something publishable by traditional routes, such as a peer-reviewed journal or presenting results at conferences. I think this is a huge barrier, and I have no simple answers. There is an huge qualitative difference between libraries that invest time and money in this way and those that do not, which is only really apparent with experience.

There are alternative routes. For me the actuality of dissemination and sharing always trumps the esteem in which a particular route of communication is held. Some of the most effective project communications I have seen are blog posts that summarize progress and demonstrate the momentum and trajectory of the work simply with clear and engaging text. This is the kind of work I or a team member would expect to do as part of internal project communication, so text can be reused with reworking for an outside audience.

For events, workarounds such as unconferences and similar events are a lower-cost approach, but these aren’t without issues. Events with no registration fee still favour those privileged in various ways. Even if one can attend unconferences these don’t always attract an audience with enough specialist knowledge, or are not promoted in a way that will attract such an audience. This may sound contrary to the Open Space Technology principle that “Whoever comes [are] the right people” (Owen, 2008), but in practice OST principles depend on a lot of work behind the scenes and careful management on the day to be effective.

Culture

By this I mean the overall stance of the library towards engagement with their sector and understanding of the value of sharing, as a cultural factor. Having a budget for conference attendance and other development activity is not enough if your workplace does not value this type of professional engagement in and of itself, or does not have confidence in staff ability or value the work. In practice this may manifest more subtly in general discouragement and ‘lack of permission’ from managers in the organization, or not sending staff operationally involved in project work to events to speak and network with peers.

Competitive edge

I have heard an argument that we should not give away findings that could help competitors elsewhere in our sector, sometimes scaffolded by belief that as contemporary universities exist in a competitive market they should behave more like private companies. I disagree with the ideological foundation of this argument, but it is logical in acknowledging the reality of a market that has been deliberately created and fostered by government. At Library Camp in 2012, Liz Jolly and I argued that:

Universities have a culture of sharing both internally and externally, and also between those working in the same disciplines across institutions. Furthermore, both within and without higher education, librarianship is a particularly collaborative profession.

(Preater, 2012)

We could remove some of the ideological focus, and simply ask if the investment of time and effort to communicate our work might be less worthwhile than the other things we could spend it on. Above I argue that communication strategies in projects or otherwise should provide you with reusable material, but looking at this strategically I think skipping communication is ultimately detrimental to your library.

To be sure, there are benefits to individual staff in building their professional profile and to the library in being seen as a place ‘where things happen’ and viewed as forward-thinking, including in recruiting and retaining staff. Additionally though, I see an advantage in shaping the speed and direction of thinking as a form of technological leadership in the sector, creating the ‘discursive formation’ (in the sense Foucault describes, for example in part II of The archaeology of knowledge, 1972) of user experience rather than waiting for others to do so.

In communicating our work and engaging our community in discourse we define the content of the discursive formation, of the body of knowledge, in what is still a relatively new and not yet fully-established area. Communication has power in and of itself in bringing in to existence this body of knowledge, and while the practice of user experience is contested, early movers are able to establish how the ‘truth’ of this practice is created and sustained, in our particular context.

For me this idea explains some of the meaning behind practitioners such as Andy Priestner stating, two years ago, that “UX in libraries is a thing now” (2014, emphasis mine), and from experience I would gauge this kind of engagement as putting you between two to three years ahead of libraries that are not doing so.

External validity or being ‘too special’

In this I include disbelief in the external validity of the work, or belief in the necessity of such validity as a precursor to sharing. ‘External validity’ means the extent to which the findings from particular research can be generalized beyond the specific context of the work. I first heard this argument when I was a participant in a library usability study, and naively asked the librarians how they were going to share their results – turns out they weren’t. Some libraries are indeed unusual in themselves, or attract an unusual user base, or both, but there is also a cultural aspect to this problem. Without deliberately maintaining wider awareness, we can lose perspective and end up believing it ourselves: thinking our service is ‘very complex’ or our situation ‘highly unusual’ when it is not particularly so.

My counter-argument is the commonalities between library services and membership mean ideas and concepts are often very transferable. Include some caveats or ‘health warnings’ on your results by all means, but let us weigh things up. Let us include you in our ongoing analysis. This is one reason I value making the theoretical underpinning of the approaches we use in our work explicit when describing what we are doing.

Fear of criticism, lack of confidence in the work

In this reason I include anything in the general space of a wish not to have one’s work critiqued, research methods problematized, or particular choices judged by others in the community. Perfectionism on our part can also play into and amplify this. All engagement in professional discourse includes some measure of risk-taking as there is implicit openness to criticism: speaking at a conference or using a platform or network where replying is easy invites replies. Criticism is tempting as it can be relatively easy for a clever person to say something high-impact. You could do as Ian suggests and start a blog and turn off the comments, but people can (and do) comment on your work elsewhere…

I experienced this recently (with my manager and her manager in the room) when a recording of a user from a piece of UX research was shown that could be interpreted as strongly critical of my project team’s work. This was extremely difficult to accept at the time, but on considered reflection seeing an interpreted piece of feedback was valuable, as it spurred ongoing development and provoked questioning of design choices that had seemed well-founded based on our research.

My recommendation is to trust your judgement if you are on top of your professional development, spend adequate time reflecting-on-action, and test your ideas by taking a critical stance toward your methods and work – including opinions and perspectives from peers. On this subject I recommend Elly’s post on ‘professional confidence’ (O’Brien, 2015). This is a great post that really bears re-reading as part of reflective practice in judging our own expertise and focusing where best to develop skills.

Concluding thoughts

Simon is right in that there is no central location for sharing results of our user experience work. I would love to see something like this created, with low or no barrier to entry so all practitioners could contribute. I think for academic libraries a space on the helibtech wiki could be a good starting point for collaboration. This is a slightly selfish request as in my workplace we maintain a list (a wiki page) of reports, presentations, industry white papers and so on about systems user experience, and used these in developing and shaping ideas for our ongoing research and development.

Absent such a location, I want to encourage practitioners to share their work. We want to hear what you have to say – share your methods, results, and conclusions where you can, as doing so contributes to and shapes discovery user experience as a professional practice.

References

Foucault, M. (1972) The archaeology of knowledge. New York, NY: Vintage.

O’Brien, E. (2015) ‘Professional confidence and “imposter syndrome”‘, Elly O’Brien, June 29. Available at: https://web.archive.org/web/20160921152614/https://ellyob.wordpress.com/2015/06/29/professional-confidence/

Owen, H. (2008) Open space technology: a user’s guide. 3rd edn. San Francisco, CA: Berrett-Koehler.

Preater, A. (2012) ‘Free and Open Source software and cultural change, at Library Camp 2012’, Ginformation Systems, 15 October. Available at: https://www.preater.com/2012/10/15/free-software-and-cultural-change-at-libcampuk12/

Priestner, A. (2014) ‘Why UX in libraries is a thing now’, Business Librarians Association conference, Leicester, 11 July. Available at: http://www.slideshare.net/AndyPriestner1/why-ux-in-libraries-is-a-thing-now-36899649

Free and Open Source software and cultural change, at Library Camp 2012.

Session underway, participants Tweeting hard. Photograph © Sasha Taylor, used with permission.

On Saturday 13th October I attended the ‘big’ Library Camp 2012 unconference (libcampuk12) at the Signing Tree Conference Centre, Birmingham.

Liz Jolly and I pitched a session on the use of Free and Open Source software in libraries, with a particular focus on discussing the cultural changes or cultural shift needed to develop and sustain the use of in libraries, a typically risk-averse environment. This idea came out of a #uklibchat discussion on Open Source software back in July – thanks to Adrienne Cooper for organizing that.

This session was prepared and facilitated jointly. However when I write “I”, “me”, etc. below I am talking about my own views and experience.

Introduction

In the session asked we use Open Source and Free Software as interchangeable terms that are close enough in meaning that Library Campers could use either term. I realize, and accept, there are objections to doing this. I will refer to FOSS meaning “free and open source software” below.

I explained that Open Source is a pragmatic model of software development where you are allowed access to the source code of the software, however it – and moreover the older concept of Free Software – are underpinned by a philosophy based around respecting users’ freedom and fostering community. Drawing on this we wanted to open with the “four freedoms” in the Free Software Definition (Free Software Foundation, 2012) and how they tie into our professional culture. This list is written by a computer scientist, so famously it starts from zero!

  1. The freedom to run the program, for any purpose (freedom 0).
  2. The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1).
  3. The freedom to redistribute copies so you can help your neighbour (freedom 2).
  4. The freedom to distribute copies of your modified versions to others (freedom 3).

We argued that in higher education and librarianship in particular, these freedoms are broadly aligned to our own professional culture. Universities have a culture of sharing both internally and externally, and also between those working in the same disciplines across institutions. Furthermore, both within and without higher education, librarianship is a particularly collaborative profession.

However, in the broader cultures of higher education we face various problems. In some ways the Four Freedoms are in opposition to the broader organizational culture we work in. We identify points of tension for universities and libraries as collaborative organizations working within power structures that do not necessarily agree with or support a collaborative approach. This is especially the case in our current political and financial climate, where increased competition between institutions will to an extent mitigate against a collaborative culture.

We wondered if perhaps this is mainly a problem within perceived “competitor” institutions, I asked if anyone finds themselves discussing things more openly with colleagues in sectors or institutions that you don’t consider a “threat” or competitor to your own?

FOSS and the culture of libraries and education

Culturally, one starting point is looking at where we still find institutional resistance to FOSS. By this I mean beyond myths like FOSS implying that you have to “build it yourself”, or that “you need to employ programmers”, rather I mean resistance to FOSS as a concept itself. I have seen some of this in my career in further and higher education, but I would say nowadays I think this attitude is dying off. Personally I find myself anticipating resistance to FOSS that simply doesn’t materialize – or in many cases I actually find enthusiastic approval for FOSS.

I am sure our experiences here vary widely – certainly buy-in from senior managers is essential and having one particularly pro- or anti-FOSS manager can make a huge difference either way. Several participants contributed here with examples from their own public sector experience where projects already in development had been scuppered when they were found to be using FOSS, and explained further that they did still spend time knocking down some very old-fashioned arguments about FOSS versus closed source such as needing to “have someone to sue when it all goes wrong”.

There was general agreement that certain sectors are worse at this than others, with libraries in local government and the NHS picked out as particularly difficult: public libraries having to accept whatever systems their authority decides on with limited or no change, and the NHS wanting to play especially safe.

One contradiction in higher education is we have a very long history of using FOSS for the services that underpin our systems (the concept of Free Software was born in higher education, when Richard Stallman was at MIT (Stallman, 2010) but a reluctance to actually use FOSS for campus-wide and departmental systems. What do we mean by this? At a basic level FOSS gives us the building blocks such as web and database servers, programming and scripting languages that we need to create software and services. Few of our IT and systems colleague would object to for example using a FOSS Web server or content management system – but notice how few FOSS library management systems are deployed in the UK, for example.

As a cultural aspect of this we would ask if library and education managers have enough in-depth knowledge of principles of technology, including FOSS, and how it can benefit their organisation to successfully govern projects and to engage with wider community? In universities there is an approach to promoting managers on academic excellence rather than strategic management ability, but these would be the people chairing project boards.

One example here is Moodle, a FOSS virtual learning environment – some argue that while the use of Moodle in higher education is growing, there is a relative lack of engagement with the community – possibly because of the aspect of knowledge culture in higher education of a fear of “exposure”, of not knowing? Oddly, we note that universities can prove not the best learning communities as we don’t like to admit when don’t know things! We also noted at a higher level a culture of “not invented here” exists in UK higher education (most obviously in nationally-funded projects) where we fail to learn from what others have done elsewhere. Or worse in some cases actively dismiss experience elsewhere because it is not our own idea.

How we buy software, and the “library mindset”

At this point I apologized to my fellow Library Campers for I was going to talk about… project management.

I argue the prevailing approach to software procurement and management in libraries works against FOSS. By this, I mean the approach to procurement or ‘invitation to tender’ that includes implicit assumptions that we are purchasing products from a software supplier or “vendor”. That said, we can actually specify and purchase FOSS in this way – what we are doing is buying the same support from a vendor but the product itself is FOSS. In the public sector, that support might require a tendering process over a certain threshold amount. Luke O’Sullivan pointed out here there is a procurement framework for purchasing FOSS systems available at the LibTechRFP wiki.

We noted that very few actually do this. A recent example is Staffordshire University where Dave Parkes and colleagues worked hard to research and justify choosing the Koha Open Source ILS, supported by PTFS Europe (Johnson, 2010). From a systems point of view it’s notable that Koha is quite a traditional LMS, and can go up against other similar systems using the full UK LMS Core Specification.

I would argue systems like the LMS and resource discovery are really about enterprise information, by this we mean they are among our key systems enabling learning and teaching, research, and other business activities in our universities. These systems are therefore business critical and should be viewed as such. However in universities this typically has never been the case. The LMS tends to be seen as a system that is “just there”, in the library – something that doesn’t need too much attention from IT or the broader university.

This ties in with an approach to user acceptance and testing that does not really exist in higher education, but should as the risks are that spreading around bad data between library and other systems in your university can cost you real money. We argued that librarians should look at software projects from a viewpoint of a “testing mentality”: what is it doing? What effect does it have on other parts of the system and on our other data? Librarians as information professionals should have a role to play here. This is not technical, but about information. More broadly Kate Lomax mentioned there’s a lot you can do to contribute without being a developer or a techie – for example documentation.

I argue these points about how we’ve viewed our previous systems and how we  procure them has created something of a “library mindset” in our culture. I feel that as library workers we’ve been complicit in this, and worse in library systems and IT we often take the safe option which can limit our outlook and willingness to risk new things. This is even while we’re very happy using FOSS on own our own computers, or as some participants mentioned “sneaking in” FOSS programs behind the back of unwilling IT departments.

What changes everything in our view are FOSS products in library management systems, discovery, finance, student management, and virtual learning environments that are now becoming mature and mainstream.

Several mainstream examples are:

Conclusion

As a kind of coda we explained that issues around governance, testing methodology, documentation, change management and so on applies to so-called closed-source software just as much as it does to FOSS, and we’d say good project management and software development practice applies regardless of development model use.

As a FOSS developer, Luke emphasized the importance of governance, testing and providing a stable service alongside development. He explained that FOSS is incredibly exciting because you can work with the source code to make changes to suit your local needs – but you risk getting totally carried away. Culturally this represents a real change for library workers not used to this flexibility, so there’s a danger of too much demand on programming time if the assumption is anything about the system can be altered to meet local needs.

The strategic issues here for FOSS projects are around effective management in terms of inclusivity, collaboration and transparency, project governance frameworks, quality and risk management, procurement policies, and change management. These are not specific to an FOSS approach but we argue, essential for such an approach to be successful and specifically to address the traditional weaknesses found in FOSS projects.

Acknowledgement

My thanks to Sharon Penfold, Project Manager at the Bloomsbury LMS for helpful discussion on this subject around procurement, data, testing, and project management.

References

Free Software Foundation (2012) ‘What is Free Software?’ Available at: http://www.gnu.org/philosophy/free-sw.html

Johnson, P. (2010) ‘Staffordshire University chooses Koha for its new library system’. Available at: https://web.archive.org/web/20160320053630/http://blogs.staffs.ac.uk/informationlandscape/2010/12/10/staffordshire-university-chooses-koha-for-its-new-library-system/

Stallman, R.M. (2010) ‘The GNU project’. Available at: http://www.gnu.org/gnu/thegnuproject.html