Why don’t libraries share the results of UX work?

While conferencing at ELAG 2016, Simon made this suggestion for improving collaboration among libraries doing user experience (UX) work on their systems:

I’ve been thinking on-and-off about the question implied in Simon’s second tweet – the reasons libraries do not generally share results of user experience testing of their systems. Below I discuss some of these, which are drawn from examples from people with first-hand knowledge from academic libraries.

“Not all libraries…”

I know many libraries do share their results and analyses, whether more formally in articles, books or book chapters and conference presentations, or more informally in blog posts, conversations, and social media. I am extremely grateful for any and all sharing of this type.

What about libraries that are doing such work but not reporting it? When I know about such work, though it requires extra effort I have had good success asking colleagues about their findings. People are usually generous with sharing – sometimes surprised in the interest, but delighted to be asked about their work. In my experience the trick to doing this effectively is:

  • Knowing that others have been doing something in the first place, for which an active stance towards professional networking helps enormously.
  • Finding the best way to engage colleagues on their terms, and making it easy for them to help you. If you’re nearby offer to visit, or set up a video chat; ask for an hour of conversation rather than anything in writing or a presentation. Manage the time and conversation effectively: focus on what you are asking about; be explicit about asking for honest opinions; always ask about lessons learned.

Why we don’t share

Time and money

By this I primarily mean a lack of time and money to refine the work into something publishable by traditional routes, such as a peer-reviewed journal or presenting results at conferences. I think this is a huge barrier, and I have no simple answers. There is an huge qualitative difference between libraries that invest time and money in this way and those that do not, which is only really apparent with experience.

There are alternative routes. For me the actuality of dissemination and sharing always trumps the esteem in which a particular route of communication is held. Some of the most effective project communications I have seen are blog posts that summarize progress and demonstrate the momentum and trajectory of the work simply with clear and engaging text. This is the kind of work I or a team member would expect to do as part of internal project communication, so text can be reused with reworking for an outside audience.

For events, workarounds such as unconferences and similar events are a lower-cost approach, but these aren’t without issues. Events with no registration fee still favour those privileged in various ways. Even if one can attend unconferences these don’t always attract an audience with enough specialist knowledge, or are not promoted in a way that will attract such an audience. This may sound contrary to the Open Space Technology principle that “Whoever comes [are] the right people” (Owen, 2008), but in practice OST principles depend on a lot of work behind the scenes and careful management on the day to be effective.

Culture

By this I mean the overall stance of the library towards engagement with their sector and understanding of the value of sharing, as a cultural factor. Having a budget for conference attendance and other development activity is not enough if your workplace does not value this type of professional engagement in and of itself, or does not have confidence in staff ability or value the work. In practice this may manifest more subtly in general discouragement and ‘lack of permission’ from managers in the organization, or not sending staff operationally involved in project work to events to speak and network with peers.

Competitive edge

I have heard an argument that we should not give away findings that could help competitors elsewhere in our sector, sometimes scaffolded by belief that as contemporary universities exist in a competitive market they should behave more like private companies. I disagree with the ideological foundation of this argument, but it is logical in acknowledging the reality of a market that has been deliberately created and fostered by government. At Library Camp in 2012, Liz Jolly and I argued that:

Universities have a culture of sharing both internally and externally, and also between those working in the same disciplines across institutions. Furthermore, both within and without higher education, librarianship is a particularly collaborative profession.

(Preater, 2012)

We could remove some of the ideological focus, and simply ask if the investment of time and effort to communicate our work might be less worthwhile than the other things we could spend it on. Above I argue that communication strategies in projects or otherwise should provide you with reusable material, but looking at this strategically I think skipping communication is ultimately detrimental to your library.

To be sure, there are benefits to individual staff in building their professional profile and to the library in being seen as a place ‘where things happen’ and viewed as forward-thinking, including in recruiting and retaining staff. Additionally though, I see an advantage in shaping the speed and direction of thinking as a form of technological leadership in the sector, creating the ‘discursive formation’ (in the sense Foucault describes, for example in part II of The archaeology of knowledge, 1972) of user experience rather than waiting for others to do so.

In communicating our work and engaging our community in discourse we define the content of the discursive formation, of the body of knowledge, in what is still a relatively new and not yet fully-established area. Communication has power in and of itself in bringing in to existence this body of knowledge, and while the practice of user experience is contested, early movers are able to establish how the ‘truth’ of this practice is created and sustained, in our particular context.

For me this idea explains some of the meaning behind practitioners such as Andy Priestner stating, two years ago, that “UX in libraries is a thing now” (2014, emphasis mine), and from experience I would gauge this kind of engagement as putting you between two to three years ahead of libraries that are not doing so.

External validity or being ‘too special’

In this I include disbelief in the external validity of the work, or belief in the necessity of such validity as a precursor to sharing. ‘External validity’ means the extent to which the findings from particular research can be generalized beyond the specific context of the work. I first heard this argument when I was a participant in a library usability study, and naively asked the librarians how they were going to share their results – turns out they weren’t. Some libraries are indeed unusual in themselves, or attract an unusual user base, or both, but there is also a cultural aspect to this problem. Without deliberately maintaining wider awareness, we can lose perspective and end up believing it ourselves: thinking our service is ‘very complex’ or our situation ‘highly unusual’ when it is not particularly so.

My counter-argument is the commonalities between library services and membership mean ideas and concepts are often very transferable. Include some caveats or ‘health warnings’ on your results by all means, but let us weigh things up. Let us include you in our ongoing analysis. This is one reason I value making the theoretical underpinning of the approaches we use in our work explicit when describing what we are doing.

Fear of criticism, lack of confidence in the work

In this reason I include anything in the general space of a wish not to have one’s work critiqued, research methods problematized, or particular choices judged by others in the community. Perfectionism on our part can also play into and amplify this. All engagement in professional discourse includes some measure of risk-taking as there is implicit openness to criticism: speaking at a conference or using a platform or network where replying is easy invites replies. Criticism is tempting as it can be relatively easy for a clever person to say something high-impact. You could do as Ian suggests and start a blog and turn off the comments, but people can (and do) comment on your work elsewhere…

I experienced this recently (with my manager and her manager in the room) when a recording of a user from a piece of UX research was shown that could be interpreted as strongly critical of my project team’s work. This was extremely difficult to accept at the time, but on considered reflection seeing an interpreted piece of feedback was valuable, as it spurred ongoing development and provoked questioning of design choices that had seemed well-founded based on our research.

My recommendation is to trust your judgement if you are on top of your professional development, spend adequate time reflecting-on-action, and test your ideas by taking a critical stance toward your methods and work – including opinions and perspectives from peers. On this subject I recommend Elly’s post on ‘professional confidence’ (O’Brien, 2015). This is a great post that really bears re-reading as part of reflective practice in judging our own expertise and focusing where best to develop skills.

Concluding thoughts

Simon is right in that there is no central location for sharing results of our user experience work. I would love to see something like this created, with low or no barrier to entry so all practitioners could contribute. I think for academic libraries a space on the helibtech wiki could be a good starting point for collaboration. This is a slightly selfish request as in my workplace we maintain a list (a wiki page) of reports, presentations, industry white papers and so on about systems user experience, and used these in developing and shaping ideas for our ongoing research and development.

Absent such a location, I want to encourage practitioners to share their work. We want to hear what you have to say – share your methods, results, and conclusions where you can, as doing so contributes to and shapes discovery user experience as a professional practice.

References

Foucault, M. (1972) The archaeology of knowledge. New York, NY: Vintage.

O’Brien, E. (2015) ‘Professional confidence and “imposter syndrome”‘, Elly O’Brien, June 29. Available at: https://ellyob.wordpress.com/2015/06/29/professional-confidence/

Owen, H. (2008) Open space technology: a user’s guide. 3rd edn. San Francisco, CA: Berrett-Koehler.

Preater, A. (2012) ‘Free and Open Source software and cultural change, at Library Camp 2012’, Ginformation Systems, 15 October. Available at: https://www.preater.com/2012/10/15/free-software-and-cultural-change-at-libcampuk12/

Priestner, A. (2014) ‘Why UX in libraries is a thing now’, Business Librarians Association conference, Leicester, 11 July. Available at: http://www.slideshare.net/AndyPriestner1/why-ux-in-libraries-is-a-thing-now-36899649

Feelings on critical systems praxis, for #critlib

I wrote this to reply to the ‘recommended writing’ suggestion by Kevin Seeber for the December 15 2015 #critlib chat on #feelings. I very likely won’t be at the chat itself, so I haven’t quite addressed the questions as posed.

Why practice critical librarianship?

I understand #critlib to mean critical theory approaches to library work as a whole so will say something about why I find this interesting, and why I think it is important and useful in the undertheorized area of library systems. In this piece I refer to #critlib, the idea of critical librarianship, and the critically-informed stance of Radical Librarians Collective (RLC) interchangeably.

I found out about #critlib chats when I was already aware of #critLIS, a critical theory reading group at the University of Sheffield iSchool. The students involved in #critLIS tweeted some of their discussions and I read the things they talked about. I could only join in minimally but what I saw of their discussion seemed thrilling in how they brought social critique to information work, but also intimidatingly erudite in the works referenced.

Later I attended a discussion on critical theory in LIS at an RLC event in London. On reflection at RLC I think the whole critical theory ball of wax was explained in problem-posing terms, which was both more useful, more challenging, and achieved more than a lecture on the same subject would have. At that session Kevin and Lauren didn’t offer grand solutions, but instead many new lenses with which to interpret and inspect practice and some practical ideas too (Smith, 2014). I realized I had some reading to do; fortunately, critical librarians (by all means imagine the ‘smiling cat face with heart-shaped eyes’ emoji here) are inevitably generous with suggestions.

Reading a text is learning the relationships among the words in the composition of the discourse. It is the task of a critical, humble, determined “subject” or agent of learning, the reader.

Reading, as study, is a difficult, even painful, process at times, but always a pleasant one as well. It implies the reader delve deep into the text, in order to learn its most profound meaning. The more we do this exercise, in a disciplined way, conquering any desire to flee the reading, the more we prepare ourselves for making future reading less difficult.

Freire (1994, p. 65)

I quote Freire’s uncompromising view from Pedagogy of Hope on the challenges of engaging with reading, as though I disagree with his presentation of study as an “always […] pleasant” struggle the last part chimes with how I feel on trying to be “an agent of learning” in understanding social critique and develop a #critlib approach with no background in theory. To be clear this is a personal reflection: I’m not telling anyone to do this, and I’m aware I speak from a position of privilege.

In discovering new these areas there is joy in exploring new contours of practice, and opening new discursive spaces. Given new tools for sensemaking, the materials of practice feel new themselves. I value #critlib most in providing a space to create and shape professional discourse, to disagree from a position of inclusivity and respect, and fundamentally to help us develop a reflexive praxis: that is, critically-informed action.

Well, how about it? I found Freire’s explanation of the simultaneous nature and equivalence of reflection and action particularly formative in doing this:

“[M]y defence of praxis implies no dichotomy by which this praxis could be divided into a prior stage of reflection and a subsequent stage of action. Action and reflection occur simultaneously. A critical analysis of reflection may, however, reveal that a particular form of action is impossible or inappropriate at the present time. Those who through reflection perceive the infeasibility or inappropriateness of one or another form of action […] cannot thereby be accused of inaction. Critical reflection is also action.

Freire (1996, p. 109)

Working through the implications of this allowed me to reframe reflection not as an introspective after-the-event mulling over, perhaps somewhat time-wasting given there is “real work” to getting on with, but something central to action itself and part of an overall method of praxis.

So, this is why I participate in #critlib and RLC. I want to see the loop between theory and practice closed in ways that appreciate it is not just the job of LIS academics to develop theory and practitioners to employ it, but that theorizing from our practice and experience should be an accessible process for all of us.

Theory in an undertheorized field

One of Kevin’s questions for the chat asks if we think #critlib is “worth it,” that is, does talk about theory make any difference to your own professional practice, and more broadly? I argue yes, but…

I think #critlib starts with a set of challenging positions to work from rather than a set of agreed ideological fixed ideas or ‘how to’ guides for library work, management, and leadership. Even if someone wrote such a thing, the texts we read and discuss are steeped in and reproduce ideologies and this would remain a problem to grapple with.

#critlib offers multidisciplinary ways of seeing and interpreting our practice that are necessarily critically informed and framed in terms of inclusivity and diversity, but that might not be applicable to practice in straightforward ways. This bears analysis of our situation, and for me, giving up the idea of applying #critlib with a particular framing ‘context’ as this term obscures a great deal. I’m indebted to Clarke’s work on grounded theory (2005, pp. 71-72) in understanding this last point.

In my view systems librarianship is relatively undertheorized and ahistorical in its understanding of thought and practice; not obviously fertile ground to develop #critlib. As the head of a systems team I see systems as occupying an important nexus in two areas often claimed neutral in various ways: libraries and technology. Of course neither are neutral, when put like this it’s laughable to suggest they are, but it’s easy to slip into such positions by default if we don’t interrogate our practices and how we utilize technology.

It can be easy for systems workers to think if we are satisfying the requirements of library staff as clients and library users as customers that’s enough, and we needn’t over-think the technology we employ beyond compliance with law (such as data protection legislation), our professional ethics (such as CILIP’s Code of Professional Practice), the standards of our employers, and so on. I’ve heard very reasonable positions voiced in the profession that this is “just good librarianship”.

I argue that this is not enough and our response should be non-neutral as our technology and information management is non-neutral. But to exaggerate for hyperbolic effect, critically-informed ethical approaches are more complicated than lazily rejecting every new idea as “problematic” while maintaining our own comfortable, compartmentalized silos. A systems approach to #critlib (#critsyslib?) means delving into our underlying assumptions by using reflective practice as a technique, taking critically reflective and reflexive approaches in our implementations of new systems and technologies.

For me developing a critical systems praxis means reject instrumental or mechanistic approaches to information, its storage and indexing, its use and understanding by our users, and the stack of technologies that underpin all of these things. A #critlib perspective would mean taking critical approaches throughout systems work including in selection, procurement, and implementation of new technologies and systems, and using critically-informed methods in researching them and in directing their development. For managers and leaders in systems it means role-modelling positive behaviours, and taking an authentically #critlib stance in what we do in our work, including for example line management and hiring practices.

I struggle with these things and cannot claim to do all of them or that my team do all of them, yet, but referring to the Freirean equivalence of reflection and action, taking on #critlib viewpoints and approaches begins the necessary perspectival shift to move towards a critical turn in systems praxis hence, why I #critlib.

References

Clarke, A.E. (2005) Situational analysis: grounded theory after the postmodern turn. Thousand Oaks, CA: Sage.

Freire, P. (1996) Pedagogy of the oppressed. 2nd edn. London: Penguin.

Freire, P. (1994) Pedagogy of hope. Reprint, London: Bloomsbury, 2014.

Smith, L. (2014) ‘Radical Librarians Collective (part three): critical theory’, Lauren Smith, 16 May. Available at: https://laurensmith.wordpress.com/2014/05/16/radical-librarians-collective-part-three/

Reflective practice and critical reflection recommended reading

I was recently asked for some recommended reading on reflective practice in general and critical reflection in particular, and as several people asked me to share this I thought I would do so here.

Reflective practice in general

Schön, D.A. (1983) The reflective practitioner. New York, NY: Basic Books.

A classic text, and one that should be seen in Schön’s context of time, place, and employment. Heavily critiqued and interpreted in the last 30+ years; I’d argue understanding reflection-in-action vs. reflection-on-action remains core to effective reflective practice and that this is still a challenging work when considering professional technical skills for practitioners.

Finlay, L., (2008) Reflecting on reflective practice. Available at: http://www.open.ac.uk/opencetl/resources/pbpl-resources/finlay-l-2008-reflecting-reflective-practice-pbpl-paper-52

For me Finlay is the best starting point for a review article on reflective practice. Written for the Open University PBPL CETL (which dates it if you remember CETLs…) but presenting, effectively, a wide-ranging literature review on reflective practice useful to anyone. Very helpful for an overview of different approaches or ways of doing reflective practice.

Bolton, G. (2014) Reflective practice : writing and professional development. 3rd edn. London: Sage.

A useful overview of reflective writing. Its utility to you may depend on your view of the importance of narrative, but worthwhile if you have any intention of writing reflective pieces on your experience.

Brookfield, S. (1994) ‘Tales from the dark side: a phenomenology of adult critical reflection’, International Journal of Lifelong Education, 13 (3) pp.203-216. doi:10.1080/0260137940130303

An interesting exploration of negative aspects (the ‘dark side’) of critical reflection for adult learners. Recommended reading for how to avoid or overcome these issues in practice. Sadly not available OA or free-to-read.

Argyris, C. (1991) ‘Teaching smart people how to learn’, Harvard Business Review, (May/June). Available at: https://hbr.org/1991/05/teaching-smart-people-how-to-learn

On challenges of applying Argyris and Schön’s single-loop vs. double-loop learning to professional practice.

Drucker, P.F. (1999) ‘Managing oneself’, Harvard Business Review, (January). Available at: https://hbr.org/2005/01/managing-oneself

Not pitched as about reflection, but contains useful nuggets on ways of thinking about development. Warning: contains some problematic ideas and sweeping statements.

Critical reflection or critical reflective practice

I take critical reflective practice to mean particularly the ‘two stage’ approach of analysis and deconstruction followed by reconstruction and change developed and articulated in the 2000s by Jan Fook, Fiona Gardner, and Sue White, alongside contributions from many other researchers. Their particular combination of theoretical underpinning and practical method is the one I prefer because it chimes with in with my own values and preferred approach to thinking about professional practice.

These researchers scaffold their approach with four theories, this list is paraphrased in part from Gardner (2014) and Fook and Gardner (2007):

  • Reflective practice itself, including an idea of experience or practice knowledge that I would describe as praxis, eg. articulated in Fook and Gardner (2007 p.24), “[R]eflective practice can be seen as a process of researching practice theory, by developing it directly from concrete practice”
  • Reflexivity, emphasizing a consciousness of how users perceive themselves and us, and how we perceive ourselves and each other in context as practitioners and researchers
  • Postmodernism and deconstruction, in this case particularly emphasizing concerns with attitudes to and influences of power
  • Critical social theory, including an emphasis on social justice as a concern

One challenge for us is taking these ideas out of their original context of health and social care and making them applicable to library and information professional practice. Personally, I think this is entirely possible as the idea of praxis implies:

  • Applying theory to practice deductively
  • Creating theory from practice via inductive formation of knowledge and new theories of practice

My point, is we needn’t wait for someone to write the ‘Critical reflection in library and information science’ textbook but can work on these ideas immediately. 😉

Gardner, F. (2014) Being critically reflective. Basingstoke: Palgrave Macmillan.

Gardner’s is my favoured introduction to critical reflection concepts and methods. It’s written to work for students and both new and experienced practitioners, providing practical examples while referencing more complex texts for underpinning theory, some of which are below.

Fronek, P. (2012) Jan Fook: critical reflection. [Podcast]. Available at: http://www.podsocs.com/podcast/critical-reflection/

Fook, J. (2006) ‘Beyond reflective practice: reworking the “critical” in critical reflection’, Professional Lifelong Learning: beyond reflective practice, Leeds, UK, 3 July. Available at: http://medhealth.leeds.ac.uk/download/1404/keynote_janis_fook

This conference keynote and podcast can be combined with Gardner’s book on critical reflection as an overview for ‘core’ reading. Jan Fook’s keynote for a medical education conference includes both the ‘what it is’ explanation and a retheorizing of critical reflection. I really like her summary on p. 14 for the focus on the political aspects:

“In reworking a theory of critical reflection, I would now articulate critical reflection as involving the ability to understand the social dimensions and political functions of experience and meaning making, and the ability to apply this understanding in working in social contexts” (emphasis in original)

The podcast interview is meant as an overview – Jan Fook’s style and enthusiasm are compelling, start here if you are unsure about where to dive in.

White, S., Fook, J., and Gardner, F. (eds.) (2006) Critical reflection in health and social care. Maidenhead: Open University.

Lehmann’s chapter 14 particularly interesting as this links a reflective writing approach to critical reflective practice; the ‘five questions’ she poses are also very useful for unpicking underlying assumptions.

Fook, J. and Gardner, F. (2007) Practicing critical reflection: a resource handbook. Maidenhead: Open University.

Other than Gardner (2014) I think this is the best introduction containing a more detailed explanation of the theory plus a plan for how to carry out practical reflective conversations as a group.

Vince, R. and Reynolds, M. (2002) ‘Organizing reflective practice’, Organization Learning, Knowledge and Capabilities, Copenhagen, Denmark, 28-30 April. Available at: http://www2.warwick.ac.uk/fac/soc/wbs/conf/olkc/archive/olkc3/papers/contribution115.pdf

I include this here as Vince and Reynolds link reflective practice to critical management studies as a way of developing a model moving beyond Schön. Their overall view of ‘collective reflection’ is a useful expansion of, and critique of some limits in Schön’s work.

Ghaye, T. (2005) Developing the reflective healthcare team. Oxford: Blackwell.

Does what it says: the focus is on team-based reflection in healthcare, from a UK context. Much of what is here is generalizable and I think this is a particularly good starting point for managers or leaders looking at ways of developing reflective teamwork.

Critical theory and praxis

A major attraction for me to critical reflection was how Fook, Gardner, and White combine ‘traditional’ reflective practices with critical theory. With that in mind I wanted to include a few suggestions for starting points I think are useful in this area that may help in understanding the above. Lauren’s recommended reading list below includes these and much, much more: everything from critical theory ‘big names’ to specific library and inform recommendations.

Generally I’d also recommend Foucault, but struggle to pick one particular work. If pressed I’d include Discipline and Punish due to the focus on power and how disciplinary mechanisms were extended to the 20th century (Taylorist, Fordist) workplace, and the classic chapter on panopticism. It’s what I’ve recommended to my team as a starting point.

Leckie, G.J., Given, L.M., and Buschman, J. (eds.) (2010) Critical theory for library and information science. Santa Barbara, CA: Libraries Unlimited.

In my opinion the best general introduction to how critical theory can be applied to library and information work.

Freire, P. (1970) Pedagogy of the oppressed. London: Penguin.

A foundational work on praxis and critical pedagogy, quotable and inspiring. Of its time and place (citations include Lenin and Mao) so should be taken as a starting point for developing ideas.

Day, R.E., (2000) ‘Tropes, history, and ethics in professional discourse and information science’, Journal of the American Society for Information Science, 51 (5), pp.469–475. doi:10.1002/(SICI)1097-4571(2000)51:5<469::AID-ASI7>3.0.CO;2-B

Free-to-read preprint available at: http://info.slis.indiana.edu/~roday/articles/tropes.pdf

Reading lists from our community

Smith, L. (2014) ‘Radical Librarians Collective (Part Three): Critical Theory’, Lauren Smith, May 16. Available at: https://laurensmith.wordpress.com/2014/05/16/radical-librarians-collective-part-three/

This blog post links to Lauren’s critical theory in LIS reading list (Google Doc) but is worth reading for additional background and the wonderful top ‘All-Time Must-Read Critical Theory in LIS texts’ list.

Professional identity, impostor syndrome, and performativity: thoughts on #radlib15

Image by Flickr user Michael Podger. License CC-BY. Available at: https://flic.kr/p/s2mJjL

Last weekend I attended the #radlib15 event organised by Radical Librarians Collective (RLC) in Huddersfield. I wanted to draw some threads between sessions I attended, and follow up on a few things left unsaid on the day.

RLC has come a long way in a few years and is evolving, but importantly it has maintained a safe space for discussions in LIS that are not happening elsewhere. My grateful thanks to the organizers at RLC for their work.

Professional confidence and professional identity

During the day, I felt two discussions on impostor syndrome and team working were linked by points made about professional confidence and identity, questions about radical perspectives on management, and ideas about the presentation of an authentic self.

Elly O’Brien recently wrote a compelling article problematizing impostor syndrome and professional confidence which helped inform the impostor syndrome session, with Elly’s article referenced at the beginning. I agree with Elly’s points about the unhelpful librarian tendency towards self-deprecation, and I think Kevin nailed it when he called out this “syndrome” as a political creation exerting power on the subject:

About lacking confidence, a point was raised in discussion about whether there is a psychological disconnect between our presentation of identity online, and our true or authentic selves:

I think Simon’s point on the marketized self is good, and I would expand on it and generalize from it. From a Marxian viewpoint anyone who is worker is compelled to sell labour as a commodity, representing a market relationship between the self’s potential for labour (labour power, Marx’s Arbeitskraft) formed from our own living bodies, and capital. In a sense under capital that marketized self is no more or less than one’s real self as it is reflected and understood by capital. This situation is deeply problematic.

Performativity in online identity

On Twitter, Chris followed this with a point and a question about authenticity in our online selves:

I’ve been thinking about a similar question about online identity since I read Ned’s blog post about creating online identity last year, especially his points about “consistent voice” and advice on not “adjusting who you are for other people”. Here I draw a line to the session on teams and teamwork and library managers as a potentially radical subject, relating them using Judith Butler’s concept of performativity.

Butler originally applied her Foucauldian reading of performativity in developing an analysis of gender, arguing that identity can be brought to life or made real by repeated and consistent use of authoritative speech, as:

There is no gender identity behind the expressions of gender; […] identity is performatively constituted by the very “expressions” that are said to be its results.
(Butler, 1999 p.33)

There is a subtle point here. Butler is not simply suggesting that if we talk as if we have identity x, we will have identity x, as in the “fake it till you make it” suggestion that was raised in the impostor syndrome session, but that performativity is:

[T]hat reiterative power of discourse to produce the phenomena that it regulates and constrains
(Butler, 2014 p.2)

I argue online identities of LIS workers are performative in the way that our use of language itself reifies, by repetition of authoritative speech, those identities that we create online rather than being something authentically constructed or a mere result of “being oneself”. For example, the identity of “an efficient, competent subject librarian”, or “a grounded, authentic manager”, created exactly by the discourse of those individuals on social media platforms like Twitter. I would expand this to include identities we create as managers, in that management is fundamentally performative: expressing power by a mode of authoritative speech, a case of actions embodied in “Doing things with words” (Learmonth, 2005).

On the broader question raised of library management as potentially radical, I have written before about authenticity in management and leadership at Radical Library Camp (the original RLC unconference in 2013) where I facilitated a discussion. In hindsight I’m not satisfied with that perspective, because I don’t see much difference between what I described and straightforward good management.

Instead I think we need to approach management and leadership from a critical perspective, and that there is space for a critical management studies (CMS) view of library management as Kenny Garcia suggested on the #radlib15 hashtag. CMS is something like a critique of management informed by Frankfurt School critical theory; to provide LIS focus I’d add in critical perspectives on information management and information literacy, and perhaps a Marxian lens to analyze information as a commodity in a marketized society. For much more development of the latter, I recommend Lawson, Sanders, and Smith (2015).

This is something I’m very interested in developing in future, so if you are interested in a CMS plus LIS mashup let me know.

References

Butler, J. (1999) Gender trouble: feminism and the subversion of identity. Reprint, Abingdon: Routledge.

Butler, J. (2014) Bodies that matter: on the discursive limits of “sex”. Reprint, Abingdon: Routledge.

Lawson, S., Sanders, K, and Smith, L. (2015) ‘Commodification of the information profession: a critique of higher education under neoliberalism’, Journal of Librarianship and Scholarly Communication, 3(1), eP1182. doi:10.7710/2162-3309.1182

Learmonth, M. (2005) ‘Doing things with words: the case of “management” and “administration”’, Public Administration 83(3), pp. 617–637. doi:10.1111/j.0033-3298.2005.00465.x

Photo credit

Sunset Flowers. Huddersfield‘ header image by Flickr user Michael Podger. License CC BY.

“UX for the win!” at #CityMash: open and focused coding of qualitative research data for discovery user experience

In Library Services at Imperial College London, between January and April 2015 my team completed two iterations of user experience testing of our Ex Libris Primo discovery system with a view to redeveloping the user interface to provide improved an user experience.

For the #CityMash Mashed Library unconference, Karine Larose and I are running a workshop on the methods we used in our second iteration of testing. Rather than run a ‘show and tell’ about our approach, the workshop will provide experience using our methods with some of our data in a similar way to how we conducted the research ourselves. We will provide hands-on experience of these methods, attempt to demystify the approaches used, and hope to demonstrate how exciting we find the professional praxis of systems librarianship.

This blog post explains the background and provides a practical overview and some theoretical scaffolding ahead of #CityMash. What we present is just one approach and all methods are flawed; we are extremely interested in hearing comments on or objections to our methodology around discovery user experience.

Acknowledgement

We’d like to acknowledge the hard work of George Bray, Master’s student at UCL Department of Information Studies, during a work placement with our team. George designed and undertook much of this testing during his work placement, based on our overall guidance, and we would not have been able to produce what we did without him with us.

Why we use constructivist grounded theory

The methods we chose for our user experience research were qualitative and post-positivist. They are based ideas developed by Barney Glaser and Anselm Strauss (1967) in their classic (and arguably classical, read on…) The discovery of grounded theory. Grounded theory includes:

  • Data collection and analysis as a simultaneous process
  • Analytically constructing “codes” and categories from data itself
  • The “constant comparative method” of comparing existing and new data in an ongoing process
  • Developing theory during each stage of data collection and analysis
  • Sampling to aid building theory, rather than being representative of the population
  • In pure grounded theory, the literature review comes after the analysis

This list is paraphrased from Charmaz (2012; 2014 p. 7).

The above may sound unusual to those with experience of more quantitative methods, and the idea of the literature review coming last may sound unusual to everyone. Bear with me. If you are interested in reading more I don’t necessarily recommend Glaser & Strauss as a first step. For an introduction to grounded theory at LIS Masters’s level, there is a chapter in the second edition of Alison Pickard’s Research methods in information (2013) which provides a detailed and readable outline.

Our touchstone work has been Kathy Charmaz’s Constructing grounded theory (2014) where she explains a constructivist approach to grounded theory. Core to her ideas are the acknowledgement of subjectivity and relativity in the research process, and a drive towards abstract understanding of observed phenomena within the specific circumstances of the research (Charmaz, 2008) which particularly resonated with us doing discovery research.

Charmaz is no ideologue, for her different traditions in grounded theory represent a “Constellation of methods” (2014 p. 14) rather than binary opposition. We have drawn on elements from both the empirical interpretivist grounded theory tradition, constructivist grounded theory, and the critical theory approaches that inform my thinking elsewhere in LIS. These are the differences we understand:

Objectivist grounded theory Constructivist grounded theory
Theory ’emerges’ from the data Researchers construct categories from the data
Researchers develop generalizations and explanations out of context Researchers aim to create an interpretive understanding accounting for context
The researcher’s voice has priority The participant’s voice is integral to analysis and presentation

What does this mean for user experience work?

You can see how a constructivist approach will focus on the voice of the user as an integral feature in understanding and presenting data. In my team (and I hope in your team) user experience work has never been informed by the “Librarian knowing best”, but this approach provides a particular emphasis. My experience is the voice of the user, seeing her context and affective responses, is a powerful way of making the case for making changes to our systems. This presentation can be extremely eye-opening even for those who work day-to-day in user-facing roles and know our users well.

We definitely did want to inductively develop theory from our data, but we wanted to be mindful of the user’s context and be interpretive, as we know our discovery system is just one part of a complex and shifting information landscape our users inhabit. We use the iterative and analytical approach of coding, and codes necessarily result from the researcher and data interacting (Charmaz, 2012). However our focus is wherever possible on trying to analyse the data rather than describe it. Ideally this should happen from the first moments of coding; more on this below.

Fundamental to constructivist grounded theory, the resulting ideas we develop are based on our interpretation of data and as researchers we cannot stand ‘outside’ that interpretation. What we create from the data is based on conceptualizing what we have studied and observed in user behaviours: we must stand inside and ‘own’ our analyses which will be affected by our biases, our preconceptions, and the emotional investment in the work we do.

This is not unprofessional, but an acknowledgement of the shared humanity of the researcher and the participant, and of the value of our work experience as practitioners that allows us to critically reflect on and develop theories of practice. To balance our subjectivity as researchers, a key part of the constructivist process has been to critically reflect on our preconceptions about discovery, information literacy, and users’ behaviour and expectations of doing their research using the tools we provide.

Working with qualitative data for user experience research

We are doing analysis of qualitative data collected during interviews to investigate Primo user experience. Ahead of interviewing proper, we held planning meetings with Library Services staff drawn from all sections of the library to work through starting points: primarily, what we wanted to get from the interviewing process, and what we wanted to know by the end of this round of investigation.

Extensive notes of these workshops were taken, and used by George to provide an initial focus for our interviewing. These are not quite research questions, but areas to focus on. These were:

  • The purpose, construction, and use of search and resources
  • Presentation of information in search: what matters to the user when selecting the right result?

Following this George and Karine developed an interview script for use by facilitators. This included general questions about information seeking as well as some specific tasks to carry out on Primo. This interview is structured and in grounded theory ideally would be based around open questions, helping us as researchers unpick meaning and move towards answering “why” questions in our analysis. We used a mixture of questions and posing specific tasks for users to complete. Our interview script is available: Primo UX Interview questions June 2015 (PDF).

In practice interviewers have different styles, and some facilitators stuck more closely to the script than others. This is not necessarily a problem, remember as an observer you are free to suggest places where we need to run another iteration and gather more data.

Our research data comprises the audiovisual recordings and the facilitator’s notes. The notes help understand the facilitator’s perspective on the interview and provide useful observations.

For #CityMash, we are providing a recording of the first part of an interview. In the full interviews at Imperial we did longer interviews making use of other methods drawn from web usability testing. The #CityMash data does not contain these. We gained informed consent for participant interview recordings and our written notes to be used for presentation and data analysis at #CityMash.

#CityMash technical requirements

  • You will need at least a tablet, ideally a laptop, to watch and listen to the audiovisual recordings. A smartphone screen will likely not be big enough to see what’s going on. Headphones are ideal but are not entirely necessary.
  • Sharing a device with another delegate is possible. Coding together and sharing your observations and thoughts as you go in a negotiated process would provide an interesting alternative to doing this on your own.
  • You will need a way of recording your coding and writing memos and any other notes. Any text editor, word processor, or pen and paper will work fine. (At Imperial College to facilitate collaborative coding, sharing, and to save time, we just write directly in our staff wiki.)

Beginning the process of open coding

Charmaz’s (2014, p. 116) guidance is that during initial or open coding, we ask:

  • What is this data a study of?
  • What do the data suggest? [What do they p]ronounce? [What do they l]eave unsaid?
  • From whose point of view?
  • What theoretical category does this specific [data] indicate?

Grounded theory textbooks often give examples of coding based on narrative such as diaries or written accounts and show example codes side-by-side with this. We are using audiovisual recordings instead, but the process is similar: listen to each statement and sentence spoken and the user’s behaviour as you go through the video and code piece-by-piece. Try to “sweep” through the data fairly quickly rather than spending too much time on each code. You will get better and faster at this as you go.

For codes themselves, try starting by writing down short analytic observations about the data as you experience it. Codes should “result from what strikes you in the data” (Charmaz, 2012) and should be “short, simple, active, and analytic” (Charmaz, 2014 p. 120 ). Remember you’re trying to be analytical about what you see, not just record what is happening.

Charmaz’s (2014 p. 120) ‘code for coding’ is:

  • Remain open
  • Stay close to the data
  • Keep your codes simple and precise
  • Construct short codes
  • Preserve actions
  • Compare data with data
  • Move quickly through the data

Keep the facilitator’s notes alongside you and try to understand how these relate to what she saw and understood in the interview.

Don’t worry about being perfect the first time. Coding is iterative and you are allowed to go back and rework things, and make new connections between data. Initial codes are provisional, and working quickly both forces you be to spontaneous and gives more time to go back and iterate over the data again.

It is very difficult, but try to put your favourite theoretical “lens” to one side during initial coding. It’s perfectly fine to bring in these ideas later, but for open coding you are trying to spark thoughts and bring out new ideas from the data rather than apply someone else’s grand theory.

Focused coding: refining data to begin to develop theory

Our #CityMash workshop is limited in time so we will do an initial round of open coding followed by small group discussion exploring focused coding.

Focused coding is the process of analyzing and assessing your first round of codes, and as a guide it should be a reasonably fast process. You are looking for connections and relationships between codes, and comparing them with the data and with each other. Looking at particular pairs of codes, which work better as overall analytical categories? Which give a better direction in developing an overall theory from the data?

Think about how you might create a theoretical framework later about discovery user experience to help inform changes to the system. Which codes better fit the data in allowing you to do this?

Charmaz (2014, pp. 140-151.) poses the following questions to help make choices about focused coding:

  • What do you find when you compare your initial codes with data?
  • In which ways might your initial codes reveal patterns?
  • Which of these codes best account for the data?
  • Have you raised these codes to focused codes?
  • What do you comparisons between codes indicate?
  • Do your focused codes reveal gaps in the data?

The results of George’s analysis of our focused coding was written up into a summary report of the things we needed to concentrate on in redeveloping our Primo interface. The systems team is currently working on Primo back-end configuration and front-end design to fulfill this, and these findings will be the subject of an upcoming blog post.

#CityMash slides

Our slides from our #CityMash talk are also available.

References

Charmaz, K. (2008) ‘Constructionism and the grounded theory method’, in Holstein, J.A. & Gubrium, J.F. (eds.), Handbook of constructionist research. New York, NY: Guilford Press, pp. 397-412.

Charmaz, K. (2012) ‘The power and potential of grounded theory’, Medical Sociology Online, 6(3), pp. 2-15. Available at: http://www.medicalsociologyonline.org/resources/Vol6Iss3/MSo-600x_The-Power-and-Potential-Grounded-Theory_Charmaz.pdf (Accessed: 11 June 2015).

Charmaz, K. (2014) Constructing grounded theory. 2nd edn. London: Sage

Glaser, B.G. & Strauss, A.L. (1967) The discovery of grounded theory. Chicago, IL: de Gruyter

Pickard, A.J. (2013) Research methods in information. 2nd edn. London: Facet.

Reflections on organizing the Pi and Mash conference #piandmash

Practical communications session in progress led by Meghan Jones. Photograph by Simon Barron, license CC-BY-SA.
Practical communications session in progress led by Meghan Jones. Photograph by Simon Barron, license CC-BY-SA.

Introduction

Earlier in August I had the pleasure of helping organize and run a tech-focused library unconference, Pi and Mash, at Senate House Library at the University of London. The other organizers were Simon Barron of University of London, and Ka-Ming Pang of St Georges, University of London. They were both brilliant to work with and brought enormous energy, fresh perspectives, and thoughtfulness and professionalism to organizing the day. This event was a long time in gestation, from the initial agreement back in January that we’d work together to the day itself in early August. I had previously hosted Library Camp London at Senate House in March 2013, and following that event I’d thought about running something tech-focused as a Mashed Library event. Ultimately for me Pi and Mash was that event, though full credit is to Ka-Ming for suggesting we do it, kicking off the initial discussion on Twitter to gauge interest, and starting to organize us.

As an organizer it’s always encouraging to receive good feedback, and participants said some lovely things about Pi and Mash during and at the end of the day. Examples on this Storify (This used to be embedded; I’ve temporarily disable Storify embedding because a script it loads is causing page loading to stall.)

What I learned from organizing the event

The limits of ‘unconferencing’http://radicallibcamp.wikispaces.com/

Ahead of the event we wanted to provide a programme with appeal to different levels of technical ability, and especially beyond experienced systems workers. To do this we reached out to professional contacts for session ideas and pitches so we could launch with a timetable already partly populated. This timetabling in itself introduced a contradiction to the event that was never really resolved, and caused some issues: were we running a participant-driven unconference, or a regular conference with a top-down organization? I formed an impression from feedback that on seeing our speaker lineup, some participants felt intimidated about pitching due to not feeling technically knowledgeable enough. Additionally, we presented an ‘almost full’ timetable with space for unconference-style pitching. While this helped generate buzz and encouraged people to get a ticket so they could come to those sessions, it made it easier to view the day as a traditional conference that didn’t demand participants set the agenda. A related point is I’ve noticed unconference sessions, for example at Library Camp, becoming increased sophisticated over time and more pre-planned. Sessions are often no longer discussions, but make more use of technology such as online collaborative editing, use more formal methods in research and analysis, and attempt to engage people beyond the conference for example by tying in with planned chats on Twitter. For me this increased sophistication deepens engagement, but can work against the more exciting aspects of unconference spontaneity such as pitching an idea that is not fully-formed on the day. We did get pitches ahead of time on our discussion and ideas document, but these were the only ones pitched on the day so it felt a bit more like a call for papers than pitching.

What this means for practical technical sessions

As noted we wanted to ensure broad appeal to a range of different technical abilities. We especially wanted to demonstrate practical aspects of library systems work that would give a flavour of what it is systems librarians and other systems workers do. To this end it was wonderful that many delegates saw the day as an opportunity to stretch themselves with professional development, and expressed an interest in getting more involved with systems work in future. One of the facilitators observed to me on day there is a real difficulty in how to ‘bring people along with you’ if they are at different levels at the start. This makes running sessions that rely on pre-existing technical knowledge that much more difficult. One suggestion from feedback was to provide pre-work or reading ahead of the day for sessions that would benefit from it. I have mixed feelings about this as despite having run such sessions like that at conferences, I feel participants should also easily be able to choose what they will on the day, or even move between sessions. For me, this was most apparent for the Linked Data and OntoWiki session, although I know there were issues in other sessions too. This combined with technical dependencies for participants, who needed to install software on their own computers to get the best from the practical work. In hindsight, what we needed to provide were laptop computers with the relevant software pre-installed and ready to use, so we could simply hand a machine with a ‘known good’ configuration to everyone attending the session. This would have been challenging, but perhaps could have been feasible using loan laptops from Senate House Library stock and given enough time for preparation.

Safer spaces, and an apology

Ka-Ming provided the idea of very actively promoting and encouraging women facilitators and participants. Essentially, we did not want to run yet another tech event dominated by men but rather one that better reflected how our profession is populated. It was great to get positive feedback on this aspect, and suggestions from critical friends where we erred. One point I want to apologize for is our gender binarism in the initial ticket allocations to men and women. As organizers we discussed this after it was pointed out, and learned from it. In future I will do better, I will approach gender more carefully to help avoid reinforcing bias and discrimination. I am glad we implemented a safer spaces policy, repurposed with permission from OK Café. As professionals we might prefer to believe policies shouldn’t be necessary, but I argue they help create inclusive events in the first place. Even if a policy doesn’t need to be acted on, it provides a context to set expectations and helps attendees develop confidence they will be supported in resolving any problems. I now firmly believe safer spaces policies or codes of practice are necessary for conferences.

Being the organizer

The way in which participants interact, learn, and spark ideas off each other is something you try to positively influence as a conference organizer, but ultimately much of ‘the magic’ is out of your control. It helped that we provided a space that participants found friendly and inclusive, with longer session times than normal conferences. This allowed for discursive conversations and digging into the technical ‘long weeds’ as participants wished. This was an important aspect for my own development, as I identified I need to move beyond running events successfully (without say, some disaster befalling us), to thinking more deeply about the value gained by delegates for their own development and understanding ways in which we can support and facilitate this. Overall I would strongly recommend (un)conferencing organizing as a means of professional development.

Thoughts on practical aspects

Middlesex South Reading Room at Senate House Library. Photograph by Andrew Preater, license CC-BY.
Middlesex South Reading Room at Senate House Library. Photograph by Andrew Preater, license CC-BY.

Following Library Camp London I’d reflected on what made the day a success, practically, and we implemented much of this for Pi and Mash. This is summarized here: Practical suggestions for running your own Library Camp. Some things that remain true:

  • Especially true for a technical event, your wireless absolutely needs to be working.
  • Individual bottles are better than glasses for carrying water around the library.
  • If you’re relying on someone for preparation such as moving furniture, survey the space ahead of time and prepare with the expectation your instructions will be followed to the letter.
  • One thing that was again a problem was noise, as we were using large rooms with two sessions in them noise carried. This was a limit inherent in the spaces available to us, which were provided free of charge by Senate House Library. However we would definitely have been better to provide smaller separate rooms, or found a way to screen off larger spaces to dampen noise.

We organized Pi and Mash almost exclusively online, which saved a lot of travelling time even with all of us being London-based. We used:

  • Google Hangouts to provide audio and video for meetings.
  • Google Docs / Drive for collaborative editing and sharing of meeting notes and actions lists.
  • WordPress.org for our website plus TablePress for tables. This was low cost as we could use existing web hosting and we all had practical experience with the software.
  • Gmail for email. Specifically the trick was to push the ‘info@’ domain address to my own Gmail using POP3 and set it up to allow responding from that address. This made for quick and efficient replies to questions.
  • Eventbrite for ticketing and emails to delegates. I still favour Eventbrite despite its quirks. Checkin is a breeze and mailouts are simple, and the quirks are at least quirks I’m very familiar with.
  • Twitter – of course, the place for professional engagement in libraryland.
  • Qualtrics for our post-conference survey (Imperial College London has a subscription).

We made some choices about what not to do with social media and other tools:

  • We considered use of Lanyrd for session slides and materials, but it seemed a more useful tool for larger, more formal conferences where you would want to draw together lots of different media types, session recordings, and so on.
  • Wiki. Ultimately we decided not to set up a wiki for Pi and Mash as we felt the limited amount of collaborative editing needed ahead of the event could be handed using a Google Doc. This is the model uklibchat use successfully, but from feedback some delegates reasonably expected a wiki to be available.
  • Unfortunately it wasn’t possible to live-stream any presentations. There was demand for this ahead of the event, but it was too difficult to achieve technically and with limited staff resource. I have done this for events by using Google Hangouts on Air which can provides a slick, professional solution at low cost using consumer webcams and microphones.

In hindsight, in thinking about our approaches to communication I found Ned Potter’s description of communication channels as white noise, peripheral vision, or line of sight from a recent conference presentation very helpful.

The main area for improvement I would focus on for future events are reaching those who do not routinely professionally engage with social media:

  • Mailing lists are still widely-used by library workers and I noticed rushes of interest when we mentioned Pi and Mash on mailing lists like lis-link.
  • Targeted personal communication is very effective at helping publicize the event by word-of-mouth. For example: encouraging library and information science lecturers to promote the event to students; and to our colleagues to encourage team members to attend the event for professional development.