Tag: senate house libraries

Free and Open Source Software and distributed innovation

‘The battle of the library systems’

On 28 November Senate House closed early for the University of London Foundation Day,  our annual celebration of our grant of royal charter on 28 November 1836. As the library closed down I made the short journey to Cilip HQ to attend the “Battle of the Library Systems” event organized by Bic.

This event was a panel debate between two sets of speakers in favour of Open Source software (OSS) and proprietary software. I was speaking in the Open Source “blue corner” alongside Dave Parkes of Staffordshire University and Nick Dimant of PTFS Europe. Sadly, Mark Hughes of Swansea University was ill and unable to attend and speak as planned. In solidarity the open team shared between us presenting Mark’s slides.

The house motion was:

Open source is about distributed innovation and will become the dominant way of producing software.

This is a quote from Talis – slightly modified from the original – from the Jisc and Sconul LMS study (Sero Consulting, 2008).

I will say a little about the arguments I made in favour of the house motion and what I thought were some strengths and weaknesses with the proprietary team’s arguments. Mick Fortune summed up as a guest speaker and I’d recommend reading his blog post ‘BIC Battles – Open Source or Proprietary?‘.

My argument

I opened by explaining our situation. Senate House Libraries and the colleges that make up the Bloomsbury Colleges group recently made a decision in principle to select Kuali Open Library Environment (Ole) as our next library management system. We have chosen an OSS system which will be run on a shared services model by the University of London.

Why is Open Source software a good fit for higher education?

I explained I prefer the older term Free Software to Open Source as it’s conceptually broader. Thinking in terms of one dimension – software development with access to the source code – sidelines the underpinning philosophy of community, sharing, and respecting software users’ freedom.

The audience was mainly academic libraries and I argued that our industry, higher education, has a culture of sharing and collaboration and librarianship is collaborative as profession. The same point was made by the proprietary software panel, but I go further and argue this therefore makes the software a good fit for us.

Kuali Ole is a library services platform being developed collaboratively by universities and their software development partners specifically to meet the needs of academic research-focused libraries. Ole is an enterprise-level system that we intend to use for business-critical services within our consortium. It will be cloud-hosted and managed collaboratively to provide a stable and trustworthy service – about as far as you can get from the idea of a keen systems librarian installing a Linux distribution on an old server and deciding to ‘give an Open Source LMS a whirl’.

I argued the key differences with Ole are that:

  • It is a true library services platform rather than a traditional library management system
  • As it is collaboratively-developed OSS we have the possibility of developing the software to meet our needs

It is the Free or Open Source licensing that is important here, as it effectively provides a strong position of sharing by default to the development model used in the foundation. Effectively, sharing and collaboration is baked in to the product and the processes used to develop it.

Proprietary software suppliers and Open Source software

Among the strongest arguments in favour of OSS for library systems is the range and variety of OSS used by proprietary suppliers themselves. Examples are most prevalent in next-generation discovery engines where Apache Solr and Lucene are used extensively, but in other library systems Postgres, Apache Tomcat, and of course the Apache http server are used widely.

I think proprietary supplies use OSS because it represents best-of-breed software that is stable and well-supported, and importantly flexible and free to use. It is licensed in a way that allows development and may be used for any purpose. This is why I emphasised freedom or liberty initially: while proprietary software suppliers enjoy the benefits of OSS themselves they’re not so keen on passing those freedoms onto us, libraries that buy their software and support services.

Suppliers’ use of OSS was acknowledged by the proprietary team. Jim Burton of Axiell mentioned extensive use of OSS throughout the company with an estimate of something like 500 different pieces used in their processes – though I expect this includes things like development tools and that the amount of OSS in their finished products is much less.

It is difficult for software suppliers selling systems based on Open Source to argue against Open Source. In using it in your own products you are vouching for it – and also undermining your arguments against it. For me this ubiquity in use and development is a compelling argument in favour of Open Source becoming the dominant way of producing software in the future.

Choosing software pragmatically

Jim made what I felt was the best argument against OSS for a complete library system directly relevant to my own experience in higher education. That was that the license is of secondary concern if the software does what you want and meets your needs. That software has an Open Source license doesn’t mean it’ll be a good fit for a given specification – relevant in a software ecosystem with relatively few complete OSS library systems as options.

I take from this that in practice our assessment process should lead us to choose pragmatically based on need rather than buying something because ‘it has a badge’. For many libraries that choice would mean proprietary software as best fit to a specification: perhaps an LMS with open and standards-compliant APIs allowing development work, perhaps cloud-hosted, perhaps with developer communities, perhaps itself built from OSS?

Distributed innovation

I argued as a software development method Open Source and open collaborative development methods make sense in our increasingly complex and networked world. I borrowed a term from David Weinberger here, that that nowadays knowledge has become “too big to know” (Weinberger, 2012) particularly evident in higher education with the complexity and sheer scale of research data.

It is a distributed and networked development approach that has created successful projects such as the Debian GNU/Linux distribution, and indeed the Linux kernel itself. One reason for the success of these projects is networked expertise: the ability to surface skills and knowledge from a globally-distributed community of developers. To apply this to library systems software I argued suppliers building systems based on a closed approach cannot respond to our changing needs as one based on networked expertise with ‘peaks’ of local knowledge that best understand our own situation and requirements can do.

The proprietary team emphasized software suppliers’ wish to listen to their customers. I don’t doubt their honesty in this at all. I think engaging customers and encouraging more open development such as developer communities very welcome. However, I argue any single vendor lacks the depth and breadth of knowledge that we have collectively in our own institutions and the scale that can be brought to bear by networked collaborative development. For this reason, the future is necessarily an open one.

References

Weinberger, D. (2012) Too big to know. New York, NY: Basic Books.

Sero Consulting Ltd (2008). JISC & SCONUL library management systems study. Available at: http://www.sconul.ac.uk/news/lms_report/lmsstudy/ (Accessed: 6 January 2022).

Selecting a Free and Open Source LMS – Kuali OLE at the University of London

Senate House Libraries, and the University of London colleges that make up the Bloomsbury Colleges group recently made a decision in principle to select Kuali OLE as our next library management system (LMS). This project is currently termed the Bloomsbury LMS, or BLMS.

I will blog throughout this project but for now I wanted to explain a little more about this from a strategic point of view from the perspective of the systems librarian at the central University of London.

Significance

For me the significance of the decision in principle is that:

  1. We have chosen a Free / Open Source Software (FOSS) system.
  2. The system will be run on a shared services model by the University of London.
  3. Kuali OLE is a next-generation library management system.

By next-generation I mean Kuali is one of the new breed of systems that are cloud-hosted, based on modern Web services, and engage better with online electronic content. This is the type of system Marshall Breeding terms a library services platform as distinct from the traditional LMS.

Choosing an Open Source system

Strategically it’s important to us that our next system will not be more of the same both in terms of the conceptual approach to the LMS as a platform and in terms of being developed openly and collaboratively.

Discussing cultural issues, I have argued previously at Library Camp 2012 that in higher education (HE) the core values of Free Software – the Four Freedoms (Free Software Foundation, 2012) – are in alignment with our professional culture in HE and that within and outside HE librarians are particularly collaborative profession. Kuali OLE is an LMS designed by and for academic and research libraries for managing their library services. I do think that’s significant, but I think what OLE offers goes far beyond the software license being a philosophical best fit for our culture in higher education and libraries.

OLE is an enterprise-level software package that will match the functionality of closed-source library systems and exists as part of a suite of enterprise software including among others a financial system and student record system. This is interesting for us as a potential solution to the huge problems we have in higher education of ‘siloedness’ of our campus-wide enterprise information systems and the difficulties we have making them talk to each other.

Importantly, OLE is a true library services platform rather than just being a traditional LMS. Virtually any LMS currently on the market can tick all the boxes on the venerable UK LMS core specification, but there are relatively few that move beyond this and also open up the possibility developing the software to meet our needs. It is FOSS licensing that effectively provides a strong position of sharing by default to the development model, on the other hand the Kuali governance model addresses traditional weaknesses in FOSS projects and guarantees the production quality of the finished system. Those most relevant to us at the University of London are quality and risk management, and providing a robust testing methodology.

Contact

This post is meant as a brief introduction from my own perspective. If you are interested in the BLMS project and wish to contact us please see the BLMS website.

References

Free Software Foundation (2012) ‘What is Free Software?’ Available at: http://www.gnu.org/philosophy/free-sw.html

How we added WebBridge / Pathfinder Pro links to our Encore catalogue

When we launched our new catalogue, Encore from Innovative Interfaces Inc., in June 2011 among the first problems identified by staff and library members was that it did not have a way to request journals from our closed stores (we call this the Stack Service which includes our tower and an offsite store in Egham, Surrey).

This missing functionality between the old and new catalogue was a major barrier to buy-in. Not having it meant staff had to explain three parallel systems for requesting just one type of material from the store. Our readers want to request store material online quickly and efficiently, not have to deal with navigating between two different catalogues.

A new release of the Encore catalogue software has enabled us to rectify this and in this post I’ll explain how. The third request system was paper forms, in case you’re wondering…

Link from the Encore record to request a store journal, based on Pathfinder Pro data.

Requesting journals

Historical note: Pathfinder Pro used to be part of Innovative’s WebBridge product, which included both an OpenURL link resolver for linking in, and software for presenting context-sensitive links on your record display. Systems librarians at Innovative sites often refer to both products as “WebBridge”.

Requesting journals from a closed store is problematic for an Innovative library if you are cataloguing journals in a normal way – using a holdings or checkin record to detail what you have, rather than itemising each individual journal volume on its own item record.

In our old WebPAC catalogue I had devised a way of using Pathfinder Pro to link out to a Web form that would send some bibliographic data to pre-populate a form. It’s easy to link out to a form from the record (put a link in the MARC 856 for a quick solution) but reusing the record metadata itself helps readers to not introduce errors.

In my opinion Pathfinder Pro is a good product – the tests you can apply are quite powerful including matching parts of your record based on regular expressions and the like.

The basic principle to enable this linking is:

  1. Check to see if the holdings record is in a store location. Egoist : an individualist review is coded ‘upr’, for example, and a data test in Pathfinder Pro is used see if the holdings record location field equals ‘upr’.
  2. If the record is a store location, build a link based on selecting the journal title, classmark, and the system record number from the bibliographic record. What is actually rendered on the page is a link using the same icon as we use for other types of online requesting:
Link from the WebPAC record to request a store journal, using Pathfinder Pro (circles in orange).

This generates a link to our online request form:

http://www.ulrls.lon.ac.uk/stackrequest/requestform.aspx?JtitleText=Egoist%20%3A%20an%20individualist%20review.&JlocationText=STACK%20SERVICE&JclassmarkText=PR%20Z&JbibText=b1746208a

Which leads to a neatly pre-populated form:

The problem with Encore

The thing that prevents this working in the new catalogue is Encore lacks the Pathfinder Pro “Bib Table” which allows you to place links on the record display itself. In the screenshot above the Bib Table is the space on the record that contains three buttons, including the store request button circled in orange.

This is a problem as many Innovative sites have built services around this feature of Pathfinder Pro that include placing a link prominently on the bibliographic record.

Towards a solution for Encore

The latest Encore release 4.2 allows you to customise the Encore record display by including your own JavaScript. My presentation on this from the European Innovative Users Group conference 2012 with further examples is available:

Customizing your Innovative Encore library catalogue with JavaScript – EIUG 2012 presentation.

View more PowerPoint from Andrew Preater

I decided there simply had to be a way to insert a link to the store request form into the page using JavaScript…

  • Our first thought was using JavaScript to check if the record is in the store, then building a link to the request from by plucking bits of metadata from the page. This was a non-starter as the structure of the rendered Encore page is not semantically sound enough to work with in this way.
  • Second thought was to use an Ajax call to scrape the record display of the classic WebPAC which would be easier to work with. This isn’t possible as Encore and the WebPAC run on different Web servers so you run into the same origin policy. And no, you can’t set up a proxy server on your Encore server to work around it.
  • Third thought was using a Web application with a dedicated screen-scraping library that that could pull the metadata from the classic WebPAC or Encore. We’d link to this from the Encore record display and allow it to direct the reader’s browser to the populated request form. This is almost what we implemented. Read on…

How to do it

Building a link out from the Encore catalogue is simple. The JavaScript for the Encore record is available as a Pastebin for easier reading.

What this will do:

  1. Get the system record number. The simplest way I’ve found to do this from the page itself is use the document.URL which contains the record number.
  2. Check to see if a div exists with ID toggleAnyComponent. Not that you’d know it from the name, but this div is rendered only if there is an attached holdings / checkin record which means we’re dealing with a journal record.
  3. If if exists, check to see if the div matches a regular expression for the string “STACK SERVICE”.
  4. If it does match, create a link out to our Web application and append it to the existing customTop div.

This link out appears in this form:

http://www.ulrls.lon.ac.uk/stackrequest/parse.aspx?record=1746208

1746208 is the system record number for Egoist, minus the leading ‘b’ and trailing check digit.

Web application for screen-scraping

The real work is carried out using a Web application written in ASP.NET by my team member Steven Baker. Steve used the Html Agility Pack which is a library for .NET ideal for scraping Web pages. Of course, you can use your favourite language to accomplish the same thing.

Scraping from the Encore or WebPAC record display is a complicated business and how our library has named our various locations and classmarks was not helping.

So instead of scraping the page directly and including lots of different tests to deal with the various oddities found in the markup, it’s much simpler to scrape the WebPAC record display and pick out the div containing the link rendered by Pathfinder Pro.

This link already has the metadata required for the store request form so it’s then just a matter of using this URL to send the Web browser on their way to the request form.

The first step is to load the classic WebPAC page using Html Agility Pack. The link from Encore provides the system record number to generate a link to the WebPAC screen in this form:

http://catalogue.ulrls.lon.ac.uk/record=b1746208

The Pathfinder Pro wblinkdisplay div from the classic WebPAC looks like this for the example of Egoist : an individualist review:

<div class="wblinkdisplay">
<form name="from_stack_service159_form">
<a href="" onClick="javascript:loadInNewWindow('/webbridge~S24*eng/showresource?resurl=http%3A%2F%2Fwww.ulrls.lon.ac.uk%2Fstackrequest%2Frequestform.aspx%3FJtitleText%3DEgoist%2B%253A%2Ban%2Bindividualist%2Breview.%26JlocationText%3DSTACK%2520SERVICE%26JclassmarkText%3DPR%2BZ%26JbibText%3Db1746208a&linkid=0&noframe=1');return false;"><img src="/webbridge/image/request.gif" border="0"></img></a>
</form>
</div>

You can see the URL Pathfinder Pro normally redirects the Web brower to. The next steps are:

  • Select the div with class wblinkdisplay.
  • Cut out the URL based on using “resurl=” and “‘)” at the start and end of the URL to get the indexes needed.
  • URL-decode the resulting string.
  • Redirect the Web browser to that URL.

In live use, this is so fast that the end user doesn’t realise anything is happening in between clicking the “Request from Store” button in Encore and getting to the request form itself.

Comments on this approach

There are several advantages to handling Pathfinder Pro and Encore integration this way:

  • The work to pick out metadata from the record has already been done in Pathfinder Pro and doesn’t need re-implementing.
  • This will be easy to extend to other store journals, or if journals move from open access shelves to the store. We only need set up a new location in Pathfinder Pro and it’s reflected in Encore as well. We’d have to do this anyway to enable online requests in the classic WebPAC.
  • This approach isn’t specific to journals and could be used to put any links generated by Pathfinder Pro into Encore.

However, this isn’t a complete solution as it doesn’t just give you the Pathfinder Pro Bib Table in Encore. This is what we would really like and what we have asked our software supplier for. That said, if you can pick something unique out of the Encore record to test for then you can link out from Encore in a way that replicates the behaviour of the linking in classic WebPAC with Pathfinder Pro.

Please do contact me with any questions or your thoughts, or leave a comment below.