[Mimas logo]"epub@mimas"


Towards an Integrated Digital Library; Exploration of User Responses to a `Joined-Up' Service

Ken Eason1, Susan Harker2, Ann Apps3 and Ross MacIntyre3
1The Bayswater Institute, 9 Orme Court, London, W2 4RL, UK. (K.D.Eason@lboro.ac.uk)
2Department of Human Sciences, Loughborough University, LE11 3TU, UK. (S.D.Harker@lboro.ac.uk)
3Mimas, Manchester Computing, The University of Manchester, M13 9PL, UK. (ann.apps@man.ac.uk, ross.macintyre@man.ac.uk)

© Springer-Verlag
Publication information.

Abstract

Digital library users have to deal with many separate services. This paper describes efforts in the United Kingdom to use OpenURL technology to provide `joined-up' services. The focus is on zetoc, a national electronic service, which enables users to find references in a British Library bibliographic database. zetoc now uses OpenURL technology to provide routes to services, which might give users access to electronic full text versions of references they have found. Data is provided from two questionnaire surveys and an interview programme conducted to explore user responses to these services. These evaluation studies show that users want these integrated services and are extremely positive about them when they work. However, `joined-up' services depend for their success on the access rights that each user has to full text sources in their institution. As a result, the success level in obtaining full text varies considerably between institutions. Users in disadvantageous positions have expressed disappointment and frustration; the service may be regarded as a promise not fulfilled. The paper describes the development of `joined-up services' as a partnership at national and local levels.
Keywords: integrated services, electronic full text, user evaluation, usage analysis, OpenURL, resolver.

1 Towards a `Joined-Up' Service

A positive view of the rapid expansion of digital services is that many users now have access to a powerful array of services. A negative view is that they are faced with a bewildering array of overlapping services that are in a constant state of growth and change. How are users responding to these developments? Are they delighted by the new possibilities and grabbing every opportunity as it emerges? Or are they dazed and confused by the acronyms, the passwords, the procedures? Do they cling to the simpler world of shelves of books and journals? One finding that is emerging strongly from early studies of user responses to the digital library [1] [2] is that users value the convenience of being able to access digital resources from their workstations. However, they are concerned about the fragmentation of services and the different rules and procedures associated with them. They would really like to see more integration of the services available to them. They want to be able to move material found in one service to another and, in particular, to get from a reference they have found in one service to a full electronic text version that is available in another service.

The JISC (Joint Information Systems Committee) [3], which funds digital library developments for the education and research communities in the United Kingdom, has been supporting service suppliers and universities in attempts to provide users with a `joined up' digital service. Mimas [4], a JISC-supported national data centre at the University of Manchester, provides a number of national digital services and has been at the forefront of these developments. It has been part of a community of service providers and universities who have been exploring the potential of searching and linking technologies, specifically Z39.50 [5] and OpenURL (Z39.88-2004) [6], as a means of providing users with seamless movement from one service to another. The focus of this paper is the zetoc service, a national bibliographic reference service hosted by Mimas [7]. zetoc is at the centre of attempts to provide joined up services and, since 2000, there have been several stages of technical development that have strengthened links with other services. User evaluations have been conducted throughout the development of this service to determine the impact on end users. This paper provides two perspectives on these developments; the technical and service provider viewpoint as the service has been developed, and the user viewpoint as the service becomes available to them. The service has evolved steadily since 2000 but for convenience it is described below in two phases September 2000 -- June 2002 and July 2002 -- December 2003.

2 The Technical Development of zetoc Mark One

In September 2000, the British Library [8], in partnership with the University of Manchester, made available to UK Higher and Further Education institutions its Electronic Table of Contents data, which lists journal articles and conference papers in all subjects and currently includes more than 20 million records from 1993 to date. This service, zetoc, which was developed and is hosted by Mimas [9], provides a Web interface for searching by end users, and also a Z39.50 interface. The NISO Z39.50 standard [5] for information retrieval defines a protocol for two computers to communicate and share information. zetoc's Z39.50 interface enables data interchange with other Z39.50 services, including meta-search discovery requests. The zetoc service is free to UK institutions (supported by the JISC) and by subscription to UK Research Councils, Irish higher education institutions and, since 2001, to National Health Service staff.

Since its inception zetoc has been part of initiatives to provide an integrated, `joined-up', digital service [10], a `discover -- locate -- request -- deliver' [11] sequence that would provide a seamless route to the full electronic text of an article. The initial zetoc service provided users with the opportunity to `discover' the bibliographic citation details of research articles of potential interest within its large database in a timely fashion. Early enhancements to the service provided electronic `request' opportunities primarily for non- electronic `delivery' by, for example, document supply directly from the British Library and indirectly through traditional inter-library loan (ILL) routes, tailorable for each institution. With customized Z39.50 connections users can transfer references `discovered' in zetoc to their personal bibliographic databases using software such as EndNote or Reference Manager.

An orthogonal purpose of zetoc since its introduction is as a current awareness service. Users may request alerts when new journal issues of interest, or articles that match saved searches on keywords in an article title or author's name, appear in the nightly zetoc data load. The zetoc alert function emails details of each article within the requested journal issue or that matches a saved search, including a URL that provides direct entry into the zetoc web service, enabling the alerted user to take advantage of the document delivery, and more recent linking, functionality.

The service rapidly became popular in UK higher education institutions with over 13,000 users registered for email alerts in May 2002 setting 20,000 journal alerts that typically resulted in 8,000 emails being sent out per night. On the same date zetoc database use was around 20,000 sessions and 40,000 searches per month via the web interface. zetoc has received many very positive reviews including the accolade of `800lb Gorilla of UK email alerts' [12] because of its breadth of service.

3 User Responses to zetoc Mark One

In 2001 an evaluation of user responses to zetoc was conducted by Loughborough University. This took the form of an electronic questionnaire announced on the Website and on email alerts sent to users. The questionnaire asked users to report the usage they were making of zetoc and what they regarded as its strengths and weaknesses. A full account of the evaluation is available on the Website [7] and is reported in Eason et al [13]. A summary of the main conclusions is presented below.

The questionnaire generated 655 responses from users in over 100 different institutions. The overwhelming view of zetoc was that it provided a very valuable service as a broad-based means of keeping up-to-date with developments. It was regarded as a simple service to use and the great majority of comments were positive.

The questionnaire asked users to report what aspects of zetoc they made use of. It identified 8 specific services, e.g. alerts, searching the database, ordering articles etc, 11 support and adaptation features, e.g. the helpline, website demonstrations, revising alerts that had been set, saving searches etc, and 3 linking services, e.g. to bibliographic services such as EndNote. This gave a total of 22 possible features that could be used. The analysis showed that 75% of the users had set journal alerts (an average of 13 journals per user) and that 50% had made a search using the database. Only 10% of the users had made use of the Z39.50 capabilities to link with other services. Many of the features were hardly used at all and very few users made use of the document ordering features in the system.

What was striking about the results was that there was a group of active users who made wide use of the functionality of the service and another group who made very limited use of it. The average use across the sample was 4.1 features but 108 users (17%) were in the range 6 to 18 and 543 (83%) used between 1 and 5 features. For the purposes of further analysis we called these two groups the `active integrators' and the `passive majority' and explored further who they were.

The active integrators were spread across many institutions and were primarily librarians (41%) and research staff and doctoral students (37%). The average score for the librarians in the sample was 5.2. From their reports this group were knowledgeable about the electronic services available to them and keen to exploit the resources of the digital library from their workstation. They made use of many other services and, for example, often used zetoc as a secondary resource, i.e. they used a domain specific service to reach the main journals they used (and could often get full electronic text) and they used zetoc to keep abreast of developments in a broader array of journals. It was this group that were making use of the Z39.50 facilities and they indicated that developing a seamless electronic service that enabled them to move from an interesting reference to full electronic text was a major priority for them.

The passive majority primarily used zetoc as an alerting service. As one user put it ``I set up some alerts and just let the service do its job''. To most of these users zetoc was the stream of emails they got giving them the table of contents of recent issues of journals. Some but by no means all also made occasional use of the database. They were largely unaware of the other features of the service and made little or no use of the linking services. These users reported that, when they found some references of interest, they often went to the library to check them out. These users also expressed a desire for a seamless route to electronic full text but were much more concerned by the plethora of services that existed and the knowledge they needed to make use of them. They wanted better services but they already felt overloaded and did not want anything that would further complicate their lives. There was representation of all categories of user in this group but it was noticeable that it contained nearly all of the faculty users (the average score for faculty staff was 3.2).

The conclusions drawn from the first evaluation were that zetoc was providing a valued service for a large number of users by providing them with a simple way of keeping abreast of developments in their field. For most users, however, this was all it did. When they wanted to follow up references they left zetoc and either went to other systems or visited the library. However, there was a strong desire to achieve seamless integration from `discovery' through to `delivery'. It was clear that there was, in the active users, a group of people who would be watching for developments and would be the likely early adopters of any enhancements to the service. These were likely to be librarians, who had a professional interest in developments to the `tools of their trade' and researchers and doctoral students who had reasons (and the time) to keep up-to-date with the technical resources available to them. Getting adoption by the passive majority, even if they wanted the enhancements, might be more difficult to achieve.

4 The Technical Development of zetoc Mark Two

During 2002, a series of enhancements were made to the zetoc service, the major change being the enabling of zetoc as an OpenURL [14] `source'. Alongside this service-specific activity, Mimas was part of the NISO OpenURL Committee and was proactively supporting institutions in their exploitation of OpenURL technology. This included hosting instances of Ex Libris' SFX software [17] for a number of institutions and holding a series of `Talking Shops' [18] with early implementers, both actual and potential.

An obvious enhancement to the zetoc web service was to provide access to the full text of an article when it is available electronically, and maybe also article abstracts and further relevant information. The problem of providing a link to the full text of an article is twofold if the user is to be given a link that is not a dead end. Firstly the bibliographic citation information must be translated into a URL that will link to an article, preferably using a standard, interoperable syntax. Secondly this link must, if possible, be to a version of an article that the user may access freely, maybe by a valid institution subscription.

The `OpenURL Framework for Context-Sensitive Services' is a proposed ANSI/NISO standard, Z39.88-2004 [19] that provides a standard way to describe a referenced resource, bundled together with the associated resources that comprise the context of the reference. Before the publication of the OpenURL Framework standard applications were based on the draft, `de facto' standard, OpenURL version 0.1 [20], which transports as its `payload' a description of the scholarly resource, such as a journal article, inline as the `query string' of a URL.

Typically, in a digital library context, a user will click on an OpenURL link in an HTML page, for example beside a citation within a reference list of an electronic article, or alongside a record in a service such as zetoc. The OpenURL for the reference is passed to a linking server, or `resolver', which will return to the user a selection of resources pertinent to the cited article, preferably including a link to a copy of the full text of the article that the user is entitled to access. Typically an organisation's OpenURL resolver includes a knowledge base that records holdings, subscription and preference information specific to that organization.

zetoc, now being an OpenURL source, provides the users with the potential to link from a full record of an article to `more information' about the article. This functionality is implemented using OpenURL, currently version 0.1, to provide a consistent linking syntax `from' zetoc. For zetoc users whose institution has an OpenURL resolver the OpenURL query is passed to that resolver. When a user activates the `more information' link they immediately see the menu of their appropriate resolver, as defined by their institution, which will include a link to the full text of the article if the institution has a valid subscription.

The problem when implementing OpenURL linking from an information service is that of knowing to which resolver to send the OpenURL query. An institution wishing to use zetoc as an OpenURL source can register their resolver address via the zetoc helpline. A user's institution can be determined from their login to zetoc. Several institutions had enhanced their electronic services available to their users by the introduction of OpenURL resolvers at the time zetoc was enabled as an OpenURL source, and more have acquired OpenURL resolvers since then.

For the benefit of users at institutions without OpenURL resolver software, a default `more information' facility is provided by zetoc -- an online article search using MDL's `LitLink' resolver [21], which provides links to places where an electronic version of the article is available. This cannot determine whether or not the user has rights of access, but where access is allowed the user will be able to obtain the full text of the article. Even where full text is not accessible, the user may still perhaps retrieve an abstract. In order to manage user expectations, this facility is deliberately undersold and announced as ``worth a try''.

Enabling zetoc as an OpenURL source has proved to be popular since it was introduced in November 2002 and Table 1 provides a summary of the logged usage statistics. Usage has grown considerably during the course of the first year of service. This is particularly true for the institution resolver service, an increase partly the result of more universities having a resolver accessible from zetoc (from 4 to 16 universities), and partly the result of more users using the facility.


Table 1. Usage Statistics for OpenURL Enhancements

FacilityUsage Nov 2002 Usage Nov 2003% Increase
Institution Resolver1521,755948
Default LitLink Resolver 3,3285,55767

A parallel development, though independent from zetoc, was the introduction by some institutions of software to provide integration of their electronic services using meta-searching portals that propagate a user's search request across several resource collections, generally implemented using Z39.50. As a result, many users were able to access the zetoc database from other services, for example from portals, from bibliographic software, and from links `to' zetoc via OpenURL resolvers. The introduction of these enhancements had a dramatic effect on the usage of zetoc In September 2002 there were 2,317 Z39.50 target sessions across UK academic institutions. By October 2003 it had risen to over 38,000. In March 2003 two of the institutions with meta-searching portals had achieved over 20,000 zetoc sessions in the first quarter of 2003.

A recent JISC-funded case study on the implementation of library portal software (Ex Libris' MetaLib) [22] revealed a significant increase in network database usage once the portal was launched to users. Included were databases that could be cross-searched and top of the list was zetoc, searches on which rose by 1385%.

The zetoc service has then become a technical mechanism by which users can, on many occasions, achieve a seamless link between discovering a reference and obtaining a full text electronic copy. The statistics show that these features are gaining popularity. But what view do the users take of these developments?

5 Evaluation of the Enhanced zetoc Service

In 2003 a user evaluation was undertaken by Loughborough University and the Bayswater Institute with the specific intention of examining the impact of the enhanced services. An electronic questionnaire was made accessible from the zetoc Website in order to collect quantitative information. A small number of interviews were undertaken to provide in-depth qualitative evidence of the way users were integrating zetoc with other services they used.

5.1 User Views of the Enhanced Service

There were 167 responses to the questionnaire and 26 interviews were conducted. Both data collection methods asked the same questions about usage of zetoc and reactions to the enhancements and the initial analysis was of the total of 193 users. The users were given a list of properties of digital libraries that they were asked to rate. Asked what was the most important service they were now receiving they chose `keeping up to date with current developments'. This is the primary role that zetoc fulfils. Asked what was the most important service they wanted to see they chose `getting from a reference to electronic full text'. This confirms the view that `joined-up services' are a priority for users.

Users were likely to receive different levels of service in different institutions and in Tables 2 and 3 below findings are reported for four types of institution:

Resolver Universities -- are those that have implemented their own OpenURL resolver service and have given zetoc access to it. These universities tend to have rich electronic journal subscriptions. As a consequence users in these universities using the `more information' facilities will be told whether their university has access to a full text version of the article they are pursuing.

Established universities -- in the UK there are many established universities that have rich electronic journal subscriptions but do not have their own resolver. Under these circumstances the zetoc `more information' facilities will tell them whether an electronic full text version of an article is available but not whether their university has access to the service that provides it.

New universities -- in the UK there are many institutions that have recently achieved university status but do not have access to a comprehensive electronic journal resource and do not have their own resolver.

Others -- This includes colleges, research centres and National Health Service Trusts. They tend to be in a similar position to the new universities in terms of resolvers and of subscriptions to electronic journal services.

Table 2 reports the number of users responding from each category of institution and the percentage within each category who were librarians. This is significant because the first zetoc survey demonstrated that librarians were more active users than the general population of users. For three categories of institution in the sample, for example, librarians are in the majority in the sample. It is worthy of note that the sample from the resolver universities has the lowest percentage of librarians. It appears that this sample is dominated by the `active integrators' amongst the user population perhaps because it was a specific request to report on the enhancements.


Table 2. User Evaluations of the zetoc Enhancements

InstitutionsUsers zetoc Score
 No.% Libs Overall ScoreIntegration
Resolver Universities3435 8.21.6
Established Universities4870 8.91.0
New Universities40607.8 0.8
Others, e.g.Colleges7169 7.20.6
Total / Average19362 7.81.0

Table 3. User Evaluations of the zetoc Enhancements (continued)

Institutionszetoc Enhancements
 TotalBetter SameWorse
 No.No.%No. %No.%
Resolver Universities2826931 413
Established Universities312581 51613
New Universities251456 104014
Others, e.g.Colleges341441 185326
Total / Average1187967 342954

A zetoc usage score was again calculated, this time out of a total of 29 because the enhancements introduced more functionality. The overall zetoc usage score in Table 2 is 7.8. This is higher than expected from the pro rata increase in the functionality and is probably correlated with the high percentage of librarians in the sample. Table 2 shows the part of the usage score that is the result of using the integration facilities, including `more information', seeking an article on-line, exporting results to other services, e.g. EndNote etc. The highest scores for integration are in the resolver and established universities where these opportunities are likely to be most effective.

There were a total of 118 comments about the enhanced facilities and the results are reported in Table 3. The columns `better', `same' and `worse' record the results of asking users to evaluate the enhancements. 67% of the responses were that the enhancements had led to a better service. However, much higher results were obtained for the resolver and established universities (93% and 81% of responses reporting the service had improved). The percentage of `better' is lower for the other categories and nearly half report that the service is the `same'. The numbers reported in some categories are too small for these results to be statistically significant but there is some evidence that, whilst users with rich electronic resources are benefiting from the `join-up' features, others do not find them helpful.

5.2 User Strategies from `Discovery' to `Delivery'

To explore the reactions of users to the new services in greater depth, 26 interviews were undertaken at six universities: three with and three without local resolvers. This sample is too small to yield statistically valid results; the study was undertaken to provide rich qualitative data about user strategies and the reasons for them. The users were recruited from respondents to the first questionnaire who had expressed a willingness to help with further research. The sample consisted of 12 faculty members, 6 researchers, 4 doctoral students and 4 librarians. The method of recruitment is again likely to have created a bias towards more active users. The zetoc scores of the users was assessed using the scale developed for the second interview and the users were asked to describe how they discovered references of interest to them and how they obtained full copies when they needed them. The average zetoc score for the 16 users at universities with resolvers (RU) was 7.6; the average for the 10 users at non-resolver universities (NRU) was 5.9.

From the descriptions of their usage strategies, the users were placed in four categories:-

The numbers of users in these categories is too small to draw more than the following tentative conclusions. First, that awareness of the OpenURL services is not high. The problem is that the passive users are not, by definition, listening to news about new services and, often overset by current systems, are unwilling to try new ones. Of those interested in electronic services, there is a group who have found an acceptable strategy and see no reason to experiment further. But there is a group in the universities with resolvers who are using the OpenURL services, are meeting with success, are excited by the new possibilities and are extending the range of their use of services like zetoc. Perhaps the group to have most concern about are those in non-resolver universities who discover that electronic full text is available but find they are refused access. These users mirror those in the second survey who reported the new services to be `no better' or `worse' than before. Frustration and disappointment could well turn these users away from electronic services.

6 Discussion

There can be little doubt from this research that users want to see digital libraries develop in a way which provides a `seamless' route from reference details to full electronic texts. zetoc has an established position as a leading national service that provides users with a current awareness service and it has now demonstrated that OpenURL technology can be used as a basis for offering users this seamless route. The users' response to the delivery of what they have been requesting has shown a lot of variability and this has many lessons for the evolutionary path by which the digital library is being built. There is clearly a section of the user population that is making use of these new facilities and is excited by the `reality' of a seamless digital library service. However, these users are a small part of the total population and there are a number of barriers to overcome before all users are taking advantage of these innovations.

One significant barrier is the provision of electronic resources in each university and the use of resolver services. A service like zetoc can only create a means of discovery and links to possible sources of full text; the local institution has to provide the rest. Widespread adoption of seamless link-up depends on a partnership at national and local levels. One specific need is to help users appreciate at an early stage what permissions they do have in order that they can avoid time wasting and frustrating hunts that only lead to denials of access.

Another barrier is the existence of a large number of passive users. We suspect that this is a property not just of zetoc but of all complex information services. This research has identified a passive majority who use a limited range of facilities and are not willing or able to spend time understanding new services. Many of them want a seamless, electronic service but not if it involves spending time learning new systems and experimenting with new facilities. We have noted elsewhere [23] that users of computer systems work out for themselves an implicit `cost-benefit' strategy that seeks to maximise success and minimise failure and time wasting. Many of the users in this study (especially librarians) are prepared to accept considerable effort for what may be limited return because of the great promise of a seamless route to electronic full text. Many other users, especially members of faculty, expect to get new benefits from relatively little effort and will continue to use the facilities which are familiar to them if the effort to change is considered too great or the benefit problematic. One advantage of OpenURL technology is that the effort of following a few links is quite limited and straightforward and, if it works and provides electronic full text, the effect on users is dramatic. On the basis of the interviews it seems that once they taste the `magic' of seamless delivery they will want to continue. This problem, however, is that, as passive users, they may never discover the new facilities. It could be that the local librarians could play a pivotal role in bringing them to the attention of passive users. This cannot be done by traditional mechanisms; by definition passive users do not respond to invitations to training events. But suppose there is a major service improvement, for example, registering a local resolver with zetoc. A message to end users saying `try this' with step-by-step instructions may be sufficiently well targeted, easy to do and likely to be successful to encourage many passive users into action.

Acknowledgements

We are pleased to acknowledge the funding from the British Library and the Joint Information Systems Committee that has supported the provision of the zetoc service, the `join-up' work and the evaluation studies that have been reported. We wish to thank two colleagues for their contribution to the evaluation studies; Martin Ashby, for work on the two questionnaires, and Sue Richardson, for help with the interview programme.

References


28 June 2004

[Go to Electronic Publishing at Mimas]Electronic Publishing          [Go to Mimas home page]Home Page          [Valid XHTML 1.0!]