Transforming Access to Library Resources Through Discovery Services

NISO Virtual Conference: Web-Scale Discovery Services: Transforming Access to Library Resources Update on the NISO Open Discovery Initiative Marshall Breeding http://www.librarytechnology.org/ http://twitter.com/mbreeding November 20, 2013 1

Index-based Discovery (2009- present) ILS Data Digital Collections Search: Web Site Content Institution al Repositori es Aggregate d Content packages Consolidated Index Search Results Access Open E-Journals Usage- generate d Data Custome r Profile Reference Sources Pre-built harvesting and indexing

Bento Box Discovery Model Aggregate d Content packages Search: Consolidated Index Open Access VuFind / Blacklight ILS Data E-Journals Search Results Web Site Content Digital Collections Central index & search functionality Institution al Repositories

Web-scale search problem ILS Data Digital Collections Search: Index Consolidated Web Site Content Institution al Repositori es Aggregate d Content packages Search Results E-Journals Pre-built harvesting and indexing Participating Content Sources ??? Non Problem in how to deal with resources not provided to ingest into consolidated index

Discovery Concerns Important space for libraries and publishers Discovery brings value to library collections Discovery brings uncertainty to publishers Uneven participation diminishes impact Ecosystem dominated by private agreements 5

Heterogeneous Representations Content objects represented by MARC Records for books and journal titles Citation data for articles Full text for articles Full text for books Abstracts and Indexing products Other metadata or enrichment

Discovery index issues Citations or structured metadata provide key data to power search & retrieval and faceted navigation Indexing full-text of content amplifies access Important to understand what is indexed Currency, dates covered, full-text or citation Many other factors 7

Library Perspective Strategic investments in subscriptions Strategic investments in Discovery Solutions to provide access to their collections Expect comprehensive representation of resources in discovery indexes Problem with access to resources not represented in index Encourage all publishers to participate and to lower thresholds of technical involvement and clarify the business rules associated with involvement Need to be able to evaluate the coverage and performance of competing index-based discovery products

Collection Coverage? To work effectively, discovery services need to cover comprehensively the body of content represented in library collections Why do some content providers not participate? How are A&I resources represented? Is content indexed at the citation or full-text level? What are the restrictions for non- authenticated users? How can libraries understand the differences in coverage among competing services?

Evaluating the Coverage of Index-based Discovery Services Intense competition: how well the index covers the body of scholarly content stands as a key differentiator Difficult to evaluate based on numbers of items indexed alone. Important to ascertain how your library s content packages are represented by the discovery service. Important to know what items are indexed by citation and which are full text

Some Key Areas for Publishers 1. Expose content appropriately 2. Trust that access to material will be controlled consistent with subscription terms 3. Fair Linking 4. Materials not disadvantaged or underrepresented in library discovery implementations 5. Usage reporting

ODI context Facilitate a healthy ecosystem among discovery service providers, libraries and content providers

ODI Pre-History June 26, 2011: Exploratory meeting @ ALA Annual July 2011: NISO expresses interest Aug 7, 2011: Proposal drafted by participants submitted to NISO Aug 2011: Proposal accepted by D2D Vote of approval by NISO membership Oct 2011: ODI launched Feb 2012: ODI Workgroup Formed 13

Organization Reports in NISO through Document to Delivery topic committee (D2D) Staff support from NISO through Nettie Lagace Co-Chairs Jenny Walker (Ex Libris) Marshall Breeding (Library Consultant) D2D Observers: Jeff Penka (OCLC) Lucy Harrison (CCLA) 14

ODI Timeline Target Date Milestone Status Appointment of working group Dec 2011 Mar 2012 Approval of charge and initial work plan Agreement on process and tools Jun 2012 Completion of information gathering Jan 2013 Completion of initial draft Jun 2013 Completion of final draft Sep 2013 15 Public Review Period commences Sep 2013

Balance of Constituents Libraries Marshall Breeding, Vanderbilt University Jamene Brooks-Kieffer, Kansas State University Laura Morse, Harvard University Ken Varnum, University of Michigan Sara Brownmiller, University of Oregon Lucy Harrison, College Center for Library Automation (D2D liaison/observer) Michele Newberry Publishers Lettie Conrad, SAGE Publications Roger Schonfeld, ITHAKA/JSTOR/Portico Jeff Lang, Thomson Reuters Linda Beebe, American Psychological Assoc Aaron Wood, Alexander Street Press Service Providers Jenny Walker, Ex Libris Group John Law, Serials Solutions Michael Gorrell, EBSCO Information Services David Lindahl, University of Rochester (XC) Jeff Penka, OCLC (D2D liaison/observer) 16

ODI Project Goals: Identify needs and requirements of the three stakeholder groups in this area of work. Create recommendations and tools to streamline the process by which information providers, discovery service providers, and librarians work together to better serve libraries and their users. Provide effective means for librarians to assess the level of participation by information providers in discovery services, to evaluate the breadth and depth of content indexed and the degree to which this content is made available to the user.

Subgroups for Info Gathering Level of Indexing + Communication of Library Rights Technical formats Usage Statistics Fair Linking 18

Specific deliverables Standard vocabulary NISO Recommended Practice: Data format & transfer Communicating content rights Levels of indexing, content availability Linking to content Usage statistics Evaluate compliance Inform and Promote Adoption 19

ODI Stakeholder Survey Collected data from Sept 11 thru Oct 4, 2012 Each subgroup developed questions pertinent to it area of concern 20

Survey Responses 782 Librarians 74 Publishers 15 Discovery Services 871 Total 21

Selected results Libraries: do you use a discovery service? Yes: 74%, Planning to soon: 17%, No: 5%, Don t know: 4% Smallest discoverable unit: Component title: 9%, Article: 25%, Collective work record: 11%, All the above: 50% Linking from A&I entry: 75 prefer linking to full text on original publisher s server 22

Librarians preferred Use statistics Total Number of Searches List of search query terms Referring URLs 23

Content providers (74) Contribute data: Yes-All: 44%, Some: 48%, No: 8% Current data: 12%, Current + back files: 85 Barriers to contributing: IP concerns, technology, staff resources Challenges in delivery: Complicated formats: 15%, transmission of data: 18, allocation of personnel: 23%, can t automate: 12%, None: 20% 24

Issues surrounding A&I resources Concern that A&I resources not be freely available to non authenticated users and only for subscribing institutions How to credit A&I data that contributes to search results Example: Index entry produced by enhancing full-text with A&I data Preservation of the value added by A&I in the discovery ecosystem 25

ODO Survey Report Issued January 2013 NOT the final report for ODI Survey findings, especially for those that responded to survey One source of input for the ODI final report of findings and recommended practices 26

ODI Final Report Issued for public Comment Comment period closed November 18, 2013 27

Report Topics Introduction In scope / out of scope Terms and definitions Evolution of Discovery Related initiatives Recommendations 28

General Recommendations Create oversight group Actions for content providers and discovery service creators to assert conformance 29

Recommendations for Content Providers Content providers should make items available to discovery service providers. Basic: Citations: specific metadata elements Enhanced: additional metadata + Full-text Provide to Libraries: disclosure of participation in discovery services 30

Recommendations for Discovery Service Creators Disclosure of content indexed Specific metadata fields Fair / non-biased linking Mechanisms for libraries to choose versions preferred for linking Annual statement regarding neutrality of linking or relevance Provide links to A&I services when applicable Usage statistics to Publishers 31 Searches

Report Highlights What is in and out of Scope Focus on content available to be indexed Quantity and form of content (citations / fulltext) Metadata fields contributed Role of A&I products Controlled Vocabularies Out of Scope Relevancy algorithms User Interface issues 32 APIs exposed

Technical recommendations Transfer of data from content providers to discovery service creators Make use of existing standards and protocols when possible 33

Current work Next Steps Review comments received Chairs + Workgroup members Make any needed revisions Submit for final approval by NISO D2D 34

Connect with ODI ODI Project website: http://www.niso.org/workrooms/odi/ Interest group mailing list: http://www.niso.org/lists/opendiscovery/ Email ODI: odi@niso.org 35