BISG Metadata Plan: 2018
Monday, March 19, 2018
Posted by: Brian O'Leary
The Book Industry Study Group proposes a plan to address U.S. adoption and use of the
ONIX 3.0 standard for book metadata. The plan includes several steps to be taken in
calendar 2018, including fact-finding, outreach, development of a refined business case,
persistent communication, and BISG-sponsored education. This document provides the
background, rationale, and details for the project.
The Book Industry Study Group (BISG) has played a significant role in promoting best
practices in the development and use of metadata in the book publishing industry in the
United States. The U.S. market was one of the first to move to widespread use of ONIX,
and an estimated 800 publishers representing the majority of titles published in the
market provide metadata using the ONIX standard.
ONIX 2.x has been in place for more than a decade and a half. The number of publishers
using ONIX 2.x now, as well as those providing metadata in other ways, have made the
U.S. market a difficult one to move to a new standard. Systems, procedures, and
investments made around ONIX 2.x compound the difficulty of migrating to a new
standard. The growing number of ONIX recipients, many requiring specific ‘flavors’ of
the ONIX message, is also a factor.
Within the U.S., efforts to improve metadata efficiency and effectiveness across the
supply-chain have met with mixed results. In 2012 BISG commissioned a comprehensive
study of metadata provision and receipt. An initial sunset date for ONIX 2.1 had been
announced around that time, and BISG wanted to develop a reasonably comprehensive
picture of the current state. As a result of this research, several opportunities to
improve metadata processes were identified. They included:
• Adopt the practice of comparing metadata to the actual product at the time a
book is printed or an e-book file is created (create an internal feedback loop).
• Create stronger feedback loops between recipients and senders, to improve the
quality of supplied metadata.
• Confirm shared definitions for metadata fields like page count (well-defined in
the specification but not uniformly interpreted by senders and recipients).
• For recipients, clearly articulate when updates occur and what gets updated and
• For both senders and recipients, improve the discussion about what metadata is
changed, added, and deleted throughout the supply chain.
In addition, the research uncovered opportunities to “future-proof” the use of metadata
in the supply chain. Recommendations for doing so included:
• Senders and recipients should collaborate to further automate data workflows
and compress cycle times.
• Senders and recipients should also prepare for more frequent metadata updates,
particularly concerning price.
• The processes used to prepare metadata for print and digital products should be
harmonized, prior to or coincident with a move to ONIX 3.0.
• The use of style tags within ONIX should be reconsidered; the use of such tags
often creates problems for recipients, many of whom have taken to stripping the
style tags as a matter of practice.
• Engage new supply-chain entrants and encourage them to understand and
support the use of ONIX.
Work to address these recommendations began in mid-2012, after the study was
completed. Progress was limited, and after a period of time, the work was tabled while
considering other pressing metadata-related issues (whether digital products should be
assigned unique ISBNs is an example).
Over time, data recipients have increasingly customized their use of ONIX, forcing data
providers to generate unique feeds for specific recipients. Major publishers today talk of
generating as many as eight (8) feeds for different recipients. This practice presents
several operational challenges, costs providers time and resources, and likely drives up
the expense of managing metadata across the supply chain.
“Flavors” of ONIX also complicate efforts to certify publishers’ metadata feeds. With
multiple ‘interpretations’ of the standard, publishers lack a baseline against which to
ONIX 3.0 in the United States
BISG recognizes that U.S. adoption of ONIX 3.0 lags other countries. Following the ONIX
International Steering Committee meeting that took place in London in March 2017,
BISG formed an ONIX 3.0 working group to examine options to move the U.S. to the
A number of publishers, intermediaries, and systems providers are already ONIX 3.0-
enabled. Given the size and complexity of the U.S. market, any transition will take time.
BISG can work to reduce the time and parallel resources required, and the approach
described later in this plan can be used to that end. Because publishers and retailers lack
the components of a business case to justify investment to move to ONIX 3.0, the
working group tried to develop a business case for adopting ONIX 3.0 in the U.S. market.
The primary benefits of using ONIX 3.0 in the U.S. market include:
• Improved data about series (collections). Product ‘series’ are widely recognized
as a powerful marketing tool that increases sales to existing customers.
• Additional and more reliable information about digital products. Digital products
account for most of the growth in the market over the last decade.
• More accurate and actionable market and supply detail information. This
matches the increasing complexity of international markets (and the growth in
export sales for US publishers).
• Access to the complete range of collateral a publisher may provide. Publishers
produce a wider range of (particularly digital) collateral material aimed at both
reseller and consumer.
• Enhanced ability to describe and support new business models, including models
like ‘subscription’, rental and so on that cannot be described in ONIX 2.1.
• Future benefit: Availability of block updates (reducing size of update files)
• Elimination of obsolete and deprecated data structures means fewer variations
in the data and a more streamlined ingestion process for recipients
• Availability of ‘Acknowledgement message’ to increase feedback and automation
• Better documentation and support, and commonality with ONIX markets
A move within the U.S. to use ONIX 3.0 may also benefit independent Canadian
publishers, some of whom have accepted government funding to be ready to deliver
ONIX 3.0 files to data recipients.
The working group found a range of examples that show the time and cost required to
migrate to ONIX 3.0 is manageable. Estimates are not precise, but publishers are
typically reporting that making their data workflows ONIX 3.0-ready took less than
$30,000 in dedicated IT support. The investment required of data recipients is said to be
larger, with more research needed in this area.
Medium-size and smaller presses that use intermediaries and systems providers to
manage their metadata may not incur any significant expense, as their vendor base is
likely to already support ONIX 3.0. Firebrand, Klopotek, OnixSuite, Booksonix, Virtusales,
and others (particularly those whose client profile includes non-U.S. publishers) have
invested over the past few years to support ONIX 3.0.
The extent to which data aggregators and distributors are ready to move to ONIX 3.0
needs to be confirmed. In its review, the working group found that Bowker and Ingram
will accept ONIX 3.0. Baker & Taylor is processing ONIX 3.0 for both print and digital
inbound and has the capability of sending ONIX 3.0 for digital only.
Among retailers, readiness is mixed. Digital-only retailers (Rakuten Kobo, Google Play)
are seen as ONIX 3.0-ready. The working group learned this summer that Rakuten Kobo
would like to move all of its publishers to ONIX 3.0, and it is working this fall to build its
own business case for the current standard.
Within the U.S. market, other retailers (e.g., Amazon, Barnes & Noble) are not ready to
accept ONIX 3.0 for both physical and digital products. The significance of these retailers
for many publishers creates a ‘chicken and egg’ challenge for the market. Without
retailer engagement, publishers currently using ONIX 2.1 feel little urgency to invest in
the current standard. Even if the total investment required is moderate, it is still an
expense that can be put off until retailers require an ONIX 3.0 feed.
To address this situation, BISG plans to lead a five-part, ongoing effort to support
adoption of ONIX 3.0 in the U.S. market. Core components include:
• Improved fact finding (“Moving past anecdote”)
• Deliberate outreach
• Creating a refined business case with international participation
• Persistent public communication
• BISG-sponsored education
These initiatives are presented in greater detail in the following sections.
Improved fact finding
BISG created the ONIX 3.0 implementation grid several years ago, around the time that
plans and a timeline to sunset ONIX 2.1 were announced. The grid was maintained for a
period of time, but it no longer reflects the state of the U.S. market.
Between November 2017 and the end of January 2018, BISG will revisit the grid,
expanding it to list the full range of publishers, systems providers, data recipients,
retailers, and others. The grid will capture use (or non-use) of ONIX 3.0 as well as any
timelines for a change in status.
We will also work to cover the parts of ONIX 3.0 used by publishers or supported by
data recipients. These include product types, tags, code list values, and block updates.
The more granular information is expected to give publishers a better sense of the
functionality they can expect to obtain with a move to ONIX 3.0. Greater transparency
• Where data elements newly available in ONIX 3.0 are supported
• Where senders may be putting ONIX 2.1 data into an ONIX 3.0 ‘shell’
• Where recipients are not actually ingesting any new data, but instead are just
processing the ONIX 3.0 file structure.
The apparent growth of customized feeds and ‘flavors’ of ONIX 2.x make it important to
understand how senders, intermediaries or recipients customize or ask to receive
customized ONIX 2.1 feeds. Some precision is required to understand current practices.
We will work to separate legitimate cases for individual feeds (sensitive price
information, retailer exclusive promotional titles, etc.) from other reasons, which (as
examples) could include: feeds changed to feature an older code instead of a new one;
or tags used for the ‘wrong’ data element.
To the best of our ability, we’ll also work to understand who asks for the changes, why
the customization is needed, and whether a shift to ONIX 3.0 afford an opportunity to
reduce or eliminate the need for customization. Senders and recipients in the U.S.
market may also benefit from sustained education, communication, or clarification of
best practices, some of which is already anticipated in this plan.
To keep track of existing practices, ‘exceptions’ may be listed separated or added to the
ONIX 3.0 implementation grid. To the extent that the causes of customization can be
managed, we may develop a timetable for their reduction or elimination.
After the implementation grid is updated and published, BISG’s executive director will
schedule conversations, held in person or remotely, to better understand the current
and expected positions for major retailers (Amazon, Barnes & Noble, Apple, and others)
and the largest (top 20) publishers. These conversations, which will take place between
February and April of 2018, will be kept confidential, but common themes emerging
from them will be brought back to BISG’s Metadata committee for consideration and
In addition, BISG will target publishers working with systems providers that are ONIX
3.0-ready to highlight the value of moving to the current standard. For systems
providers that are willing, we will work to create a “how to” document illustrating ways
that publishers already on these platforms can cost-effectively migrate to use of ONIX
3.0. We will work to make this happen in the same time frame (February to May)
planned for one-on-one conversations with retailers and publishers.
Where publishers, systems providers, distributors, or retailers are willing, we will share
technical roadmaps with planned timing for rollout and implementation of ONIX 3.0. We
recognize that some supply-chain participants may hold back certain information. The
intent here will be to show progress, or an intent to implement, where possible.
Refined business case with international participation
At Frankfurt Book Fair, BISG noted that we could use help from representatives in other
markets to better understand the business benefits of ONIX 3.0. We also could use help
understanding how other markets moved quickly and completely to use of ONIX 3.0. We
have a sense that the structure of the U.S. market works against rapid and complete
We expect to be able to learn the business benefits of using ONIX 3.0 from publishers,
data recipients, and retailers in markets that have moved from ONIX 2.1 to ONIX 3.0. An
example could be Germany, where a fairly rapid transition is taking place. The review
may also include a comparative view through participation of multinationals with a
presence in both the United States and the United Kingdom. Some UK companies have
already adopted ONIX 3.0 (PRH, Macmillan) and may provide useful benchmarks.
Ahead of the London meeting of the International Steering Committee (April, 2018), we
will prepare a summary of the U.S. business case, accompanied by concerns, gaps, or
objections raised during outreach conversations. We will ask the steering committee for
feedback, in writing, at the in-person meeting, or elsewhere.
Persistent public communication
BISG’s Metadata Committee will convene three discussions among (in turn) publishers,
data recipients, and (after the two separate meetings) a summit of publishers,
distributors, aggregators, and retailers. The timing of these discussions will be
determined in part by availability of key participants. We propose the publisher meeting
for May 17, the retailer gathering for July 25, and the joint publisher/retailer summit for
Tentative agendas include the following:
May 17: What do publishers need?
Publishers articulate what they need to achieve, within their businesses, to justify
moving forward. Will include:
• Retailers supporting ONIX 3.0 (meaning that recipients will take data points in 3.0
that they don't currently take in 2.1)
• Better processing consistency across recipients - that is, better adherence to the
standard and best practices. Publishers hate having to make many and various
exceptions / changes to the ONIX output for recipients. This includes taking 3.0 for
all product types, not just for eBooks, as well as eliminating non-standard
interpretations of the supplied data
• Address concerns with perceptions of ONIX 3.0’s flexibility. To meet expressed
business requirements, the ONIX 3.0 standard can carry many different types of
data. As well, the range of types can be extended easily. While individual data
elements are no more ‘flexible’ than in 2.1, and ONIX 3.0 elements are often better
and more precisely specified, both data senders and data recipients in the U.S.
market feel they need more direction/standardization before moving to ONIX 3.0
• Review of international use cases, with comparisons done by multi-nationals where
possible, plus maintenance and supply of data elements that meet individual
• Estimate of cost and effort to implement 3.0.
July 25: What do recipients want?
Retailers articulate what data they want publishers to supply in 3.0, in order for the shift
to be meaningful and worthwhile. Likely to include:
• Better Series Data (Collections information)
• More information about digital products
• Accurate market and supply detail information
• All the marketing collateral available from a publisher
• Estimate cost to implement 3.0.
Participants will be asked to share their publisher guidelines for ONIX 3.0 to help identify
areas in which best practice may not have been implemented.
In migrating to ONIX 3.0, recipients will be asked to help the supply chain reduce or
eliminate the number of variations publishers must support using ONIX 2.1. This can
also provide benefits for data recipients in several ways (avoiding the need to build
multiple extracts; consistent data mapping; and uniform or shared parsers).
September 17: Data Recipient / Publisher summit
• Create agreement on the problems to be solved
• Overcome barriers / reluctance on both sides
• Establish realistic goals and timelines
• Develop agreements on implementation best practices (don’t migrate bad practices
used to manage ONIX 2.1 to ONIX 3.0)
• Identify opportunities to use ONIX 3.0 as a baseline for certification
• Develop and confirm target dates for implementation
Throughout this period, the implementation grid will be updated as new information is
received, and the content of the grid will be summarized for the BISG community on a
monthly basis. BISG will also develop, publish, and update an “ONIX 3.0 FAQ”, a
document that will go live before the first (publisher) gathering.
An update on the work completed and the progress made through September 2018 will
be provided in writing to members of the ONIX International Steering Committee ahead
of the October 2018 meeting at the Frankfurt Book Fair. It may be discussed there, at
the discretion of the committee.
Throughout 2018, BISG will provide or identify resources capable of providing support
(e.g. training events, consultation, in-person hand-holding, etc.), on aspects of ONIX 3.0
that are a significant selling point for the standard (e.g. e-book metadata).
Over that same time, a weekly BISG newsletter (planned for launch in January 2018) will
offer regular tips that support 3.0. We will look to EDItEUR, the BISG Metadata
committee, and publishers using ONIX 3.0 to offer content in areas like:
• Did you know that in 3.0 you can associate an author with a city – something
you can’t do with 2.1 – and retailers could use that info for ‘local author’
• Having difficulty communicating your rights data? Have you seen how ONIX 3.0
handles each market?
BISG and its Metadata committee anticipate focusing resources throughout 2018 on
making a case for adoption of ONIX 3.0 in the U.S. market. Full implementation requires
progress in areas BISG does not directly control, but we will make it a priority to
communicate the benefits and implications throughout the coming year, as outlined in
Final draft prepared for International Steering Committee review: December 27, 2017