Friday, January 17, 2014

EPUB has a Steep Road Ahead (Notes from Open Book Hack 2014)

People I talk to about ebook technology belong to one of two camps.
  1. EPUB3 is the future of ebooks. 
  2. EPUB3 has lost the ebook format war because no one is supporting it. 

Here I'm helping Fortitude fix a bug.
(photo by Ray Schwartz)
So it was very enlightening to join a group of developers at New York Public Library last weekend at Open Book Hack 2014. Sponsored by NYPL Labs and the Readium Foundation, the event was convened to examine the challenges of the "Open Book" on the Web. Since I had written that another publishing hackathon "pretty much ignored ebooks", I felt compelled to attend, and write a report.

Open Book Hack was the first event I've been to where developers were actually working on EPUB, the format that is emerging as the underpinning of the digital book industry. This was both encouraging and discouraging. Encouraging because you could start to glimpse possibilities that EPUB will enable, and discouraging because the road ahead looks so steep.

Open Book Hack attracted a very stimulating mix of developers from around the world (who were in New York for Digital Book World) and local developers looking for interesting problems. Notably, there was a very impressive contingent of students from New York's Flatiron School. The projects are listed on a github wiki.

I was very eager to meet the folks from Berkeley's iSchool that have been working on epub.js, a javascript package that lets you read EPUBs from the web in your browser. One of them, Jake Hartnell, is also on the team building, a tool designed to let everyone annotate the web (and one of the sponsors of the event). (The others were AJ and Fred.) Sooner or later you'll see these tools added to But there's still a lot of work left to do.

For example, the project at Open Book Hack with the highest ratio of usefulness to impressive-sounding-ness was the project to add scrolling to epub.js (check it out, it works!). That's right, they added an option (mostly) to make chapter 2 come below chapter 1. You will soon be able to use ebooks on the web using an open-source 9th century reading interface.

There was one prize awarded. The winning project, "See and Read" allowed two people with two screens to interact via an ebook. Very nicely executed and well-deserved, but if I told you that I had invented something that allows two people to interact with a book, you would say "oh?" because you don't really need 100 billion transistors and two glowing screens to interact with a book.

Another project "Breadcrumbs and Beanstalks", extended Harvard's StackLife interface to enable ebook browsing similar to that found in a physical collection of books, with 2 dimensional browsing (the 2 axes being publication date and the general-to-specific axis of subject headings. It looked a bit clunky, but thar's gold in there somewhere.

Hugh MacGuire, Max Fenton, Jean Kaplansky, and Fendi M were doing something brave with linking in  PressBooks, but having already done too much linking in my day, I decided not to understand it. A team from Sophia University in Japan was doing something clever with textbooks and Readium. There was work on converting PDF to EPUB and a project to use Phonegap to make apps from EPUBs.

The project I joined up with focused on making a book club application around a shared EPUB reader. (It works, too!) We based it on epub.js and I wasn't very useful to the effort, because the three Flatiron-trained Ruby-on-Rails developers in our group were too awesome for my plodding-but-powerful python to compete with. So I helped with exploration and documentation of the API, and finding bugs and deployment gotchas in epub.js. I now know how to configure CORS for buckets in S3. Yeah that was my weekend. I also fixed a bug by staring, in an intimidating way, over someone's shoulder. Ah, good times.

What became apparent to me in working with these tools was that the freshly trained developers got everything to work by un-EPUB-ing everything. The web platform just works, with the one exception being that centering text blocks in CSS just doesn't, unless you look away from the screen. The EPUB platform always throws something in your way, for reasons that even StackOverflow doesn't explain. Ruby Zips won't unzip. Cross Sites won't request. Java Scripts won't bind.

EPUB's competition isn't Amazon and KF8 fixed layout, it's the web and HTML5 and its huge gravitational pull. For 90% of ebooks, the benefits of EPUB over HTML are scant (because EPUB is based on HTML!) and the development barriers are significant. It's been years, and still EPUB authoring tools aren't mature or mass market. Deployment tools are barebones.

Don't get me wrong. I'm still betting big on EPUB, but dammit, Publishing Industry, for an $80 billion pillar of modern society, you're investing a nanoscopic amount on your basic infrastructure (i.e. EPUB), despite the herculean efforts of the people I met last weekend.

  1. I was really impressed with the Flatiron School students I worked with. If the rest of the students are anything like Edina, Tiff, and Dan (Ivan helped a bit, too), they are going to have a huge impact on the New York area economy. Maybe I should learn RoR.
  2. Bill McCoy has done an amazing job bringing people together under the IDPF and Readium umbrellas. Imagine what he could do with financial support commensurate to his task. Perhaps he should take up bootlegging.
  3. Jake wrote up his impressions, too.
  4. As have Virginie Clayssen and Camille Pène, in French.
  5. Would have posted sooner, but MILESTONE IN UNLUE.IT.

Enhanced by Zemanta

Wednesday, January 1, 2014

In 2013, eBook Sales Collapsed... in My Household.

2013 was not the year the ebook industry was expecting. We hoped that ebooks would continue their explosive year-on-year revenue growth, and that the replacement of print by digital would proceed apace. We suspected that the growth of ebook sales might moderate, because, as HarperCollins CEO Brian Murray told Publisher's Weekly, "Nothing grows by triple digits for too long." But just as CDs replaced vinyl and digital downloads replaced CDs, it seemed obvious that the the age of the printed book was nearing its end; the century of the ebook was dawning.

We got a few things right. Internationally, ebook sales growth was strong. Print continued its slow decline. Bookstores continued to close. But for some reason, ebook sales in the US stopped increasing. And even started declining!

There are many possible explanations for this turn of events. There are technicalities with the data collection, particularly with publishers such as Amazon's imprints that don't report their sales numbers. Young Adult sales dropped steeply, as there was no smash hit to follow on the huge success of Hunger Games. 50 Shades of Gray didn't turn out to be a lasting relationship. And there's been a downward trend on prices, particularly as publishers start to use dynamic pricing to stimulate sales. But it seems to me that something in the environment is changing, more than just a market maturation.

Amazon probably has enough data to understand what's happening, but they're notoriously opaque about reporting numbers. On the other hand, they're quite good about reporting to customers what they've bought. So I decided to analyze my own household's Amazon data. I had the impression that my family was spending less on ebooks, but I wasn't sure, because they still seem to spend all hours of the day reading. The results were kind of shocking.

The graph shows my household Kindle ebook purchases from 2009-2013. As you can see, 2013 marked a steep drop from the 2009-2011 peak years of about $1000 per year.

I don't buy Kindle ebooks myself (I buy ePub only, so I can hack on them) but other members of my household have bought quite a lot. The average price paid is about $7, and this has held quite steady. But in 2013, Kindle purchases stopped almost completely, and they were not replaced by purchases on other platforms.

Based on detailed "interviews" with the subject ebook purchasers, here are some non-factors in this collapse:
  1. "Netflix-for-Books" services. Nobody but me has heard of them.
  2. Kindle Owner's Lending Library. Despite an well-used Amazon Prime subscription, they haven't figured out how to use it for ebooks.
  3. Our public library. Nobody but me has used it for ebooks.
  4. Piracy. As if!
The two main reasons for this spending collapse turn out to be:
  1. The Kindle acquired in early 2009 reached end-of-life due to a cheaply made power cord, and was replaced by an iPad. The lack of in-app purchase for the Kindle App has resulted in a significant impediment to Kindle purchases. The iBookStore has not attracted a single ebook purchase.
  2. The iPad owner now spends the vast majority of her reading time in fan-fiction websites, mostly and Same for the iPad borrower, but a different mix of websites.
I find it worrying that the Justice Department pursued an big antitrust suit against Apple and 5 of the big 6 publishers, won, and despite making an issue of Apple's in-app purchase ban in iOS, it seems to have lost the argument with Judge Cote. We'll see how that turns out.

It's worth paying close attention to the fan fiction sites. After all, 2012's biggest revenue engine for the book industry, 50 Shades, was a repackaged fanfic. On an iPad with a decent internet connection, the fanfic sites work better than ePubs. They link and they script. Just try making a link from one ePub to another and you'll get my point.  They deliver content in smaller, more addictive chunks, and they integrate popular culture MUCH more effectively than books do, for reasons relating primarily to copyright. The authors are responsive and deeply connected to readers; they often ARE the readers!

There's a fanfic site to appeal to every reader; I highlighted Wattpad earlier this year. ("AO3"), a project of the Organization for Transformative Works, a non-profit, experienced the growth in 2013 that was missing from the ebook sector. The number of works hosted by AO3 doubled to just under a million works, covering almost 14,000 "fandoms". (A good example of a fandom is the "Dragonriders of Pern" fandom, which currently hosts 534 works)., an advertising supported site, hosts almost 2000 fandoms and over 1.3 million works, more than half of which are in the Harry Potter or Twilight fandoms. Game oriented discussion forums also engage in fanfiction. (Popular in my house is

My anecdata might be completely anomalous, although Amazon, a very data-driven company, seems to be aware of the same phenomena. They've been making the Kindle into a full featured tablet to go head-to-head with the iPad. They've also launched a fanfic site called Kindle Worlds, which has 15 worlds and 341 works.

Early stage venture capitalist Josh Kopelman says that many of the best opportunities for startups are not those in expanding markets. "We love investing in technologies and business models that are able to shrink existing markets. If your company can take $5 of revenue from a competitor for every $1 you earn – let's talk!",  he has written on his firm's website. Kopelman founded in the early days of the internet, a company which shrank the book market by getting people to resell the books they had just bought for a fraction of the price of a new book. Microsoft's Encarta shrank the Encyclopedia business from $1.2B to $600M before Wikipedia shrank the business by another 90%.

In 2014, I'm guessing it's the book publishing industry's time to shrink. A convergence of tech startups, tech monsters, and tech non profits seems to be ready for the assault. The fanfic sites, the Wattpads, the Project Gutenbergs and the Manybooks, the Readmills, the Leanpubs and the Smashwords (and I hope the Unglue.its); these are people building the foundations of a creative industry that will flourish even if the ebook sales collapse that I see around me spreads to your house as well.

Happy New Year!
Enhanced by Zemanta