Sunday, 14 April 2013

Could Big Publishing put up a website to compete with Amazon's?

Someone on KBoards started a thread about whether readers would flee to the Big Six (actually the Big Five now we have Random Penguin) for guidance as to what to read, faced with a flood of indie offerings of mixed merit. Perhaps they might band together and create their own website to rival Amazon, selling only quality books from approved writers? KB Burke's comment was so interesting I asked his permission to quote it here:

Just some observations from a software developer ...

The big publishers haven't figured out discoverability. Their talent pool means little when readers can't find the books. Just because they put 1,000 vetted writers on a site, doesn't mean I can find the 12 that I want to read.

To do that, they need years of buying behavior. The science behind 'also boughts' is called Collective Intelligence, and making the website is 10% of the battle. Basically, you model the mass behavior of people, and then identify patterns. 16 year old guys who bought Warhammer books, also bought Halo books. If you're 16, and have bought one of these things, you might like the other. And so on. It's statistics, but you need data to build the model.

This is why Amazon bought Goodreads, for the data. If the Big Six understood software engineering they would have bought that network years ago. Websites don't sell books. Data sells books, and the big publishers don't have the data to compete. They are 10 years late to the party.

This why Amazon is always tweaking their algorithms. They get more data and adjust their models. It's no different than a presidential campaign modeling an election by 'likely voters.'

Dozens of tech companies, with big time talent, like Apple and Google and Sony, have failed to compete with Amazon. They don't have data, but they do have some of the most talented engineers in the industry. Think about that. No one in New York will have anything like Google's resources, and Google isn't hurting Amazon at all.

As far as quality goes ...

There might be 100,000 bad indie titles, with quality issues, but it is probably a bell curve. Some percentage are high quality, 5-10%, that compete with the big publishers.

This has been true since the 1930s and the beginning of pulp. There is an ocean of crap, and a small handful of standouts. Who curates that crap doesn't matter. This is why word of mouth sells books. Amazon has made significant leaps in this regard, with their algorithms, but no one else is close.

The ocean is bigger today, but the model is the same as Edgar Rice Burrows. His books sold, despite that ocean of crap.

Collective Intelligence is really why Amazon dominates the book industry. The traditional publishers have a team of editors telling me they found another Edgar Rice Burrows. They are telling me what I should read. Meanwhile Amazon is telling me what people do read. People who like Burrows have also bought x, y and z. This helps me find my tribe, so to speak.

This is the now. The automation of white collar jobs, like book curation, and it won't ever go away. New York thinks good taste can't be automated, but that's because they don't understand the science.


  1. Yeah, the Big Five (isn't it now, since Random Penguin?) are no threat as an online store. No one is within a mile of Amazon when it comes to predictive analytics.

    But even they could become a lot better. They still recommend that I buy my own books, for example, just about every week, and their algorithms can be manipulated (e.g., John Locke's review buying). Now if DARPA and Amazon ever got together then they'd be able to tell you who you should marry (or divorce) and what exactly you want for dinner tonight...

  2. New York thinks good taste can't be automated, but that's because they don't understand the science.


    Bingo. I use to work for a company involved in the ebook industry, (I write full time now) and customer behaviour and discoverability is the holy grail.

    Thanks for posting this, Lexi!

  3. Eric, I think the Big Five are all too aware they will never catch up, and this is what makes them so vindictive towards Amazon. I can't judge their algorithms, as I frequently look up books I don't intend to buy from KB sigs out of curiosity.

    Maia, I wouldn't be at all surprised if Amazon found the holy grail, given its other achievements.

  4. "customer behaviour and discoverability is the holy grail"

    Amen to that. There's a great deal of cutting edge research going on into the subject. So, while I could imagine that while Amazon may well end up owning the answer, I suspect it will be thought up by some pointy-brained geeks somewhere else... and sold to the highest bidder. It will be Amazon because the big five still don't really understand the value of the information.

    Alternatively, if anything beats Amazon, I should imagine it will be some kind of dating site-based algorithm used to match readers with their favourite books, which will all be linked form Amazon anyway. ;-)

    I like the convenience of Amazon but I suspect their corporate ethics are... not the same as mine.



  5. Plih. Why aren't I a pointy-brained geek with an answer to sell to the highest bidder, instead of a boring old jeweller/writer?

  6. I'm in your corner, Lexi - all this number-stuff is no doubt how it works, and if I were really good at selling my books I'd understand the system and make it work for me. But hey ho, I'm not - give me my notebooks and I'm happy!

  7. Good Reads has data. But buying it also nips in the bud one viable competitor of Amazons- which pretty much is what all the single platform software providers try and do. It's why Microsoft was told that it had to stop bundling so many aps with their OS.

  8. Russell, surely Goodreads, as it does not sell books, isn't a viable competitor to Amazon, just a useful source of data? Though I suppose it's a useful source of data that would have benefited Big Publishing had they had the wit to see it. (What's an OS and who told Microsoft to stop?)

  9. Windows is Microsoft's OS. By bundling apps with the operating system Microsoft were stifling competition.

    As well as Goodreads I think that Amazon has bought Ivona which has some of the best computer voices available.

    Alternative on-line booksellers are going to need some awesome innovative ideas to compete!

    Actually I think the Amazon system could be improved a lot by allowing readers to search interactively. Perhaps a reader could specify a favorite author and search for new authors with closely similar characteristics.

    These characteristics need to be quantified somehow. Here readers would need to participate by contributing their assessment (marks out of ten) for genre, plot, writing style, characterization etc.

    This interactive input would greatly enhance the value of the statistical databases.

    It needs a lot more research and I fear it will always be difficult for debut writers until enough reader input is obtained!

    There may be some book-loving genius in a back room somewhere hatching up something along these lines right now. A literary version of the young Bill Gates.

    It would be fascinating to watch it emerge! LOL

  10. What about Amazon also-boughts, Q? And I suppose tagging was a primitive version of what you suggest. But authors collaborated to tag each other (and some tagged successful authors' books with their own names). Now tags are no more.

    Life has always been hard for début authors. It's something that these days they can go direct to readers rather than pursue agents and publishers, often fruitlessly.

  11. I find also-boughts quite useful in throwing up suggestions to investigate further. However it is a very crude filter. When people review books I would like more than just an overall *-rating. Incentives to write reviews would also help, eg. discounts on next purchase.

    Statistics always yield averages and of course Mr average does not exist. The sort of profiling that I would like could give averages more closely aligned to my personal preferences.

    Début authors really need some sort of label to act as a beacon in the 'Tsunami of crap'. Finalist in a writing contest (Golden Heart for romance perhaps).

    There might also be scope to set up something where the new writer could get there book profiled by a team of readers. A fee would be necessary to filter out the less serious. There could then be a separate listing of new authors with a 'stamp of approval' and detailed ratings.

    This could be a way of providing the beacon to attract readers and give a fast track to the top for the truly talented.

    Just some of my random musings! LOL

  12. I'd have expected you to be more Darwinian, Q. As it's so hard for a book to attract attention, most indie offerings disappear to the dungeon depths of the Amazon chart, never to be seen again. Sometimes one hears the odd muffled shriek or groan...

    What you propose is to replace one lot of flawed gatekeepers with another bunch, likely to be equally flawed. Let the reader decide, I say.

  13. most indie offerings disappear to the dungeon depths
    Let the reader decide, I say

    It appears that the readers aren't reading the stuff in order to decide.
    Evolution sometimes needs a little help!

    My suggestion of a panel of readers to provide a seal of approval works with second hand cars. Its a natural progression to extend it to books! LOL

  14. I think there's more of a consensus in what we want from a car than a novel. What criteria could be used when one reader finds a book riveting that another says is boring?

    I wouldn't pay for my books to be judged by a panel. There's some Indie book award, can't remember its name, where you have to pay to be considered. So it's meaningless to get that award - winners are chosen from a self-selected small pool.

  15. Phew! I'm having a break from tidying my garden. What better place to come and play. LOL

    Lexi, scientists have coped with peer review for a long time. It isn't perfect. For example the mighty Isaac Newton had a paper rejected by one Robert Hooke. Newton subsequently destroyed Hooke! Likewise Einstein had a few problems, but on the whole it is better than nothing. If anyone could publish in respected journals without some sort of filtering we would be swamped with work by cranks and nut-cases making life very difficult. I think the same is true of all academic disciplines. Why should popular fiction writers be different?

    I envisage that members of the peer review panel would each display their credentials and preferences in literature including a list of favourite authors and favourite books. Submitting authors would then choose two panel members to review their book. The system would then evolve towards some optimum.

    Lexi, you don't need this as your visibility is already well above the parapet. But the poor souls who have sold no books might find it attractive if they have confidence that their work would sell if widely visible.

  16. I agree about the problem, but not the solution. I'd like Big Publishing to stop moaning and blaming Amazon and actually do something to provide a better service for authors and readers. Right now, for most writers, Amazon is the only game in town, and its algorithms aren't as favourable as they were back in August 2010 when I first published.

    It's hard to get noticed, even for those of us who started early and know the ropes.

  17. Q, I like your idea of being able to search for authors with similar characteristics.

    Because my first book is a time travel romance, Amazon keep sending me other time travel romances which they think I might like. Hunks with no clothes on and that sort of thing :0) However, my time travel romance isn't like the others - or so my readers are telling me. I don't know because I've never read any time travel romances.

    Those readers who have mentioned similar authors seem to me to be nearer the mark of what I would like to read and that has to be true because I've already read those authors.

    You are on to something there, Q.

    I'm not so sure about paying to have my book reviewed by so-called experts. We have to remember that most indies have been slushing around misunderstood for decades now.

  18. Oh dear, my last sentence was a bit rushed and is ambiguous. I didn't mean that most indie writers have been trying for decades, more that a succession of indie writers started in the slush piles of big publishers.

  19. Anna, I looked for your book on Amazon and found 'Shaman in Stilettos'. It does indeed look very unusual for time travel .... make that unique!

    I think we understood the slush piles bit. LOL

    Its good to find a kindred spirit searching Amazon. *smile*

  20. Ah... That Anna Hunt is not me. As Lexi, and a very few others know, that was my YouWriteOn username which was, if not unique, certainly not being used by another writer on the internet - that I could see. My next try is currently unique - Anna Faversham.

    My book 'Hide in Time' has suffered the same fate as my username. Some inconsiderate scientists have discovered how to hide in time and so the phrase is taking most of the top spots on search engines.

    Ho hum.

  21. Actually, Anna has three names, but nothing would make me divulge the third.

    (Except life-changing amounts of money. I'm sure in that case, Anna would understand.)

  22. Three pseudonyms and only one book. Anna must have paranormal powers!

    I looked up the book on the kindle page and was very surprised to see 'Ice Diaries' immediately beneath 'Hide in Time'. Now how did you two seduce the Amazon search engine to do that I wonder.

    Or could that third name possibly be Lexi?
    Both blonde, no dates on photos .... I love a good mystery!

    I downloaded the free sample. When I read it I'm sure that I will be able to recognise Lexi's voice.

    Don't worry Ladies, your secrets will remain safe with me! LOL

  23. I've noticed that, Q. I think it may be because as writing friends of long standing we sometimes check to see how the other's book is doing, thus establishing a connection in Amazon's abstruse calculations.

  24. Only two pseudonyms, Q. I was quite happy with the first one but it took so long to write the book that the other Anna Hunt became ubiquitous before I got to press that 'publish' button.

    I'm sure we'll both be fascinated to hear if you think we do write similarly. I'm convinced we don't. Lexi is economical with words: I fling them around.

  25. My initial impression is that Anna Faversham is a detailed plotter with a little pantsing thrown in, while Lexi's emphasis is the reverse. I like the voice and will try and read the full novel when time allows.

    It seems that the mystery of the third name remains. LOL

  26. Well, you've got me right, Q. I wonder if Anna agrees with your analysis?

  27. I can only say that I try for a good plot and interesting characters. What those characters get up to when I'm not looking but the reader is, leaves me aghast. Lawless lot, characters.

    I'm glad you like a good mystery, Q.