Of book reviews and selection bias

Publishers send me books to review. I guess I’m on the list of potential reviewers, which is cool because I often enjoy reading books. And, even if I don’t get much out of a book myself, I can give it to students. A book is a great incentive/reward for class participation. For any book, if you have a roomfull of students, there will just about always be somebody who’s interested in the topic.

Anyway, most of the time I don’t write or publish any review, either because I don’t have anything to say, or I don’t find the book interesting, or because I think I’m too far from the intended audience, or sometimes just because I don’t get around to reviewing it—this happens even for books I really like.

The other day I received a book in the mail on a topic that does not fascinate me, but I took a look anyway. The book was boring—even given his topic. The book had some historical content which seemed unbalanced (too much focus on recent events and a shallow, episodic treatment of what came before), also it had what seemed to me to be a smug tone—not that the author seemed like a bad person, exactly, just a bit too complacent. I just didn’t like the book, enough so that I didn’t even give it to a student, I just set it outside somewhere for some stranger to read.

I forgot about this book entirely, and then I happened to be looking at a blog post by someone else who, like me, receives a lot of books from publishers, reads a lot of books, and reviews a lot of books. He reviews more than I do, actually, often quick one-paragraph blurbs. And I noticed that he mentioned this book! I was surprised: did this blogger actually like that book? Maybe not: it’s not actually clear, as his one-paragraph review was descriptive but not actually complimentary.

Anyway, this brings us to selection bias. Most of my book reviews are positive. Why? For one thing, when I don’t like a book, often it’s on a topic I don’t care about and don’t know much about, so I’m not so interested in writing a review and I don’t feel so competent to write one either. Also, when I get a book for free, I guess I feel like I owe something to the publisher.

I do sometimes write negative reviews of books I receive for free, but there’s some selection bias here.

The funny thing is, I often write negative reviews of journal articles that people send to me. Why is that different? For one thing, books are usually sent to me by their authors or publishers, whereas I’ll often hear about a journal articles from a third party who already doesn’t like the article. Also an article is typically making a smaller, more specific point, so it makes more sense to criticize it on specific grounds.

16 thoughts on “Of book reviews and selection bias

  1. I would like to find Shannon Brownlee author of Overtreatment offer up an essay on Andrew’s blog. Overtreatment was fascinating. Undoubtedly one of the most intelligent persons I’ve come across. Currently she is SVP at the Lown Institute and has an account on Twitter.

  2. “Selection Bias” ?

    … there are millions of available books that ‘could’ be read but only a very few that actually can be read in a lifetime, due to physical time constraints.

    Thus, the selection-bias in reading any specific book is of staggering magnitude.

    • What refuge is there for the victim who is oppressed with the feeling that there are a thousand new books he ought to read while life is only long enough for him to attempt to read a hundred?
      — Oliver Wendell Holmes Sr. 1809-1894, Over the Teacups (1891)

  3. I’m also more likely to write a positive review of a book as opposed to a journal article. If a publisher asked me to review, it was probably because I was recommended by the authors. I would likely have a favorable opinion for the author in that case. Also, if the book was already published, I’m not sure what a negative review would accomplish – it wouldn’t change anything at that point.

    • Roy:

      Indeed, I wrote some mildly critical things about a book last year, pointing out some very specific errors—and I even sent the criticisms to the publisher who said they’d forward it on to the author—and nothing ever happened. That really made me mad. I have no idea if my comments ever reached the author, but it’s possible that he did hear about the mistakes but didn’t want to fix them because that would complicate his simple story of the world. The whole episode really annoyed me.

      • I have had two curious experiences with reviewing books recently, one book for a journal, one out of my own interest. Both books were good, although one had rather too many errors and the other a few. I sent the authors detailed error lists and both replied with grateful emails. Over three months later the first book’s webpage still has a blank erratum page. The review of the other book was more recent and the author regularly updates his webpage with corrections but has not included mine.

        Incidentally, there is a good section in Halmos’s Automathography on the trials and tribulations of being a book review editor—worth a look (as is the rest of the book).

  4. I recently listened to a book that and some good information but was generally a bad book. It had to do with genetics. The author kept writing things like “I’ll save you the explanation. It’s complicated”, which I found really irritating. Judging by some explanations that were provided, seems like the main reason for skipping the explanation was that it was too complicated for the author, not the audience, which was almost certainly going to be all science educated people.

    Also at the beginning of the book the author spent several chapters introducing four different people, and I thought he was going to use the experience and lives of those people to track the progress of certain genetic concepts. But after the introduction of each person, they almost completely disappeared, leaving me to wonder why the author spent so much time laying out the story in the beginning – except that one person was a childhood buddy who I guess he really wanted to talk about.

    Ultimately I got a lot out of the book, but the main reason was that I already had substantial knowledge of the topic, so I could more or less fill in those “complicated” explanations.

    When I was a kid I bought a book for my dad, an avid reader, about the history of the Saudis. My dad faithfully read it cover to cover. He believed every book should be read. When I was older I picked up the book because I thought it would be interesting. But after about 10-15 pages of absolutely erratic discussion of familial relationships, I gave up. A completely terrible book. A complete waste of time. If the author can’t be bothered to explain things clearly, why should I read h/her stupid book?

  5. Maybe you could maintain a list of “Books I chose not to review”. It won’t eliminate positive bias in your book reviews, but it would serve as a (vague) datapoint about what you think, or decided not to say you think, about those books.

  6. In an ideal world, reviews would be entirely voluntary; the only reason to write a review is that you want to do so–whether to praise it, point out problems with it, or do a combination of both. I do that sometimes.

    In a somewhat less-than-ideal world, authors and publishers solicit reviews but leave it completely up to the individual whether or not to respond. The playing field remains more or less fair, and despite a small amount of pressure, reviewers may still choose whether or not to do it. But since this is true for all books, no one is at an immediate disadvantage. (I have written a few reviews on friendly request–and while I found it less pleasant than reviewing a book on my own initiative, I recognized that I could have said no.)

    In the world as it is today, there’s a class divide of sorts between books that come with aggressive, powerful publicity drives (where the publicists and publishers have connections with the big media outlets and secure reviews long before a book’s publication) and books whose authors have to request reviews on their own or pay a publicist out of their own pockets. And even a publicist paid by an author is at losing odds against the hype machine (unless the author is rich and pays an awful lot).

    When my first book came out, I hired a publicist who had been highly recommended to me. She got a couple of reviews for the book but found, in many places, that people were reluctant to take it on because “everyone was talking” about another book, on a distantly related subject, that had come out in the same month or so. In other words, they wanted to get in on the buzz, rather than take a risk with a book that hadn’t been buzzed about yet. Over time, though, the book got some nice attention. My second book went under the radar, for the most part. Maybe that’s because of its inherent flaws–I see things I would do differently in it today–but I suspect that it has at least as much to do with the way publicity works.

    Speaking of publicity, on an unrelated topic, I thought this might be of interest to readers here: “Nobel Prize Winner Demonstrates the Best Way to Apologize.” When one of her papers was retracted, the chemist Frances Arnold admitted outright to the mistakes and apologized. (If this comment doesn’t go through, I’ll resubmit it without the link; the article is published in The Guardian, January 6, 2020.) https://www.theguardian.com/science/2020/jan/06/nobel-prize-winner-demonstrates-best-way-apologize-chemist-frances-arnold

  7. I don’t know why voluntary reviewers won’t have the same selection bias problems. I would think that a better situation would be to have non-academic scientists (scientists in industry) with relevant expertise review books from academics. Even journalists doing reviews by collecting the comments of other experts would be better. That way the journalist will feel compelled to get some negative comments from some expert, and those of us relying on the review can see that the book may have some fundamental flaws.

  8. I am seeing a lot of pointless angst from busy people who are not being compensated for their time and expertise. So first, give yourselves a break, and second, if publishers were to make it worth your while over and above “You can keep the book,” what is your price point? Fifty dollars? One thousand? At the rate they keep sending you books and articles, how much would it take to ditch the science thing and become a full-time reviewer?

  9. I amused myself (if no one else) by a crude taxonomy of book reviews, wide sense.

    I can’t believe it is complete.

    Before publication:

    1. Author(s) ask you for review of draft(s) as an act of friendship or collegiality; usually rewarded by a mention in the book, sometimes gifts
    in kind. (If the author is your partner, details may vary.)

    2. Publisher asks for opinion (meaning, is it a good idea? how about structure, content, level, accuracy, likely sales, etc.) on book
    proposal. Usually rewarded by money or free books. (Often the book never materialises, variously that the proposal was rejected or the book never written.)

    3. Publisher asks for opinion (meaning, as detailed a review as you want to send) on draft chapters or entire book. As above. (Sometimes the book is even killed by publisher at this stage.)

    Exception. One notorious publisher of computing books, known for shoddy treatment of authors and very low quality, offers only the incentive of
    being written up in the book.

    4. Publisher asks for blurb for book cover, web page, etc. No personal experience of this one. (Presumably negative blurbs don’t see public
    light.)

    5. Editors ask for review of chapters. May be highly formalized, akin to review for academic journals. Sometimes a paper (chapter) is killed at
    this stage, or otherwise fades away (e.g. if authors miss a deadline for revision).

    After publication:

    6. Academic journals and other serious periodicals commission a review for publication. Usually you keep the book (I’ve known exceptions to
    that). Academic journals don’t usually pay you; other periodicals will (if it’s published). Some censorship may be exercised, but I’ve always
    found that decisions didn’t involve anyone but the editors.

    7. You publish a review unilaterally, usually via a blog or a site such as Amazon. No-one pays for anything, but often there is scope to revise
    a review or even delete a review you regret. Naturally your personal agenda can dominate, all the way from this is a wonderful book that deserves yet another puff to Public Service Announcement: A bad book to be avoided.

    8. You send a review in the form of errata and/or suggestions for the next edition to the author(s). Strong cultural and personal variations
    here, from sincere gratitude (serious statistical authors often maintain errata pages) to much less.

Leave a Reply to Torquemada in Training Cancel reply

Your email address will not be published. Required fields are marked *