“Who’s bigger”—the new book that ranks every human on Wikipedia—is more like Bill Simmons than Bill James

I received a copy of “Who’s Bigger?: Where Historical Figures Really Rank,” by Steven Skiena, a computer scientist at Stony Brook University, and Charles Ward, an engineer at Google. Here’s the blurb I gave the publisher:

Skiena and Ward provide a numerical ranking for the every Wikipedia resident who’s ever lived. What a great idea! This book is a guaranteed argument-starter. I found something to argue with on nearly every page.

Here’s an argument for you. Their method ranks obscure U.S. president Chester Arthur as the 499th most historically significant figure who has ever lived. William Henry Harrison, who was president for one month, is listed as the 288th most significant person. This seems ridiculous to me. We’re considering all people who have ever lived (who are on Wikipedia), including inventors of drugs, discoverers of physical laws, founders of countries, influential religious leaders, explorers, authors, musicians, newspaper editors, etc etc. Surely there are many thousands of people who are more historically significant than these two minor figures who just happened to be president of the United States for brief periods.

How did this happen? I can’t be sure, but think about this: Right now, as you read this, there’s probably a 7th grader somewhere who’s googling Chester Arthur as part of a class project on American presidents (and is not allowed to write about Washington, Jefferson, or Lincoln). And somewhere else there’s a 5th-grade teacher who has assigned a different president to each of the students in his or her class. These kids are all going on Wikipedia, and lots of other pages link to these presidents’ pages.

In his historical baseball abstract, Bill James ranks approximately 1000 of the best baseball players. The ranking is, of course, ultimately subjective (in that James has to make choices in what data sources to use and how to combine them) but it has a clearly-defined goal: wins. James is rating players based on how many wins they contributed to their team. In his historical basketball book, Bill Simmons ranks approximately 100 of the best pro basketball players. Simmons, like James, is entertaining, intelligent, and thought-provoking, but one thing his ratings don’t have is a clear external goal. One could say that James is ranking based on an (imperfect) estimate of a somewhat well-defined outcome, whereas Simmons is ranking for the sake of ranking. As with the notorious “U.S. News” college rankings, the ratings are not an estimate of anything but themselves. That’s fine, it’s just they way it is, I’m not trying to slam Bill Simmons, I’m just trying to make this distinction.

Skiena and Ward, like Simmons, are producing a ranking that is not an estimate of any externally-defined goal. Again, that’s fine—it’s all one could possibly do here, I think.

But . . . I found it grating how Skiena and Ward kept on referring to their rankings as “long-term historical significance,” “historical greatness,” etc. They write, “Our algorithmic rankings help focus attention where it should properly be focused. Any person interested in livestock production would miss the point if they study how to raise geese and Angora rabbits at the expense of cows, pigs, and chickens. Our rankings help answer the question ‘Where’s the beef?’ in history in a rigorous and effective way.” I don’t think Chester Arthur is very beefy at all, but I do believe that he gets lots of links because the U.S. educational system is so focused on presidents.

The part I like is when they write, “what we are really trying to do here is study what shapes the process of historical recollection.”

In summary, I have mixed feelings about this book. I like the idea of quantitatively studying the process of historical recollection. But it’s hard for me to take seriously any method that represents Grover Cleveland as the 98nd most historically significant figure who’s ever lived.

31 thoughts on ““Who’s bigger”—the new book that ranks every human on Wikipedia—is more like Bill Simmons than Bill James

    • Its the first quote on Amazon, so I think the publishers understand the book is supposed to raise debate. Using any single method to rank “greatness” will miss some great ones and allow some less-great ones through (remember the debate from the blog several weeks ago here on stats papers based on citations! Do we trust the 7th grader more than the journal paper writer?).

  1. The book may – since I’ll never read it – illuminate something interesting: the difference between notational form and substance, a version of noise and signal. Chester Arthur is bound to get a ton of mentions because he’s in the list of US presidents even though he’s historically unimportant. Harrison gets even more mentions because he died so quickly, meaning his only significance in 99+% of the mentions is as trivia, which is the essence of noise. In my house, btw, Chester Arthur is highly significant because we have a cat named Chester. He was originally Chessie when he was living in the yard but Chessie – see Chessie Cat – is a girl so I switched the name and now he’s a trivial reminder of a trivial entry on a list that happens to get repeated often. Might as well say my cat is historically significant if only he were President. (Think about that.)

    Is trivia significant? I have no idea if the book addresses that, though ranking Harrison that highly suggests they don’t filter much (or well?). If we applied the same kind of reasoning to signal to noise ratios, imagine how poorly we’d have done developing radio!

    Bill Simmons’ book was, to me, a guy who has built a career as the ultimate sports “guy” showing that he can argue persuasively in bar bets about sports. It isn’t geekery, the way Bill James’ work is – as you note. It’s at the level of educated fan conversing over beer while the game is on when you draw on napkins to prove your point.

    • I, too, have a cat named Chester, but he is named after Chester Nimitz. He is so named in order to match him with our cat Nimitz, whom we found (evidently abandoned) on the Nimitz Trail in a nearby park. You could switch which Chester your cat is named after. Just a thought.

  2. You could make an analogous criticism against Google’s Page Rank (I think). Yet it’s turned out darn useful.

    • Rahul:

      I think the difference is that Page Rank has a useful circularity: it uses the popularity of webpages to rank the popularity of webpages, for the purpose of users who are looking for the most popular pages. To put it another way, a high-ranked webpage is likely to be of interest to a lot of people, because these are the webpages that people are linking and clicking to. Using the same principles to rank historical figures leads to the ridiculous conclusion that William Henry Harrison is the 288th most significant person who’s ever lived, whereas really what’s going on (I think) is that a lot of 5th-grade teachers who assign a different president to each student in their class.

      • Yes, but isn’t that an exception? Aren’t you using an outlier to judge the rule? Are all (most?) of the others on the list equally ridiculous?

        Once in a while even Page Rank can do silly things: remember circa 2005 when searching for “moron” used to bring up Geogre Bush’s webpage?

        • Rahul:

          I don’t have the book in front of me but I’m pretty sure that there are other problems with the list beyond the 3 that I mentioned above. In any case, I think what you’d want to do in such a case is not just say, Hey, mistakes happen! but rather to treat these errors as information, as model checks, as canaries in the coal mine and use them to improve your procedure.. Sort of like what I did when someone pointed out problems in my election maps.

        • I feel there’s utility in distinguishing clearly between two cases: (a) an algorithm that gets it mostly right but screws up a few cases versus (b) an algorithm that just gets it mostly all wrong.

          From your critique it is hard to tell if the situation is (a) or (b).

          PS. Something like, (say) “Of the 30 top entries 20 did not make sense” might be more useful than presenting three gross outliers.

        • the problem is “gets it right” isn’t well defined. it’s not an argument over what’s right, it’s an argument over the definition of what’s right.

        • Perhaps for a purist. But not for Andrew (I think). Clearly, if Arthur & Harrisson in the list are ridiculous one has a (subjective) opinion about what’s “right”.

  3. Andrew,

    Your example certainly suggests that Skiena & Ward’s results fail to ring true in a broad historic perspective. As you point out, it’s very difficult to judge merit that’s not intrinsically based on popularity. Back in the day, I laughed reading that Bill Gates, on one of his self-study forays, decided that “studying film” could be efficiently done simply by watching every year’s “Best Picture” Oscar winner!

    I would think that a Watson-class team of programmers, perhaps armed with a different book of Skiena’s — his truly important & classic “Algorithm Design Manual” — could do a better job of mining the semantic depths of significance. I’m not sure the problem would quite rise up to IBM’s “Grand Challenge” threshold, though..? Maybe if we reframed the question as something like: most influential ideas / inventions / creations..?

  4. Because “greatness” and “historical significance” are meaningless concepts (at least without adding more context) it’s not surprising that the method of their measurement is also flawed. At best, they can measure the amount of buzz created by various historical figures, which might be all they want.

  5. Doing any analysis without much context is a futile excercise. It’s interesting and fun, but essentially answers no question and is potentially detrimental to learning. The danger here is that Who’s bigger is going to get quoted exponentially, simply because folks love ranks (who doesn’t?) and senseless information is going to spread like a genetic disease, legitimized by its popularity.

  6. I hope they at least extensively discuss what historical significance and why their algorithm is a good metric of it.

    If we look at the algorithm in light of what it is supposed to do and consider it very reasonable on theoretical grounds, we might we be willing to accept some surprising results. Something like Page Rank doesn’t seem compelling though may produce decent results because what it measures (popularity?) is related to historical significance.

  7. > Skiena and Ward provide a numerical ranking for the every Wikipedia resident who’s ever lived.

    Well, I’ll give them this much, if you’re going to bring the stupid, bring it.

  8. “One could say that James is ranking based on an (imperfect) estimate of a somewhat well-defined outcome, whereas Simmons is ranking for the sake of ranking.”

    I’d disagree. Simmons’ “Book of Basketball” is clearly oriented toward judging players based on their contributions to their teams’ post-season success: Bill Russell (11 championships) thus ranks #2 all time, while Wilt Chamberlain (many remarkable individual statistics) is relegated to #6. Also, please note that James was in the analysis business for a quarter of a century before promulgating his Win Shares all-purpose statistic of everything. James’ first “Historical Abstract” from the 1980s ranked baseball players at least as subjectively as Simmons’ basketball book from a half decade ago.

    I don’t think their methodological philosophies at similar points in their careers are all that different. Both read a lot of antiquarian journalism on their sport, read players’ memoirs carefully for their judgments on other players, and so forth. Simmons also does something James in his prime couldn’t do: he watches a lot of old games on Youtube, which proves quite helpful.

    Simmons isn’t a statistical innovator like James, but they aren’t really all that different.

    • For example, I have a theory that many of the most famous creative artists in history rebelled against their family’s plan for them to study law. Murray’s lists of top 20 composers or top 20 writers, combined with Wikipedia, are useful in seeing how correct this hypothesis is.

  9. And to be a devil’s advocate, being president was not the only thing William Henry Harrison and Chester Alan Arthur did in their lives.

  10. Personally, I think amusingly wrong attempts to quantify a thing can still be useful, if their wrongness tells us something about how to ask the question better next time. I’d say this qualifies!

  11. Pingback: Who’s Bigger?: Where Historical Figures Really Rank: Steven S. Skiena, Charles B. Ward: 9781107041370: Amazon.com: Books | High Noon GMT

Comments are closed.