Skip to content

Creating a Lenin-style democracy

Mark Palko explains why a penalty for getting the wrong answer on a test (the SAT, which is used in college admissions and which is used in the famous 8 schools example) is not a “penalty for guessing.” Then the very next day he catches this from Todd Balf in the New York Times Magazine:

Students were docked one-quarter point for every multiple-choice question they got wrong, requiring a time-consuming risk analysis to determine which questions to answer and which to leave blank.

Ugh! That just makes me want to . . . ok, I won’t go there. Anyway, Palko goes to the trouble to explain:

While time management for a test like the SAT can be complicated, the rule for guessing is embarrassingly simple: give your best guess for questions you read; don’t waste time guessing on questions that you didn’t have time to read.

The risk analysis actually becomes much more complicated when you take away the penalty for guessing. On the ACT (or the new SAT), there is a positive expected value associated with blind guessing and that value is large enough to cause trouble. Under severe time constraints (a fairly common occurrence with these tests), the minute it would take you to attempt a problem, even if you get it right, would be better spent filling in bubbles for questions you haven’t read.

Putting aside what this does to the validity of the test, trying to decide when to start guessing is a real and needless distraction for test takers.

In summary:

The claim about the effects of the correction for guessing aren’t just wrong; they are the opposite of right. The old system didn’t require time-consuming risk analysis but the new one does.

The time control is the key. Without time control, you look at each question and make your best guess. It’s that simple. And if everybody attempts every question, then taking off points for wrong answers has exactly zero effect. But if not everyone gets to every question, then the new format creates a clear incentive for people to spend time filling in bubbles to questions they haven’t looked at.

Palko has more here on news media reports with horribly confused discussions of the SAT.

I also clicked through to Balf’s New York Times magazine article and noticed this:

When the Scholastic Aptitude Test was created in 1926, it was promoted as a tool to create a classless, Jeffersonian-style meritocracy.

Huh? Is a Jeffersonian-style meritocracy the system where everyone is equal and so we all own slaves? I guess if this article were written in Russia it would all about how the SAT was promoted as a tool to create a Lenin-style democracy.

The problem I see here is that Balf seems to be dealing in images and impressions rather than thinking through his ideas. “Jefferson” and “meritocracy” have positive images, so they go together, the old SAT was bad so therefore it required “a time-consuming risk analysis,” etc. One journalistic convention that I really can’t stand is the push toward giving every story a single coherent perspective. In Balf’s article, College Board president David Coleman is the hero and so everything about him has to be good and everything he’s changed has to have been bad. Another article by someone else might take the opposite stance. I’m not saying the article needs “balance,” I’d just like to see a bit of questioning.


  1. numeric says:

    When the Scholastic Aptitude Test was created in 1926, it was promoted as a tool to create a classless, Jeffersonian-style meritocracy.

    It was actually created to get Jews into the Ivy League schools, something you should appreciate. In fact, while Milton Friedman famously stated that discrimination was economically irrational, if there is one constant in American society (as you note in your parenthetical comment about slaves), it’s discrimination. Of course, we now know it takes 10 to 15 generations for regression to the mean to eliminate inherited advantage. So for most groups, even the elimination of discrimination (if such a thing is possible) will still portend a long period of time to overcome the effects of such.

    I do think that it’s somewhat unfair to apply current beliefs in social organization to past historical figures. Jefferson was progressive, even radical, for his time. That he was a racist (and slaveowner) is undeniable, but his instincts were towards equality rather than hierarchy, which is where a reductionist view of economic status would have placed him. We don’t knock Athenian democracy because it excluded most of the inhabitants of Athens.

    • Andrew says:


      Jefferson did a lot of great things. I just didn’t think the term “Jefferson-style democracy” was so appropriate in the context of a discussion of social exclusion.

      • Steve Sailer says:

        Jefferson carried through all sorts of technical legal and bureaucratic reforms to make land ownership more widespread. For example, he biased inheritance laws against the English custom of primogeniture. And during the Articles of the Confederacy period Jefferson set up the system of surveying and selling Western land that allowed smalltimers to buy 160 acres directly from the federal government. Thus, the U.S. typically didn’t have to follow the King of Spain’s practice of granting giant, vaguely defined holdings to big men, which remains one of the pillars for the widespread inequality in Latin America today, 500 years later.

        The psychometricians who developed modern admissions testing tended to be from land grant colleges like Iowa (home of the ACT) and Indiana (where Louis Terman, who brought the IQ test to America, 100 years ago, was educated). It sounds pretty Jeffersonian to me.

    • Steve Sailer says:

      Jews and smart farm boys, both — Robert A. Heinlein’s 1950s juvenile sci-fi novels are full of overlooked smart young men from the middle of nowhere. After Sputnik, everybody agreed with Heinlein in his enthusiasm for testing for about ten years. So the last of the anti-Semitic quotas were taken down (probably Yale, 1965 was about the last to go — Harvard got a permanent jump on Yale by moving to meritocracy about a decade before). Somewhat similarly, the space race facilities in Houston and Huntsville were largely manned by engineers from previously overlooked backwaters.

      A friend of mine, Jim Chapin, was a history professor at Yale while George W. Bush was there. Bush got in, barely (he thought he was going to the U. of Texas), in 1964. By 1966, my friend said, the atmosphere of the campus had changed dramatically toward the intellectual due to the huge influx of smart Jews due to the lifting of quotas in 1965 admissions. He felt that what we think of as The Sixties had a lot to do with Jews suddenly making up critical masses on elite campuses.

    • Eli Rabett says:

      Amusingly the Ivys had small quotas for Jews into the 1960s so Eli guesses that did not work very well (or to be honest that there is much truth to it). It has been said that the primary beneficiaries wrt college admissions of the Civil Rights Movement were women and Jews. For Blacks, of course, they went from essentially no slots to a few.

      One should also bear in mind that intelligence tests given to draftees for WWI were interpreted to show that Jews as a class were not the brightest bulbs in the land.

      Brigham wrote a book in 1923 called A Study of American Intelligence. This was based on his work on the Army Alpha Test. He analyzed the test results by race and found–as people who do that have always found–that people of color, Jews, Mediterraneans, anybody who wasn’t a kind of what he would call a Nordic, was inherently intellectually inferior. And that the country was in big trouble because two many of these people were coming into the country. So this book is a kind of very ripe, racist book by today’s standards, typical of establishment thinking of the time, although Brigham, you know, bothered to write it down. And it just stands up very well as an offensive piece of writing. Now, Brigham renounced it within about five years. To his great credit, he specifically disowned the book. He changed his mind, he broke with the eugenics movement and by the end of his life, was really one of the leading critics, of the eugenics movement. So he came around and deserves a lot of credit for that.

      Frankly most of these comments should be deleted with extreme predjudice.

    • Popeye says:

      I do think that it’s somewhat unfair to apply current beliefs in social organization to past historical figures.

      I disagree. I mean, it’s not particularly worthwhile to try to change Jefferson’s mind about slavery, he’s dead. But we praise him for “all men are created equal” and it’s not like there wasn’t an abolitionist movement during his lifetime. Never mind “How is it that we hear the loudest yelps for liberty among the drivers of negroes?” If you had articulated an anti-slavery argument to Jefferson, he would have been perfectly capable of understanding you. It wouldn’t have been like trying to explain quantum mechanics to a Babylonian farmer.

      • numeric says:

        Popeye disagrees with my comment:

        I do think that it’s somewhat unfair to apply current beliefs in social organization to past historical figures

        while Andrew states:

        The racism of decades past always seems laughable, but the racism of today always seems like common sense.

        For 6 points on the SAT writing exam (now being dropped), write an integrative analysis that compares and contrasts these two opinions.

        Let’s see. Popeye states that Jefferson was well aware of counterpoints to slavery, while Andres alludes to today’s racism. Is there a current situation of inequality that seems like common sense that both Andrew and Popeye are familiar with, benefit from, and make no effort to change? There is!

        I refer to the current Athenian democracy situation in academia, where a very few are allowed to make decisions (the tenured faculty) and lead an economically satisfying life (oh they do, no matter how much they complain), whereas the rest of the helots, hoi polloi, support staff, grounds crew, gypsy academics who teach the classes, etc, live a life of mass desperation. Now, strictly speaking, this isn’t racism (so minus one point), but the parallels to a racist situation are clear–there is a class of individuals on top enjoying rents and there is, for most fields, a much larger group of identical qualifiations that would love to have their “positions” but instead are relegated to a lifetime (or a large chunk) of freeway flying to their next 3 hour course. So when we extol academic freedom, include the freedom of penury.

        Now, this argument only half applies to Andrew, since the statistician in him is producing students with real compensatable skills in today’s market. The political science side (which he voluntarily aligned himself with when he didn’t have to), alas, the situation I described obtains in full measure. And, of course, the indivdiduals who clean his office and empty his trash are on the cusp (if not in) poverty, and many of them are people of color (this ties in racism–so give me back my minus one–or at least a half-point).

        A direct reply to Popeye is that I stated Jefferson was progressive (in the sense of moving society towards is current state, a definition that has problems when you consider the German academics who undermined the Weimar Republic and help pave the way to Hitler’s rise to power). My comment about Jefferson was that he did not let existence determine consciousness, thereby rising above reductionism. In our current political culture, much of which seems based on a crude social Darwinism that would be more fitting to the Gilded Age, that’s something. Jefferson was well aware of anti-slavery arguments and he had to resort to crude rationalizations on the intellectual inferiority of his slaves to justify the supremacy of the white over the black. The same process works (crude rationalizations) for tenured academics who view the plight of their unpositioned colleagues. So to get back to Andrew’s point, the inequality of today seems common sense.

        • Andrew says:


          I was not expressing agreement or disagreement with your comment. I was just making a general reflection inspired by what Eli had posted in his comment above.

          In any case, yes I agree that, along with Paul Krugman, Paul Ryan, Meg Ryan, Meg March, Jo March, Joe Paterno, etc etc etc., I live a comfortable life while others, less well-situated, struggle. Indeed we hear about very few people who voluntarily relinquish a comfortable lifestyle. The only historical figures who come to mind here are Mohandas Gandhi, George Orwell, Ralph Nader, and various well-born nuns. I’m sure there are others but such behavior is not common; instead, standard practice seems to be for people to keep their comfort, whether they criticize economic inequality (Paul Krugman) or accept it (Paul Ryan). I don’t think this has much to do with my job as university professor, as I’d be in a similar economic position if, for example, I worked at some company or government agency fitting statistical models.

          • numeric says:

            Your comment fell under this little RSS I seem to have started (though having to page down mulitple times means I may have lost the indexing). So I had every right to integrate Popeye’s and your comment. I lay no claim to any originality in my thoughts–the culpability of those who engage in what we now consider egregious violations of present-day morality has fascinated many. I recall “Reagan Defends Cemetery Visit : Says German Dead Are Also Victims of Nazis” (google this for link to LA Times newspaper story) and the controversy that created for his statement that German soldiers were victims of Nazism as well. On the other hand, Confederate soldiers who died defending a slave society are seen as heroes in a large part of this country.

            My integration was that we all (including me) participate and profit from a system that, when looked back on in 50-100 years, our descendants will say “How could they”. I have no idea what your views are on the structure of society but you do not have to “relinquish a comfortable lifestyle” to be progressive in how I am using the term (and how I used it for Jefferson). For example, to deal with the surplus of political science Ph.D.’s, you could be in the forefront of a movement to limit the number of graduate students admitted to these programs (or even support such a movement–does it exist?). To deal with poverty or near-poverty among support staff, solidarity with strikes, job actions, etc would be a step towards what I call progressive. If you want to be considered progressive by future generations, the above are a few things you could do that directly relate to your position in society.

            Saying you have to “throw down your things and follow me” or live the comfortable life style is a false dichotomy. The early Christians soon changed this particular directive into a matter of faith in the hereafter rather than a call to change in their current life, and it was then converted (in its most extreme Calvinistic form) into the belief that those who were most worldly successful were the most holy. More nuanced Christians attempted to help those less fortunate while maintaining a very pleasant existence (this seems to be the emphasis of the current Pope, which is why progressives are delighted with him). My point is it is possible not to go “all in” and still be progressive. This was my point about Jefferson, but it applies more generally to everyone, particularly those who consider themselves progressive.

  2. Wonks Anonymous says:

    Jefferson advocated a “natural aristocracy” of talent. So perhaps that’s where the “meritocracy” angle comes in. HIs conception of democracy depended on separating the natural aristocrats from the “pseudo-aristoi” of inherited land & church hierarchy.

  3. Eric Tassone says:

    Well, at least Jeffersonian democracy, with its agrarian emphasis, allows us to mention Fisher and work of his like “Statistical Tables for Biological, Agricultural and Medical Research” on this blog! (Shall we avoid discussion of any commonalities between Jefferson’s slave ownership and Fisher’s enthusiasm for eugenics in stuff like “The Genetical Theory of Natural Selection”?)

    • Mark Palko says:

      There’s a more direct and even uglier eugenics connection through Carl Brigham, the father of the SAT. Brigham did disown these theories later in his career but not before giving a big boost to the movement through his writings psychometrics.

      • Steve Sailer says:

        I’m increasingly come around to the view that the most overlooked division in the intellectual life of the English speaking world is between smart country boys and smart city boys. Jefferson, Darwin, Galton, the Termans, father and son, W.D. Hamilton, E.O. Wilson, Charles Murray, and Richard Dawkins were all country boys interested in nature, agriculture and breeding. Fisher was a city boy who moved to an experimental farm.

        On the other side, the folks like Gould and Lewontin who have striven so energetically to blacken the reputation of the country boys as demonic eugenicists tend to possess the city dwellers’ fear that the peasants will be coming after them with pitchforks any moment now. They remind me of poor Mr. Salter, Lord Copper’s editor in Waugh’s Scoop, who is forced to venture deep into the terrifying English countryside to sign William Boot, Countryman to a lifetime contract.

      • Steve Sailer says:

        Silicon Valley was largely founded by eugenicists. There are two main theories of who is the Father of Silicon Valley:

        — The more popular one traces back to William Shockley, who brought together the young executives who would go on to start Intel and other crucial firms.

        — The more persuasive one points to Stanford’s dean of engineering Fred Terman who a generation before mentored Hewlett & Packard and many others.

        Fred Terman was the son of Louis Terman, creator of America’s first IQ test, the Stanford-Binet. Louis ran the Terman’s Termites study of high IQ children that’s just winding down now. (Shockley just missed qualifying for it.) This study persuaded the public that high IQ children weren’t misfits. Fred Terman and Shockley were friends and Terman gave Shockley a professorship after Shockley’s disgruntled employees went on to bigger and better things.

      • Ian Fellows says:

        There is an ugly eugenics connection with the entire early history of statistics. Pearson, for example, was the editor of the Annals of Eugenics.

        • Steve Sailer says:

          Or, Galton or Spearman.

          Similarly, much of modern genetics and modern evolutionary theory is an outgrowth of the eugenics movement. Of course, we’ve all read our SJ Gould so we all know it’s just “the pseudoscience of eugenics,” so, I guess, statistics, genetics, and biology aren’t real.

          • Phil says:

            Steve is right as usual. What right-thinking person could possibly object to this Pearson quote?

            “Taken on the average, and regarding both sexes, this alien Jewish population is somewhat inferior physically and mentally to the native population. It is not so markedly inferior as some of those who wish to stop all immigration are inclined to assert. But we have to face the facts; we know and admit that some of the children of these alien Jews from the academic standpoint have done brilliantly, whether they have the staying powers of the native race is another question. No breeder of cattle, however, would purchase an entire herd because he anticipated finding one or two fine specimens included in it; still less would he do it, if his byres and pastures were already full.”

            Yes, Pearson sure was right about the Jews. You can’t argue with science! Andrew, you should be very proud to have risen so far above the norm of your race.

        • P says:

          There is no “ugly eugenics connection.” Rather, statistics as a science was to a considerable extent developed to study problems in eugenics. Annals of Eugenics was renamed Annals of Human Genetics in the 1950s, and remains a well-respected genetics journal.

          • Andrew says:


            I think Phil’s comment above indicates some of the ugliness.

            • P says:

              I don’t find it any more ugly that Pearson’s Marxism. In any case, that hardly invalidates the correlation coefficient.

              The medical branch of early 20th century eugenic thinking is today totally accepted and uncontroversial. It goes by the name “genetic counseling.”

              • Andrew says:


                You wrote, “There is no ‘ugly eugenics connection.'” In response, I pointed to an ugly eugenics connection that Phil had noted in an earlier comment. That’s all. Nobody was saying that this invalidates the correlation coefficient.

  4. Steve Sailer says:

    In general, conspiracy theory thinking remains highly respectable when it comes to denouncing the SAT, cognitive testing, and even the concept of intelligence. Grassy knoll obsessives have a more logically coherent world view than the stuff you read in the newspapers about how testing is a racist conspiracy by right wing college professors to keep my kid from getting into Harvard like he deserves.

  5. FredR says:

    I took Jefferson-style meritocracy as a reference to his famous letter about a ‘natural aristocracy’

  6. Rahul says:

    the minute it would take you to attempt a problem, even if you get it right, would be better spent filling in bubbles for questions you haven’t read.

    What if you added a button “Random Guess leftover questions”? On the electronic test, of course.

    • Andrew says:

      Sure, but in that case it would be more statistically efficient to just give the expected score, which is what is obtained by taking points off for wrong answers (or, equivalently, by giving partial credit for questions not attempted). Instead of randomly filling in bubbles, it would make more sense to give 1/5 of a point for each question not filled in.

Leave a Reply