Skip to content
 

So much of academia is about connections and reputation laundering

There are two ways of looking at this:

1. Statistics / numerical analysis / data science is a lot harder than you, the reader of this blog, might think.

2. Academia, like any other working environment, is full of prominent, successful, well-connected bluffers.

Someone sent me this:

and this followup:

I clicked around a bit, and there was a lot of appropriate distress this this sort of bad statistical analysis was being done at the highest levels of government.

But here I want to focus on something else, which is the general level of mediocrity, even at the top levels of academia.

When he is not “A/Chairman @WhiteHouseCEA,” the author of the above tweet is a tenured professor of public policy:

In his tweet, Philipson introduces the useful concept of “economist turned political hack.” My problem here is not whether Philipson is a hack but rather that someone can reach the heights of academia and government while being so ignorant as to think that this above curve fit could be a good idea.

OK, part of this is straight-up politics. There will always be prestigious and well-paying jobs for people who are willing to say the things that rich and powerful people want to hear.

But I think it’s more than that. Let me go back to points 1 and 2 above.

1. Statistics is hard. Sure, a decorated and tenured professor of public policy, “one of the world’s preeminent health economists.” Tyler Cowen says that graduate students in top econ programs all have “absolutely stellar” math GRE scores, and Philipson graduated from the University of Pennsylvania, which I think has a top program . . . But, again, statistics is hard. Really hard for some people. You could get a perfect math GRE score but not understand some basic principles of curve fitting.

Michael Jordan was a world champion basketball player but couldn’t hit the curve ball. So you’re surprised that someone who was good at taking multiple-choice math tests doesn’t understand statistics? C’mon.

Further evidence of the difficulty of statistics comes from Philipson’s economist colleagues from celebrated institutions around the world who famously flubbed a regression discontinuity analysis by massively overfitting—so, yes, the same sort of statistical error made by the Council of Economic Advisors and discussed above—in an analysis that was so bad that it motivated not one, but two papers (here and here) exploring what went wrong.

2. Academia, like any other working environment, is full of prominent, successful, well-connected bluffers. The striking thing is not that a decorated professor and A/Chairman @WhiteHouseCEA made a statistical error, nor should we be surprised that a prominent academic in economics (or any other field) doesn’t understand statistics. What’s striking is that the professor and A/Chairman doesn’t know that he doesn’t know. I’m struck by his ignorance of his ignorance, his willingness to think that he knows what he’s talking about when he doesn’t.

Some of this must be because bluffing is rewarded: it’s one way to advance in the world. If I wanted to be really cynical, I’d say that being not just wrong but aggressively wrong (as in the A/Chairman’s above tweet) sends a signal of commitment to the cause. I’m not quite sure what the cause is here, but maybe that’s part of the point too.

Again, this is not a problem that’s special to economists. I’m sure you’d see the same thing with sociologists, political scientists, psychologists, etc. Economists may be more of a concern because of their current position of power in government and industry, but the difficulty of statistics and the ease of bluffing is a more general issue.

P.S. More on the specific curve fit here.

224 Comments

  1. Wonks Anonymous says:

    Perhaps it just shows my statistical ignorance, but I don’t understand why Philipson is wrong. The cubic isn’t presented as any sort of “true model” of the phenomena (which is why, unlike the IHME projections, it doesn’t continue past the existing data), but instead another way of visualizing the trend in the existing data without the bumps. The IHME model (the actual focus of his tweet) can’t be expected to forecast all those fluctuations, so it makes sense to compare it to a more smoothed out version of the data.

    • Garrett says:

      “The cubic isn’t presented as any sort of “true model” of the phenomena (which is why, unlike the IHME projections, it doesn’t continue past the existing data)”

      Then what is the red dotted line supposed to be?

      • Ea says:

        It’s bad to display the red dotted line, but the original tweet does explicitly stress that it is only meant to be a visualization of the data.

        • Michael Nelson says:

          Why would anyone think that the cubic curve even looks like it fits the data at all, much less “fairly well”? The only place it appears to match the actual trend is at the far left…and that’s only because that part of the graph is too small to see discrepancies. On the right side, it misses most of an entire week’s worth of deaths, and that’s before you get to the pink dots. So there’s no visual rationale. There’s no theoretical rationale for fitting a symmetrical, bell-shaped curve, other than the Central Limit Theorem…which applies to frequencies, not growth.

          The only rationale for choosing to show people this “visualization” is that it reinforces an unfounded bias toward believing the inevitable decrease in deaths will come sooner rather than later, and faster rather slower.

    • Zhou Fang says:

      Ignoring the dotted red line business, *how* does a fitted cubic polynomial help “visualise the trend in the existing data without the bumps”? You might be imagining a spline is being used, but no, we are talking about a straight cubic function.

    • Joseph Candelora says:

      In addition to the comments above about the red dotted line that extends to the future (making it a projection beyond existing data), it’s a terrible visualization of the trend in existing data.

      Let’s ignore the red dots and focus just on the red dashes. The cyclicality is due to day-of-week reporting effects; a 7-day rolling average would be a much better visualization. Looking at the red dashed line, what do you think has happened with case counts? Their cubic fit would tell you the case count peaked rose rapidly, peaked about 2.5 weeks prior to last actual data point, and has dropped as rapidly as the rise in the period since then. That’s not all what happened in real life, which on a rolling-average basis would show a much lower peak and only the slightest drop since peak, much slower than the equivalent rise.

      I can’t think of a single good thing to say about the visualization. What about the red dashes looks like a useful simplification of reality to you?

      • Zhou Fang says:

        Yeah. Consider their statement:

        “As shown, IHME’s mortality curves have matched the data fairly well”

        Is that clearer to see when you compare the green IMHE projection line to the red cubic fit line than to the black raw data line? Obviously not. A projection that better fitted the “smoothed out version of the data” would be a *worse* fit to the data itself.

        • Zhou Fang says:

          I mean I’d argue that the CEA and Philipson have it exactly the wrong way round. A cubic fit is *only* suitable for use as a model based fit (albeit, implying pretty Strong assumptions about the model that you’d better back up). It makes no sense for use as a general data smoothing methodology, no more than fitting a linear trend line, a sine wave, or a picture of an elephant.

          • Joseph Candelora says:

            But it goes up-down right? And that’s the answer you want, up-down. A straight line is just straight-straight, and that can’t be right. Neither can up-down-up-down-up-etc. But elephant works cuz it’s mostly up-down with just a few wiggles.

            And as I think about it, a few wiggles could really give it that air of authenticity. Hey man, I say publish Zhou Fang’s picture-of-an-elephant Covid fit and you could land a plum spot on the task force!

    • GabbyD says:

      I agree with this. never understood the bother. its just a smoothing for data viz, no? But the pink dots dont have a legend attached to it, which is the real problem.

      • JFA says:

        The question is “What is the purpose of the visualization presented?” Seems like there is a lot of obfuscating. CEA presents 3 of IHME’s *projections* and then say the display of cubic fit as just “data smoothing”. Curve fitting on the 7-day rolling average would have been better. At best, someone doesn’t know how to present data effectively because that cubic fit business is just worthless and adds nothing to the graph.

        They might have been trying to defend the general use of IHME’s model. The weird thing is the IHME model has come in for criticism because it was too *optimistic* about deaths, usually projecting lower than the actual even for next day predictions. I can’t square that criticism with the 3/27 projection actual doing a decent job of predicting through the end of April and the 4/6 projection prediction more deaths than occurred.

    • Wonks2 says:

      >Perhaps it just shows my statistical ignorance, but I don’t understand why Philipson is wrong. The cubic isn’t presented as any sort of “true model” of the phenomenon

      Here let me put it into perspective:

      Fake news editor 1 to fake news editor 2: “this doesn’t really show that he thought the cubic model was better than anything else. He labelled it as a cubic model, didn’t call it “our model” or anything…

      Fake news editor 2: let’s lie and say he did. The way we lied and said Trump told people to drink bleach.

      Fake news editor 1: won’t they just see that the chart doesn’t say what we’re saying?

      Fake news editor 2: nah, we can lie all day. They just read the headline. Here let me show you: “Trump’s advisers released a ‘beyond stupid’ mathematical model of coronavirus deaths created in Excel”

      Fake news editor 1: who is the headline quoting?

      Fake news editor 2: nobody. We made it up. Literally a New York Times columnist made that quote up. We’re quoting fake news itself.

      See for yourself:

      https://www.businessinsider.com/trumps-coronavirus-team-relies-on-stupid-model-by-controversial-economist-2020-5

      Fake news editor 1: really? Why don’t we just make up a story saying the white house is refusing to release the real projections and drew one by hand instead?

      Boss, walking by: you’re promoted!

      Headline next day, quoting fake news itself: “Trump ‘actively hiding’ scientist’s projections; team ‘draws their own'”.

      Fake? No problem. Dishonest? No problem.

      Get 1 million views: cha-ching!

      • Andrew says:

        Wonks2:

        The issue is not whether someone’s claiming that the cubic model “was better than anything else.” The problem was, first, that it was fit at all, as it’s an entirely inappropriate model for this sort of problem and, second, that the model was reportedly used for decision making.

        • Wonks2 says:

          >second, that the model was reportedly used for decision making.

          literally a fake news report, like my example that the white house is actively trying to hide and suppress the models. it’s fake news because the tweet literally says “As shown, IHME’s mortality curves have matched the data fairly well.” pretty much the opposite of actively hiding and repressing.

          here, let me bring my boss into this.

          Hey boss! This guy Andrew isn’t willing to lie enough for us. Could you get him on message?

          Boss comes in.

          Andrew, I think you need to remember what our goal is as an organization here. We are here to get pageviews. As long as, somewhere, the actual news is accessible to people, we can lie all day about it.

          Andrew, see your comment here? Look at this:

          >the model was reportedly used for decision making.

          you are using the term “reportedly” rather than outright lying. Please put your scruples aside. Stop saying the word “reportedly”. Don’t become the message. Make up the message. You need to start lying a lot more if you want to make it in this businesss. Where did you get your degree?

          You: University of —

          Boss: — Wrong! Harvard and Princeton. That was a test and you just failed it. You need to practice lying a lot harder. Practice after me. Just repeat these:

          -> The white house actively represses models.

          -> The white house uses a linear model to predict deaths will be negative by June. (I know we only have evidence of a cubic model, but a linear model reads stronger. Everyone knows a = mx+b. Do you remember the formula for a cubic model? Don’t answer that. Just say they use a linear model.)

          -> You are not on message because our message is that the white house uses HAND DRAWN models.

          Just to let you know our thinking in the editorial office, we considered lying that a Republican advisor’s grand-daughter drew a model in crayon and that this became the cover of Trump’s reopen america plan.

          So please, Andrew. If you want to work here get back on message.

          If you really want to get into the statistics and such you are free to join the CDC as a statistician, but while you’re here writing news your job is to lie, lie, lie. Please get on message. Take a creative writing class if you need to. we’re not here for facts, we’re here to tell a story. people can always read the facts somewhere, all they have to do is a simple Google search for one of the wire services where we get our facts. they can see the actual tweets or watch the press conferences. Our job is to embellish those facts and lie about them. Think of a tabloid. Which headline would you click: “Trump fakes Fauci’s figures” or “On reopening schools, Trump says he disagrees with Fauci, citing high youth survival rate.” I guarantee the second one isn’t going to get 1/100th the page views of the first one. I didn’t get to where I’m sitting today by telling a lot of facts.

          I want a comment from you in my inbox, saying the white house uses a linear model that predicts negative deaths by june, actively suppresses models, and that they are using a hand-drawn model. If you’re not willing to do that then I don’t know what to tell you. We’re here to get page views, not write footnotes in history books.

          Now get out of my office.

          you: uh, you came here? This is my desk.

          Boss: (walks away, ignoring your inconvenient fact.)

          • Andrew says:

            Wonks2:

            You need to have your own blog where you can write as much of this sort of story as you’d like.

          • Zhou Fang says:

            It sure is easy to rebut everything by saying “you are just lying”.

          • Joshua says:

            Thanks God the truth nanny has arrived.

            She has a lot of work to do.

            > “I think one of the things we’re most proud of is, this just came out — deaths per 100,000 people, death,” Trump said. “So deaths per 100,000 people — Germany and the United States are at the lowest rung of that ladder. Meaning low is a positive, not a negative. Germany, the United States are the two best in deaths per 100,000 people, which frankly, to me, that’s perhaps the most important number there is.”

      • Phil says:

        Wonks2,

        I didn’t see any news article that said Trump told people to drink bleach. What I did see was news articles that quoted what Trump actually said, which was this: “”And then I see the disinfectant where it knocks it out in a minute. One minute. And is there a way we can do something like that, by injection inside or almost a cleaning?”

        For the record, the answer to Trump’s question is “no.”

        And yeah, the news also reported that manufacturers of disinfectants felt the need to tell people not to drink disinfectants. But that was not fake news, the manufacturers really did tell people that.

  2. Garrett says:

    Statements like this are frustrating because I might say to a friend “don’t look at that projection, it’s not useful” and the response might be “oh so you think you know more than the head of the CEA?” When you say that someone’s wrong about something, people interpret an underlying relative judgment between the critic and the subject.

    By the way, there were nearly 2000 reported deaths from covid in the US yesterday. But the CEA implied there should’ve been less than 500! What happened!?

    • zbicyclist says:

      1500 didn’t get the memo. :)

      Note Tyler’s contention “graduate students in top econ programs all have “absolutely stellar” math GRE scores”, which is likely true.

      But one problem with math and stat education is that both multiple choice tests and ‘story problems’ involve problems where you know there’s an answer, and furthermore an answer where the information in front of you is both correct and sufficient.

      The real world (and certainly COVID-19) is substantially different.

  3. dhogaza says:

    Garrett

    What happened? As Andrew said about MJ, they couldn’t hit the curve :)

  4. jim says:

    Hilarious and tragic on so many levels.

    But maybe this is the takeaway: the different projections taken together give a rough boundary of the possibilities, which in turn about what one might guestimate by eye. So whether he understands statistics or not, nothing is lost or gained by these projections.

    But it would be fun to get paid what the creators get paid for making them and probably even more fun to get paid what the person touting them is getting paid for touting them and probably at least a little fun to get paid what the critics are getting paid for criticizing them.

    So at least people are getting paid for nothing.

    But I don’t despair! In the end it will be useful if Bank of America’s mortgage portfolio is skewed to this particular crowd, none of whom are likely to default on their mortgages, since I have a large holding in BAC.

    So in the end it’s all just stimulus funding.

  5. dhogaza says:

    Andrew:

    “Michael Jordan was a world champion basketball player but couldn’t hit the curve ball. So you’re surprised that someone who was good at taking multiple-choice math tests doesn’t understand statistics? C’mon.”

    The MJ example is perhaps even more instructive than you suggest.

    Because MJ did improve his ability to hit the curve ball towards the end of that one season of AA ball. Apparently he spent hours daily in the batting cage with the curve-ball throwing pitching machine, on game days as well as off days, working his ass off to improve, and it showed on the field. He wasn’t stellar, but he did improve, and most importantly he understood the need to work hard (of course his insane attitude towards work is one reason why he became such a legendary basketball player, becoming an excellent defensive as well as offensive player, and an effective three-point shooter, especially in the context of the times, as well, all after he’d become recognized as one of the best to play the game).

    “Tyler Cowen says that graduate students in top econ programs all have “absolutely stellar” math GRE scores, and Philipson graduated from the University of Pennsylvania, which I think has a top program . . . But, again, statistics is hard.”

    So perhaps, unlike MJ, these folks aren’t particularly self-aware and don’t recognize that perhaps they need to work hard to better their performance at statistics? MJ understood that native born talent wasn’t enough by itself. Perhaps if these folks weren’t so proud of their math GRE scores, they’d understood that native born intelligence isn’t enough, and that hard work’s needed and the need for this doesn’t stop once you’ve got your PhD, or tenured spot in academia.

  6. Roy says:

    If you look into this more, 538 made a bet before the details came out, that they just stuck the data into Excel and used its cubic trend function. It turns out that this was indeed what they did. You could perhaps even justify doing this if along with the graph there was a full explanation of what was done, the limits of what was done, and what parts should have been ignored (basically any of the fit beyond the last actual data point should not have been shown). Given what they ex post facto claimed they wanted to show a simple smooth would have done the trick.

    And it would be nice if this was just harmless, but you have to wonder how many people saw this and thought great, the infections have peeked, are going down, and will be near zero shortly, when in fact if you remove 3 states the number of infections in the US is increasing sharply. Without getting into some of the very heated debates recently on this blog, it does show the importance in really consequential situations like this of open data, open code, and discussions of the limits of the data gathering processes, the analyses done, the limits of those analyses, and what all of these say about the uncertainty of any conclusions or estimates.

    • Joseph Candelora says:

      Wow, I hadn’t seen that.

      The scariest part is the unsourced statement in the WaPo article about how the WH is using that “cubic model”:

      “White House officials have been relying on other models to make decisions on reopening, including the IHME model and a ‘cubic model’ prepared by Trump adviser and economist Kevin Hassett and the Council of Economic Advisers.

      …Even more optimistic than that, however, is the ‘cubic model’ prepared by Trump adviser and economist Kevin Hassett. People with knowledge of that model say it shows deaths dropping precipitously in May — and essentially going to zero by May 15.”

      For other readers:
      Here’s the May 4 Washington Post article that I quote from above: https://www.washingtonpost.com/health/government-report-predicts-covid-19-cases-will-reach-200000-a-day-by-june-1/2020/05/04/02fe743e-8e27-11ea-a9c0-73b93422d691_story.html

      Here’s Nate Silver’s Twitter thread from the same day, betting that Hassett just used Excel’s cubic fit on the actual data: https://twitter.com/NateSilver538/status/1257476755574718470

      And then Silver tweets simply “LOL” the next day, when CEA tweets the chart at the top Andrew’s blog post.

      • dhogaza says:

        “…Even more optimistic than that, however, is the ‘cubic model’ prepared by Trump adviser and economist Kevin Hassett. People with knowledge of that model say it shows deaths dropping precipitously in May — and essentially going to zero by May 15.”

        So THAT’S where that crazy prediction came from. I was very surprised to see Dr. Birx quoted as saying that deaths would start dropping steeply towards the end of May.

      • Øystein says:

        Yes, how the “model” was used by the White house is the real issue. Given Philipson’s background, it seems implausible that he did not understand what the cubic function was actually doing. I think putting the cubic fit on is confusing and obviously possibly misleading, but if it helps him think about the data better, great. However, he works for the WH and has some responsibility for how they interpret it. We do not know how the cubic fit was presented internally, but the quotes indicate that people actually used it as a forecast, regardless of whether it’s intention only was as visualization.

    • Eric B Rasmusen says:

      “It turns out that this was indeed what they did.” Cite? Why do you think they used Excel? The 538 cite doesnt’ give any reason. Presumably any software can generate a cubic curve, even the fanciest.

  7. anonymous commenter says:

    Reminder: Andrew Gelman uses his credentials to perpetuate anti-Asian discrimination at US colleges.

    http://bostonreview.net/law-justice/andrew-gelman-sharad-goel-daniel-e-ho-what-statistics-cant-tell-us-fight-over

    • Andrew says:

      Anonymous:

      My article that you link to, coauthored by Sharad Goel and Daniel Ho, has the title and subtitle, “What Statistics Can’t Tell Us in the Fight over Affirmative Action at Harvard. A group seeking to ban affirmative action has sued Harvard for discriminating against Asian Americans. The core issues won’t be resolved by statistics alone.” Nowhere do we “perpetuate anti-Asian discrimination.” Nor, for that matter, do we “use our credentials.” If you have a specific aspect of this article that you disagree with, feel free to share this.

  8. anonymous commenter says:

    The byline in the article links to the author’s bio, http://bostonreview.net/andrew-gelman

    The first line of the bio is “Andrew Gelman is Professor of Statistics and Political Science at Columbia University.”

    Now what do these credentials allows him to do? They allow him to cite with approval a statement that refers to Asians as “pawns”: “As a dean at the University of California, Berkeley, has said, Asian Americans are ‘being used as a pawn in a chess game.’ “

    “Pawn” is a word that casually denies agency to Asians. It perpetuates the stereotype that Asian Americans are passive and naive.

    “Pawn” is a degrading slur even if Andrew Gelman is “Professor of Statistics and Political Science at Columbia University.”

    • Andrew says:

      Anonymous:

      I agree that we should not have reported that quote from the dean in that way. We should’ve removed the word “As” from the statement you quoted, so instead it would’ve said, “A dean …” Reporting that quote is fine, but we should’ve made clear that by reporting that quote, we were not endorsing it. So good catch on your part. I would not say that this one quote is enough to say that our article “perpetuates anti-Asian discrimination,” but I guess that’s a judgment call.

      Also, this is a more minor issue, but, no, a link to a bio with my academic credentials does not mean that we “used our credentials” to make our points. Sharad, Daniel, and I are who we are, and we make that information available.

      P.S. You keep signing your comments with the user name “Boston Review” and with a link to Boston Review which isn’t accurate because you’re not Boston Review, so that’s why I’ve changed to “anonymous commenter.” I don’t have the patience to keep fixing this. So in future comments, please either use your name or a non-misleading pseudonym.

      • Simple question:

        Is it “reputation laundering” when you approvingly cite a “dean at the University of California, Berkeley” in casually referring to Asians as “pawns” ?

        • Andrew says:

          Anonymous:

          First, as I said above, it was a mistake for us to copy the quote approvingly. We should’ve removed the word “As” from that sentence. But, beyond that, no, it’s not “reputation laundering” to quote an official from a selective university when writing about admissions policies at selective universities, any more than it would be reputation laundering to quote an official from Facebook when writing an article about Amazon, or to quote an official from the LAPD when writing an article about the NYPD.

          I’m guessing that you disagree with what Goel, Ho, and I wrote in that article, and that’s fine—disagree all you’d like!—but this seems all pretty much irrelevant to the topic of today’s post. I recommend that if you have more to say on the topic, that you (a) say something about the actual article beyond that one secondhand quote, and (b) put it in the appropriate comment thread, which is here.

          • Referring to Asians as “pawns” is not something where we can agree to disagree on.

            It’s a casual racist slur. And you approvingly quoted the slur without any supporting data.

            I guess you don’t need data when you can invoke the exalted reputation of the “dean at the University of California, Berkeley.”

            • Andrew says:

              Anon:

              As noted above, I agree with you that we shouldn’t have reported that quote approvingly. The quote is what it is. I don’t think there’s anything wrong with reporting the quote or its source; we just shouldn’t have used the word “As” in that sentence. We’re in agreement on this one so I don’t know why you keep bringing it up. I guess I could contact Boston Review and see if they can fix that sentence. Again, if you have anything else to say about this article, I recommend you post on the relevant comment thread where thee’s a lot more said on the topic!

              • Andrew says:

                By the way, this is one reason I prefer blogs to twitter. On twitter, someone can just do a hit-and-run, call you a racist or whatever, and then watch the likes stream in. On the blog, I can respond directly in comments, and anyone who sees the original statement can see the follow-up discussion.

                Again, I agree that we should not have cited that quote approvingly. I’ll ask the Boston Review people to remove the word “As” in that sentence.

              • Removal of the word “As” doesn’t really soften the blow of that “pawn” quote from the Berkeley dean.

                It’s the third-to-last sentence of the article.

                You’re still letting that prejudicial quote stand as the last word on the topic.

            • Phil says:

              Actually, referring to Asians (or any other group of people) as ‘pawns’ is something we can agree to disagree on. I agree with Steve (several comments below) that if people are being used as pawns then it should be OK to say so.

              A Google News search’ finds current claims that doctors and nurses are being used as pawns; refugees in Turkey are being used as pawns; Australian fire survivors feel that they are being used as pawns; children in the PWCS (whatever that is) are being used as pawns; some Premier League soccer players feel they are being used as pawns; and on and on and on.

              I suppose one can argue that the term ‘pawns’ should never be used to refer to people, that it devalues them as human beings or something. I would disagree but it doesn’t seem ridiculous. But that doesn’t have anything to do with racism, really. As the examples above illustrate, racial groups are not the only groups ‘used as pawns.’

              If, instead, you’re fine with everyone else being called ‘pawns’, but for some reason not Asians, well, that just seems bizarre.

              • There are 150 Asian organizations supporting the lawsuit against Harvard.

                Using the term “pawn” implies these groups are brainwashed and have no minds of their own. This is a vile racist stereotype.

              • Zhou Fang says:

                It doesn’t imply anything of the sort.

                It implies the lawsuit is being done in their name but by someone ultimately uninterested in their well being.

              • Exactly, the meaning of “being a pawn” is that someone else is in control of your destiny and they don’t care very much about you.

                It has almost nothing to do with the person being called a pawn. It’s a statement about the person in control, the manner in which they are treating others.

              • yyw says:

                That was just a bad quote and imo Andrew has addressed it appropriately.

                An ambulance chasing lawyer might be using victims as pawns. It doesn’t mean that it isn’t in a victim’s interest to work with him/her. It’s also bad form to impute ulterior motive without providing clear evidence.

              • Martha (Smith) says:

                “PWCS (whatever that is)”

                Prince William County Schools?

    • Zhou Fang says:

      Hello, I am an Asian person, stop using me as a pawn in your fight against proper statistical methodology during a pandemic, thanks.

    • Another anonymous commenter says:

      Comments made by the “anonymous commenter” highlight some interesting aspects of these social justice types. I should preface this by pointing out that I would certainly see myself as a “progresseive” when it comes to social justice issues[1]. My beef is with the “social justice” culture that lives on such sites as Reddit and Tumblr, and indeed in blogs and comment sections such as this.

      First: nothng can vindicate the crime of the accused. I think Andrew’s comments are more than sufficient, but nothing seems to have changed in the attitude of the accuser. They are as aggressive as ever. They are driven more by search for (online) drama than anything else. For them this is NOT an issue to be discussed, or to be resolved; it is an issue to be ruminated on, something to be enraged over for online entertainment. There is absoutely nothing to be done at this point, and my guess is that they will continue posting their “reminders”. As I will point out later, discussion for these people is seen as being problematic itself; yes, just the mere fact that people dare to discuss things is seen as oppressive.

      Second: as already suggested in the first part, there is no discussion to be had. This is usually achieved through extreme abstraction. For example everything about this case that is not about racism has been abstracted away: “anonymous commenter” makes it clear that for them no other dimension exist other than “this is racist” or “this is not racist”. As many people have pointed out, there are many other dimensions to this problem, all sorts of stuff about metaphors and how to discuss issues pertaining to marginalized groups, about intentions of writers and interpretations… this is all flushed down the toilet. Either you see the issue as racist or not-racist, there is no other discussion to be had.

      Third: initial knee-jerk reactions are seen as the end of the discussion. In this case “anonymous commenter” had the impression that the expression about pawns was racist and that’s it. My guess is that for most people rading this blog this would a starting point for self reflection: did I interpret that correctly, even if there the expression CAN be seen like that (it sure can) is that a reasonable interpreation, if this is to be interpreted as racist, how far is it reasonable to go with that interpetation and so on. Maybe these ideas would further be refined in a discussion with other people. But for people like “anonymous commenter” this is poison.

      They don’t see that kind of self reflection as positive thing, they see it as oppression. The initial emotional reaction to something is taken as the ground truth: if one gets the feeling that Andrew is racist, then any discussion beyond that is seen as devaluing that person’s opinion and oppresive if it’s people from a majority group[2] telling a marginalized person “how they should feel” or that “their experiences aren’t valid”. Obviously no one is telling them “how to feel” but to go through completely normal cycles of self reflection, but this is how this is usually framed.

      One annoying aspect in all of this is that there is grain of truth in many of these things. For example when it comes to telling people that their intuitions about something being racist is invalid IS a tactic used by racists online. Just couple days ago I saw this in action on Reddit: there was a “clever” caricature of Anita Sarkeesian as the jew from the famous nazi propaganda picture, but any comments pointing this out were deflected by saying it doesn’t matter, that they are just overthinking it and so on. That was a clear example of racists/sexists hiding their attitudes by nullifying criticisms; maybe even indeed “gaslighting”: “you are only IMAGINING those nazi connotations”, I think was their main deflection.

      This also applies to the initial complaint about describing people or organizations as pawns. Many commenters here have brought up interesting aspects to this, and indeed there is a discussion to be had. But, as I already said, this is all thrown away, discussion is framed as being a form of oppression in itself. This leads to obvious problems, since no matter how much “anonymous commenter” and the likes would like us to believe that these are binary issues about “racism” or “human rights”, the issues themselves — in this case communication, it’s implications etc. — are much more nuanced and complex. Pair that sort of nuancedness, complexity with the aforementioned propensity to frame any sort of self-reflection or discussion as problematic/oppression, and we are in quite the pickle.

      I would like to end by pointing out that racism, sexism, homophobia, classism and so on are important issues. Often these sorts of criticisms are deflected by implying that people, like me, who dare to criticize social justice culture, would be operating under the idea that such problems don’t exist or if they do thay aren’t worth discussing. Quite the contrary. I’m so annoyed by social justice culture BECAUSE I think the problems mentioned ARE important and I see people like “anonymous commenter” doing a great disservice and devaluing the gravity of such issues with the strategies they use in online discussions.

      [1] One more demonstration of the impossibility of discourse is that this statement WILL be nullified. It will be seen as lying, I will be branded as a racist, who is just trying to masquerade as a progressive. The idea that someone could be progressive and not side with them completely, is completely foreign to them, and they will not accept anything less than 100 percent compliance with their opinions.

      [2] If the person is from a minority group, they are usually diagnosed with having “internalized racism” and “perpetuating oppression”.

      P.S. Can’t find spell checker for the new version of Notepad++. I will blame all of my typos on that. Also on the fact that I simultaneously had a discussion (about music) with my spouse. Now I know what Stan has to go trough when I use it in the parallel mode.

      • You plead for nuance and reasoned debate.

        Yet you don’t seem to mind when one of the sides in the debate is casually dismissed as a “pawn.”

        If you traffic in ad-hominem attacks, do not expect a genteel response.

        • They’re not being “casually dismissed”.

          If I say “original commenter is being used as a pawn by a mortgage broker cabal” it doesn’t mean I think you aren’t valuable… it means the mortgage broker cabal is treating you as if you aren’t valuable. In fact the person saying this is usually saying it *because they think the ones being treated as pawns are more valuable than the actions of the chessmasters imply* you are getting this *exactly backwards*. Saying “asian students are being treated as pawns by such and such university” is a *condemnation of the university* not a condemnation of the asian students.

    • Terry says:

      My first guess is that “anonymous commenter” is a troll.

      Second guess: “anonymous commenter” is a Social Justice Warrior practicing Social Justice Battle with training wheels on. True, these first two are often observationally indistinguishable because neither appears to care about the truth of what they are saying.

      Third guess: (sarcasm alert) “anonymous commenter” is an Asian Supremacist seeking to perpetuate perceived Asian hegemony via institutional power mechanisms embedded within an intersectional web of power dynamics, inequality, sexism, racism, and homophobia. (See how easy it is to spin a BS Social Justice narrative?)

  9. A perfect math GRE score puts one in the 96th percentile (not sure how they handle individuals taking multiple tests). Hardly a sign of mathematical genius, even among the pool of students taking the GRE.

  10. MP says:

    As too often happens, once politics becomes involved, the quality of discussion dives. And politics seems to get involved in everything. And politics on twitter just makes it worse.

    I agree with Wonks initial comment, the cubic is just presented as a smoothing function. It isn’t extrapolated on the chart, and it isn’t meant to be extrapolated. And yet Furman jumps on it with his “lowest point in history” comment, saying it was superficial and misleading to do something that wasn’t even done. Phillipson then comes back with a semi-valid defense followed by a childish, unnecessary, and (I think) inaccurate personal attack.

    That said, the cubic seems like a really poor choice for smoothing this data. It doesn’t actually look like it shows a very good fit to the IHME curves. And even if it had, it wouldn’t outweigh the other criticism I’ve read of the IHME model.

    Furman is right to say that the stakes are high. I’d like to see more people take that as a reason to act better, not to get angrier faster.

    • David says:

      Actually, if you look closely, you can see the cubic curve is extrapolated on the chart – that’s the pink dots. It’s not *correctly* extrapolated, as the actual cubic curve would cross the X axis, but I suppose showing the correct extrapolation would just highlight how ridiculous the “model” is. Instead, they made it symmetrical like a bell curve.

      The problem is that even if it wasn’t extrapolated, and even if it perfectly fit the data, and even if it was made entirely clear that the curve wasn’t being used for projections, a cubic curve has no business on that chart. Curve-fitting for data visualisation just isn’t a thing. For data visualisation you either use trend lines or data smoothing. Polynomial fitting is something you do to build a model, which you only do if you have theoretical basis for believing that the data should fit that type of model. For instance, you might try fitting an exponential curve or a logistic curve because there is a theoretical basis for relating these functions to an epidemic. Polynomial functions just make no sense in that context, even if they happen to fit the data well over a short time span.

  11. Anonymous says:

    There are respected prize winning academics who’ve written multiple books on statistical inference, authored many dozens of papers on how stats should be done, invented new stat tools, are widely listened to as stat gurus, regularly lecture on the subject, and yet who’ve seemingly never made a statistical or scientific inference in their life and probably couldn’t do a cubic curve-fit in Excel.

    Stats just seems to be that kind of endeavor.

  12. ia says:

    Philipson is a fairly well-known hack, and his hackery started long before he came to the CEA: https://www.propublica.org/article/big-pharma-quietly-enlists-leading-professors-to-justify-1000-per-day-drugs

    • Andrew says:

      Ia:

      Interesting. Philipson does not just work for the controversial consulting firm that is discussed at that link, he’s also a co-founder, according to the article.

      Also I noticed this:

      When AbbVie funded a special issue of the American Journal of Managed Care on hepatitis C research, current or former associates of Precision Health Economics wrote half of the issue. A Stanford professor who had previously consulted for the firm served as guest editor-in-chief.

      I wanted to see who the Stanford professor was, so I clicked on that second link. Unfortunately it sent me to a redirect, but the link itself has the text “bhattacharya-jena-and-batt-added-as-phe-principal-consultants.” Some googling reveals that, yes, it’s Jay Bhattacharya, who’s come up in the news recently.

      Small world!

      Just to be clear, I’m not trying to make some general criticism of industry consulting. How could I? I’ve done lots of industry consulting myself, including for pharmaceutical companies. I don’t think industry consulting automatically makes one a hack. I have no comment on the particular consulting described in the above-linked article, as I’ve not looked into the details.

    • JFA says:

      Is a drug that cures Hepatitis C not worth $1000/day? If we want pharmaceutical companies to come up with cures, shouldn’t they get paid for the value that produces? I don’t have time to read the very long and I’m sure very detailed article, but whatever hackery Philipson has been involved in, saying that medical cures are valuable probably shouldn’t included in the list.

  13. anonymous says:

    Philipson has published in Biometrika (twice), Econometrica (twice), AER (seven times), QJE (once), JPE (once), restud (once), and a bunch of other papers. Some of these are very good, serious empirical papers.

    I highly doubt that Philipson doesn’t understand extrapolation.

    It is far more likely that Philipson understands about as much as we’d expect someone with his CV to understand, but is now willing to let political motives drive empirical analyses.

    You like to imagine that everyone else is an idiot, but sometimes people just lie.

    • seperate anonymous says:

      Possibly, but another hypothesis is that as people become more senior, or more involved in politics/administration/management their skills atrophy.

      How well does Zuckerberg code now for example?

    • Andrew says:

      Anon:

      I agree that there are political motivations, and Philipson could just be lying. I thought about this when writing the post. But I think that, in addition to any political factors, he doesn’t understand what polynomials can and can’t do in fitting and extrapolation. One reason I say this is that there’s a recent history of economists being overconfident about predictions from polynomial models (see the last two links in my above post), which makes me think they just haven’t thought these issues through. Regarding Philipson’s publications: I’ve not looked at them, but I can well believe that he has done good work, both theoretical and applied. It’s possible to do good work without fully understanding how all your tools work. That’s just how things go: we are working on important projects, so we use sophisticated tools that are at the edge of our understanding.

      • Joseph Candelora says:

        As I learned after digging into Roy’s comment above, it appears that the cubic fit isn’t Phillipson’s work, but rather that of Kevin Hassett, another former CEA chair and an economist who Trump brought in as senior advisor on economic policy for COVID19.

        It looks like Hassett’s career has been entirely political — some time in government, 20 years (I think) at the American Enterprise Institute, and then two gigs under Trump. So I’m not sure how well your academia thesis fits here. I guess it does if one considers thinktanks to be a branch of academia, but this seems like pure politics. Hassett gave Trump the projection he wanted, and Phillipson went along with it.

        If you want some more cringe, there’s a good 5/2 Washington Post article about Hassett’s model https://www.washingtonpost.com/politics/34-days-of-pandemic-inside-trumps-desperate-attempts-to-reopen-america/2020/05/02/e99911f4-8b54-11ea-9dfd-990f9dcc71fc_story.html :
        “The epidemiological models under review in the White House Situation Room in late March were bracing. In a best-case scenario, they showed the novel coronavirus was likely to kill between 100,000 and 240,000 Americans. President Trump was apprehensive about so much carnage on his watch, yet also impatient to reopen the economy — and he wanted data to justify doing so.

        So the White House considered its own analysis. A small team led by Kevin Hassett — a former chairman of Trump’s Council of Economic Advisers with no background in infectious diseases — quietly built an econometric model to guide response operations.”

        Many White House aides interpreted the analysis as predicting that the daily death count would peak in mid-April before dropping off substantially, and that there would be far fewer fatalities than initially foreseen, according to six people briefed on it.

        Although Hassett denied that he ever projected the number of dead, other senior administration officials said his presentations characterized the count as lower than commonly forecast — and that it was embraced inside the West Wing by the president’s son-in-law, Jared Kushner, and other powerful aides helping to oversee the government’s pandemic response. It affirmed their own skepticism about the severity of the virus and bolstered their case to shift the focus to the economy, which they firmly believed would determine whether Trump wins a second term.”

        The May 4 WaPo article I linked to above in response to Roy’s comment is the one that says the cubic fit “model” is Hassett’s model:
        https://www.washingtonpost.com/health/government-report-predicts-covid-19-cases-will-reach-200000-a-day-by-june-1/2020/05/04/02fe743e-8e27-11ea-a9c0-73b93422d691_story.html

        Now I suppose it’s possible that Hassett (former CEA chair under Trump) and Phillipson (current CEA chair under Trump) have both independently instituted cubic fits to the observed data. Alternatively, it’s possible that WaPo misunderstood and mischaracterized Hassett’s model, and that he’s doing something other than the cubic fit they described. But that all seems very unlikely, so I’m going with Hassett’s cubic fit as being what has been adopted by Phillipson. And I think the fact that the chart the CEA tweeted includes a red dotted line showing projection from the cubic fit gives away the game — this is in fact the “model” that the President’s handpicked advisor developed, and has been using for internal projection purposes.

    • Steve says:

      How many of these are solo papers? I don’t know, just asking. But I think that is what Andrew had in mind with #2. Some people can network their way to the top and yes, that includes publications. And it is only getting worse, as now hardly anyone publishes solo.

    • Andrew says:

      OK, I was curious so I looked up a few of Philipson’s papers, using Google to find his papers in Biometrika and Econometrica.

      Geoffard and Philipson (Biometrika, 1995): Proves a mathematical property of a certain model for infectious diseases.

      Philipson and DeSimone (Biometrika, 1997): Analysis of randomized experiments when participants can try to figure out which treatments they’re getting.

      Koijen, Philipson, and Uhlig (Econometrica, 2016): Theoretical and empirical analysis of the link between financial and real health care markets.

      Philipson and Hedges (Econometrica, 1998): Analysis of randomized experiments when participants can try to figure out which treatments they’re getting.

      Philipson (Econometrica, 2001): Benefits of incentives in survey research.

      They look like solid work to me. So I don’t imagine that Philipson is “an idiot” as the anonymous commenter above claimed. Nor did I say anywhere in my above post that he was an idiot, or anything like that.

      What I said is that statistics is hard. Even for people who know a lot of statistics!

  14. David Young says:

    What you are pointing to Andrew (misuse of statistics and numerical analysis) is in my view a growing problem.

    40 years ago it was possible to build large interdisciplinary teams and keep them funded. As science has grown, these teams become essential to do high q

  15. David Young says:

    What you are pointing to Andrew (misuse of statistics and numerical analysis) is in my view a growing problem.

    40 years ago it was possible to build large interdisciplinary teams and keep them funded. As science has grown, these teams become essential to do high quality work. No single individual can be familiar with even the majority of the knowledge to do a competent job of building a new modeling tool for example.

    In medicine, with large studies, it is the usual practice to get a professional statistician to help with study design and data analysis. That’s not the case in many other fields but should be. When I delved into uncertainty in CFD over the last decade I found 2 fantastic statisticians to do a lot of the heavy lifting. They were fantastic to work with. Some fields are pretty bad in this regard. Climate science has had a problem in this regard and some of the abuses of statistics have become famous as outside statisticians have come in and gotten some practices changed. The use of uniform priors is in my mind the most consequential one.

    With regard to numerical analysis, that’s a mature field in its own right with numerical PDE’s a big part of it. Yet, most applications codes are built and used without these specialists on the team. This is partly an artifact of the growing soft money pressure everyone is feeling which makes building and sustaining teams difficult. But also the constant pressure to publish a large volume of papers means there is tremendous pressure to get the job done as quickly as possible. The advent of R means anyone with technical training can “roll their own” and most do.

    I also think this new “culture” has caused a deceleration of progress. In CFD, the problem is bad. Decades of colorful fluid dynamics and dramatic over selling of codes, models and methods has given laymen the thought that CFD is a solved problem. Nothing could be further from the truth. The selection bias is intense and everywhere. You virtually never see simulations that disagree with data shown, particularly to outsiders who control your funding. The result is that progress, for example on time accurate eddy resolving simulations, has ground to a halt. Similarly, progress on turbulence modeling has ground to a halt, with top experts growing cynical about the entire field and its mass delusions. Negative results should be required in any paper about a model or code.

  16. Fafa says:

    Philipson’s papers on infectious disease modeling in the 90s were good, as was his work on cost estimates of burden of disease and adherence. They emphasize the importance of accounting for private averting behavior when individuals perceive health risks. In other words, he would seem to be the type of economist you would want to see in a prominent government position right now. But I suppose the desire to win the race to humiliate one’s self quickest is hard to resist. I thought I was sufficiently cynical, and I don’t know anything about him personally, but it is stunning to see this type of hackery from him. The money better be great.

    • Andrew says:

      Fafa:

      If we’re gonna speculate, I’d go with duty rather than money as the motivation, and overconfidence rather than hackery as the way he got there. We can assume that Philipson’s talked with enough like-minded people to be convinced that his policy positions are correct. If your positions are correct and the stakes are high, you’ll want to shout. And if you consider yourself a statistics expert but you haven’t thought too hard about forecasting, you might not fully see through the problems of one particular model.

  17. paul alper says:

    Andrew wrote:

    “What I said is that statistics is hard. Even for people who know a lot of statistics!”

    The same goes for astrology:

    https://www.nytimes.com/2018/10/17/style/astrology-exam.html?action=click&module=RelatedLinks&pgtype=Article
    ———————————————–
    “There’s an exam, and it involves math. In fact, there are many exams [including the odd trick question].

    Interpretation is one of many skills that the ISAR CAP tests. It includes an essay portion and about 600 multiple-choice, true-false and short-answer questions, which cover chart calculations, the history of astrology, basic astronomy as applied to astrology and forecasting skills. Sample questions include: What is the Sun’s greatest distance from the celestial equator? What is the harmonic of a quintile aspect, and how many degrees is it? And how often are Mercury and Venus trine? (Trick question! A trine is a 120-degree angle between two planets, which never occurs between Mercury and Venus!)”
    ———————————————–

    On occasion, as in statistics, some forecasts need to be modified as the data roll(s) in:

    https://www.nytimes.com/2020/05/09/style/coronavirus-astrology-predictions.html?campaign_id=9&emc=edit_nn_20200510&instance_id=18372&nl=the-morning&regi_id=77532059&segment_id=27142&te=1&user_id=d7e3e90dc8fbbcc2d51df749fc62495f

    “The prominent astrologer Susan Miller placed responsibility for the coronavirus on the distant dwarf planet [Pluto] in March after her earlier, pre-pandemic prediction of a “great” 2020 seemed to miss the mark.”

    ————————————————
    Just as statisticians and economists can still remain unscathed despite being very wrong,

    “the fact that astrologers did not see the virus coming hasn’t made their practice any less popular. In fact, horoscope sites have reported rising traffic as people look to the stars to give shape to a formless quarantine life.”

    • Andrew says:

      Paul:

      No kidding. If astrology was easy, everybody’d be doing it.

      • Victor says:

        Dude I think you missed the point entirely. I think Paul’s point is that any kind of exercise that involves using a 40 day time series to predict the future in the context of an unprecedented event is about as pointless as astrology. Y’all are bickering about using a cubic function or splines but the reality is all we have is piss poor measurements and uncertainty. You might as well be arguing between using tarot cards vs a crystal ball.

        • Andrew says:

          Victor,

          I disagree with you. Decisions have to be made about all sorts of things, ranging from travel restrictions and school closings to budges for testing and vaccine development. Science-based forecasts are not perfect, but good forecasts map from assumptions to conclusions, and when the forecasts are (inevitably) wrong, we can learn from them. None of this is true for polynomial extrapolations, tarot cards, or crystal balls. As the saying goes, the shitty is the enemy of the good.

  18. somebody says:

    Having undertaken some graduate study in economics, I would say the problem is more with the general educational philosophy of the discipline than with statistics being hard. You make fun of the GRE statement, but about a third of my classmates had straight As from an undergraduate program in pure math. And they got a work out in analysis at least, some topology and differential geometry as well depending on who you get as a teacher. If you’re a non economist wondering how all this pure math could possibly helpful to a practicing economist, I still don’t know the answer and I got As! It really felt like a sort of hazing ritual economists put their students through to make sure that if a noneconomist asks questions, they can reply with enough math that people stop listening.

    Or, in other words, we were taught a whole bunch of theorems about economic models so that we could abdicate from engaging with them critically or philosophically, which is to me the really interesting bits in statistics. Andrew has previously blogged about Mostly Harmless Econometrics puzzled about why so much of the book was dedicated to proving complicated theorems. My feeling is that economists learn the theorems so if someone has an objection you can quote a theorem at them. There was very little in the way of visualization, exploratory data analysis, statistical programming, or examining full distributions, it was less “embrace uncertainty” and more “applying CLT with the delta method means you don’t get to say anything.” Probably the most embarrassing bit is that despite proving theorems about utility functions for half a year, I learned about doing formal decision analyses over posterior distributions here.

    Probably the most egregious example is learning about how Schnonnenstein, Mantel, Debreau shows that general equilibria are probably not attractive, then being told without justification that I should use general equilibrium based models anyways. Instead of “push your model, see how far it goes without breaking”, you “describe your model mathematically and close your eyes immediately.” In short, the problem isn’t that statistics is hard. It’s that economists are taught statistics as a discipline of self-defense.

    • Kyle C says:

      This is so well expressed I want to save it to show other people so I don’t rant at them.

    • ia says:

      wow…you sure didn’t actually internalize anything from your first year! I learned a lot more than this, which sounds more like a caricature of how non-economists describe economists.

    • Fred says:

      “If you’re a non economist wondering how all this pure math could possibly helpful to a practicing economist, I still don’t know the answer and I got As!”
      This sentence says more about you (and perhaps your program) than Econ as a field in general.
      No, I don’t use most of the stuff, pure math or not, I learned in year one of grad school and I probably won’t.
      But who does?
      Don’t get me wrong.
      Economics as a discipline has a LOT of serious problems but so many of the criticisms (from non-Economists) tend to be misguided and it unfortunately serves as a distraction from discussing actual problems that Economics needs to address.

      • somebody says:

        > This sentence says more about you (and perhaps your program) than Econ as a field in general.

        It’s possible, the program had a reputation for being elitist and navel-gazey. However, I’ve had a number of experiences on and offline that go something like this:

        1. Economic historian asks “Why are you using this GE model? I don’t really see a reason to believe we’re in general equilibrium now or at any given point in time.”
        2. Economist responds “You think we don’t know about this? That’s just the Sonnenstein, Mantel, Debreau theorem, we’re all already aware of this, we teach it to freshmen. All models are wrong but this is the best we have.”

        Never anything more concrete than that, a reason to believe we’re in general equilbrium at any given point in time or that the errors are theoretically bounded or that the models have some kind of conspicuously strong fit to any historical dataset. After a while it started to feel like the purpose of learning the theorem was so that I could tell people “I already know this, I learned it my freshman year.”

        I learned quite a lot of theorems and didn’t learn anything about statistical programming (stata doesn’t count), data visualization, or exploratory data analysis. I’m not sure if I ever even heard “overfit” or “sensitivity analysis” in an actual class, and I’m certain I never heard “predictive check” or “model check.” We didn’t read Tukey, Tufte, or anything that didn’t have a proof in it. For the alumni I know, none of them credit any of their actual economics classes for any practical skills in data science. Everything they know they picked up along the way from research, not infrequently with some horrendously convoluted workflow. I’m curious why you think not teaching anything relevant to practical research, while teaching an endless parade of theorems proceeding from obviously false axioms, is not an incorrect way to teach an ostensibly empirical science. Why not just do away with the econometrics classes entirely and just teach theory if everyone is just going to pick up the real skills on the side?

        • somebody says:

          The last paragraph is just about econometrics by the way. Of course the theoretical content is very important for making sense of economic events with a shared language. But econometrics being a set of “pragmatic” courses with an apparent focus on research skills, but the content of which turns out to be completely useless navel gazing once you actually want to work with real datasets, seems like a tremendous waste.

        • Andrew says:

          Somebody:

          This reminds me of my pet peeve, the definition of risk aversion as a nonlinear utility function for money. Actually you see risk aversion with small amounts of money, on a scale where a nonlinear utility function makes no sense (see section 5 of this article). This is a pretty basic example. But risk aversion is still defined that way all the time. Economists know about this sort of example but they treat it as some sort of weird anomaly.

          I guess they’d probably say that they’re no different than physicists teaching classical mechanics using the concept of frictionless pucks, or statisticians assuming the normal distribution even though nothing’s really normally distributed in practice. And maybe they’re right, I dunno.

        • Peter Dorman says:

          I’ve spent the better part of a career wondering why economists put GET (general eq theory) at the center even though it has been ravaged by theory since the 1970s. My pet explanation is that it is required for welfare analysis: you can’t interpret market outcomes or proposed interventions according to the welfare yardstick unless you believe that the economy is in a (constrained) equilibrium in which prices have these desirable properties. I strongly urge economists to abandon welfarism, period. It is theoretically indefensible, intellectually naive, and there are better ways to make economic analysis relevant to social decision-making.

          That’s my rant.

          • Dale Lehman says:

            I agree completely. Along the same lines, I’ve wondered why 2nd best theory seems to have disappeared from the field. It undermines most of the analysis that economists do (I guess the reason is not a mystery after all) but is hardly mentioned any more.

        • Fred says:

          > It’s possible, the program had a reputation for being elitist and navel-gazey.

          I’m not sure what being elitist has to do with this.
          If anything, I’d expect elitist programs to do a better job at outlining how the math they learn in year one is related to more advanced courses and research.

          As pertains to the GE model, I do agree with lot of your criticisms, but(or therefore?) I am not a macroeconomist so I don’t really consider myself qualified to discuss this issue in detail.

          >I learned quite a lot of theorems and didn’t learn anything about statistical programming (stata doesn’t count), data visualization, or exploratory data analysis. I’m not sure if I ever even heard “overfit” or “sensitivity analysis” in an actual class

          I guess this is the point where I shrug and say YMMV. If that has been your experience, I am frankly a little appalled but that hasn’t been my experience nor my friends’ experiences in other schools.

          I will say this. There is a reason why they don’t award you Ph.D after the first couple of years after you have taken most of the lectures.
          Graduate school is not a place where everything you need to know will come from classes.
          That you need to pick up a lot of practical skills from actually conducting research (with the guidance of faculty) is not a bug but a necessary feature in my opinion.
          There is only so much you can learn by learning without doing.
          And that is hardly exclusive to Economics.

          > I’m curious why you think not teaching anything relevant to practical research, while teaching an endless parade of theorems proceeding from obviously false axioms, is not an incorrect way to teach an ostensibly empirical science.

          I reject the notion that Econ grad schools don’t teach anything relevant to practical research period.
          Maybe not a whole lot in year one. But again, that is to be expected in most fields.
          I can’t help but get the feeling that you have a narrow idea of what ‘practical research’ consists of.
          Econ Ph.D program’s main goal isn’t to develop data scientists, but Economists.
          I don’t think it is fair to blame its curriculum for not being optimized for data scientists.

          • somebody says:

            > I reject the notion that Econ grad schools don’t teach anything relevant to practical research period.
            Maybe not a whole lot in year one. But again, that is to be expected in most fields.
            I can’t help but get the feeling that you have a narrow idea of what ‘practical research’ consists of.
            Econ Ph.D program’s main goal isn’t to develop data scientists, but Economists.
            I don’t think it is fair to blame its curriculum for not being optimized for data scientists.

            I do think lots of economics courses are great! They’re necessary to make formalized statements about economic phenomena, which provided a shared unambiguous language for discussion and testable hypotheses. My question is more directed at econometrics classes and their failure to teach economists to engage empirically. Econometrics courses don’t really have what I would call “economic content” except as motivating examples. They seem intended to be econ grad school’s “practical research class,” except all I learned was a bunch of applications of the delta method, plus running OLS in stata. My only explanation for spending so much time on things that both aren’t useful pragmatically and also aren’t economically informative is that doing proofs about uniform convergence of estimators lets economists defend themselves against statistical criticism.

            And fine, if they don’t teach all the practical details in the courses, you pick them up on the job. But even if economists can teach themselves R or python or whatever, there’s a broader philosophy of thinking critically about the relationship between a model and the dataset that seems missed. The real lifeblood of statistics isn’t in the p values or in the cluster robust sandwich estimators of standard errors, it’s in looking at pretty plots for funny patterns that should or shouldn’t show up if you’re on the right track. Economics papers seem to like centering the former over the latter under the illusion that the numbers and formulas are more concrete.

            • JFA says:

              All my econometrics courses covered very useful topics like measurement error, causal estimation, censored data models (many of which were derived from labor economics models), among other econometric models. Some field courses (like IO) covered structural equation modeling (most of which were derived from game theoretic models… most of which were over my head). We even covered Bayesian estimation.

              Quality varies from program to program. I was in a top 50 program, so not a tip-top program. I do think they spent too much time on theory rather than practice, but overall, I learned quite a bit of knowledge that I use everyday.

              • somebody says:

                The irritating thing is that I took these courses at a top 5 program (full disclosure, I was an undergrad in something else), and the attitude really seemed to be that the more topology and differential geometry someone invoked the better they were, and the more empirically oriented the research is the less respect it deserves.

  19. Willard says:

    Speaking of connections and reputations:

    Neeleman is feeling the hit. On top of starting JetBlue in 1999, the entrepreneur founded Azul Brazilian Airlines, cofounded WestJet of Canada and Morris Air, and holds a major stake in TAP Air Portugal.

    On April 7, he vented in an op-ed for the Daily Wire, the right-wing news website helmed by political commentator Ben Shapiro. “Since the outbreak, I have spent all my days and a lot of my nights trying to find a solution to save as many as possible of the 40,000 jobs I am responsible for and do what I can to help avoid an economic catastrophe in the making,” Neeleman wrote.

    His “search for a solution,” he continued, had led him to “three amazing and dedicated professors and scientists from Stanford University School of Medicine with impeccable credentials”: Jay Bhattacharya, Eran Bendavid, and John Ioannidis. “I have come to know them personally,” Neeleman added.

    https://www.buzzfeednews.com/article/stephaniemlee/stanford-coronavirus-neeleman-ioannidis-whistleblower

    • Joshua says:

      Willard –

      That’s intersting. Maybe John’s being the 17th author doesn’t really give a clear sense of how influential he was in the project?

      I might also wonder, since he’s personal buds with a rightwinger who has such strong feelings about the government mandated shelter in place orders, there is some political connection to John’s extensive PR campaign on right wing media promoting his study and saying that their findings support the conclusion that COVID-19 is basically just like the seasonal flu.

      I have to say, if there is a political connection it would help me to understand why he felt that it’s OK to extrapolate from a projected I *infection* rate among a non-random, non-representative sample to make broadly applied conclusions about a generalized *fatality* rate – in particular when there are some reports that some of the researchers involved quit the project after expressing concerns about the quality of the tests.

      If that turns out to be true, I hope that John offers a clarification.

      • Joshua says:

        > Others had received an email from Bhattacharya’s wife, falsely claiming that an “FDA approved” test would definitively reveal if they could “return to work without fear,” as BuzzFeed News has reported.

        How is that not an ethics violation? How could it possible pass an IRB review?

        > One email, without a visible timestamp or sender that was sent to Bogan’s and Neeleman’s addresses, read: “David, I think you should write Taia a note and tell her you’ll support her lab if she validates this kit.” Bendavid confirmed that he put Neeleman and Wang in touch.

        > And Neeleman did write to her. “First and absolutely most importantly, we have to establish without any doubt, the efficacy of these tests,” he wrote. “I am frustrated by what appears to be the lack of urgency.”

        Jesus.

        > Bendavid mentioned that he was particularly worried about the test’s rate of false positives. If the test generated more false positives than the scientists were expecting, the results would throw off their infection estimates and affect what they could tell people about their antibody status, he wrote.

        Well, that might put John’s work in research methodology in a slightly differ light, I’d say.

        > Bendavid seemed “resistant to the idea” that people with positive results should be contacted and retested, Boyd continued: “Is this because it would take some time to do so?” Furthermore, he noted, Wang had told him that she did not think her experiments “validated or verified the accuracy of the Premier Biotech kits at all.”

        Jesus. Is this is true, Benavid needs to be fired.

        > Ioannidis also pointed out that it was a preprint, not a published study, and therefore subject to further revisions.

        Whaaa? He went on a national TV campaign based on that preprint. This just keeps getting worse.

      • Joshua says:

        I said weeks ago that they violated basic scientifix principles.

        Little did I know (if this is true).

        Just to point out, I think that the potential conflict of interest in the fundkng is the least of it.

    • David Young says:

      From a scientific point of view this is irrelevant. There are by now perhaps 10 serologic studies around the world showing roughly the same thing. Actual infections have been at least 10 times and perhaps 80 times the reported “case” numbers. In Los Angeles, Miami Dade, and Arizona the implied IFR’s are consistent with the Santa Clara study. While this makes for good outrage porn, it means essentially nothing.

      • Willard says:

        > From a scientific point of view this is irrelevant.

        Come on, David. That’s just false. *Even* from a scientific point of view due diligence must be paid to conflicts of interest.

        How many of your comments must I fish out from your ClimateBall past to show how infelicitous are your current contributions?

        • Joshua says:

          Willard –

          Personally, I find the conflict of interest part the least troubling aspect. I mean I could quite easily believe that there was no explicit bias from the funding.

          But the other stuff? Wow!

          I mean either the IRB at Stanford is a joke (and I find that hard to believe) or they lied about their recruitment practices.

          • Joshua says:

            Just to elaborate:

            The researcher’s wife sent out an email to recruit, and either they omitted the description of that aspect of their recruitment (a stone cold violation), or they lied about their recruitment practices (an even worse stone cold violation), or the IRB approved a stone cold violation of a recruitment practice.

            My biases (having seen IRBs be quite stringent in their reviews at other institutions) tell me that the last option is the least likely. The other two should have the researchers fired, in my opinion.

            • Joseph Candelora says:

              I have real problems with the study, but this isn’t one of them.

              Bhattacharya claims he didn’t know his wife sent out that email, and I believe him. They were heavily oversubscribed from the main target of her email (white women), there was no reason to be moreso.

              And the incoming view of Bhattacharya, Ioannidis and Bendavid was that infection prevalence was much higher than everyone thought (in their op-ed, Bhattacharya and Bendavid posited a ‘true’ IFR of 0.01% — yes, one hundredth of one percent).

              If anything, that recruiting email lowered the observed prevalence in their study, because it went out to the people who were doing the most social distancing. So I would bet he’s pissed that his numbers were brought down.

              • Joshua says:

                Joseph –

                > Bhattacharya claims he didn’t know his wife sent out that email, and I believe him.

                Let’s say that’s true (I have no particular reason to think he’s lying. Or that he’d telling the truth.) How does thst happen? How does a wife of a researcher send out an email to recruit? How does she not know that’s an ethics violation? How does he not find out after the fact that she sent out emails once people start responding? In other words, he’s the first author and he has no idea what’s going on in the study he’s conducting to assess the state of a pandemic that’s killing people. And then he goes on a wide media campaign to promote the findings of a study that he didn’t even control closely – to tell the public what he advocates for the best policy to address a pandemic that’s killing tens of thousands.

                At best, it’s terribly sloppy science and incredibly irresponsible.

                > If anything, that recruiting email lowered the observed prevalence in their study, because it went out to the people who were doing the most social distancing. So I would bet he’s pissed that his numbers were brought down.

                Purely speculative about the direction of the effect of the emails. And do you really think that the emails in themselves had a significant effect on their findings? I doubt the signal would be big enough. And if it did, their recruitmwnt methodology was terrible. And if he knew about that after the fact, he had an obligation to redo the study.

                I see no way to think that this wasn’t horrible science, at best.

                I won’t pass judgment one way or the other on whether he was lying. It doesn’t really matter to me, and there’s no way to know, anyway. But whether or not he’s lying doesn’t speak to the science.

              • David Young says:

                Josh, You are posting long and repetitive comments. I think what is more likely to have happened is that someone who was initially involved didn’t like the initial hypothesis that the IFR was very small and felt so strongly about it that they leaked anything they could find to try to discredit the work before it could get published and properly evaluated. That’s called by political hacks “getting out in front of the story”. Merchants of doubt with political motivations do this all the time. You seem to believe in swallowing whole a one sided document dump.

                What’s the “ethical” violation with someone who is not an author sending out a solicitation email? I don’t know what the basis of the claim of FDA approved is. As long as this is disclosed in the final paper, I’m not sure why its a problem.

                They are continuing to improve the data on the test used. When the paper is published we will know a lot more.

              • Joseph Candelora says:

                > How does thst happen? How does a wife of a researcher send out an email to recruit? How does she not know that’s an ethics violation? How does he not find out after the fact that she sent out emails once people start responding?

                Presumably, she got excited and wanted her friends in the PTA or whatever to have access to testing. And from what I read, they didn’t track how people found the study, they just signed up responders based on zip code.

                > Purely speculative about the direction of the effect of the emails. And do you really think that the emails in themselves had a significant effect on their findings? I doubt the signal would be big enough. And if it did, their recruitmwnt methodology was terrible. And if he knew about that after the fact, he had an obligation to redo the study.

                Point being, I doubt there was active malfeasance on their part — if they’ve put their thumb on the scale with the email, they’ve put it on the wrong side.

                Yes, very sloppy work. Compounded by a terrible decision by his spouse.

        • David Young says:

          As all scientists know, soft money is almost always involved in modern research. Usually in the paper there is an acknowledgement of that. It’s not a problem or a “conflict of interest” except to outsiders who are perhaps unfamiliar with modern research or playing at merchants of doubt. It is irrelevant to whether the paper will ultimately be proven correct.

      • Joshua says:

        David –

        > From a scientific point of view this is irrelevant.

        That is a stunning statement. From a scientific viewpoint, it’s incredibly relevant. Someone who has been a symbol of scientific integrity has been a part of extraordinary violations of basic scientific principles. It’s actually hard to believe, both what they did (assuming the article is accurate) and that you’d dismiss the scientific relevance.

        • David Young says:

          It may be stunning to you Josh, but its true. Concilience of evidence gradually shows us the truth.

          • Joshua says:

            David I’ve made it abundantly clear why I think your characterization of the “consillience” is wrong.

            But that’s not what I’m stunned about. I fully expect that from you. In fact, I’d be stunned if you didn’t do that.

            I’m stunned at your dismissal of the scientific issues discussed in that report. Just as I was that Andrew dismissed them.

            The ethical issue are one thing. Even the methodological issues. But those were fundamental violations of scientific principles as far as I’m concerned – like extrapolating from non-representarive sampling. Imo, that isn’t just a methological error. And these were worse.

            • Andrew says:

              Joshua:

              I did not dismiss the scientific issues in that report. For example, I discussed the nonrepresentative sampling in items 1 and 4 of this post.

              • Joshua says:

                Andrew –

                > I did not dismiss the scientific issues in that report

                I apologize for not being clearer.

                By “report” I meant the BuzzFeed article. And you did say that you felt it didn’t present issues of scientific relevance, did you not?

              • Joshua says:

                And just to further clarify, by “representative” I was referring to extrapolation from a Santa Clara study of infection to a national level characterization of fatality.

            • David Young says:

              Josh, I think you will be in a very small minority of scientists who think this affects the science and the final outcome. If you have ever published a paper you will know that authors often forget to include information in the first draft that is later included. Scientists are human too and can overlook or forget things too. In fact, it is usually the case that more work is done after the first draft that clarifies any remaining issues. Absolutely standard practice.

              • Joshua says:

                David –

                I can’t know for sure that the BuzzFeed article is accurate, but as described, any reasonable scientist that does research with human subjects would read the section that describes the interactions with Boyd and immediately recognize not only massive ethical problems, but scientific problems as well. If the description is accurate that’s why two of the scientist withdrew from the project. Any of the others involved should have done likewise. If they didn’t, they should be fired. Boyd apparently documented the problems and they paid him no heed and proceeded anyway. There no excuse.

                The interactions with Wang fall into the same category. It’s unfortunate that rhe article tends to focus a bit more on the conflict of interest issue. Itz too, is legitimate, but it’s of less clear relevance to the science.

                Any reasonable IRB would reject their recruitment and testing procedures. If the methods they submitted to an IRB were detailed as described in the study, it beggars belief that those methods would pass a review. Any scientists experienced with human subject research would recognize the problems as Boyd described them with notifying participants about their testing. If they employed different methods than those described, that’s even worse.

                I can only assume that you have never done any research with human subjects, or you’d see the problems immediately as well.

              • Joshua says:

                And David –

                A word of friendly advice. Your spinning in this reflects very poorly on you. You are stuck in a rit of justifying poor science and unethical scientific practices because of your ideological biases. It’s an understandable tendency, but at some point unless you have evidence that what’s detailed in the BuzzFeed article is false, it’s just an embarrassment for you. At least just caveat your rationalizing by saying you don’t think the srcike is acxurare, but if it is, you agree there are serious issues (even if you don’t think they’re as bad as I do).

                Read the sections about Wang and Boyd. If the report is accurate, it couldn’t be clearer why they quit the project. Any scientists who stayed on despite the issues they objected to (and withdrew because of), should be held accountable. If they could credibly establish that they didn’t know what was going on, it would be sloppy science but not necessarily a fundamental breach of ethics and proper scientific practices.

              • Joshua says:

                And with that, spin away to your heart’s content. I’m not going to try to help you salvage your self-respect any further

              • David Young says:

                Josh, You are repeating over and over the same statements without really addressing anything I said. I have no idea if you have any expertise in any area since you are are an anonymous internet hack. It’s stunning that you think anyone should take what you say seriously. Whjy?? It’s a serious quesion.

              • Zhou Fang says:

                David Young, who are you? Stop calling people anonymous hacks when the only piece of credibility you seem to have is that you use a surname. Are you the novelist? The wrestler?

              • David young says:

                Josh, You didn’t respond to what I said about the process of doing science. You just repeated with much repetition how shocked you are. And you are back into reading people’s minds. It’s arrogant and condescending. And that despite you promising not to do that. It is an unethical game you play here as all professionals know.

                I’ve published a lot of papers over the last 40 years.

              • Andrew says:

                To all in this sub-thread: enough has been said on this topic! Give it a rest, as it overwhelms our comment threads. Please agree to disagree, or take the disagreement elsewhere. Thank you.

              • Joshua says:

                Andrew –

                Apologies. Won’t happen again.

              • David Young says:

                I may have been too harsh above with Joshua. Sorry Josh.

      • Andrew says:

        David:

        There’s a big difference between 10 and 80. I wrote a bit about the Santa Clara study: the data are consistent with exposure rates between 0 and 3%, or something like that. We’ll have to see what happens going forward. I agree that the details of funding etc. aren’t so relevant to the scientific conclusions. Looking forward, though, it’s good to have funding and motivations as clear as possible.

        • Joshua says:

          Andrew –

          > I agree that the details of funding etc. aren’t so relevant to the scientific conclusions.

          No, but if the other stuff is true… How about the way they dealt with the reliability of the testing? Do you think that isn’t relevant to their scientific conclusions? To the point where, possibly, two of the researchers withdrew from the project specifically because of how they approached validating the testing?

          And what about this?:

          > > Bendavid mentioned that he was particularly worried about the test’s rate of false positives. If the test generated more false positives than the scientists were expecting, the results would throw off their infection estimates and affect what they could tell people about their antibody status, he wrote.

          He’s worried about the false positives because if they don’t get the results they were expecting it would affect what they’d tell people about the study?

          You can’t possibly believe that isn’t relevant to the science of their findings, can you?

          • Joshua says:

            And even this:

            > > Others had received an email from Bhattacharya’s wife, falsely claiming that an “FDA approved” test would definitively reveal if they could “return to work without fear,” as BuzzFeed News has reported.

            Recruitment from an email from the researcher’s wife? Along with a false claim about the significance of participation?

            I mean sure, that false claim that is a flat out ethics violation if true, but it’s more than that. It’s a scientific violation. That’s an inherently biased recruitment method.

            I’m stunned that you think what that article describes isn’t relevant scientifically.

        • David Young says:

          Well, the bottom line here is that even assuming that we should discount the Santa Clara study an assumption I disagree with, there is plenty of data showing pretty much the same thing, viz., IFR is between 0.15% and 0.4%. In a previous comment I went through the numbers. Josh also has pointed to a Danish study in blood donors that found an implied IFR that is even lower.

          In any case, because of the high differential between age cohorts IFR’s, population study IFR’s will need to be looked at in terms of the age distribution of the population studied. At least the Santa Clara study tried to adjust for this. Miami Dade went further and tried to contact a random sample.

          It’s pretty hard to claim that all these studies were strongly biased in the same direction. That’s called conciliance of evidence. There is also past experience with epidemics. In all cases IFR estimates have declined over time, often dramatically so.

          Most scientists take soft money from lots of people and groups. While they usually disclose that, its kind of conspiratorial to assume that they dishonestly skew their results to reflect this funders views.

          • Andrew says:

            David:

            I don’t think we should discount the Santa Clara study. It is what it is. Sure, there’s selection bias in the data, but just about all studies have selection bias. As I’ve said in other contexts, I don’t think it’s so useful to talk about IFR in general as it depends so strongly on how many people over 85 have been exposed. I have no reason to think the Stanford and US researchers dishonesty skewed their results. As I wrote in a post a couple weeks ago, I think they interpreted their results in the context of their existing expectations. I think we should be aware of financial conflicts of interest without thinking that that a conflict of interest makes a study useless. The conflict of interest is just one piece of metadata that can help us contextualize a study.

            P.S. Just to clarify: I’m not saying I disagree with you regarding the general claims about how deadly the disease is. I don’t really know about that, so I’m trying to restrict my statements to what I can say from the data I’ve looked at.

            • Joshua says:

              Andrew –

              In case you’re interested. I’d be curious to hear your take:

              > > After exclusions, there were 13 estimates of IFR included in the final meta-analysis, from a wide range of countries, published between February and April 2020. The meta-analysis demonstrated a point-estimate of IFR of 0.75% (0.49-1.01%) with significant heterogeneity (p<0.001).

              https://www.medrxiv.org/content/10.1101/2020.05.03.20089854v1

              • Andrew says:

                Joshua:

                I took a quick looked at the abstract, which seems to describe the meta-analysis in clear and sober way, with the following admirably-restrained conclusion: “However, due to very high heterogeneity in the meta-analysis, it is difficult to know if this represents the “true” point estimate. It is likely that different places will experience different IFRs. More research looking at age-stratified IFR is urgently needed to inform policy-making on this front.”

              • David Young says:

                Josh, I’m surprised you would cite this without looking at the details. I would raise a couple of issues.

                1. 4 of the studies were modeling studies which I would not trust due to the ill-posed nature of models of epidemics.
                2. Perhaps of the remainder half were from February or March. I didn’t see Los Angeles or Miami Dade perhaps because they haven’t been officially published yet. Or Arizona either.
                3. IFR estimates will be very sensitive to the age structure of the population that is infected. You can correct the seriological data for any nonrandomness in the sample. The New York data is probably biased because no testing was done among those who are most vulnerable.
                4. IFR estimates always decrease over time in virtually all epidemics, in some cases dramatically so. There are a number of reasons for this including that early fatalities are often skewed to those most susceptible. The remaining population will not be nearly as vulnerable.

                So more research is needed but this study seems to me to need updating using later datasets.

                The Danish sample was blood donors and all were <70 years old. Since one is measuring infections it could perhaps benefit from an age adjustment. I don't know if they did that or not. It would be a good follow up to test some older age cohorts. It's quite possible that more of them have been exposed.

              • confused says:

                Yeah, I think the heterogeneity is pretty critical here.

                I would like to see IFR estimates for a given state/country (with a known age distribution of population) under different assumptions about who gets infected (IE – all ages equally, biased towards younger, etc.)

                I think that might be pretty important, policy-wise, given how dramatically the risk rises with age.

              • Joshua says:

                Andrew –

                Yes, restraint is good. Good point I completely agree and I shouldn’t have left that off of my excerpt.

            • Willard says:

              Andrew,

              David knows about contextualizing studies. Here’s the very first thing he said in a series of 14 comments at AT’s:

              Well, We have no idea if Ferguson was telling the truth but I’m willing to give him the benefit of the doubt. The problem here is that his science was wrong and his track record seems to be to mostly err on the alarmist side.

              https://andthentheresphysics.wordpress.com/2020/05/09/attacking-scientists-who-tell-the-truth/#comment-175731

              It just so happens that this time the metadata ain’t on his side.

              And speaking of metadata, David just called me a liar at Judy’s for stating an area of expertise one can read on his Researchgate page.

          • Joshua says:

            David –

            > Josh also has pointed to a Danish study in blood donors that found an implied IFR that is even lower.

            Did you even read the paper? Did you read about the ages of the people included in the study?

            If not, you should do so.

            Also, I pointed you to meta-analysis. Why don’t you mention the results of the meta-analysis?

      • somebody says:

        > While this makes for good outrage porn, it means essentially nothing.

        To use an extreme analogy, if I randomly throw 3 bowling balls from the empire state building and I happen hit nobody and damage no property except the speeding getaway vehicle of a bank robber, I’m a perfectly fine and reasonable citizen and there should be no consequences for what I did.

        As far as I can tell, your general position is that epistemological errors up to and including lying and bribery don’t matter as long as your main takeaway happens to be right.

        • David Young says:

          You are lying about my position of course. I said the issue of funding is scientifically unimportant. Andrew says we don’t know if the Santa Clara study is right or not.

          You are aware that falsely accusing 18 scientists of a serious crime is libel. But I forgot you are an anonymous internet persona with no qualifications. So you wear the typical hood used by those who want to avoid consequences of their actions. That’s rather cowardly.

          • someone says:

            I’m not accusing them of libel, I’m responding to your position. I’m not saying they did it or even that I think they did it, I am responding to your assertion that it’s “scientifically unimportant” as in, unimportant to the process of science, and also your much stronger assertion that it’s “essentially meaningless.” You’re saying that even if they did do it, it wouldn’t matter. I can see now that you mean “it wouldn’t change what we think about IFRs”, which is defensible, but that isn’t what you said at all. You should express yourself better!

          • Joshua says:

            Andrew –

            You should check that stuff out. Looks to me like some good stuff about dodgy statistics.

            Also, I thought this was good (if very basic):

            https://twitter.com/CT_Bergstrom/status/1261148217401503744?s=20

          • Andrew says:

            Joshua:

            Twitter threads are so exhausting to read! But I agree with Bergstrom that just because a study is peer reviewed, that doesn’t mean it’s any good.

            • Joshua says:

              Andrew –

              I agree about Twitter threads and I don’t usually follow, but you may want to make an exception here because it’s a great discussion/example about the use of statistics. It’s right up your ally.

          • Carlos Ungil says:

            The good news is that extrapolating from currently numbers 95% of the population has been already infected!

            Anyway, while the methodology is bizarre (I didn’t even try to understand it, to be fair), this 30% prevalence mid-April doesn’t seem so ridiculuous compared with that 60% in March, which produced by actual epidemiologists: https://www.ft.com/content/5ff6469a-6dd8-11ea-89df-41bea055720b

            • Joshua says:

              Carlos-

              Paywall on that. But really? 60% in March?

              • Carlos Ungil says:

                https://www.medrxiv.org/content/10.1101/2020.03.24.20042291v1

                “Importantly, the results we present here suggest the ongoing epidemics in the UK and Italy started at least a month before the first reported death and have already led to the accumulation of significant levels of herd immunity in both countries. […]
                Three different scenarios under which the model closely reproduces the reported death counts in the UK up to 19/03/2020 ​are presented in ​Figure 1​. Red and green colours represent solutions attached respectively to transmission scenarios with R​0​=2.75 and R​0​=2.25 (reflecting variation in estimates of R​0 in literature) with the proportion of the population at risk being distributed around 1%. The model output (posterior) for time of introduction (the start of transmission) place this event a couple of days after the first confirmed case in the country, and over a month before the first confirmed death ​(Figures 1E-F)​. In both R​0 scenarios, by the time the first death was reported (05/03/2020), thousands of individuals (~0.08%) would have already been infected with the virus (as also suggested by ​[5]​). By 19/03/2020, approximately 36% (R​0​=2.25) and 40% (R​0​=2.75) of the population would have already been exposed to SARS-CoV-2. Running the same model with R​0​=2.25 and the proportion of the population at risk of severe disease being distributed around 0.1%, places the start of transmission at 4 days prior to first case detection and 38 days before the first confirmed death and suggests that 68% would have been infected by 19/03/2020.”

              • Joshua says:

                Carlos –

                Thanks. Almost enough to make me join the chorus of old men yelling at clouds.

    • Shiva Kaul says:

      It’s OK for the JetBlue CEO to donate money for research. But why was he actively participating in their discussions about antibody test performance? How could Ioannidis be “not personally aware” of his financial contribution?

      • David Young says:

        It was a large team and each member may have little time to invest in learning everything there is to know about the project beyond doing their part of the work.

        • Shiva Kaul says:

          OK, let’s say the senior author Ioannidis was not involved in obtaining funding. Nor was he involved in the crucial discussion of antibody test performance, which led to the departure of colleagues. Nor was he involved in the erroneous statistical analysis, which remains erroneous in the second draft.

          With that level of involvement, was it appropriate for him to appear on headline news, lending the credibility of himself and his institution?

        • David Young says:

          Well Shiva, All we have to go on is a leak by a whistleblower and a media report. We don’t have the other side of the story. An objective position would be “lets wait until the evidence emerges.” What I think is happening here is what so often happens in our harsh and corrupt media environment. Leaks are reported as fact and they later turn out to be wrong. The leaker can “select” what to leak to deceive too.

          Why are you so concerned with this? It will resolve over time as we get more facts.

          • Shiva Kaul says:

            The report has named sources – Taia Wang and Scott Boyd – who had direct involvement with the research, apparently possess concrete evidence supporting their complaint, and have initiated an ethics review at Stanford Medicine. I haven’t been hoodwinked by a sketchy media claim.

            The Santa Clara study, boasting many esteemed authors and a custom statistical analysis, could have been a template for lots of future seroprevalence surveys. Unfortunately, it seems to be a template for something very different.

          • David Young says:

            We will see in time what happened when everyone weighs in. Right now its a little early to declare then guilty until proven innocent. Likewise its too early to say if the scientific findings and methods are right but there is growing evidence that they are in the ballpark that I think I mentioned above (or somewhere here).

            • Andrew says:

              David:

              Of course the Santa Clara study is in the ballpark! When analyzed carefully, the data show an uncertainty interval of approx (0%, 2%) for the prevalence: that’s consistent with just about everyone’s story. The statistical analysis methods in that study are wrong—it’s not “too early to say” that, this is just math, they made mistakes. But the conclusions are broad enough that they can’t distinguish among various hypotheses. That’s fine. Future studies can take more care in data collection and use better statistical analysis. We learn from our mistakes in science: it’s good they did the study, and it’s good that others criticized it so that future researchers can do better.

              • David Young says:

                I largely agree Andrew. Hopefully they will update their statistics in the final version. BTW, Ioannidis said on a recent video he really appreciated the constructive criticism. Perhaps you were among those he had in mind.

              • Joshua says:

                Andrew –

                I held off after your admonition. But seems that the convo is continuing and as such, I think you might want to know who it is you’re conversing with, and what his quality of reasoning is. David Young is dpy6629, and dpy6629 is David Young:

                dpy6629 | April 19, 2020 at 11:29 pm |

                Josh, This Gelman is a nothingburger. He admits he’s not an expert on serological testing and that he doesn’t know if the Ioannidis paper is right or not. I think I’m done with your low value references.

                =====

                And when someone explained to David that you’re a respected statistician, and said the following about your post on the Santa Clara study:

                > Gelman has written what I shall call a “precautionary warning” that the result of the Stanford study should not be regarded as definitive. In consideration of how hard it is to get any studies up and running, I think it should be regarded as first step and a teaching example of what has to be improved upon in the next study of the same topic.

                =======

                David had the following to say…

                dpy6629 | April 20, 2020 at 5:15 pm |

                Gelman looks like someone who likes to hold forth on subjects he is ignorant of such as serologic testing. He then tries to shame other scientists who know much much more than he does. Typical blog thrill seeker whose conclusions can’t be trusted.

                https://judithcurry.com/2020/04/14/in-favor-of-epistemic-trespassing/#comment-914924

                https://judithcurry.com/2020/04/14/in-favor-of-epistemic-trespassing/#comment-914978

              • Andrew says:

                Joshua:

                Jeez, that’s pretty rude of him. The whole thing is just so frustrating. Why can’t people just accept that a particular study is not definitive?

                And if someone thinks that posting blog entries on Bayesian inference and hypothesis testing is “thrill seeking” . . . that’s just sad.

              • Joshua says:

                In my opinion, the problem is not so much with the Santa Clara study in itself.

                The problem is with Ioannidis et al. going on a national TV publicity campaign to say that the Santa Clara study justified an extrapolation at the national (or perhaps global level) level, and proved that COVID-19 is about as much to worry about as the seasonal flu. And then advocating for public plicied under the assertion thst COVID 19 is “nothing to be scared of.”

                That, is inexscusable – and about the same ethical level, if the BuzzFeed story is true, as testing study participants for antibodies after they were told that a positive test would be liks a back to work passport, and then balking when a colleague said they should be retesting those participants who were positive while informing them that even if they tested positive they might still be infectious because of fslse positives.

              • Joshua says:

                Just to be clear. I have no issue with an epidemiologist going on national TV to advocate for COVID policies. That is their right.

                I have an objection to them doing that with a scientific justification that the Santa Clara study could be extrapolated to determine that COVID 19 is about as much to worry about as the seasonal flu.

                In particular given that we have vaccines for the seasonal flu.

              • David Young says:

                Andrew, Joshua is quote mining a long comment thread with lots of other comments. What I meant to say is that your post is in my opinion a nothingburger. I don’t know about the statistics part but it seems to me the critical issue is the serologic testing. In that comment thread it is also stated that you are a world class statistician, a comment I agree with. I apologize for the thrill seeker comment.

                Joshua has been heckling me on blogs for several years. He shows up with very repetitious and often unscientific comments and always brings up his motivated reasoning ad hominem. It just gets very frustrating to have every interaction taken down the same road into internet diagnoses of my “reasoning” and state of mind. In addition its unethical.

              • Joshua says:

                And in case anyone missed it, the same researchers went on to conduct another study with a larger sample, of MLB employees.

                They noted that the test they utilized for the more recent test was extremely accurate.

                They found antobodies in 0.7% of the participants.

                It should noted that when they found a lower infection rate then they anticipated, one of the authors reverse engineered to conclude that their sample must not have been representative of the surrounding community.

                > “I was expecting a little bit of a higher number,” Bhattacharya said. “The set of people in the MLB employee population that we tested in some sense have been less affected by the Covid epidemic than their surrounding communities.”

                I have to wonder if that “little bit higher” might have been on the order of 2 or 3 or 4 times higher?

                Looks to me like a classic case of adjusting how you talk about your research findings to fit with your priors. And it’s interesting how he focused on the representativeness of the sampling when talking to the press about the MLB study that showed a lower infection rate than the Santa Clara study.

                Prolly just a coincidence.

              • David Young says:

                You have no evidence that they adjust how they talk about their findings to suite their priors. It would be an adult and ethical thing to stop reading their minds.

              • confused says:

                Yeah, I think there is far too much extrapolation of studies and experiences to larger populations that are very different.

                And I think this works both ways to a degree (IE… both under-stating and over-stating impacts).

                Clearly the Santa Clara study (even if it wasn’t affected by selection bias or false positives) can’t be extrapolated to the US as a whole. That would greatly under-state the impacts.

                But there was a lot of talk, at least in the non-specialist media, in March that states that didn’t lock down quickly were going to see Lombardy/Madrid/New York City style results, and that hospitals would be overwhelmed in lots of places. That clearly didn’t happen – only NYC, and maybe Detroit and New Orleans, even got close to overwhelming their systems. So it seems like the impacts, at least in the short-term*, were over-stated by expecting that Lombardy/Madrid/NYC were representative of the US as a whole.

                *IE – Texas and Florida might end up with a comparable number of deaths if this goes on for 2 years at the current rate, or if there is a really bad second wave. But the rapid explosion of cases and overwhelming of hospitals now seems exceptionally unlikely.

              • Zhou Fang says:

                It is really amazing how David can go “I don’t know about the statistics part” on a statistical modeling blog and still think people want to know that he has an opinion nevertheless.

              • somebody says:

                @confused

                I’m not sure what places you’re talking about that didn’t do a lockdown and also didn’t get overwhelmed. It seems like everywhere had at least a slight formal lockdown and there’s a lot of evidence the mobility began to decrease before government action.

              • Joshua says:

                confused –

                > Yeah, I think there is far too much extrapolation of studies and experiences to larger populations that are very different.

                So for me it isn’t so much the size of the different populations, but two other aspects. The first is that they extrapolated for Santa Clara from a non-random sampling.

                But the 2nd is a much bigger problem: they went from an *infection* rate for a Santa Clara population (which may indeed be accurate in the end) to go on national TV to extrapolate a national/global *fatality rate*. How can that be done without a representative sampling at the national or global level? It can’t.

                They justified their extrapolation that crossed category boundaries (infection rates to fatality rates) based on a selective discussion of the uncertainties involved. They haven’t engaged with the science about fatality rates and yet pontificated on it anyway. And they’re continuing to do it. The Hoover Institute just put out an interview with Bhattacharya where he continued a propagandistic, rather than scientific, treatment of the uncertainties. I could elaborate, but here, see for yourself: youtube.com/watch?v=289NWm85eas&t=2521s

                And note that all of a sudden Bhattacherya is VERY focused on the representativeness of their sampling with the MLB employees returned a 0.7% infection rate.

              • Willard says:

                > Joshua is quote mining

                Interested readers may wish to read David’s comments at AT’s:

                https://andthentheresphysics.wordpress.com/2020/05/09/attacking-scientists-who-tell-the-truth

                He made 13 comments on that thread until he got caught telling a second porky.

                Readers may also be interested in the comment thread on a post dedicated to him:

                http://julesandjames.blogspot.com/2020/04/euromomo_10.html

                David made more than 25 comments over there.

                Readers might also wish to read David’s comments at Judy’s:

                https://judithcurry.com/2020/05/06/covid-discussion-thread-vi/

                David made more than 30 comments under his other sock.

                David’s modus operandi should be fairly obvious.

                Caveat emptor.

              • confused says:

                @somebody

                “I’m not sure what places you’re talking about that didn’t do a lockdown and also didn’t get overwhelmed. It seems like everywhere had at least a slight formal lockdown”

                I was thinking of Sweden, South Dakota, Arkansas, etc.

                Well, I guess it depends on what you mean by ‘lockdown’. In general people seem to be referring to stay-at-home orders of one type or another, usually combined with closing all ‘nonessential’ businesses. The places I had in mind didn’t do either of those, but did do some social distancing measures (like banning mass gatherings and perhaps closing schools, hair salons, etc.).

                “and there’s a lot of evidence the mobility began to decrease before government action.”

                This is true. I think that this, and the fact that social contacts were already much lower in South Dakota or Arkansas than in a mass-transit-heavy dense city like NYC, is why those places aren’t as badly impacted as would have been expected beforehand.

                @Joshua: “So for me it isn’t so much the size of the different populations, but two other aspects.”

                Yeah, I phrased that wrong. By “larger” population I meant extrapolating from Santa Clara County (or NYC) to the entire US. Yeah, if those smaller populations were representative of the whole US, there’d be no problem, but they aren’t.

                I think the claims about the Santa Clara study started with an unstated assumption that the IFR will be pretty much the same everywhere. But there doesn’t seem to be much reason to think that would be true.

              • Carlos Ungil says:

                > banning mass gatherings

                Defined as gatherings of over 10 or 50 people.

              • Joshua says:

                confused –

                > I think the claims about the Santa Clara study started with an unstated assumption that the IFR will be pretty much the same everywhere. But there doesn’t seem to be much reason to think that would be true….

                Even if it were true. Or even if the infection rate nationally is DOUBLE what they found in Santa Clara, say 6%. At 92k deaths already, the IFR would be on the order of 0.46%. They said, based on their Santa Clara study, the national IFR is like 0.1%.

                WTF are these people doing?

              • confused says:

                I do think they were wrong — what’s happened in the Northeast is just not compatible with a super-low IFR.

                But I don’t think it was totally irrational, from the information that was available in early April, if they started with the assumption that IFR will be the same everywhere. (But that assumption is still wrong.)

                It’s not really fair to compare current deaths to a prevalence estimated that long ago; it’s been well over a month, so the prevalence could have increased by much more than 2x.

                It’s now clear that places other than just NYC have seen death rates incompatible with a 0.1%-0.2% IFR. And we now know the seroprevalence in NYC is closer to 20% than 80%. But I don’t think either of those things were known when the first pre-print was published.

              • Joshua says:

                confused –

                > But I don’t think it was totally irrational, from the information that was available in early April, if they started with the assumption that IFR will be the same everywhere. (But that assumption is still wrong.)

                I don’t agree. I think it was entirely unreasonable. As soon as I saw what they were doing, I said so. I said, immediately, you can’t extrapolate nationally from a non-nationally representative sample. Can’t. Do. It! Epidemiologists must know this.

                I knew it and I’m not an epidemiologist and plus I’m a dummy and they’re obviously very smart people.

                I bet they even got really good scores on their GREs!

                > It’s not really fair to compare current deaths to a prevalence estimated that long ago;

                Well, yes and no. Along with the growing deaths, there is a growing % of infections. That’s why I doubled it from their estimates.

                > it’s been well over a month, so the prevalence could have increased by much more than 2x.

                That is something they should have anticipated when they went all over national TV to say that COVID is nothing more scary than the seasonal flue. They had to know at the time that the deaths would grow as a % of the public. They were obviously banking on a growth of infections decoupled from growth in the number of deaths. And that, was obviously wrong! And many people said so at that time. Lot’s of people were saying that the fatality rate was much higher than the flu and they came out and said, no, the rate was about the same as the flu and they said that the Santa Clara study supported their conclusions. That was obviously wrong at the time. Santa Clara was a hot spot. Santa Clara isn’t representative by race/ethnicity and SES, among other important metrics (like access to healthcare, and most likely health behaviors, # of comorbidites, etc.) Plus, there was plenty o’ evidence at the time of a striking age stratification in deaths largely do to death rates in LTCFs. Did they check to see if Santa Clara is representative in that all important statistic? We could go back to the first cases in Washington to know that representativeness of LCTFs would affect overall fatality rate.

                They didn’t do their due diligence, obviously. They were pushing a political agenda. It’s obvious.

                > It’s now clear that places other than just NYC have seen death rates incompatible with a 0.1%-0.2% IFR. And we now know the seroprevalence in NYC is closer to 20% than 80%. But I don’t think either of those things were known when the first pre-print was published.

                Lot’s o’ people knew there were problems with their estimated fatality rate when they went on their publicity campaign pushing the “just like the seasonal flu” propaganda. And in fact, plenty of people knew that the fatality rate was higher, and they said so, well before they pushed their agenda-driven messaging. The timing of the paper doesn’t hold up as an excuse – sorry.

                It’s ok to have an agenda. It’s not ok to take a one-sided approach to uncertainty when you’re doing so.

            • confused says:

              Yeah, I agree it was misrepresented.

              But I really do think that at least the upper end of their range (0.2% or so) was not totally unreasonable until the NYC serology results came out, which was after the pre-print for the Santa Clara study. I mean, the IFR could be 0.2% and there could still be 400,000 deaths in the US, if enough of the population was infected. It was only when we saw that NYC was closer to 20% than 70% that I was really comfortable with saying it *couldn’t* be that low US-overall (sure, NYC itself would be higher even at 70% seroprevalence, but I think there’s some reason to think NYC would have a somewhat higher IFR than the US average).

              And Santa Clara certainly is not nationally representative… but I wouldn’t focus too much on race/ethnicity and SES. Those are likely proxies for mobility/contact patterns, prevalence of underlying conditions, and maybe access to healthcare rather than having much of a “real” effect separate from those other factors, since they aren’t biologically meaningful in themselves.

              I think the % of people in each age group infected is primarily driving differences in observed/calculated IFRs. Spain actually seems to have a higher seroprevalence among the older population; other places have found the opposite.

              • confused says:

                Probably I’m being too kind, actually.

                It mostly seemed plausible to me in early April because it seemed to explain why Florida and Texas weren’t nearly as bad off as expected. In mid-March people were suggesting that Florida and Texas would end up much like New York in 2-3 weeks after Spring Break. When I saw that study, I figured maybe we did have as many infections as expected, they just didn’t turn into hospitalizations and deaths.

                But yeah, now it’s clearly wrong.

                (Likely what really happened with Florida and Texas is that outdoor interactions are less risky + Spring Breakers are young enough to have low risk.)

              • Joshua says:

                Confused:

                Ioannidis meta-analysis if IFR. Shocker, he finds it Lowe than other analyses.

                https://www.medrxiv.org/content/10.1101/2020.05.13.20101253v1

                This is unreal:

                > For the other studies, healthy volunteer bias may lead to underestimating seroprevalence and this is likely to have been the case in at least one case (the Santa Clara study)19 where wealthy healthy people were rapidly interested to be recruited when the recruiting Facebook ad was released. The design of the study anticipated correction with adjustment of the sampling weights by zip code, gender, and ethnicity, but it is likely that healthy volunteer bias may still have led to some underestimation of seroprevalence. Conversely, attracting individuals who might have been concerned of having been infected (e.g. because they had symptoms) may lead to overestimation of seroprevalence in surveys.

                So he ignores many of the ways that the Santa Clara study might have been an overestimation because of the recruitment processes.

                But then he doubles down to ignore the many reason why that Santa Clara would be an overestimate – do to higher median income, lower minority population, etc. with respect to a broader exptrapolation beyond Santa Clara.

              • Joshua says:

                Another example of John’s thumb on the scale:

                >Locations with high burdens of nursing home deaths may have high IFR estimates, but the IFR would still be very low among non-elderly, non-debilitated people.

                He ignores the uncertainty in the other direction; i.e., does Santa Clara have *fewer* long term care facility residents than what would be nationally representative? He consistently looks at the uncertainties only in one direction.

                As someone who has long respected Ioannidis, I am having a hard time understanding how poorly he’s approaching the uncertainties in all of this.

              • confused says:

                Yeah, I just saw that.

                It seems to say that the Denmark and Netherlands blood donor studies are actually calculating an “IFR for the population below age 70” so the really low numbers might actually be right for those… just not comparable to the others.

                I would expect to see a lot of heterogeneity in IFRs* – and if for example Idaho hadn’t infected any LTCFs or other very vulnerable populations at that time, ~0.2% might not be totally unreasonable in those conditions – but some of those numbers seem really low. Kobe and Brazil maybe have false positive issues? Those prevalences are fairly low…

                But what’s the deal with Oise, France? 25.9% seroprevalence seems way too high for false positives to matter much…

                *apparently the recent Spain serology study actually showed somewhat higher seroprevalence in people in their 60s and early 70s than in the overall population. That could make a big difference to IFRs.

              • Joshua says:

                confused –

                > Denmark and Netherlands blood donor studies are actually calculating an “IFR for the population below age 70” so the really low numbers might actually be right for those… just not comparable to the others.

                I knew that about the Denmark study. Hadn’t seen anything about the Netherleands study.

                I can’t believe that not once in the paper does he really discuss the representativeness of the sampling.

                You took issue with the degree to which I characterized the study populations as “hotspots” – but clearly many of them were, and he basically only talks about representativeness from the perspective that they might be underestimating seroprevalence.

                It will be interesting to see how this all plays out. Betting against Ioannidis prolly isn’t a good idea, but I don’t see how he isn’t treating uncertainty selectively to confirm his bias.

              • confused says:

                It may be significant that Ioannidis is in California.

                The burden of this pandemic in the US has been overwhelmingly in the Northeast and Midwest. There are more than twice as many deaths in NYC as in all Western states combined.

                NYC has seen over 2 deaths per thousand population, New York State as a whole and New Jersey are over 1 per thousand. California is less than 1 per 10,000… and it’s one of the relatively harder hit Western states.

                If you’re west of the 100th parallel, it’s easy to believe it is like the flu.

              • David Young says:

                I’ll just point out that this new meta-analysis agrees well with my estimates of IFR stated here and elsewhere. The number is 0.01%-0.4%. Naturally, it will be highly variable depending on the age structure of those exposed.

                I’ve taken a lot of abuse on this issue both here from the usual suspects and at James Annan’s blog from the usual anonymous bullies. Some of them should apologize for all their insults and sneering.

                “Infection fatality rates ranged from 0.03% to 0.50% and corrected values ranged from 0.01% to 0.40%. Conclusions The infection fatality rate of COVID-19 can vary substantially across different locations and this may reflect differences in population age structure and case-mix of infected and deceased patients as well as multiple other factors. Estimates of infection fatality rates inferred from seroprevalence studies tend to be much lower than original speculations made in the early days of the pandemic.”

        • Shiva Kaul says:

          I haven’t seen a single seroprevalence study which has adjusted for both demographics and test performance in a convincingly correct manner. (That includes the 2nd Santa Clara draft.) How can the meta-analysis be trusted?

          The very prominent mistakes in the Santa Clara paper are likely being repeated, despite being simple and correctable. The unusual scrutiny of the paper was meant to prevent this debacle, not advance political agendas.

  20. Steve says:

    Let me just say that I personally believe that higher institutions are discriminating against Asian Americans. I wanted the lawsuit against Harvard to succeed as a father of an Asian American boy. As a lawyer, I also disagree with the position that Andrew took in the article that you linked to. I think that in the law we should be following a rule where discrimination can be shown with a statistical data and the burden shifted to the defendants, otherwise, it is impossible to prove in a court. And, I even agree with you that the argument that Asian Americans are being used as pains is offensive in some sense because it implies that there are not plenty of intelligent Asian Americans that supported the lawsuit. But, calling Andrew racist for quoting someone else who wasn’t being racist but seemly demeaning is sort of over the line. There is a real benefit to trying to imagine that your opponent in a political or intellectual debate is as moral and decent as you imagine yourself to be. It will make you smarter in the end if you try it.

    • Curious says:

      Sometimes ignoring the immoral premises upon which an argument rests just makes one blind to reality. In this case it is the reality of what is not being measured by these tests, what level of precision is absurdly assumed, and the faulty reasoning with which it is argued.

  21. Ryan J. Cassidy says:

    Are you saying Legendre and Gauss were wrong?

    I feel so betrayed… ;).

  22. Sam says:

    The math GRE is hard enough to probably disqualify candidates that could have succeeded in an econ program, but at the same time much too easy to demonstrate that you’re actually good at anything (assuming that we’re talking about the ordinary math gre, as opposed to the math subject test, which is actually hard).

    I’ve never heard an economist cite gre scores as evidence that their colleagues are worth paying attention to, but if they do, that’s concerning.

  23. Roy says:

    Just want to remind everyone to get out some champagne tomorrow because the number of deaths will be zero!

    Would like to think that some people’s reputations will take a real hit after all this shakes out, but nah I don’t think so. Not the people who made honest efforts but whose results or their interpretation of the results might be off (that probably has happened to all of us more often than we care to remember), but the people who let politics and pre-desired outcomes influence what they did.

    IRL COVID-19 deaths in the US are likely to shoot past 100,000 by the end of May. It will be “interesting” (in a macabre sense) to see what happens in about mid- to end of June with all the re-openings happening now.

    • Andrew says:

      Roy:

      It might take awhile for the reputations of Richard Epstein, Cass Sunstein, and those Council of Economic Advisors dudes to fully recover from this one. On the other hand, people can go far with damaged reputations. I don’t think Kevin Hassett’s reputation ever recovered from “Dow 36,000” but it didn’t stop him from getting an influential government position. Reputation is important, but it’s only one part of the story. Connections and ideology are important too.

    • confused says:

      >>IRL COVID-19 deaths in the US are likely to shoot past 100,000 by the end of May. It will be “interesting” (in a macabre sense) to see what happens in about mid- to end of June with all the re-openings happening now.

      I think it’s almost guaranteed to exceed 100,000 (unfortunately), given that we’re already over 85,000 and that there is a significant lag from infection to death (today’s deaths are probably mostly from infections at the end of April), and there is also a delay in reporting, which introduces even more lag. So even if there were no more new infections, we’d likely *still* hit 100,000 deaths.

      But beyond that, I don’t think any real prediction is possible, because too much of the data is bad. Reporting lags are different between different states, and also are likely to vary within a state between hospitals vs. nursing homes vs. other places. Early testing and reporting (both cases and deaths) focused on hospitals, so many cases were missed, and some early deaths in e.g. nursing homes are only now being reported; now prisons and such are being tested.

      So we really don’t know what the shape of the curve is up to now (the “cases” curve is practically useless because testing capacity was so low early on, but even the “deaths” curve is probably very wrong unless deaths are reported by date-of-death rather than date-of-reporting).

      Then we add to that uncertainty…

      – the uncertainties around transmission (how important is surface vs. aerosol? Outdoor vs. indoor? Casual contact vs long-term exposure in “superspreader” events? If COVID-19 was here earlier than we thought, does that mean the R0 was lower than we thought?)

      – the uncertainties around how people will actually behave when orders are relaxed further

      – the uncertainties around seasonality of the virus. (I think this one is huge. Most respiratory viruses are seasonal, so why shouldn’t this one be?)

      I am cautiously optimistic; I’d expect to see evidence of a spike in Georgia and possibly Texas by now if re-opening was going to be a huge disaster. (New cases are up a bit in Texas, but testing is *way* up, and hospitalizations are not rising – so I think the apparent rise of cases is a result of greatly increased testing; Texas’s % positive tests is pretty low.)

      • David Young says:

        I largely agree with this. I do believe that early failures with regard to nursing homes have dramatically inflated the death count. I think 40% of US fatalities have been among residents of nursing homes.

        Using Ferguson’s IFR numbers means that between 1-2 million would die if everyone was exposed. I personally think this is too high by a factor of 3-4, but fatalities would still be a large number. I still believe that most of these fatalities would be among the most vulnerable, i.e., those already seriously ill, but we don’t have good statistics on this.

        So we will have to learn to live with this threat for a long time. It’s nearly impossible in the US to do the kind of strict contact tracing they did in Korea but I think most people will take common sense measures to protect themselves. I just doubt we will be able to prevent a scenario with a rather constant rate of new infections for months. Another intriguing possibility is some newer models that are showing that in a population with a highly skewed susceptibility, herd immunity can be reached at much lower infection rates, perhaps as low as 20%. Nic Lewis has a post on this at Climate Etc.

        • confused says:

          There will probably not be one IFR for the whole US, except as an overall average (total deaths/total infections). I wouldn’t be surprised to see a factor of 2 difference or more between states.

          The median age difference between Utah – the youngest state, at 31 – and Maine – the oldest, at 44.9 – is nearly 14 years!
          And how people live (e.g. multigenerational households or not, density, proportion of people living in apartment complexes vs. houses, etc.) varies dramatically across the US too. That is likely to affect how infections are distributed among age groups.

          >>I just doubt we will be able to prevent a scenario with a rather constant rate of new infections for months.

          By our own actions, in most states*, I certainly agree. I think there is some hope for seasonality, though.

          And the rate does seem to be declining now, accounting for the increase in testing. Lags in reporting (infection-to-testing, testing-to-reporting-results, death-to-report-of-death) may make the curve look “flatter” (lower peak / slower decline) as well. So who knows.

          *Due to their isolation, Hawaii – and possibly Alaska – may actually be able to eradicate the virus on a New Zealand style model. But it could still be reintroduced from elsewhere.

          >> Another intriguing possibility is some newer models that are showing that in a population with a highly skewed susceptibility, herd immunity can be reached at much lower infection rates, perhaps as low as 20%.

          It’s possible, but the question is if those kinds of contacts / susceptibilities are common in the real world.

          Supposedly some towns in northern Italy hit 60%+ seroprevalence, so it’s not universal. But contact patterns in, say, the Dakotas or Arkansas will not resemble those in northern Italy (population density of Lombardy is comparable to New Jersey). So – who knows.

          • Brent Hutto says:

            If someone wanted to do an analysis I would find convincing, they would need to model the long-term steady state case where it is completely endemic and the total number of deaths reflects almost entirely the product of person-specific IFR and demographics.

            Because absent a highly effectively, widely-available vaccine that’s the long-term outlook, isn’t it?

            Rates of testing, rates of new infections, time to symptoms, time to death, asymptomatic cases, all of that determines how long it takes to reach an endemic state and how much damage is done during that build-up. In the long term, you need a stratified or multilevel model that reflects fatality probabilities for various population subgroups and the size of those subgroups. I think that means Mr. P type models.

            • Long term a vaccine at least as effective as a typical flu vaccine is practically guaranteed. Also long term mutation of the virus to a less virulent form is also practically guaranteed. Of course, it depends on what you mean by long term. I suggest 3 years out there’s no question a vaccine is widespread in the 1st world. I’d guess that mid next year a vaccine will be available which is rationed mainly to the at-risk population. It really isn’t that hard to get a vaccine that works against this virus. There are multiple candidates already that have been shown to work in monkeys etc. The big issue is *showing* it’s safe and effective not making it safe and effective.

              • confused says:

                I would agree; historically vaccines have taken ages to develop, but the current efforts are not following historical timelines. Some are already in human testing.

                I think we will see that modern biotech/medical technology can do more things faster than we thought, when there is sufficient motivation to take a bit more risk.

                However… pandemics historically don’t last that long. The 12-18 month timeline to get a vaccine is comparable to how long flu pandemics have lasted. Now, this isn’t a flu virus, and we’re doing more social distancing measures than we did even in 1918-19, so flattening the curve may drag it out longer… But still…

                In 2009-10 flu pandemic, we had a vaccine in less than a year (it was available in sufficient quantities for the general public by November), but the second wave was already dying down on its own by then.

                I think a vaccine will help a lot for the “endemic” aftermath (the 2009-10 flu pandemic virus became a regular seasonal flu virus, which still circulates), but I doubt it will change the pandemic itself that much.

                (And I would imagine the vaccine will be better than flu vaccine. Influenza virus comes in many strains that mutate rapidly, cycle through different animal species, etc. This virus is mutating enough that genetics can track whether outbreaks in the US came from Asia or Europe, but it doesn’t seem to be mutating at influenza levels, from the limited and preliminary data we have.)

            • confused says:

              That is quite possibly true, but I don’t think it’s certain, and I also don’t think it is the answer to the same question most people are asking.

              We probably will have a vaccine very soon (by historical vaccine timelines), which will greatly mitigate the “endemic” virus. So the more interesting question is what will happen over the next year or thereabouts, before a vaccine can become available.

              (Also, I don’t think it’s guaranteed that the deadliness/behavior of the virus as an endemic virus will correspond to how it behaves as a pandemic virus — past flu pandemics haven’t shown that. 1918-19 died out on its own with no vaccine. 2009-10, while far milder, showed a vastly different curve of risk by age than seasonal flu, but seasonal flu since then hasn’t changed that much. Of course this isn’t a flu virus…)

          • Carlos Ungil says:

            > some towns in northern Italy hit 60%+ seroprevalence, so it’s not universal

            In fact both statements could be true at the same time:

            A) seroprevalence reached 60%

            B) the reproduction number was below 1 when seroprevalence reached 20%

            I don’t say it happened in those towns, just that it’s not logically impossible: most people could have been already infected by the time 20% of the population had developed antibodies.

            • confused says:

              A good point. If there is a big super-spreader event that infects most of a small town, or if the spread is just very fast, everybody might be infected in a very short time frame, and herd immunity might be totally irrelevant.

              This could be what happened in some prisons that have shown very high infection rates.

              I didn’t intend to say that 60% or whatever is the “real” herd immunity number… just that some places have shown very high prevalences, so we can’t simply say when we hit 20% we’re safe.

              I do think that movement/social contact patterns and such are making a huge difference. It’s hard to see what else could account for how (comparatively) lightly affected much of the interior-west US has been, including places that took much less strict measures than most of the US (e.g. South Dakota), while the urban Northeast and Midwest were hit very hard, comparable to much of Europe.

              So… I would not at all be surprised if some places really do reach herd immunity at fairly low seroprevalence, like 20% or whatever. But not everywhere.

          • Interesting perspective Confused

  24. Xiang says:

    The validity of this general statement is something worth debating, for sure, but in this particular case, it doesn’t seem like that Chairman Phillipson “doesn’t know what he’s talking about”, as the author seems to be suggesting. But rather, he’s trying to defend his political position, for some political aims (remember that he’s a politician now, instead of an academic publishing a paper).

    In many situations, it’s not that the person doesn’t know what they’re talking about or “bluffing”, but that they are deliberately presenting a position that fits their current role and benefits them in some way. I totally agree that the former does happen, but those two things are really distinct, and if the author conflate those two and throw a blanket claim that “stats is hard”, it’s not really helpful.

    • Andrew says:

      Xiang:

      You could be right. “Statistics is hard” is, I think, a pretty good explanation of what went wrong with that polynomial-regression air-pollution-in-China study that was conducted and publicized by some well-situated economists. I think it’s also a possible explanation of what happened with Philipson here. Another possibility is what you’re suggesting, that Philipson understands the statistical issues here and is knowingly taking a weak position for political reasons. It also could be something in between, where he’s never really thought through these issues and, for political reasons, is not trying too hard to get things right here.

  25. waffletower says:

    As much as I would like to find fault in the cronies of the Trump administration I think the detractors of this specific CEA tweeted graph are lost. Nowhere in the tweet is the cubic fit data used to produce an estimate of Covid-19 deaths. Sure you can tunnel in on the motivations behind including the red cubic curve, and entirely miss the point of the tweet, but IMO its inclusion simply elucidates that curve fitting was employed on each of the IMHE model datasets. The only death estimate (134,475) provided comes from the latest IHME covid death projection, the green curve. This can be inferred by visiting https://covid19.healthdata.org/united-states-of-america which provides additional visualizations of the IHME model. The August 4th death projection reported by the IHME model today is still higher at 147,040. I found the graph helpful, as it shows that deaths predicted by the model have been steadily increasing. The tweet is not an attempt to minimize projections of Covid-19 deaths.

    • Andrew says:

      Waffletower:

      From the above-linked Washington Post article:

      White House officials have been relying on other models to make decisions on reopening, including the IHME model and a “cubic model” prepared by Trump adviser and economist Kevin Hassett and the Council of Economic Advisers. . . . People with knowledge of that model say it shows deaths dropping precipitously in May — and essentially going to zero by May 15. . . .”

      The article also points to forecasts of 3000 deaths per day, which didn’t happen either. Anyway, it seems that this cubic curve was used as a forecast, at least by some people. Beyond this is the point made by several above commenters, which is to ask what was the point of the cubic fit in the first place? But we could ask that about polynomial fits more generally. Again, statistics is hard, and even people with “stellar GRE scores” can get confused. The problem is not when people mess up, but when they stubbornly refuse to see what they did wrong, even to the extent of attacking people who point out their errors.

      • waffletower says:

        If such a cubic model was indeed used as a forecasting tool, I agree that constitutes malpractice and quackery. But I don’t think this tweet shows a model generated from a cubic function, I believe it to be more plausible that the red graph is simply a smoothed representation of available Covid-19 death data in the US as of 2020-05-04 as the tweet in isolation directly indicates. I believe it was provided to allow better comparison in the context of the 3 other IHME model plots which are also curve-fitted. But, perhaps instead the tweet was intended to downplay reports of actual usage of a cubic model in the White House as claimed in the Washington Post story. The tweet in the context of the CEA twitter feed doesn’t provide evidence of this that I can find, however. Such is speculation. I am happy to see many eyes looking critically at the data that the White House is using. Unfortunately, I believe we have grave interpretation issues among the White House leadership — data model quality notwithstanding.

    • Joseph Candelora says:

      “As much as I would like to find fault in the cronies of the Trump administration I think the detractors of this specific CEA tweeted graph are lost. Nowhere in the tweet is the cubic fit data used to produce an estimate of Covid-19 deaths.”

      What do you think the red dotted line in the tweet is? Note how it extends to August.

      • waffletower says:

        Again, the only quantitative estimate provided in the tweet (134,475 deaths by August 4, 2020) comes from the latest IHME covid death projection — the green curve. The red curve is not offered as a quantitative alternative. The tweet clearly give precedence to the IHME model and if anything, is an attempt to explain or downplay the news concerning a cubic model as reported in the Washington Post.

        • Zhou Fang says:

          No one is claiming the tweet (which by the way was most likely written by someone other than the person who made the graph) says the cubic fit is an alternative. People don’t think the cubic fit would be on the graph *at all*, and in contrast to the original explanation (that the cubic fit is an alternative) which is merely inappropriate, the new explanation (the fit helps to data smooth and visualise how well the IHME model fits) makes no sense whatsoever.

  26. Joshua says:

    With respect to comparing the effects of a “lockdown” in one country to voluntary social distancing in another, (or, I might add, extrapolating a national fatality rate from an infection rate in a non-random sample from one locality that isn’t nationally representative on such basic metrics such as SES and race/ethnicity) ::

    > On the other hand during the past month I have witnessed, on Facebook, Twitter, Whatsup, mailing lists, etcetera a rather widespread attitude in a large number of colleagues, who have started to entertain themselves with publically available data on contagions, deaths, hospitalized patients, and related geographical information. It began as a trickle of graphs posted on Facebook or Twitter, showing an exponential function overlaid to a few data points, and then it became very quickly a flood of plots of all kinds, where the data were tortured to confess they wanted to plateau somewhere; this was invariably done using this or that single-dimensional parametrization, picked up without justification, “because the chisquare looks good”.

    I saw curves describing data from one country overlaid with other curves describing data from other countries, shifted according to ad-hoc criteria;…

    https://www.science20.com/tommaso_dorigo/the_virus_that_turns_physicists_into_crackpots-246490

Leave a Reply