Skip to content
 

Tony nominations mean nothing

Someone writes:

I searched up *Tony nominations mean nothing* and I found nothing. So I had to write this.

There are currently 41 theaters that the Tony awards accept when nominating their choices. If we are being as generous as possible, we could say that every one of those theaters will be hosting a performance that fits all of the requirements for an award. The Tony awards have 26 different categories. There are 129 nominations this year, not including the special categories. For a play in this day and age to not get a single nomination is just a testament to its mediocrity. Plays or People can even get multiple nominations in the same category. The Best Featured Actress in a Musical has these marvelous nominations:
Lilli Cooper, “Tootsie”
Amber Gray, “Hadestown”
Sarah Stiles, “Tootsie”
Ali Stroker, “Oklahoma!”
Mary Testa, “Oklahoma!”
People will frequently get nominated twice in the same category for different pieces!

According to the official Tony Awards website, “A show is only eligible in the season when it first opens, no matter how long it runs on Broadway.” This immediately gets rid of many current shows, and leaves only 21 shows by my counting. I may be slightly wrong, but that is still a very small number. If there are 129 possible nominations for your piece, and you are only 1 out of 29 possibilities, receiving a tony nomination is not a badge of honor, but a badge of shame. There was recently an article in the New York Times about how King Lear, a show that received mixed reviews, was disappointed that it only got 1 nomination. I’d like to see if anyone else can help me figure this out.

My reply: OK, so here’s the question. Why so many Tonys for so few shows, which would seem to reduce its value?

The most natural answer is that Tonys and Tony nominations give value to “Broadway theatre” more generally: the different shows are in friendly competition, and more awards and more nominations get the butts in the seats.

But that doesn’t really answer the question, as at some point there have to be diminishing returns. The real question is where’s the equilibrium.

Remember that post from a few years ago about the economist who argued that the members of the Motion Picture Academy were irrational because they were giving Oscars to insufficiently popular movies: “One would hope the Academy would at least pay a bit more attention to the people paying the bills. Not only does it seem wrong (at least to this economist) to argue that movies many people like are simply not that good, focusing on the box office would seem to make good financial sense for the Oscars as well”?

The discussion there led to familiar territory in econ-talk: How much should we think that an institution (e.g., the Oscars, the Tonys) is at a sensible equilibrium, kept there by a mixture of rational calculation and the discipline of the market, and how much should we focus on the institution’s imperfections (slowness to change, principal-agent problems, etc.) and suggest improvements?

One comparison point is academic awards. Different academic fields seem to have different rates of giving awards. It would just about always seem to make sense to add an award: for example, if the Columbia stat dept added a best research paper award for its Ph.D. students, I think this would at the margin help the recipients get jobs, more than it would hurt the prospects of the students who didn’t get the award. On balance it would benefit our program. But we don’t have such an award—or, at least, I don’t think we have. Maybe we should. The point is that it doesn’t seem that statistics academia has reached equilibrium when it comes to awards. Political science, that’s another story: they have zillions of awards, all over the place. Equilibrium may well have been reached in that case.

Dan Simpson or Brian Pike might have more thoughts on the specific case of the Tonys. Maybe someone could “at” them?

P.S. When I was a kid, nobody cared about the Tonys, Emmys, or Grammys. But every year we watched the Oscars, Miss America, and the Wizard of Oz.

26 Comments

  1. Anonymous says:

    Nobody cares about the Tonys, Emmys, or Grammys now either.

  2. Dan Simpson says:

    Oh I have a lot of feelings. I do t really care about the premise (literally if it brings someone joy let them have the opinion), but for the highlighted category, I’ve not seen Tootsie but I’ve see the be others and Mary Testa and Ali Stroker both borough so much to their roles and they very much deserve the nom (I think Rebecca Naoimi Jones got snubbed for best actress and I have FEELINGS about that even though Stephanie J Block should win and really should’ve won for Falsettos). I think the featured actress prize is a serious battle between Amber Gray and Ali Stroker who and both incandescent in their parts.

    Anyway. I’ll be watching this in a queer bookshop with drag queens on Sunday because that is correct.

    Go see that Oklahoma!

    • Andrew says:

      Hey Dan. Good to see I’m not the only one wasting time on the internet after 11pm!

    • Dan Simpson says:

      Also, like, there are lots of categories. 4 acting, writing, directing , at least one music, sets, choreography, costumes, lighting, sound, etc etc etc. There are 4 show categories and 8 acting categories, 2 director, 2 writer, 1 music, 1 lyrics that really make a commercial difference. There aren’t many shows but not every show is nominated for the big things and some years there are as few as two nominations in a category.

      And we will not forget the Drama Desks giving their best musical revival to the Yiddish version of Fiddler (which was well deserved but ineligible for Tonys). These are not coronations

  3. Thanatos Savehn says:

    Re: your “P.S.”: same at my house. And I’ll always remember the year my folks finally got a color TV. When Dorothy woke up over the rainbow … an epiphany for my sister and me.

    As for the rest, maybe modeling the subjective experiences of others, like my memories of Oz, is a hopeless endeavor.

  4. Anonymous says:

    Quote from the blog: “One comparison point is academic awards. Different academic fields seem to have different rates of giving awards. It would just about always seem to make sense to add an award: for example, if the Columbia stat dept added a best research paper award for its Ph.D. students, I think this would at the margin help the recipients get jobs, more than it would hurt the prospects of the students who didn’t get the award. On balance it would benefit our program. But we don’t have such an award — or, at least, I don’t think we have. Maybe we should.”

    Please don’t add more academic awards.

    I reason academic awards might be unscientific, and (in at least some cases) invalid. I wrote down some of my thoughts, and reasoning concerning handing academic awards to people in a little pre-print (https://psyarxiv.com/pju9c/). I think it could perhaps be fitting for this blogpost, and/or possible entertaining in a way, and/or provide a viewpoint from someone who is not a part of academia (at least in the traditional sense).

    Some quotes from the pre-print:

    1) “If psychological societies and organizations want to keep handing out awards to individuals, why stop at recognizing “uniquely creative and influential scholarly productivity at or near the peak of one’s scientific career “or a lifetime of significant itellectual contributions to the basic science of psychology”. Why not go all the way, and really make it resemble the Oscars or the MTV Movie Awards! To name a few possibilities: “Best Bayesian re-analysis” (cf. “Best adapted screenplay”), “Best co-author” (cf. “Best actor/actress in a supporting role”), “Best gift-authorship” (cf. “Best cameo appearance”), “Best attempt to get the word count of a to be submitted paper just below the journal’s limit” (cf. “Best film editing”), “Best ‘comment on/‘reply to’-exchange” (cf. “Best fight”), and “Best additional value of publishing in an ‘official’ journal instead of only posting a pre-print” (cf. “Best makeup and hairstyling” or “Best visual effects”).”

    and,

    2) “Psychological scientists build on each other’s work, and should be reminded of that every time they read or cite something that has been written by someone else. This view of building on previous work by others is perhaps best captured by the metaphor of dwarfs standing on the shoulders of giants, and Isaac Newton who wrote: “If I have seen further it is by standing on ye sholders of giants” (Newton, 1675/1676).

    Concerning Psychological Science and individual awards: perhaps standing on the shoulders of a giant does not necessarily imply the dwarf has been looking in the right direction, perhaps the giant later turns out to be a dwarf or the dwarf later turns out to be a
    giant, and perhaps there are many giants, and many dwarfs, who are standing on each other’s shoulders. If any of these things are possible, it shouldn’t be about which giant, or which dwarf, is on top at a certain point in time. It should be about what can be, and has been, seen.”

    • Mikhail Shubin says:

      +1

      > “If I have seen further it is by standing on ye sholders of giants” (Newton, 1675/1676)

      The metaphor I like to use, is that modern scientist stands on a mountain made of dwarfs.

      • Anonymous says:

        Quote from above: “The metaphor I like to use, is that modern scientist stands on a mountain made of dwarfs.”

        That is one way of looking at it. Perhaps that fits nicely with the idea that there may be many giants, and many dwarfs, that are standing on each other’s shoulders.

        What i also find interesting to think about is how a lot of science seems to me to be dependent on things like language, mathematics, logic, etc.

        If this makes (any) sense, aren’t all scientists standing on the shoulders of the people who “invented” or “came up with” things like language, mathematics, logic, etc. And if so, were these people “giants” or “dwarfs”, and were they even “scientists”? Who even were “these people”, and/or can they even be distinctively separated from other people to even warrent describing them as a “giant” or a “dwarf”?

        • Mikhail Shubin says:

          Well, this relates to the whole debate of free will
          (Usually it is framed as “can somebody be blamed for committing a crime given we all are products of our environments”, but it could be as easily framed as “should somebody be praised for achieving something given…”)

          And even if you dont believe in free will and responsibility, you can still vote for punishing criminals (because it creates a deterrence, i.e. create an environment in which less crime is possible) and giving gold medals to scientists (because it creates an incentives)

      • TGGP says:

        I believe it was Greg Cochran who said “If I have seen further than others, it is only because I’m knee-deep in the dwarves.”

    • Great excuse to quote a quote on CS Peirce – from Peirce: A Guide for the Perplexed by Cornelis De Waal

      “The interaction with fellow inquirers is crucial for filtering out the various idiosyncrasies that individual inquirers bring to the table—it allows us, as he puts it, “to grind off the arbitrary and the individualistic character of thought … error plays a central role in science, as it is precisely when we are wrong that we are given the strongest evidence that there is a world out there that is independent of what we may think it to be, and with which we are somehow in direct contact. Hence, there is an important sense in which error is to be encouraged rather than avoided. This too speaks to the idea that science is a communal enterprise, and one that should involve people with different backgrounds, inclinations, and talents, so that the greatest variety of angles is explored. Part of what is at play here are Peirce’s conceptions of self and mind, which are decidedly anti-Cartesian.

      Peirce defines the private self not in terms of anything exquisite or divine, but in terms of error and ignorance. What makes our private selves unique is that we differ from others in that we are wrong about different things and that we are ignorant about different things. Hence, for Peirce, scientific inquiry—which seeks to alleviate error and ignorance—is in essence a process of self-effacement.”

      Now making awards an integral part of academic career progress certainly makes self-effacement both risky and costly. Something to be avoided at all cost?

      • Andrew says:

        Keith:

        You write, “making awards an integral part of academic career progress certainly makes self-effacement both risky and costly. Something to be avoided at all cost?”

        Good point, and it’s related to why I wasn’t thrilled with the proposal to have a Nobel-like prize for statistics. But it’s the usual prisoner’s dilemma or tragedy of the commons. If nobody gave awards, that might be great. But if you add a prize to your field or your subfield or your institution, then your field/subfield/institution can benefit.

        I complicating factor for me in writing this is that my collaborators and I have received buckets of awards, and I always enjoy it. So I’m not quite sure what to think.

      • Anonymous says:

        Quote from above: “This too speaks to the idea that science is a communal enterprise, and one that should involve people with different backgrounds, inclinations, and talents, so that the greatest variety of angles is explored’

        I am not so sure i agree with (how i interpret) the Peirce quotes (or the quotes of an intepretation of Peirce). I don’t necessarily see how exploring the greatest varieties of angles leads to the best science. This (kind of) reasoning can easily lead to unscientific things like thinking something is, or is not, “good” science because some particular sub-set of “different” people all worked on it. Or using appeals to majority (“argumentum ad populum”), or something similar. I fear this is already beginning to become “a thing”.

        I also do not agree with (how i interpret) sentences like “Hence, for Peirce, scientific inquiry—which seeks to alleviate error and ignorance—is in essence a process of self-effacement.”. I reason science, and scientific principles, should be most important. More important than for instance, the individual’s possible “exquisitness or diviness” but also more important than the individual’s possible “error and ignorance”. Perhaps i don’t agree with (the gist of) this sentence because the focus is still too much on a person, not scientific things like logic, reasoning, facts, etc..

        I looked up the word “self-effacing”, and this is a definition i found: “having or showing a tendency to make oneself modestly or shyly inconspicuous”. I think i don’t necessarily agree that scientists should be “modestly or shyly inconsicuous”. For instance, if scientists see someone claim that 2 + 2 = 5, they should not “self-efface” in my reasoning. They should (try and) make clear that 2 + 2 does not equal 5 for this and this reason.

        I agree that science can be viewed as a cummunal enterprise, but to me this does not mean that everything in science is equal, or that some “average” should be taken of all views, or that some “consensus” should be reached. If A says 2 + 2 = 5, and B says 2 + 2 = 7, science and scientsts should not take some sort of average and say “well we’ve had great discussions, and our committee has decided to agree that 2 + 2 = 6”. There can be a lone scientist that is not part of the committee that should say that 2 + 2 = 4 for this and this reason.

        I also reason some scientist are better than others, because they are better at scientific things like reasoning, logic, hypothesis forming, etc. I reason that these possibly “better” scientists should not “self-efface”. They should speak up when they (think they) see something that is not scientific, or not in line with scientific values, principles, etc. I just reason that these (possbily) “better” scientists should not get awards.

        I am not sure if the following analogy makes sense (entirely), but i think it could perhaps make something clear. Perhaps you can compare my point of view concerning giving awards to indivual scientists to running in the 4 x 100 meters in the Olympics. You shouldn’t (and don’t) give the gold medal to JUST the final runner, but if this final runner has run the fastest 100 meters, you still should want this final runner to be in your team.

        • Chris S says:

          While I can see how the Peirce quote supports multiple interpretations, I think the spirit of it was more about exploring a variety of angles in conditions of moderate-to-high uncertainty (as opposed to something like the arithmetic examples you gave, or climate change denialism).

          In social science in particular, I think bringing multiple angles/experiences to the table is especially useful when designing measurements, since the operational definitions of many constructs may vary in nuanced ways according to social context.

          Also I think the quote was using “self-effacement” in a more philosophical sense, as opposed to the dictionary definition. As in, the result of engaging with other scientists and being wrong on occasion is a reduction in the number of idiosyncratic beliefs that distinguish one self from another. I don’t think the implication was that scientists shouldn’t criticize one another’s claims.

          And +1 on some scientists are better than others, and that they should criticize wrong-headed claims. Although I do think learning through error is an unavoidable part of the scientific reasoning process.

  5. There is the producer’s perspective…

    As a producer I used to work with before they started producing plays put it – “these investments are just mainly made for tax write offs. But if a play ever becomes very popular and runs for a long time it like winning the lottery – costs are mainly written off and its mostly profit.”

    e.g. Come From Away (as of Jun 2, 2019) Total Current Gross: $136,846,695

    > “A show is only eligible in the season when it first opens, no matter how long it runs on Broadway.”
    So after the first season, no one needs an award because they are amply being rewarded. So much better to “salt the mine”.

  6. Padang Itik says:

    Maybe the Tony’s can give a sense of “comparative advantage”, in the sense that every play gets some nominations but all for different things. This one has good actors, this one has good makeup, this one has a good score, and so on. When I saw “War Horse” a few years ago everyone was excited about the puppet technology used for the horse as it grew up, but no one was necessarily raving about the story or the acting. Similarly with The Lion King, people like the way they portray the animals with puppets, costumes, and makeup, but no one really said it had the best director or was the best in other ways. I personally don’t care much about sets and costumes, but I love plays that have good music. So seeing the “worthless” Tony nominations can tell me where to find the best music, while other people can look up the costume nominations to decide what they should see. Of course this is an expensive way to signal comparative advantage but I think it makes Tony’s a little better than worthless.

    Re: academic awards, when I was in grad school my department sponsored an award that was theoretically open to any student in the university, but somehow every PhD student in my department won it every year plus a few token winners from outside the department. It was a small prize, like $500, and it went to fund research that you were already drawing on your advisor’s budget for, which was coming from the same source as the prize money, so it seemed pretty worthless. Of course all of us put on our resumes “won XXX Fellowship 5 years in a row” as padding and dutifully went to the awards ceremony where old professors toasted each other. At least having events like that now and then broke the monotony of research and gave us practice writing proposals, so even that award was not totally worthless.

  7. Jonathan (another one) says:

    My wife spent two decades as a Tony voter and occasionally sat in on meetings of the administration committee (which makes those crucial decisions like who is eligible for best actress and who is supporting or is a play new or a revival) and I can tell you that there are certain pressures to (a) only nominate shows and performances that are still performing and (b) only give awards to shows that will do well on tour. Winning Tonys has a definite but not major effect on the revenues for a show on Broadway itself (though losing or not being nominated very often causes a show to close). Gerry Schoenfeld, the late chairman of the Schubert Organization was quite outspoken to the effect that every show running ought to get time on the Tony show whether they were nominated or not and that “everyone ought to get at least one.”

    One other thing. The phenomenon of near-eternal runs for shows is pretty new. The paucity of possible shows contending is a result of this phenomenon. It is almost unheard of for a category to be dropped because there are too few contenders, although we certainly have the phenomenon where the number of contenders is sharply reduced (two nominees this year for best musical revival, once a hotly contested category.)

  8. Michael Nelson says:

    Statistics needs at least one big award. There are a couple of worthwhile things that awards can (and often do) achieve: 1. Supporting people whose work is important (or has great potential to be) but who remain obscure and underfunded, and 2. Motivating people in the field to do a certain kind of work that hasn’t received enough attention. In the case of the Oscars, we see #1 when a nomination for an indy film gives a big boost to its creators, cast and crew, so that they can make more such films. We see #2 when the big commercial studios invest big bucks in “award bait” films despite anticipating very low box office numbers, and big stars join those projects (with salary cut) for much the same reason.

    In the case of academia, the MacArthur Fellowship (aka “genius grant”) arguably accomplishes #1 while the Millennium Prize does #2. A MacArthur specifically for stats might not be workable since, as far as I can tell, there’s really not enough intellectual diversity in our institutions and published research to be able to pick out a “genius” every year. Statistics is driven by (i.e., receives funding for and gets published for) work on highly practical problems within relatively narrow parameters (no pun intended). Researchers challenging orthodoxy (like those who kicked off the replication crisis, or Andrew’s friend who tried to build a network of open journals but ended up having to sell them) seem to be more the exception than the rule. Then again, an award might attract more statisticians to do that kind of work, or even uncover ones already doing it.

    A Millennium Prize for stats seems much more likely to work: lots of us would love to work on, for example, developing a model-free research design that doesn’t involve randomization, or figuring out how to conduct a series of small lab studies that are just as generalizable as one big field study, or finding an approach that lets us look at our data pre-analysis and do something useful with that knowledge without turning it into post hoc trash. An incentive, in terms of funding or prestige, would help motivate not just researchers but also academic departments and journal editors. Technically, we do have “awards” like this, in the form of things like fellowships and grants, but these will never garner great respect and general admiration: they’re given out to too many people, for criteria that are too general, and almost always have strings attached (you receive it as a salary for attending a program, you promise to conduct a specific study, etc). Fellowships make it possible to do the work that is in demand, while prizes make it possible to do the kind of work that is extraordinary. The work that is, well, worthy of a prize. :)

    • Anonymous says:

      1) Quote from above: “There are a couple of worthwhile things that awards can (and often do) achieve: 1. Supporting people whose work is important (or has great potential to be) but who remain obscure and underfunded, and 2. Motivating people in the field to do a certain kind of work that hasn’t received enough attention”

      I sincerely doubt awards do any of these 2 things. I would guess they do just about the exact opposite. Could be interesting to investigate this. My guess is that the results will be in line with those findings about peer-review and truly “innovative” and “paradigm breaking” work: it is exactly that kind of work that does NOT get rewarded via a publication (and possibly an award).

      2) Quote from above: “Researchers challenging orthodoxy (like those who kicked off the replication crisis, or Andrew’s friend who tried to build a network of open journals but ended up having to sell them) seem to be more the exception than the rule.”

      I am not sure who the people are that “kicked off the replication crisis”. My best guess would be people like Bem and Stapel and the people who wrote the “Levelt” report on the Stapel case did a lot, if not most, of the work in “kicking off things”.

      I have been wondering about the people who AFTER those things happened (all of a sudden?) talked about how “replication” is super important, etc. I have recently started to wonder if that perhaps simply was the best way forward for the field: it could have moved the opinion from “everything is BS” to “let’s replicate to see if everything is BS”.

      A great move perhaps, but i am less convinced about those people possibly being the “real” people who are challenging orthodoxy. This opinion for me is backed up by many of the proposals that have been, and are being, made to “improve” matters. They to me are possibly not truly changing matters, and could merely be a different version of the same problematic stuff.

      3) Quote from above: “Then again, an award might attract more statisticians to do that kind of work, or even uncover ones already doing it.”

      I am finding this award stuff more puzzling by the minute. Why the f#ck should i care about a freaking statue or plaque (or whatever) that some people decided i should have for doing something what they think is “good”.

      To me it’s similar to going through peer-review: i just can’t take that process seriously anymore after finding out all that might be wrong with the journal-editor-peer review model. To quote Bruce Lee: “I am not in this world to live up to your expectations and you are not in this world to live up to mine”.

      If i were to ever be handed an award, i intend to (and hope that i) refer to some stuff written in my pre-print (or hopefully better sources) above and say something like “Thank you, but i (want to) politely decline. I think handing out awards might be unscientific, and might perhaps be an insult to science, and scientists, in general. Thank you again.”

      • Michael Nelson says:

        1) I agree these are empirical questions, and would also like to see them investigated. Perhaps we could motivate the research by offering a prize for it? :) Seriously, though, there is strong correlational/observational evidence that both of these things happen with the Oscars. Producers put money into non-money-makers, some admit they’re doing it for the awards/prestige, they pour more money into promoting the film with Academy voters, they show up for the awards, then they take out full-page ads congratulating the films (and themselves) after the ceremony. As for whether academic awards have the same effect, I have far less evidence, but it certainly is the stated goal of the MacArthur fellowship to dig up and support diamonds in the rough, and the Millennium prize (and its predecessors like Hilbert’s problems, and its cousins like the XPRIZE) to attract attention to important problems.

        2) I agree the issue of who kicked off the replication crisis is fuzzy (frankly, I think Bem at least deserves to be a co-winner for taking psychology’s practices seriously enough to reveal their ridiculousness, if only inadvertently). Which only underlines the difficulty of finding/doing “breakthrough” or “watershed” work (as opposed to incremental extensions that collectively move us forward) in a field so wedded to application. Seems like most of our progress in the last couple of decades has been a result of the massive increase in computing power and algorithmic cleverness, and much less so a deeper understanding of theory. There are exceptions–I think type M and S errors are both novel and crucial (and frequently ignored), as was the classification of different types of randomness (less often ignored)–but most research focuses on creating new and better tools, not on reconceptualizing what statistics is. Fields Medal winners, as a rule, do this within their respective branches of mathematics. Nobel Prize-winning physicists and chemists don’t really redefine their fields but they do discover new things about the universe (although new ways to measure things also get recognized). I have digressed mightily, but my point is, I agree–it’s a difficult question, what kind of statistical work *deserves* $1,000,000?

        3) Truly, a noble (not Nobel) soul would not be very motivated by what others think. There are some great quotes from Grigori Perelman’s on his Wikipedia page about this: “[The Millennium Prize] was completely irrelevant for me. Everybody understood that if the proof is correct, then no other recognition is needed.” and “I’m not interested in money or fame. I don’t want to be on display like an animal in a zoo. I’m not a hero of mathematics.” Similarly, Andrew Wiles was not motivated by cash prizes to solve Fermat’s last theorem. But statisticians, in addition to appreciating the beauty of statistics, are by and large applied researchers, and have few (paid) opportunities to work on problems that they are unlikely to solve. The potential money and prestige might give one just enough of an extra incentive to make a significant personal investment in such work. Consider that Wiles literally stopped all other math work and was only able to keep his career going by publishing his old stuff piecemeal. Surely he would not have done that if he had not fallen in love with the problem as a child, and surely no child would be aware of such a problem if not for the popular fame brought to it by having no fewer than four major cash prizes offered for its solution over the previous 150+ years.

        • Anonymous says:

          Thank you for the comment!

          I would just like to make a final note. You wrote: “I agree these are empirical questions, and would also like to see them investigated”.

          I reason, and wrote, it could be interesting to investigate matters concerning awards and the things you proposed they could result in. I do not reason, and wrote, that the results of that investigation should necessarily influence a decision concerning whether to hand out awards or not.

          As i pointed out in a recent discussion on this blog, i reason science and scientific results are not the only things to keep in mind with regard to proposing, and/or doing things. I think there are reasons to simply not hand out awards, regardless of whether they show a “positive” effect in some way or form.

          To try and illustrate my point:

          I would reason that it is sub-optimal (at least in some way) when researchers leave out statistical analyses in their paper compared to what they wrote in their pre-registration for instance. Now, i could reason that when researchers do this, they should stand at the entrance of a supermarket, holding a sign in their hands that reads something like:

          “I am a scientist, and i wasn’t able to read and/or follow what i wrote down beforehand. I hope you grocery shoppers will do a better job with your shopping list for buying your groceries than i was able to do with my pre-registration for my scientific study.”

          I would reason that mandating this in the case of “sloppy” pre-registration related work could positively influence accuracy concerning the pre-registration and subsequent presented information in the paper. However, even if my “meta-scientific” research concerning the possible positive effects of this proposal would show an increase in accuracy, it may not be scientifically desirable, or a good idea from a scientific perspective…

  9. Anonymous says:

    Slight correction:

    I wrote: “I have recently started to wonder if that perhaps simply was the best way forward for the field: it could have moved the opinion from “everything is BS” to “let’s replicate to see if everything is BS”. “

    I think i should have written: “I have recently started to wonder if that perhaps simply was the best way forward for the field: it could have moved the opinion from “everything is BS” to “let’s replicate things”.

    That’s (part of) the possibly usefuleness of framing things in terms of “replication” and “replicable”, instead of talking about whether something is “BS”, or “true”, or “valid”, or something like that.

    (A possible extra bonus of keep talking about, and emphasizing, “replicability”, might be that you can then use that to in turn do other stuff. For instance, stuff like not even actually replicating things but coming up with some algorithm that determines for you how “replicable” something is so you don’t even have to perform replications anymore!).

    • Anonymous says:

      Quote from above: “(A possible extra bonus of keep talking about, and emphasizing, “replicability”, might be that you can then use that to in turn do other stuff. For instance, stuff like not even actually replicating things but coming up with some algorithm that determines for you how “replicable” something is so you don’t even have to perform replications anymore!).”

      Another possible result of talking about “replicability” instead of whether something is “true”, “valid”, etc. is that it may lead to possibly problematic actions, and interprerations.

      For instance, a focus on large data sets with many variables that can be analyzed by spliting them up into “exploratory” and “confirmatory” sets can be considered to be all in line with all this “replicability” stuff but i wonder if it’s all that it’s cracked up to be from a different (arguably more important) perspective. I am not sure if the following makes sense, but i reason that the larger the data set, the higher the chance that what is p-hacked, or data-mined, in the “exploratory” set will be found in the “confirmatory” set as well.

      Now, if this is (largely) correct, i reason that a focus on “replicability” instead of whether something is “true” or “valid” (or whatever the best term is) might be problematic. If you take into account that many possible nonsensical, and/or hard to interpret, correlations may be present in large data sets with many variables, i reason you could end up fooling yourself (and others) with this focus on “replicability” instead of whether something is “true”, or “valid”, etc.

      Possibly see Standing, Sproule, Khouzam (1991) “Empirical statistics: IV. Illustrating Meehl’s sixth law of soft psychology: Everything correlates with everything”, and https://tylervigen.com/spurious-correlations in this regard as well.

Leave a Reply