What does it take to repeat them?

Olimpiu Urcan writes:

Making mistakes is human, but it takes a superhuman dose of ego and ignorance to repeat them after you’ve been publicly admonished about them.

Not superhuman at all, unfortunately. We see it all the time. All. The. Time.

I’m reminded of the very first time I contacted newspaper columnist David Brooks to point out one of his published errors. I honestly thought he’d issue a correction. But no, he just dodged it. Dude couldn’t handle the idea that he might have ever been wrong.

Similarly with those people who publish all those goofy research claims. Very rarely do they seem to be able to admit they made a mistake. I’m not talking about fraud or scientific misconduct here, just admitting an honest mistake of the sort that can happen to any of us. Nope. On the rare occasion when a scientist does admit a mistake, it’s cause for celebration.

So, no. Unfortunately I disagree with Urcan that repeating mistakes is anything superhuman. Repeating mistakes is standard operating practice, and it goes right along with never wanting to accept that an error was made in the first place.

This bit from Urcan I do agree with, though:

For plagiarists, scammers and utter incompetents to thrive, they seek enablers with the same desperation and urgency leeches seek hemoglobin banks.

Well put. And these enablers are all over the place. Some people even seem to make a career of it. I can see why they do it. If you help a scammer, he might help you in return. And you get to feel like a nice person, too. As long as you don’t think too hard about the people wasting their time reading the scammer’s products.

44 thoughts on “What does it take to repeat them?

  1. Not to mention how difficult it is in some disciplines (see: Criminology) to retract research which is likely fraudulent. This paper is making its rounds and is really quite shocking: https://retractionwatch.com/2019/07/17/criminologist-posts-27-page-article-explaining-why-he-asked-for-one-of-his-papers-to-be-retracted/

    Current top journal in Criminology has a policy to only retract papers for “legal issues”. Oh, and one of the authors of the paper is an editor for the same journal.

  2. Quote from the blogpost: “This bit from Urcan I do agree with, though:

    For plagiarists, scammers and utter incompetents to thrive, they seek enablers with the same desperation and urgency leeches seek hemoglobin banks.

    Well put. And these enablers are all over the place. Some people even seem to make a career of it.”

    This is in line with my own conclusions concerning academia, and/or the poor state of parts of social science.

    I am also reminded in this regard of the “Levelt report” about the Diederik Stapel fraud case, and most noticeably page 48 in which the following can be read:

    “Another clear sign is that when interviewed, several co-authors who did perform the analyses themselves, and were not all from Stapel’s ‘school’, defended the serious and less serious violations of proper scientific method with the words: that is what I have learned in practice; everyone in my research environment does the same, and so does everyone we talk to at international conferences.”

    I have wondered whether large part of people working in parts of social science are BOTH the enablers AND the plagiarists/scammers/utter incompetents. Conferences, people starting professional e-mails with “dear colleagues” or even “dear friends”, hierarchical things like the journal/editor/reviewer model of scientific publishing (and the related “hierarchy of journals” that comes along with it), the professor/PhD student -dynamic and relationship, the large part of often very young students that are “involved” with something as important as science, etc.

    All this “cliquish” and “hierarchical” stuff that seems to me to be part of many things in social science/academia annoys the sh@t out of me. More importantly, i reason it’s an intricate part of the possible problematic processes that have led, and can lead, to enabling things that shouldn’t be enabled. As long as the foundational stuff is not changed, i reason and predict this all will all happen again. Perhaps not even necessarily in a different form.

    • Quote from above: “All this “cliquish” and “hierarchical” stuff that seems to me to be part of many things in social science/academia annoys the sh@t out of me. More importantly, i reason it’s an intricate part of the possible problematic processes that have led, and can lead, to enabling things that shouldn’t be enabled. As long as the foundational stuff is not changed, i reason and predict this all will all happen again. Perhaps not even necessarily in a different form.”

      I believe this “cliquish” and “hierarchical” stuff is now being called “collaboration” or “crowdsourcing”.

      If i understood things correclty, some people seem to think something is a “collaboration” when you give most of the power, and benefits, to a small group of people. I believe this usually is accompanied by repeatedly saying stuff like “we have to change the incentives” or “we have to incentivize this”.

      Whether or not this is all truly a collaboration, or whether or not this is all truly crowdsourcing, or whether or not this is all even “good” (for) science seems to me to be of little concern given the amount of discussion i came across about this all. But perhaps that’s because the people writing the blogposts, or tweets, that get all the attention are part of the small group that receives the most power, and benefits the most…

      I am starting to wonder how much i could make if i set up some sort of Center For Excellence, touting my own “collaborative” efforts involving hundreds of labs across the world. If i play my cards right, i could out-compete everyone for resources. I could be a “director” of the Center For Excellence and give myself a nice salary. Let’s face it, i earned it. After all, i am truly improving matters!

  3. In my experience, ‘cliquish’ types are a nuisance to deal with b/c some subsets are clearly users. I was reprimanded a few months ago for giving them any credit or attention. It’s catching their errors at an opportune time & in a creative way which to me is a sign of exceptional talent.

    It’s that subsets learn to avoid confrontation when it is most demanded. Or move the goalposts. It really requires practical humility in the sense that we should expect obstacles to the best of thinking and see it translated to different objectives in not so good ways & contexts. Overgeneralization is a key factor in all this.

  4. Quote from the blogpost: “For plagiarists, scammers and utter incompetents to thrive, they seek enablers with the same desperation and urgency leeches seek hemoglobin banks.”

    I think the following might also be relevant here: “How academia resembles a drug gang” by Alexandre Afonso

    https://www.researchgate.net/publication/272247082_How_Academia_Resembles_a_Drug_Gang

    If academia resembles a drug gang, perhaps SOME professors can be seen as pimps. SOME professors may let (often young) students work for a meagre salary if a salary at all, while they abuse their power and position, and probably being the only person who truly benefits most from it all.

  5. I remember a classroom debate around the best definition of “scientist,” in which the class concluded that you don’t qualify as a scientist if you go through all the stages of the scientific method but then stop before publishing, because you have denied the scientific community the opportunity either to profit from the knowledge you have produced or to critique it. Scientists, by definition, must move science forward. I’d now extend that disqualification to anyone who publishes but then rejects all reasonable criticism. They, too, fail to move us forward.

    • “Scientists, by definition, must move science forward. I’d now extend that disqualification to anyone who publishes but then rejects all reasonable criticism. They, too, fail to move us forward.”

      Nice one!

      I like to add that perhaps they do not only “fail to move us forward” but may also be “actively preventing us to move forward”.

      I would also like to add that reasoning about criticism, possibly countering it, etc. should be done according to scientific values/rultes/etc. I reason scientists should reason, and debate, like scientists and not like politicians, lawyers, or high school students in a debate club.

    • I’d argue that publication isn’t entirely required. I mean, if you discover something and then use it to build a product line and sell a new effective medicine or device or whatever you’re clearly moving society forward. Lots of people doing science do it on behalf of a company for example.

      But your other point about publishing junk and defending this junk failing to actually be science is correct. That’s the point of Feynman’s “Cargo Cult” science. The idea of Cargo Cult is that at some point in the past people did things, and it was associated with good things for their society, and they continue to do those things, even though those things are just the trappings of the process, not the actual causal mechanisms of the good outcomes….

      So, wearing headphones and talking into microphones and clearing long strips of runway aren’t going to make cargo planes land on your little island… And collecting poorly controlled data, calculating p values, and making up stories that explain the p values isn’t going to make you learn real scientifically valid facts…. but it sure looks like the real thing…

      • This reminds me of some of the great minds whose innovative work for the military in areas related to computer science, cryptography and information theory was classified, then independently discovered and published by others. Clearly, the originator was just as much of a scientist as the published researcher, though nobody may know it for decades. Maybe a partial motivation for Gosset (Student)? As I recall, though, we were thinking more of hobbyists and armchair philosophers.

    • Scientists, by definition, must move science forward.

      Not any definition I’ve heard. To begin with, “move science forward” is probably too vague to be useful.

        • I mean my estimate is that in fields like biomed research ~99.99% of the publications are actually impediments to progress at this point. I got that number from walking around conferences and seeing a couple good projects hidden amongst tens of thousands of mindless NHST ones that generate questionable conclusions that no one will ever replicate.

        • That Fisher quote:

          “We are quite in danger of sending highly trained and highly intelligent young men out into the world with tables of erroneous numbers under their arms, and with a dense fog in the place where their brains ought to be. In this century, of course, they will be working on guided missiles and advising the medical profession on the control of disease, and there is no limit to the extent to which they could impede every sort of national effort.”

          Fisher, R N (1958). “The Nature of Probability”. Centennial Review. 2: 261–274.

        • I sometimes wonder whether actual, and good, (social) scientists became extinct somewhere in the 70’s or something.

          The quote above from Fisher from a paper published in 1958 reinfrces that wonderment, and wondering, in me.

        • Anon said,
          I sometimes wonder whether actual, and good, (social) scientists became extinct somewhere in the 70’s or something.”

          I don’t think good social scientists became extinct somewhere in the 70’s or something — but I think that (probably starting around then) there was an increase in (mass-produced?) poor social scientists. Maybe it had something to do with the flourishing of easier-to-use software that seduced people to rely more on “what the computer said”, to the neglect of things (like good design and careful thinking?) that the computer couldn’t do. Another possible contributing factor may have been baby-boomers increasing the number of graduate students, which may have led to lessening of quality in order to handle the increased quantity?

        • Quote from above: “Hey—I’m a social scientist, and I think I’ve done some good work!”

          Yes, thank you for the comment.

          I was exaggerating of course, but i nonetheless should have used a different word than “extinct” even in my exaggeration.

          Perhaps i should have used something like “rare”. So, “actual, good” scientists may have not became “extinct” after the 70’s like the Dodo bird, but may have become increasingly more “rare” like a Pangolin, or a Seneca White Deer, or an Elephant Shrew (i looked up “rarest” animals and these names came up).

          For any possible further useful thoughts i like to refer to the reply by “Martha (Smith)” who makes some possible excellent points, and brings some nuance i was unable to do in my 1st comment.

  6. Many, many fields have evolved methods with which the participants in this blog discussion totally disagree. To the extent that use of those methods is a virtual requirement of publication and being taken seriously by ones peer group.

    There always needs to be a distinction drawn between “fraud” which comprises falsification of data or conclusions under the methods to which the author purports to be operating versus this other thing. We need a good term other than “fraud” for doing exactly what you were taught in grad school and what nearly every reviewer of manuscripts you submit will expect you to do, even though a more enlightened deep thinking considers the methods completely bogus.

    I perceive a rather porous line between cases where the label of “fraud” is justified and where it is only an intensifier to signify extreme displeasure with the norms in a field not ones own.

      • Tunnel vision?
        Field-specific (or field-induced?) tunnel vision?
        Viewed through field-colored glasses?
        TTWWADI? (That’s The Way We’ve Always Done It)
        That’s what the Bible of my field says to do?

        • I borrowed this phrase from a group of high school math teachers who were active in improving math teaching. They even had buttons made that had TTWWADI with a slash through it.

    • Quote from above: “We need a good term other than “fraud” for doing exactly what you were taught in grad school and what nearly every reviewer of manuscripts you submit will expect you to do, even though a more enlightened deep thinking considers the methods completely bogus.”

      I believe the “Levelt report” called it “sloppy science”.

      I am not sure if that also means that the people that engage in sloppy science can be called “sloppy scientists”.

      I am also not sure if there is a point where “we” could say that the scientists who engage in “sloppy science” had a responsibility to know that what they did was wrong. I often feel these discussions seem to remove all individual responsibility, and acountability, from the discussion. It feels like some sort of academic/scientic version of “willfull ignorance” or something like that. I don’t like that, and more importantly i think it doesn’t lead to optimal improvements.

      Also, as a result of your comment i also wonder what happened to those that engaged in more enlightened deep thinking and considered the methods completely bogus. Perhaps they left academia/science, or got shunned by the “fakers” and “enablers”. I wonder how we could get those people back in science/academia, as i reason they should be there instead of the “fakers” and “enablers” and “sloppy scientists”…

  7. So it’s pretty clear that it’s almost impossible to get people to admit their mistakes. We can bang our heads on that wall by exhorting people to admit their mistakes, but that isn’t likely to be any more successful in the future than it has been in the past.

    So what does work? Answer: having other people point out mistakes. Who has the motive and means to do this? Answer: other academics with different viewpoints or agendas.

    So if you want to actually do something about this problem, encourage other viewpoints in academia. Think of academia as a courtroom where biased opponents fight it out and we are the judge, rather than a celestial choir where we exhort everyone to be good and pure.

    That approach has actually worked to some extent in economics. The efficient market people fight with the behavioral people. The free-marketeers fight the market-failure people.

    • Terry:

      Two things. First, yes, it’s almost impossible to get some people to admit their mistakes. But there are other people who have no problem admitting mistakes. I admit mistakes all the time. I’d like to say it’s cos I publish so many things that I recognize that mistakes are inevitable, but for whatever reason, I’ll admit mistakes and I’m happy when people point them out to me.

      Second, I agree with your general point, which is related to the “research opposition” point that I made a couple years ago. I argued that one reason why certain junk science in psychology persisted for so long, and continues to persist in many influential quarters, is because traditionally this work has no active opposition.

      A similar issue arose years ago when Phil and I were doing research on decision making for radon exposure. There were various stakeholders who were OK with exaggerating radon risks, but not a lot of people pushing on the other side. We discuss some of that here.

      • Very interesting to see some serious work on radon. I remember the radon foo-faw a while back. It had the feel of a minor hysteria. (There is always the suspicion that self-righteousness makes people exaggerate the facts.) I looked into it a little and was actually surprised at the rationality of the analyses. But, as you point out, things weren’t fully rational, and there were thumbs on the scale that distorted decision-making. Alarmists especially like the zero-threshold risk assumption because they always get to say that some positive number of people are dying. Radon testers could also game the tests to get a higher reading and another remediation.

        Relatedly, it is looking like recycling was (mostly) a giant waste of time and effort. NPR has two good podcasts on this. (All this was known in the 80’s, but good intentions can be overwhelming.)

        The garbage-barge story (highly recommended): https://www.npr.org/2019/07/09/739893511/episode-925-a-mob-boss-a-garbage-boat-and-why-we-recycle

        Should we recycle? https://www.npr.org/2019/07/12/741283641/episode-926-so-should-we-recycle

        It will be interesting to see how long this cartoon coyote can continue running on thin air.

        • Terry:

          I have mixed feelings on the recycling-is-a-waste-of-time thing. On one hand, yes, I would not be surprised if recycling as currently practiced is counterproductive. On the other hand, I’m guessing that the solution is not to just throw things out, but to use less (following that “reduce, reuse, recycle” slogan in which Recycle is a distant #3 choice). So to the extent that mandatory recycling is a practice that makes it more difficult to just throw things out, it could be a step in the right direction. I’m not really sure; I just want to push against the implication that, just because a particular suggested improvement is not a good idea, that this means the status quo is ok.

        • My impression is that many recycling systems have turned out to have unintended negative consequences. But this doesn’t mean that the whole idea should be abandoned; it just points out that there is a need for improvement — in particular, a need for learning from mistakes to improve the process (a la Deming, Shewhart, et al).

        • I’m pretty sure for plastic and paper the optimal disposal method is to chop up the material in small bits and then feed it into a furnace designed to produce steam to power turbines. Small quantities of natural gas as a pilot fuel ensure enough heat to produce complete combustion, the outflow is basically pure CO2 and H2O and the material reduces the rate at which we burn fossil fuels. Since someone pays you to take the fuel away from their house, you win there, and then you sell the energy in the form of electricity and you win there. So it works extremely well and is commercially viable with enough incentives to do the investment needed to process the garbage and build the appropriate furnace equipment.

          This has been known since the 70’s but high horses and self congratulatory virtue signalling together as well as some kind of subsidization for taking plastic waste in China won out big time.

          When it comes to Aluminum, steel, and glass, recycling is already very effective.

        • I think we largely agree. Some forms of recycling make sense, aluminum and steel especially. These were recycled long before recycling was cool and will continue to be recycled if public recycling mandates end.

          But recycling became weirdly religious. A lot of people get a lot of warm fuzzies from it. … Which may be a good thing. Channeling all that religious zealotry into recycling is mostly innocuous. If it were channeled elsewhere, it might cause real harm.

          My most basic objection to recycling is that it assigns zero cost to homeowner’s time. We are talking pennies worth of material for all that labor and water and gas-powered trucking and recycling center staffing. Raw materials are incredibly cheap. Even at Chinese wage levels the labor cannot be justified (the Chinese will no longer take our recycling). The main payoff is, therefore, warm fuzzies. (Which is fine with me if someone cares a lot about those warm fuzzies.) For some recyclables, the payoff is actually negative — you have to pay someone to haul it away. Question: why don’t people steal your recycling? Answer: because it’s not worth anything! It’s just garbage that you spent time and water cleaning. All your efforts just turned dirty garbage into clean garbage that no one wants.

          I wish it were otherwise. I feel the attraction of being thrifty. I like feeling virtuous. I pick up litter. I save coupons. But, with modern technology, it just doesn’t make sense.

          I also find the aversion to landfills a bit odd or even silly. It’s just a big pile of garbage with, eventually, some grass and bushes on top. Sometimes, they are kind of scenic. Some people don’t mind living near them. If you do, then don’t live near one. Win-win. There is lots of room for garbage, at least in the U.S.

        • Actually around here (Los Angeles county) people routinely steal recycling in large quantities, this is extremely perverse because now instead of one garbage truck going around picking up the recycling there is one garbage truck and like 6 pickup trucks and 4 guys on bikes scrambling around trying to be the first to steal the recycling… of course they only steal the aluminum cans and glass bottles that have a 10 cent redemption value or whatever built into the price of the product. Sad really…

          The real damage of recycling is all this wasted effort including fuel etc, when we could be burning it as fuel and generating power and reducing our consumption of other fuels.

          Basically plastic is just oil from a fuel perspective, and paper/cardboard is just wood, also a decent fuel. Even pizza grease is just basically un-processed bio-diesel.

        • Daniel,

          Agree about people stealing the “good” recyclables aluminum and steel. That is how aluminum and steel used to be recycled. You put a deposit on them and the homeless would go through the garbage collecting them, and kids would bring them to the store to get some spending money. I assume the deposits are set higher than the actual raw material value, so its still not clear it is worth the homeowner’s time to clean and put it out on the curb.

          If you want to get all virtue-signally, you could righteously claim that we should landfill plastic and paper to fight global warming because landfilling them safely sequesters that carbon dioxide and keeps it out of the atmosphere. Burning it would release the carbon dioxide. A silly argument perhaps, but it has some warm fuzzies. In environmental rhetoric, the objective function keeps shifting.

          Fun facts I learned from the NPR podcasts: Much of our recycling is unusable because we don’t clean it well enough. Did you know a plastic bottle can’t be recycled if you leave the top on? That a pizza box with a pizza stain cannot be recycled? That jars and bottles should be rinsed three times? That if you put badly cleaned recycling in with other recycling, it renders the entire lot unrecyclable? That China dumped a lot of our low-quality recycling in the ocean, contributing to the Pacific plastic problem?

        • More fun facts:

          Aluminum scrap is worth about $0.30 per pound (delivered to the recycler)

          31 aluminum cans weigh 1 pound.

          So, each aluminum can you recycle is worth about a penny, less the cost to deliver it to the recycler.

          Kind of puts things in perspective.

        • I always wondered if it was worth the cost of the hot water to clean plastic bottles…

          LANDFILLS ARE NOT PLEASANT. If you live near one your yard and neighborhood will constantly be full of trash blowing out of the landfill, plastic bags especially. They’re also detectable via smell for miles.

        • I’m guessing market signals will kill off recycling soon. When a town’s recycling bill goes from $20/ton to $100/ton, a lot of towns will nope out.

        • As both you and Daniel pointed out there once was a robust market for recycling in the US. There still is a great market for metals. Paper though is pretty much irrelevant.

          IMO in the end the market “knows” best how to do most things. Prices really do signal the consumption of resources.

          I think burning the waste is the best choice. Aside from being stinky and unsightly, landfills frequently have toxicity issues. Lots of waste is already shipped by rail to landfills in remote places. Why not ship it to giant powerplants? The large scale should push down the cost of controlling emissions.

        • Could be. I wouldn’t mind if the market directed things that way. It would still require unpaid homeowner labor for tiny, tiny economic benefits. So maybe its not correct to call it a market solution … unless you consider the warm fuzzies to the bottle-washing homeowners as their payoff.

    • Quote from above: “So if you want to actually do something about this problem, encourage other viewpoints in academia. Think of academia as a courtroom where biased opponents fight it out and we are the judge, rather than a celestial choir where we exhort everyone to be good and pure.”

      I agree with this as i think this is part of how i view science should work, at least to some extent. Recent years, and recent academics, have messed this all up i think. I get the feeling it seems almost “rude” to some people to simply point out that someone’s reasoning may be faulty, or something like that.

      Regardless, i reason that there may in fact be alternative viewpoints but i think they DON’T HAVE ANY CONSEQUENCES. Due to things like politics, power, egos, the journal/editor/reviewer model of publication, money, etc., the current people in academia can basically do just about anything they want. There are almost no consequences to saying and writing the most absurd, and incompetent, and unscientific, things.

      I sometimes view academics as sort of the kings and queens of days gone past. They have almost a monopoly on power, money, and influence compared to “non-academics” concerning science, and scientific information. This however in turn influences everyone. They can do or say just about anything without negative consequences. They hold on to their “thrown”, and don’t let other people sit on it for even a day. Only “scientific offspring”, or friends, can sit on the thrown.

      My “solution” to this possible problem is partly in line with your proposal in that i think 1) other people should present criticm, but i reason they do NOT necessarily have to be academics. And 2) scientific values, principles, and responsibilities should be made clear over and over and over again (and will hopefully therefore be adhered to by all parties involved). When “non-academics” are more educated about the scientific values, principles, and responsibilities i think they can function as a control mechanism in some way or form, and can perhaps function as scientists as well. Science does not belong to academia, it belongs to everyone.

      I think 1) and 2) can both be helped to be achieved by blogs and other non-academic communication. Of course there will be “bad” blogs that delete critical comments, or have a network of “friends” that all like and re-tweet eachothers blogs, or say they are all about discussion but don’t let everyone join in, or write unscientific things, etc.

      But that’s where 1) and 2) kick in again. It should be tried to make clear which blogs do and do not filter comments. It should be tried to make clear why something that is written is, or isn’t, in line with scientific values, and princples, etc. Although it seems impossible to “right” every scientific “wrong” this way, i see no other option but to simply (try and) stick to scientific values, principles, and responsibilities and whatever happens, happens. In line with this, i also think it could be scientifically useful to invent some sort of mechanism that makes it impossible to filter comments on blogs. I think this could be done, and i think this could in turn become some sort of a “quality” marker for scientific blogs.

      I would hope that would all somehow lead to some (in my opinion) needed restoration of science, and the quality of scientific discourse, but whether that’s the case or not i don’t know.

  8. “There are almost no consequences to saying and writing the most absurd, and incompetent, and unscientific, things. ”

    Well, I think that might be a stretch! There are definitely constraints on what academics can say and write in papers and both internal and external criticism impacts what people are willing to claim. That might not be true for every person but on the whole substantive criticism does seem to have an impact.

    • “That might not be true for every person but on the whole substantive criticism does seem to have an impact.”

      That would be great!

      I personally have largely given up on “substantive criticism” being able to make an impact in academia/science. It’s mostly about who says what and when. Sometimes things align in such a way that a small step forward can be made, but i have concluded this is largely due to coincidents.

      When i wrote that there are almost no consequences to saying and writing the most absurd, and incompetent, and unscientific things i was also thinking about professional consequences like losing your job, or not getting grants anymore, or not being able to publish at a “top journal” anymore, or not being promoted, etc.

      Only the people who truly mess things up like Stapel or Wansink suffer true consequences, and i reason that’s done reluctantly on the part of universities, and possibly mostly due to media attention these cases received.

Leave a Reply to Anonymous Cancel reply

Your email address will not be published. Required fields are marked *