Mark Zimbelman has the story:
Over the past few days, a 2012 paper coauthored by someone who is arguably the most prolific psychology researcher on (ironically) honesty, Dan Ariely, was identified as containing fraudulent data and is being retracted . . .
Sadly, I have had some suspicions about some experiments that Professor Ariely ran using a shredder that was modified so it looked like it was shredding but really wasn’t (see this link for more details). After purchasing several shredders in an attempt to modify them so we could do what he claimed he did and even getting a BYU engineering student who was the lead TA in the Mechanical Engineering lab involved in the effort, we emailed Professor Ariely to ask him how he did it. He provided a voicemail response that was not satisfactory. He claimed it was “quite simple” to convert a shredder by breaking the teeth in the middle with a screwdriver but that he doesn’t use that method any longer. . . .
We were unable to break any teeth on the shredders we purchased but ended up finding a way to remove some of the teeth in the center by taking the shredder apart. Unfortunately, when we did this the papers would no longer go through the shredder without getting turned to one side or another and they inevitably got stuck because the shredder no longer had enough teeth to pull them through. We concluded that it was impossible to modify any of the shredders we bought and I filed the entire experience in my mind by putting an asterisk next to Dan Ariely’s name as someone who made a claim regarding his research that seems potentially suspicious.
Too bad the original shredder got lost, huh?
I’m thinking it could be cool for someone to set up a Museum of Scholarly Misconduct, which would have various artifacts such as Marc Hauser’s monkey tapes, Brian Wansink’s bottomless soup bowl and his 80-pound rock, Diederik Stapel’s survey forms, Mary Rosh’s survey forms, Michael Bellesiles’s probate inventories, the National Geographic video that Matthew Walker cited as a source for his false WHO claim, the famous Surgisphere dataset, and of course the shredder discussed above. We could also include some items that aren’t misconduct, exactly, but are scholarly embarrassments, such as the survey forms filled out by the experts who assessed North Korea to be kind of a democracy, some of the documentation from that classic ESP study, the original cold fusion apparatus from 1989, and something or another from the original slow-walking priming experiment.
I like the idea of a real-life museum with physical artifacts—or, if necessary, replicas, for example if Hauser still wants to hold onto his tapes or Wansink won’t share his 80-pound rock with the general public.
This came up in our discussion of the soup bowl experiment, where there was a fascinating case of a real-life replication of a nonexistent study. Kind of like if someone were to breed a unicorn.
P.S. A colleague points us to these recent Wall Street Journal columns, all of which appeared after the scandal of the dishonest dishonesty paper (see also here):
There must be lots of people who’d looove to have a regular Wall Street Journal column—but they pick someone whose studies don’t replicate and who can’t trace his data?
OK, I guess that makes sense. If they choose someone who’s careful with his data, then he can’t write such exciting columns. Being restricted to the truth is a real inhibitor.
Hardware setups are an area where research transparency hasn’t made much progress yet. It’d be interesting to define what “open experiment materials” means when an experiment involves more than images and code.
Maybe journals should require a hardware diagram if there is custom hardware involved?
Steve:
In laboratory biology they sometimes require a video of the experimental setup, right? So in this case maybe they could have a video of the shredder in action. I’m not sure the best way to document the 80-pound rock—maybe a video of someone standing on a bathroom scale, then picking up the rock and getting back on the scale? All in one take, of course! Experimental setups such as the soup bowl seem more conceptual than real; maybe if the video setup were required, researchers would be less likely to pretend that these fake experiments really happened.
A video requirement could help, but wasn’t there even kind of documentation including pics and short description of the syphon-like mechanism for the soup bowl setup?
Awesome! It has to be at UPenn, right? So we can start with Samuel Morton’s skulls?
Wasn’t Morton vindicated against Gould’s calumny?
Oh gosh, you might be right. We are going to have to make it an interactive exhibit, “Gould or Morton,” where you get to measure copies of the skulls yourself, with seed and shot options …
Re “Gould or Morton”: Perhaps you are not aware that there was an American composer whose name was Morton Gould.
Martha:
I looked up him up on wikipedia. Good stuff:
– He’s from Richmond Hill, Queens. That’s where my mom grew up!
– He married a woman named Shirley, then a few years later they divorced. The next year he married another woman named Shirley. They divorced too.
I was curious about his music so I looked it up on Youtube. I wasn’t sure where to start. I picked “American Concertette for piano and orchestra” from 1943, which kinda sounded like what you’d expect an American classical orchestral piece from that era. I guess it would work just fine as movie music.
The hundredth monkey.
All the tour guides will be the daughters of beautiful parents.
Or sons of ugly parents.
I’m assuming this museum would be an empty room with stark white walls.
At least then all the visitors will stop stereotyping and discriminating against one another.
Is the hypothesis that being in a stark, white environment primes us to see everyone as more white and male thus reducing discrimination? Just asking for a paint by numbers scientist friend.
It’s even deeper than that! Though we might need to limit attendance to only those with a high need for structure.
https://www.science.org/doi/10.1126/science.334.6060.1202-a
Mary Rosh … good one.
There is a follow-up post to the shredder article: https://fraudbytes.blogspot.com/2021/08/my-experience-with-arielys-modified.html?m=1 It suggests that Ariely (or his group) never modified any shredders at least for their published work, and that the idea of modification only exists in Ariely’s public statements.
> Dr. Ariely has made numerous claims to use a modified shredder in his matrix experiments. I am unable to find any published papers by Dr. Ariely that use a modified shredder. Modifying a shredder to do what he has claimed appears to be very unlikely.
Or wax sculptures of the canvassers from LaCour’s study! Oh wait…
And a plaque on the wall, with the famous line which seems to apply to all cases of scientific misconduct and just plain incompetence:
A jar of Benveniste’s memorable memory water ?
And the original cold fusion generator! And the set-up for every ESP experiment ever conducted . . .
> it was “quite simple” to convert a shredder by breaking the teeth in the middle with a screwdriver
I don’t know why this is so believable to me but I wouldn’t have stopped to question it had I read this in another context.
There are some cool little mechanical tricks — like hitting a jar lid on the edge with a spoon, or whatever. That’s really cool that it makes the lid come off easier! In the spirit of breaking metal, soda cans are pretty cool. But from a practical perspective, the soda cans seem different — it is their intended purpose to be broken like this. The jar lid may be constructed with this in mind, but it probably isn’t the intended way to use the jar. I don’t expect the shredder manufacturers were making allowances for these sorts of modifications at all.
Looking at the picture of the shredder in the article too, it looks like those teeth are really well attached to the spindle things they’re on. I wouldn’t expect that to break off cleanly at all.
I guess this is just filed under, sounds simple but really isn’t. It reminded me yesterday of the cut operator thing that has come up before: https://statmodeling.stat.columbia.edu/2016/02/20/dont-get-me-started-on-cut/. I remember wanting such a thing and it seems so simple! But the whole thing is a lot more complicated than it seems on the surface.
Ben:
Yeah, it makes sense that the screwdriver thing should sound simple. I’m guessing that what happened is that the researcher thought about the shredder thing as a possible research gimmick but then it was too difficult to put together (or maybe they didn’t even try), but it was such a good story that he couldn’t resist talking about it in interviews as if it were real. Similar to the soup bowl thing: such a great idea that the dude couldn’t resist talking about it. It would’ve been cool if they’d said, “Hey, what a great idea this seemed to be, but it didn’t work”—but I guess that kind of thing doesn’t get you a Ted talk or a billion dollars of VC funding and Charles Schulz on your board of directors or whatever.
Well, see the Crystallography Horror Museum created in 2020 from a series of totally crazy papers published and built up from mixtures of figures plagiarized, and imaginary impossible compounds…
https://cristal.org/CHM/CHM.html
Unlikely that any of the great ’70s/80s artifacts have been preserved though. Would love to see Summerlin’s magic-markered mice from Sloan-Kettering, or John Darsee’s uninfected (and therefore non-radioactive) hearts from the Brigham.
Sorry meant ‘un-injected’…. hearts were supposedly injected with a tracer radionuclide; the lack of traceable radioactivity put paid to the legitimacy of published findings
You should package the museum piece digitally – into an NFT – and auction it.
Hi Andrew
How about some of those new materials as promised by Jan Hendrik Schön?
Cheers Oliver
I tried to clear a jam on the shredder at our house yesterday. When I looked at the shredder, it appeared quite easy to remove the teeth using an angle grinder. No disassembly would be required. There would be a chance that the chips from the grinder would jam the mechanism. I am virtually certain that, without much effort, I could remove the teeth from a two or three inch region in the middle of the rollers.
Bob76
Bob:
An angle grinder, sure. But in the audio clip (follow the last link at this post), Ariely claims he used a screwdriver, so that’s the method that Zimbelman tried.
Zimbelman stated “. . . We concluded that it was impossible to modify any of the shredders we bought [to appear to work but to pass the middle unshredded] . . .” I did not read that statement as limited to using a screwdriver.
The picture of the shredder rollers in Zimbelman’s web post looks quite similar to those in our shredder. I think it would be easy to grind off the little teeth. I suspect you would have to be superman to knock them off with a screwdriver.
I think it might be possible to disable a strip-cut type shredder with a screwdriver. I could not find any good diagrams of a strip-cut shredder however.
Bob76
Ariely’s shredder. Is this fake?
https://www.youtube.com/watch?v=AmMe0T1Wf-g
Anon:
I have no idea. All I know is the information linked to above. Following the link in the above post yields this article from 2016, Ariely says they modified the shredder in 2002. The description is, “we modified the shredder! We only shredded the sides of the page, whereas the body of the page remained intact.” He also said:
Which is interesting given the fraudulent projects he was involved in. Interesting if he was doing fraud and interesting if he was the unfortunate victim of fraud.
And he said, “Ethics is like health and therefore something we need to invest in, monitor, be mindful of and continuously consider – as individuals and as a community.”
Ariely’s movie, (Dis)Honesty – The Truth About Lies, came out in 2015, and in the Youtube link you give, Ariely states that the shredder in the video is the same one that was used in the experiment from 2002. That video shows a dramatization of the experiments. In that dramatization, papers are put into the shredder and then it shows them opening up the bottom of the shredder to reveal pages that are shredded only on the edges.
In the answering-machine message given in the above-linked post, Ariely says they bought the shredder at Home Depot. In the video you link to, the shredder is labeled Staples. I didn’t think they sold Staples brand products at Home Depot, but I’m not an expert on these stores.
There are several possibilities here:
1. The 2002 experiment happened as was described, they really modified that shredder as claimed, they kept the shredder around, it was still operating in 2015, they demonstrated it in the video, and it worked as advertised. Also, that shredder in 2002 had actually been bought Staples, and Ariely just had a lapse in memory when he said they’d bought it at Home Depot.
2. The 2002 experiment happened as was described, they really bought that shredder at Home Depot and modified it as claimed, but then the shredder was lost, or discarded, or broke. When they made the movie, they went to Staples and bought a new shredder, modified it (using that approach which Ariely said is simple but which Zimbelman in the above post said is actually difficult or impossible), and it worked just as planned. Under this scenario, Ariely is lying when he writes that the shredder is the same as in the original experiment, but, hey, it’s just a movie, right? The Coen brothers had that title card saying that Fargo was based on a true story, and it wasn’t—that doesn’t mean they were “lying,” exactly!
3. The 2002 experiment happened as was described, they really bought that shredder at Home Depot and modified it as claimed, but then the shredder was lost, or discarded, or broke. When they made the movie, they went to Staples and bought a new shredder, but at this point they didn’t bother to try to modify it. It was just a dramatization, after all! Why ruin a perfectly good shredder from Staples? Instead they just filmed things just as they literally look in the video: they put paper in the shredder, then off screen they mangle the edges of some other sheets of paper, put them in the bin in the bottom of the shredder, and then turn on the video again and take the partially-mangled papers out. The way I wrote this, it seems kinda complicated, but if you think about it from the standpoint of someone making a video, this is much easier than getting some modified shredder to work!
4. The 2002 experiment never happened as described. Ariely or one of his collaborators had that cool idea of modifying the shredder, but it wasn’t so easy to do so they gave up and just faked the study. Then when making the 2015 movie they just said they lost the shredder, and the moviemakers just did the reenactment as described in option 3 above.
5. I’m sure there are other possibilities I didn’t think of!
Is the video “fake”? I doubt it! I assume the video is either a clip from the movie or was filmed at the same time as the movie. It’s a reenactment and is clearly described as such. In a reenactment if you show some sheets of paper fed into a shredder and then later you show some sheets of paper removed from the shredder, there’s no reason that they have to be the same sheets of paper. Similarly, if a movie reenacts a flight from New York to Chicago, and it shows a shot of a plane taking off from LaGuardia, followed by a shot of a plane landing in O’Hare, they don’t have to be the same plane. It’s just a reenactment! The fact that someone made a video reenacting a scene with a shredder does not imply that this shredding actually happened in the video—there’s really no reason to do it that way. I agree that possibility 1 listed above is a possibility; it’s one of several possibilities consistent with the information that is currently available to me.