58 thoughts on “Some will spam you with a six-gun and some with a fountain pen”

1. Huh. I’m now wondering if there’s a good case to be made here for the ethics of not debriefing at all. After all, in both cases, that’s the point when your blood pressure really hit the roof… whereas if you’d never heard any more about it you’d have forgotten about the incident completely!

   I’m joking. I think.

   Reply ↓

   • +1

     It’s a stupid study yeah, but as an aside I wonder how one rates the relative annoyance of various other social science survey studies. Most surveys take a lot more time than an email to answer too.

     Reply ↓

     • But one isn’t duped into completing the surveys, whereas the email from the purported student was a deception.

       Reply ↓

2. Without getting into the merits (or lack thereof) of this particular study, I can see how studies of this general type can be useful, telling us things we would have a hard time learning another way. Is there any way that you, Andrew, would consider this form of study to be acceptable? Maybe you get an email that says “We are studying an issue of practical importance that is related to the education of students from other countries. If you agree to participate, then at some point in the next year we will send you a single, short email without identifying ourselves; your response to that email (if any) will constitute the entirety of your participation. We can’t tell you more than this without affecting the results. If you’re willing to participate, please reply with ‘yes’.” Would that be OK? They’d still be “wasting” your time merely by asking you to participate, and for the subset of participants who say Yes they will have wasted more of your time, since you’ll now be reading their emails twice, not once.

   Reply ↓

   • Phil:

     Yes, this came up in our blog discussion a few years ago on that earlier study. I think pre-asking would be ok (even though, yes, it would waste more time). Another option would be to pay the participants of the study. One problem, I think, is that sending people emails is free (to the sender), thus nothing is stopping them from spamming thousands of people. Jeremy Bentham would not be pleased.

     Reply ↓

     • What are practical IRB situations where people think that not pre-asking is OK? Any examples?

       Reply ↓

3. I am confused by the point of the second email (unless the study is still ongoing). If the data are about your first reply, then once you have replied, you get the debrief that apologizes that you changed your busy schedule around to make sure that you had 9:50 free. IRBs are usually very particular on having as little lying as possible. So why was it necessary for them to make you think Alex had a family emergency? I guess if they were not wanting publicity to get around they had to delay the debrief, but there seems lots of other ways to design this (and Phil’s suggestion which is used in lots of social psychology is better anyway). Why make you feel sorry for whatever emergency Alex had to deal with (I’m feeling sorry for him now!).

   Reply ↓

   • Dan:

     It’s wheels within wheels within wheels, man. I’m still waiting for the debriefing that explains why the first debriefing was a fake, and that actually there’s a 1.2 million dollar bank account in Nigeria waiting for me if I just co-sign a few forms . . .

     Reply ↓

   • This whole thing brings up the IRB problem. By that, I do not mean to dump on people who serve on IRB’s. A big part of the problem is that they are volunteers who don’t really get any compensation for their participation, so can’t reasonably be expected to do a thorough job since they are juggling their IRB responsibilities with lots of other responsibilities. In that sense, it’s similar to the one of the problems with peer reviewing: Reviewers just don’t have the time to do a good job all the time.

     I don’t know a good solution, but I think part of it could be a joint effort of various professional societies and funding agencies to try to come up with guidelines that IRB’s could follow; with recommendations to institutions about how to provide quality IRB reviews (it necessarily will cost — you can’t expect to get high quality for free.)

     Reply ↓

4. Very bad attitude from Xian Zhao, Monica Biernat, and their institutional IRB.

   Reply ↓

   • Well, to be fair, Alex did have a family emergency. Or don’t tell me that was made up too!

     Reply ↓

     • Family emergency is totally plausible. After all, it can be hazardous to have a student in the family, especially for grandmothers.

       http://www.improbable.com/airchives/paperair/volume5/v5i6/GrandmotherEffect%205-6.pdf

       Reply ↓

       • I know of a grandmother of three who died the Saturday before finals week. So it can happen!! (Ironically, she was a college professor, and had three children who were also college professors.)

5. If it were me, I would withdraw my response. While “spite” is probably a good enough reason, I think there’s a pretty good principled reason to. Practically all the costs of this study are borne by the (unwilling) participants. As you point out, sending these emails cost pretty much nothing, while all the people who received them will have wasted at least a small amount of time and had their trust in communications of this sort damaged, at least in a small way. Withdrawing responses is the only way to shift some of the burden back on the researchers. I see this the same as telemarketers: I would never buy something from a telemarketer on principle, even if I was sure it was a good deal, because that would only increase the profitability (and therefore the incidence) of telemarketing.

   Reply ↓

   • Agreed. My perception of academia in the USA is that there is already a serious problem with people ignoring sincere but unsolicited email (indeed, that’s what motivates their study). The effect of the study can only be to make the problem worse – they are harming thousands of students across the globe.

     Reply ↓

     • I wonder what would have happened if Andrew’s first response was

       Hi Alex,
       It would be great to meet with you. I was going to be away those days because of a family emergency, but I have changed my plans. Changing the flight was only a few hundred dollars, but it will be worth meeting you!
       Really excited,
       Andrew

       I assume the authors knew that some people would change plans so I would hope they had something planned for replies like this.

       Reply ↓

     • They should attach a $20 starbucks coupon to the follow up email.

       Andrew: If they had, would you have still have been pissed off about it? :)

       Reply ↓

       • Rahul:

         Yes, even a $10 coupon would’ve been ok to me, then I’d have felt fairly compensated. Actually, I don’t drink coffee so I would’ve given out the coupon to one of the students in my class. But that would’ve been just fine, I would’ve felt happy about the whole experience.

       • As a long time reader and first time commenter…

         You don’t drink coffee? Seriously? Wow. You sure get a lot of work done.

       • Lots of people don’t drink coffee. In our case, it’s only an inconvenience when we have guests and can’t offer them coffee in the morning. But no point in keeping it around for those rare occasions since coffee doesn’t “keep” very well (at least, good coffee doesn’t)

       • Count me in as another non-coffee drinker. (But I don’t get as much work done as Andrew.)

6. As a social psych PhD, I have seen how studies like these are adored by the field because they use a behavior (rather than paper-based) dependent variable, they seem to have “real-world” implications, and they are methodologically easy compared to either laboratory studies or intensive field world.

   However, now that I’ve left that world for a non-academic career, I have to admit that these studies look more and more ridiculous to me from an outsider’s perspective.

   Reply ↓

7. Andrew,

   I’m curious as to why you think the original study was “crap-ass”. Experiments like these seem to avoid most of the major issues of social psychology. Do you think the statistics were dodgy?

   I can see why you’d be a little annoyed, though.

   Reply ↓

   • This type of approach is actually used all the time in discrimination cases. An individual (or couple) is (are) created with certain characteristics in common and one different–race, ethnicity, sex or some other legally protected status. Then the behavior of, say, the rental agent is observed to see whether it is identical for the two types of constructed individuals (or couples) (offering identical apartments at the same rental rate, for example).

     I can see why rental companies might not like this type of behavior but it is really the only way to demonstrate pervasive discrimination, as it really is an experiment (keeping all conditions constant but changing one). But it’s a very useful technique. Andrew’s position seems to be it shouldn’t be done on him because it is a waste of time but really, there are almost no academics carrying out this type of research and the amount of time he spent was minimal. And it does provide an indicator of general societal attitudes towards legally-protected groups. I wonder if he would be happier if he received this e-mail from the Department of Education and then had a lawsuit filed against him, if, say, he, for whatever reason, denied the Chinese-surnamed student 10 minutes but gave a non-Chinese-surnamed student the 10 minutes? But that is how we monitor other decision-makers (see the above rental agent example).

     Reply ↓

     • The reason it shouldn’t be done is not so much because it wastes respondents’ time but because it conditions them against responding to future emails, thus hurting the very people the study is purporting to help.

       Reply ↓

     • The problem is that you don’t, and can’t, keep all conditions constant. This is true in any field, which is why we replicate (to average out all the varying factors we can’t, or don’t know to, control). To go with your rental agent example: were the individuals (or couples) exactly as punctual as one another? Were they equally polite? Were they wearing the same clothes, or near enough? Were they driving the same car? Was the rental agent in the same mood on both days? Was the weather the same on both days? Are there really identical apartments available? If we are monitoring decision makers this way without quite a bit of replication, then we are doing it wrong.

       From the researchers’ followup, it sounds like their sample size was 30 respondents times however many “social science and science” departments they decided to cover. That sounds pretty small to me. If even two of the 15 people in a set happened to be on vacation or at a conference the next week that would make a huge difference. They say they selected one faculty member that “appeared to be a US citizen and white” from their website picture at random. Even if we assume they actually made the selection at random (i.e. listed all the possible names and made a truly random selection from the list), they are skewing (off the top of my head) toward faculty who have a website photo and one the basis of their determination of who is “white”. From the way the email is worded it seems as though the US Citizen determination was made from the picture, which seems odd. Could that mean they skewed towards people with English-sounding names? Why limit themselves to the top 30 departments when email is free? That would imply that they are skewing toward the busiest professors. The English used in the initial email is pretty good, but not perfect–how does that interact with the name? Presumably the “you can call me by…” clause was pasted into the “Alex” set. Could that clause have an effect? It’s pretty clumsily worded (being inserted before the “a student from China”)–would a better worded email have made a difference? What about if the Chinese name wasn’t presented at all?

       Even if this study was large enough and the respondents were well-selected, the only thing being tested is whether the presence of an English name makes a difference in people’s responses to the email, so there’s no way to tell whether there is a difference between the treatment of either group with any other ethnicity. Even if there were a difference, what would that even mean? It’s hard to see it as discrimination, considering that the ethnicity of the student is obvious in both cases. Considering that people with a certain name aren’t a protected group, how is this useful? Finally, is one’s response to a short-notice request for a meeting really evidence of anything?

       There are just so many holes and confounding factors in this study that it really does seem like a complete waste of time. Even if you waste five minutes apiece of 12 people’s time, that’s still an hour wasted. And I absolutely fail to see how this study being conducted could in any way prevent or even reduce the likelihood of any anyone being sued for anything. I still say the best course of action is to withdraw consent from any study like this to keep from encouraging it to catch on.

       Reply ↓

       • There is an effort made to make the couples (say) as close to each other as possible, and they are looking for relatively overt acts of discrimination (such as offering certain properties to the non-minority couple that are not offered to the minority couple). Here is a write-up of such efforts:

         http://www.propublica.org/article/no-sting-feds-wont-go-undercover-to-prove-housing-discrimination

         I’ll leave it to you to decide if exact matching outlined by you is necessary in the cases described in this article. For those that don’t want to wade through this article, the bottom line is that housing discrimination happens all of the time (at least anecdotally) and there are relatively few resources devoted to uncovering it.

         On a more general note, there is a great divide in how minorities perceive the incidence of racial discrimination versus non-minorities. It influences research efforts also–in political science, empirical analyses of elections typically focus on issue voting and party id and are typically approached in a color-blind manor, whereas practitioners (at least in racially mixed areas), pay much greater attention to race and ethnicity. Part of it is difficulty with measuring such things, but part of it is that researchers (particularly of the dominant social class, which most academics are), don’t like to analyze in terms of immutable characteristics (it’s worse than Marxism in many ways–at least one can change one’s class, but one can’t change one’s race or gender). I don’t know whether Devane’s objections to rental stings are influenced by his race, and “check your privilege” has apparently become a way of ending any discussion in academic circles, but it helps to inspect one’s own beliefs/actions for unconscious biases.

       • I’d slow down before turning to the class and analyzing people’s motivations too much. In the context of the topic at hand, your hypothetical rental agent analogy was a useful way to talk about the potential problems with underpowered studies where confounding factors are not sufficiently controlled for nor sufficiently understood. It’s funny how describing areas of concern with an approach amount to “objecting” to it. I specifically objected to performing such an approach without sufficient replication (I should have also said without the proper controls). The fact that something is difficult to measure is no excuse for measuring it poorly. Hopefully, in the real world, rental agent studies are carefully controlled and replicated. The study at hand, though, seems to me to be underpowered and poorly controlled. It uses unwillingness to schedule an appointment as a proxy for prejudice, I guess? While given enough power, any main effect they might show may be valid, but they appear to want to also look at the interaction of that effect with department, and given their sampling approach I very much doubt that such an approach is valid. Even if they did show a valid, significant effect, it would seem to only relate to the use of an English name by Chinese students. You seemed to imply earlier that it might root out elements of discrimination, but is that really the case? In both cases, the email clearly discloses the nationality and Chinese name of the student. It’s hard to see how a difference might be a discriminatory matter, considering that it would be known to the people responding to the email (I think) that the English name was something intentionally chosen by the student.

       • I’m not “turning to class”, just pointing out that pretty much every study in psychology, sociology, and political science (that looks at it rather than ignores it) finds evidence of racial differentiation (showing a picture of an African-American as opposed to a white before asking about punishment for a hypothetical crime results in a desire for heavier punishments, etc). These biases are amenable to explication through real-world experimentation (the rental agents, say). How many social science phenomenon can we say that about (markets are an obvious one–raise prices, consumption goes down, lower prices, it goes up)?

         Getting back to this particular study, I was addressing Andrew’s claim that his time shouldn’t be wasted. My argument is that there are relatively few studies of this type and that it is a very important social phenomenon, so it comes down to I think he’s complaining too much. Social psychology, like all soft sciences, is very susceptible to fads, and this type of study is certainly faddish these days. And doubtlessly it can be done better. But measuring attitudes towards race/ethnicity/nationality in what is attempting to be a multi-racial society is important and if one “wastes” a few minutes in a year being measured, the tradeoff for society is worth it (and I think he’s wasting a lot more time reading John Updike novels rather than doing statistics, but that’s just my evaluation).

       • I actually meant physically “turn to the [rest of the] class” as in “the other students in your lecture hall”, referring to the way you addressed me, then referred to me in the third person while speculating on my motivations. It was probably a little more snide than necessary, and easily misunderstood in this context.

         “But measuring attitudes towards race/ethnicity/nationality in what is attempting to be a multi-racial society is important…”

         I think it’s worth clarifying our differences here. I don’t disagree with this statement (I wouldn’t anthropomorphize “society”, but saying “in a society that is becoming more multi-racial” doesn’t fundamentally change it). However, I don’t believe that this statement (though true) is a valid defense of this study.

         1) It (the study) explicitly doesn’t measure attitudes toward race, ethnicity, or nationality per se. It simply purports to measure attitudes (among faculty who “appear to be white”) toward the use of an English name by Chinese students. It cannot indicate what those faculty member’s attitudes towards Chinese students generally are. It also cannot make any statements about whether the respondents’ attitudes differ from the attitudes of those of other ethnicities. It purports to be able to determine whether these attitudes vary among departments, but I have serious doubts about whether that would be valid, given the way the study is constructed.

         1a) This sort of study seems ripe to me for unjustified claims. That is, even though it does not and cannot test attitudes toward Chinese students in general, I worry that claims of that nature might ultimately be made. I realize this is rank speculation on my part, and there is every possibility that the researchers will be scrupulous in their claims, but I am a bit weary of unjustified headlines, especially in the social sciences.

         2) I don’t think I’m misrepresenting you to characterize part of your argument as, “Even if this study is flawed, even deeply flawed, the subject matter is important enough that the benefits (though perhaps smaller than would derive from a better constructed study) still make it worth the costs.” Specifically, in this case, the costs are the unwilling participants’ time. Here is where I disagree with you. To analogize, drug discovery is an incredibly important field, but that is no reason to stand for shoddy research in that area. In fact, I would argue that the more important an area of study, the higher the standards of research should be. Poor work in drug discovery can lead to tremendous wastes of time and resources on dead ends, both in the original work and in follow-up work. It can cause the risks of certain drugs to be poorly understood. Perhaps most importantly in the long run, it can damage the public’s (and funding agencies’) trust in the value of the field. That is how I view a study like this. Through poor construction, it contributes very little information to the field, can yield results that are prone to misapplication, damages the trust people have in communications of this variety, and reduces the trust that others have in the field in general. The point of criticizing a poor study is not that it improves the field in a smaller fashion than it could, but that it can actively damage the field.

         I’ll go back to the rental agent analogy. While the subject matter being tested is important to test, and difficult to test otherwise, a poorly constructed sting has very real costs, with few benefits to show for it: it decreases the level of trust among people in general, generates a level of resentment among those “stung”, can miss real cases of discrimination, and can result in people being falsely accused of discrimination. If such studies were generally conducted in this manner, they would eventually generate widening cynicism about the approach, that ultimately could bleed into cynicism about the cause.

         3) Obviously, the unwilling nature of participation is problematic. While it is clear that such an approach is necessary at times, it confounds the disadvantages when the study in question is poorly constructed. If this had been a well constructed study (larger sample set, better blocking, better selection of participants, use of multiple names, etc.) I would have less of a problem with this aspect. As it is, like other spammers, the researchers are free-riding on a system that is largely based on trust. That would not be so bad if society really benefited from their work, but I suspect the biggest benefit will be to the researchers in the form of a CV line.

         4) I’m less likely than some to discount small costs that are spread widely. Although any one respondent may have only wasted 5 minutes, if there were a couple hundred participants, that amounts to two days lost.

8. Ah, payback’s a #@&*!. Years ago I remember reading in one of Abbie Hoffman’s books where a bank pissed him off so he opened up a safe deposit box (under a false name) and filled it with a pot roast. (Actually, I can’t recall if he said he did it or if he was just floating the idea. No matter.) His speculation was that it would be about a week before the stink got so bad that the bank would have to drill the locks and remove it. The challenge of course would be to determine which box contained the roast, otherwise they’d have to starting drilling locks until they found it. Ouch.

   PS I like the study invitation protocol/etiquette that Phil suggests above. If the invitation were couched that way then I’d probably accept.

   Reply ↓

9. “Xian” looked to me like a post-Christian Christian name for “Christian,” but then I was a big fan of the 1980s LA punk band X, fronted by Exene Cervenka (formerly Christene Cervenka).

   Reply ↓

   • No kidding? I saw them at the House of Blues in Boston about a year ago. They were great. Given their ages, I wonder how much longer they’ll keep touring.

     Reply ↓

   • Steve, I can swear I was not involved in this experiment. Thanks for the punk reference!

     Reply ↓

   • ‘Xian’ as a Chinese name is pronounced approximately as ‘shyen’. The ‘x’ is not exactly like English ‘sh’, however.

     If you are interested, go here:

     http://www.mdbg.net/chindict/chindict.php?page=worddict&wdrst=0&wdqb=先

     and click on the double arrow button >> to bring up a menu with a speaker. Hit the speaker and it will pronounce the word for you.

     Reply ↓

     • Actually, I returned to this and noticed that Christian writes the name of his blog as “Xi’an” with an apostrophe in the middle.

       This is actually pronounced differently in Chinese from what I wrote above. It is the name of an ancient capital city in China. To here it spoken, go here:

       http://www.mdbg.net/chindict/chindict.php?page=worddict&wdrst=0&wdqb=西安

       and click on the double arrow button >> to bring up a menu with a speaker. Hit the speaker and it will pronounce the word for you.

       Reply ↓

       • Bill,
         My understanding (but I can’t locate the reference where I read it) is that Steve’s version of what Xi’an stands for (i.e., the first name Christian) is essentially correct.

       • Martha,

         I’m sure Steve is correct, but I think that Christian may have intended to pun on the Chinese place name (Xi’an) since his blog title has that apostrophe in that particular place, Christian is a world traveller and has an interesting sense of humor. Normally one would write ‘Xian’ to mean ‘Christian’ when using the ‘X’ abbreviation…no apostrophe.

10. How about this version?

    Original Email:

    Dear Professor Gelman,

    My name is [Name Treatment] and I am a student at [University Treatment].

    I [subject-Verb Agreement Treatment] to graduate programs in [Program Treatment], and I will be visiting Columbia in [Time Horizon Treatment] as part of my [Diction Treatment].

    I [Organizational Capacity Treatment] final dates, but [Confidence Treatment] email again before my trip and [Casualness Treatment] with you
    for [Time Demand Treatment] if [Qualifier Treatment] when I visit.

    [Deference Treatment].

    [Closing Treatment],
    [Nickname Treatment]

    Immediately upon receiving a reply to the original email, this email is sent back from the experimenter to the unwitting participant:

    Dear Professor Gelman,

    My name is [Real Name], and I am [title] at [Institution]. The email to which you just responded was part of an experiment I am running at the [Real Institution/Lab].

    I apologize that I deceived you, but the project required the small deception. Details of the experiment and hypothesis tests can be found here [link] and a project description is provided below. The linked page also includes a seal of authenticity from the [Real Institution IRB] which confirms their approval of the experimental design and also confirms that no further deception is occurring.

    I hope you will understand that your time was used for a legitimate research purpose. In order to compensate you for your time and for deceiving you, I will donate $3 in your name to a fund that will be used solely for the purpose of providing tuition and fee remittance for students from low income households [link].

    If you do not wish to have your responses included in our analysis, click the following link and your entire record will be removed from our database and the donation will not be made [link]. I ask that you do not do this, and allow us to use the information you have provided. Obtaining high quality data is, as you know, the foundation of any high quality quantitative research. I assure you that all identifying information will be purged from any publicly available data released as part of this project. The project privacy policy can be found here [link].

    Your colleague,
    [Real Name]

    *Description of the Project
    **Boilerplate IRB/Legal stuff.
    ***Re-copy the links to the study design, privacy policy and opt-out pages.

    Reply ↓

    • Jrc:

      Yes, this is much better, partly because it does not involve any scheduling and partly for the $3 (which I don’t think would be enough if the treatment were to perturb me but would be just fine in this case). Indeed, I wouldn’t see the need for the last part (“If you do not wish to have your responses included in our analysis . . .”).

      One reason that particular email bugged me so much is that I’ve been spending lots of time lately answering calls from criminals (or whatever you want to label them) who call our home phone claiming to need to fix our computer or some other scam. Which leads me to the choice of wasting time with these slimy people or else screening my calls and losing some actual personal content. The principle is the same—a public channel is being abused for private gain. The only difference is, in this case the private gain is some pair of bozos trying to get a publication on their C.V., at minimal effort and cost to themselves.

      Reply ↓

      • The allowing your responses to be excluded is standard practice especially when you have not given informed consent prior to collection. You are basically in that way given post hoc informed consent. And you always have the right to withdraw from any study at any time. In some cases when interviews are about particularly sensitive topics I’ve seen IRBs require a re-confirmation of consent at the end of the interview. However I think they should give the $3 no matter what since you gave them your time and data under false pretenses.

        The second fake email is totally unnecessary (unless as you said it’s part of a second even more sneaky Obedience like study where you think the study is about discrimination based on names, but it’s really about whether people will blog about the annoyance of participating in studies like this). When I was on the our IRB we would not have approved the second email.

        What they sent you doesn’t even meet what I would consider minimal IRB standards at our institution since they haven’t told you how you were selected. In this case I bet they spammed some mailing list rather than take a reasonable sample size given the lack of sophistication of the study. I like Phil’s idea, an then they actually could do repeated measures.

        Reply ↓

11. There is a huge lack of replication problem in the Social Sciences.

    “Now could you stop wasting all of our time?” is inaccurate and the whole description is unkind.

    Are you proposing that we never replicate because it is a waste of time?

    Reply ↓

    • Evelyn:

      I’m not sure what you’re saying? Are you implying that the host of the blog would be opposing replications?

      Reply ↓

      • I’m saying that calling studies ‘crap-ass’ is one thing, but calling attempts to replicate ‘Copycat’ is more emotional than helpful.

        Reply ↓

        • Maybe not the ideal wording but

          > In all seriousness, this is how the world of research works

          Unfortunately that is an excellent description of some of the groups I have interacted with.

          Replicate has many meanings and often this is not well discussed.

          The caveat JG Gardin used add to redone by an independent third party was something like given a real interest – was it something that _deserved_ redoing. Just redoing basically the same flawed study that produces the same misleading biased finding likely does more harm than good.

          An example I am aware was a researcher who asked me once – “Do you know a paper that argues that you can decided on whether to do fixed or random effects analysis in a meta-analysis with a test.” My answer was yes but its a flawed and problematic approach that seldom if ever should be used. He answered, “I know, but I just need something to get the paper by the journal reviewer, and that would do the trick’. They used the reference, the meta-analysis was published and given they were a well known researcher (now in the list of the top XXX biomedical researchers in the world), many others starting copying and using that method in their meta-analyses.

12. This reminds me of “Inception”. In a couple of weeks you will get a comment on this post saying something like “Thank you all for participating. That email was part of our field experiment about how people react [on blogs and forums] to [experiments performed on other people] about Chinese students who present themselves by using either a Chinese name or an Anglicized name.”

    Reply ↓

    • Based on Bem’s work Andrew probably received that email last year.

      Reply ↓

13. Pingback: A week of links | EVOLVING ECONOMICS

14. Pingback: Contacting Ph.D programs prior to application season

15. … and some will spam you with a mailer purporting to come from the state secretary of state.

    http://www.kbzk.com/news/stanford-apologizes-for-supreme-court-mailers/

    Reply ↓

16. Love it. Most researchers think about ethics in terms of whether harm is involved. Considerations of wasting people’s time rarely make waves. I’m glad you’re making waves about it.

    Reply ↓

17. Pingback: Was it really necessary to do a voting experiment on 300,000 people? Maybe 299,999 would've been enough? Or 299,998? Or maybe 2000? - Statistical Modeling, Causal Inference, and Social Science Statistical Modeling, Causal Inference, and Social Scienc

18. Wow, not even close to ethical. An IRB might not stop your study to mislead volunteers during a study. But they certainly don’t have the moral authority to “approve” interfering with and lying to people who did not volunteer for your study.

    Reply ↓

19. Pingback: Scientists behaving badly - Statistical Modeling, Causal Inference, and Social Science Statistical Modeling, Causal Inference, and Social Science

20. Pingback: Crowdsourcing Data Analysis 2: Gender, Status, and Science - Statistical Modeling, Causal Inference, and Social Science Statistical Modeling, Causal Inference, and Social Science

21. Pingback: Oh, it's so frustrating when you're trying to help someone out, and then you realize you're dealing with a snake. - Statistical Modeling, Causal Inference, and Social Science Statistical Modeling, Causal Inference, and Social Science

22. Hi ! The article based on this spam has now been published :
    http://www.sciencedirect.com/science/article/pii/S0022103116304620
    In Study 1, an email from a Chinese student requesting a meeting about graduate training was sent to 419 White professors with the name of the sender being varied (Xian versus Alex). Use of the Chinese name led to fewer responses and agreements to meet than using the Anglo name.

    Reply ↓

    • Baptiste:

      Awesome! This paper hits all the right notes:
      – Published in an Elsevier journal
      – Forking paths (“The preference for Anglo (Chinese) names over Chinese (Anglo) names was apparent among those high (low) in assimilationist and low (high) in multicultural ideologies.”)
      – Use of p-hacked findings to make general claims (“These findings point to an important interplay between partial intergroup membership and acculturation ideologies of perceivers in predicting bias.”)
      – And, ummm, I’m not sure what else: the article is paywalled and the hell I’m gonna pay $35.95 for an article that itself is based on data that I supplied for free!

      Maybe these people will hit the real jackpot and get a Ted talk. Also, now that Mark Hauser and Amy Cuddy have retired, I guess there are two open professorships at Harvard!

      Reply ↓