Ethics and the Replication Crisis and Science (my talk Tues 6pm)

I’ll be speaking on Ethics and the Replication Crisis and Science tomorrow (Tues 28 Feb) 6-7:30pm at room 411 Fayerweather Hall, Columbia University. I don’t plan to speak for 90 minutes; I assume there will be lots of time for discussion.

Here’s the abstract that I whipped up:

Busy scientists sometimes view ethics and philosophy as “touchy-feely” concern that scientists worry about only after they are too old to do real research. In this talk I argue that, on the contrary, that ethics and philosophy are practical tools that can make us more effective scientists. Many of the traditional discussions of statistical ethics are outdated, but we can move to a more modern understanding of ethics in statistics—and in science more generally—by looking more closely at the goals and practices of quantitative research. The current replication crisis in science motivates much of this discussion, but our discussion will consider broader issues too.

13 thoughts on “Ethics and the Replication Crisis and Science (my talk Tues 6pm)

  1. Waiting till you are old to dabble in statistical philosophy / ethics may be a feature instead of a bug.

    If you’ve never gotten your hands dirty doing science it gets artificial to do the philosophy / ethics of science.

    • I may be overstating this a little, but it’s my sense that the reason I never really got traction in social science/management studies, was because I started with a pretty well-developed philosophy and ethics. (I got my MA in philosophy, specializing in the philosophy of explanation.) For this reason I never really “got” what social scientists were doing. They weren’t just getting their hands dirty, I thought, they were often getting blood on them. They also never really got my criticism, since I seemed to be (and sometimes was in fact) denouncing the received practices in the field. That was before the replication crisis. At this juncture, I don’t what I would recommend to early career researchers. After all, in order to get their hands dirty (and get published), they seem bound to engage in some already, or soon-to-be, discredited practices.

      • I think it is perfectly fine (even advisable) to *learn* ethics at an early state. Or even philosophy.

        I guess what I was trying to say was it may be counterproductive to try and make contributions to the fields of ethics / philosophy *before* one has gotten one’s hands dirty doing science. IMO philosophers that make this mistake tend to dwell too much in the abstraction which is divorced from the realities of practice.

        By all means learn the tricks of how not to get your hands bloody early in your career. But don’t attempt to write a novel treatise on safe-tool-use unless you’ve spend a decade getting your hands dirty actually using the tools.

        • Rahul,

          Your argument suggests it is impossible to see the problem without that level of experience, which is simply ridiculous. It also suggests that the only source of a solution is from within a system that is built around a corruption designed to defend the illogic of said system.

          I don’t find this type of argument compelling within any domain. It is the equivalent of saying, “We do it this way because we have always done it this way, unless I decide not to do it this way.”

        • It isn’t a generic argument. I’m just saying that in the science philosophers I see today experience in doing science seems to help. Just an anecdotal observation.

        • If people are becoming “science philosophers” without actually working in science, there’s a problem (unless they focus, say, on ancient science). But if non-scientists are taking interest in the issues without claiming expertise, that seems fine.

        • I do agree ” philosophers … tend to dwell too much in the abstraction which is divorced from the realities of practice” part of the same phenomena that drives the replication crisis in other disciplines (Susan Haack has written on this).

          But where have you guys been?

          In my first professional engagement as a statistician I was told I had to find an analysis of the data set that made the research look like a success or they threatened they would ruin my career.

          This happened a couple weeks before I started in the Biostatistics program at U of T when the administrator/programmer of the Stats Consulting Center asked if I could do this analysis for a researcher that needed it right now. Why were they hiring me out as an expert? (I read through Stephen Fienberg’s Analysis of Cross-classified Data to do it – a wonderful book especially given I could read through all the 150 pages while doing the analyses.)

          The in the next summer, one of the other students in Biostatistics revealed that they had been asked to alter the data – and did – thinking that was just one of things one had to go along with.

          OK selective sample but I do think most working in any statistical capacity run into ethical stuff very early in their careers.

          At least if they have the right training to recognize it (e.g. should they know to verify the data quality and ask about how the data came to be [the first third times we did this experiment it was not promising but this last run really seemed good so we just brought that data forward] or ask about how many similar studies have been published, ect., ect.

  2. My interest in this topic is purely personal. I have no professional qualifications at all in statistical research, other than a few basics that I remember from my undergraduate university courses. My interest in this started when I was trying to decide how much confidence I could have in projecting the opinion percentages from popular opinion polls onto the target populations. Intuitively, I suspected “zero,” but I’ve been trying to clarify my reasons for that, and to get some ideas from some researchers in survey methodology.

    After a few days of poking around, I finally came up with some fruitful search terms, like “researcher degrees of freedom” for example, and in the process I learned about what people are calling a “replication crisis.”

    I agree with Mr. Gelman and others that it might be unfair, and even backwards, to single out psychology in discussing the problem. It even seems to me to do credit to the field, that it was in the front lines of publicizing the problem. I also agree with Mr. Gelman and others, in treating it at least partly as an ethical issue. In fact I would go beyond that and say that it’s part of a widespread and growing moral and spiritual crisis in all of society, affecting all professions and all the institutions of society. I don’t think it’s all, or even mostly, in the morality of researchers.

    I’ll say what I think is an indispensable part of the solution, that anyone who wants to can do to help:

    1. Practice and promote continual self-improvement for the benefit of all people everywhere.

    2. Help with the growth and spread of healthy community life at the grass roots level, in every corner of the world and every corner of society.

Leave a Reply to Samoylovich Cancel reply

Your email address will not be published. Required fields are marked *