This is Jessica. Recently a team associated with the machine learning conference Neurips released an interesting document detailing a code of ethics, essentially listing concerns considered fair game for critiquing (possibly even rejecting) submitted papers. Samy Bengio, Alina Beygelzimer, Kate Crawford, Jeanne Fromer, Iason Gabriel, Amanda Levendowski, Inioluwa Deborah Raji, Marc’Aurelio Ranzato write:

Abstract: Over the past few decades, research in machine learning and AI has had a tremendous impact in our society. The number of deployed applications has greatly increased, particularly in recent years. As a result, the NeurIPS Conference has received an increased number of submissions (approaching 10,000 in the past two years), with several papers describing research that has foreseeable deployment scenarios. With such great opportunity to impact the life of people comes also a great responsibility to ensure research has an overall positive effect in our society.  

Before 2020, the NeurIPS program chairs had the arduous task of assessing papers not only for their scientific merit, but also in terms of their ethical implications. As the number of submissions increased and as it became clear that ethical considerations were also becoming more complex, such ad hoc process had to be redesigned in order to properly support our community of submitting authors.

The program chairs of NeurIPS 2020 established an ethics review process, which was chaired by Iason Gabriel. As part of that process, papers flagged by technical reviewers could undergo an additional review process handled by reviewers with expertise at the intersection of AI and Ethics. These ethical reviewers based their assessment on guidelines that Iason Gabriel, in collaboration with the program chairs, drafted for the occasion. 

This pilot experiment was overall successful because it surfaced early on several papers that needed additional discussion and provided authors with additional feedback on how to improve their work. Extended and improved versions of such ethics review process were later adopted by the NeurIPS 2021 program chairs as well as by other leading machine learning conferences. 

One outstanding issue with such a process was the lack of transparency in the process and lack of guidelines for the authors. Early in 2021, the NeurIPS Board gave Marc’Aurelio Ranzato the mandate to form a committee to draft a code of Ethics to remedy this. The committee, which corresponds to the author list of this document, includes individuals with diverse background and views who have been working or have been involved in Ethics in AI as part of their research or professional service. The first outcome of the committee was the Ethics guidelines, which was published in May 2021.

The committee has worked for over a year to draft a Code of Ethics. This document is their current draft, which has not been approved by the NeurIPS Board as of yet. The Board decided to first engage with the community to gather feedback. We therefore invite reviews and comments on this document. We welcome your encouragement as well as your critical feedback. We will then revise this document accordingly and finalize the draft, hoping that this will become a useful resource for submitting authors, reviewers and presenters.

The idea that AI/ML researchers need to take certain values (fairness, environmental sustainability, privacy, etc.) seriously so as to avoid real-world harms if their tools are deployed has been pretty visible in recent years, but these conversations have been mostly “opt-in” until recently. Documents like the code of ethics and requirements like broader impacts statements attempt to introduce values that haven’t been baked into the domain the way that caring about scalability and efficiency are. Naturally things get interesting. It’s hard not to acknowledge that knowledge of how to evaluate or deliberate about ethics is sparse (not to mention how to institute guidelines). Some adamantly oppose any ethics regulation on this ground, or for other reasons, e.g., arguing for instance that tech is neutral or that the future is too unpredictable to evaluate work on downstream consquences. Documenting what constitutes an ethics concern at the organization level requires deciding where to draw lines and what to leave out in a way that the individual papers about ethical concerns don’t have to contend with. I’ve heard some people refer to all this as a power struggle between the more activism-minded critics and those who are content from having benefitted from the status quo.

At any rate, among ML conferences, Neurips organizers have been pretty brave in stepping up to experiment with ways to establish processes to encourage more responsible research, whether everyone in the community wants them or not. The authors of this doc took on a difficult challenge in agreeing to write it. 

My initial reaction from reading the code of ethics is that if there is a power struggle around how big the role of ethical deliberation should be in AI/ML, this is a rather small step. It’s an interesting mix of a) what seem like basic research integrity suggestions that should not ruffle any feathers and b) Western progressive politics made explicit. I have more questions than strong opinions. 

The biggest question, as some of the public comments on the draft get at, is to what extent the guidelines can be interpreted as suggestions versus mandates. It seems reasonable to think that if Neurips is going to do ethics review, there should be some log somewhere of what kinds of things might lead to an ethics flag being raised that is taken seriously in review, and the abstract above makes clear this was part of the motivation for the code of ethics. But being transparent about the content of the guidelines is not the same as being transparent about the intended use of the guidelines. I think the authors do attempt to clarify – they write for instance that “this document aims at supporting the comprehensive evaluation of ethics in AI research by offering material of reflection and further reading, as opposed to providing a strict set of rules.” At the same time, most Neurips authors know by now that it is within the realm of possibility that their paper submission could be rejected if there are ethics concerns that aren’t addressed to satisfaction, so naturally they are going to pay attention to the specific content. 

From my read I expect that this code is not meant to have the same power as the ethics reviewers themselves; its a reflection of stuff they care about that can change over time but which will probably always be incomplete and secondary to their authority. Language used throughout the document makes it read like a tentative set of suggestions. There’s a lot of “encourage”, “emphasize,” “to the extent possible” to imply it’s more like a wish list on the part of Neurips organization. Though there are a few sentences that use slightly stronger terms, e.g., saying the the document “outlines conference expectations about the broad ethical practices that must be adopted by participants and by the wider community, including submitting authors, members of the program committee and speakers.” There’s also a reference to the code of ethics being meant to complement the Neurips code of conduct, which as far as I know is treated more like a set of rules that can get one reported, asked to leave events, etc. So not surprising to see some confusion about what exactly its role will be.

When it comes to the specific content, many of the suggestions about general research ethics seem like fairly well-accepted best practices: paying participants (like in dataset creation) a fair wage, involving the IRB in data collection, trying to protect the privacy of individuals represented in a dataset, documenting model and dataset details (though I’m not sure templates are necessary, as they suggest, so much as making the effort to record everything that might be important for readers to know). 

Suggestions about research ethics that struck me as slightly bolder include aiming for datasets that are representative of a diverse population and using disaggregated (by sub-population) test sets when reporting model performance. I guess fairness has graduated from being a sub-area of AI/ML to being an expectation of all applications. I wonder about a weaker suggestion that could have been made, that authors simply be more mindful in making claims about what their contributions are capable of (e.g., don’t claim it can detect some attribute from images of people’s faces if your training and test data only included images of white people’s faces). The idea that it’s not enough to ask for transparency and accurate statements about limitations, that instead we should evaluate the potential to do good for the world, is admittedly where I struggle the most with ethics reform in CS. The less value-laden premise that researchers should be transparent and rigorous in their claims seems at least as important to express as a value. Do these exist somewhere else? I glanced at the code of conduct but that seems more about barring harassment or fraud.

The second part of the code provides guidance on how authors should reflect on societal impact. It says that all authors are expected to include reflection on societal impacts in a separate paper section, so that reads like a requirement. The high level idea that authors should to the best of their ability try to imagine ways that the technology might be used to cause harm does not seem contentious to me. Requiring sections in all papers on societal impacts does however raise the question of who they are for – readers, authors, both? When I first heard proposals a few years ago that AI and ML researchers need to be stating broader impacts of what they build, I wondered why anyone would expect such reflection to be useful, given how much uncertainty exists in the pipeline from research paper to deployment, and the fact that computer scientists have often taken very few humanities classes. Having paid attention to attempts to codify ethics over the last few years, I now have the impression that the motivation behind requiring these statements is driven more by a desire to normalize the idea of reflecting on ethics, i.e., to encourage awareness in a general way rather than producing concrete factual statements. Still, it’s often hard in reading these proposals to tell what the ethics advocates have in mind, so how the expectations about downstream impacts are phrased seems important. You don’t want to imply predictability so much as an expectation that researchers will not intentionally produce tools for societal evil. 

Related to this, early on the doc states that:

In general, what is legally permissible might not be considered ethically acceptable in AI research. There are two guiding principles that underlie the NeurIPS Code of Ethics. First, AI research is worthwhile only to the extent that it can have a positive net outcome for our society. Second, since the assessment of whether a research practice or outcome is ethical might be difficult to establish unequivocally, transparent and truthful communication is of paramount importance.

“Positive net outcome” is a strong statement if you try to take it literally as a criteria we should use to judge all research in the near term. The second principle seems to acknowledge this, so I doubt the authors intend this phrase to be taken very seriously. Still the wording does make me cringe a little, since it implies that there is some ultimate “objective” perspective on what is positive versus negative. And the word “net” makes it seem like something that’s at least theoretically measurable, like we can calculate the definitive goodness of a project, it’s just that we can’t observe it perfectly as humans. My mind inevitably goes to wondering how many research projects submitted to Neurips explore techniques that don’t feed directly into real world deployments, but maybe help the researchers learn important lessons they apply in later projects, but could still be said to have opportunity costs in terms of taking effort away from what seem like more obviously beneficial applications (say, more efficient methods for diagnosing certain illnesses)… etc. Researchers can be pretty good at making up problems, or exaggerating problems we think we can solve, and we can learn from these projects, but the impact is sort of meh. I feel that way about some of my own work actually! I would hate to have weigh in on whether it was “net positive” or not. I don’t think the authors are advocating for thinking like this, but I guess I would prefer to avoid the positive / negative labels altogether and frame it more generally as an expectation that researchers will be careful in considering and acknowledging risks. 

One guideline in the societal impacts section that stood out to me is worded less like a suggestion:

Research should not be designed for incorporation into weapons or weapons systems, either as a direct aim or primary outcome. We encourage researchers to consider ways in which their contribution could be used to develop, support, or expand weapons systems and to take measures to forestall this outcome.

It would seem from this that one should expect mentioning weapons as an application to get their paper flagged for ethics review. I wonder here how the authors decided that weapons systems were off limits but not other uses of tech. I’m not disagreeing, but I’ve seen people call for bans on other types of applications, carceral for instance, so I wonder why only weapons system are ruled out. In line with transparency, it would be great for Neurips to also publish more on the process for producing this document: what were the criteria that the group had in mind in thinking through how to write this guidelines, including both what types of concerns to include and how to phrase the directives? I imagine there might be some lessons to be learned from reflecting on the process for creating guidelines like this, where there may have been disagreements, etc. 

It seems like it could also be an interesting exercise to collect many versions of a document like this, representing what different members of the Neurips community think is worth explicitly citing as community guidelines that could inform issues flagged for ethics review. There are some comments about how the vision in the document doesn’t really capture an international perspective as much as a Western one, for instance. It makes me wonder more about what the part of the abstract that says the committee includes “individuals with diverse background and views,” like along what dimensions views differed.