This post is by Lizzie.
Many years ago I received an email from some ecologists I had never met who wanted to use data from my PhD (remember my PhD field work that I told you about the other day)? Oh glorious day! Me and my shrubs were out in the sunlight of other researchers’ interests. And, better than that, all my data was already public so when they asked for the data I just pointed them to a link and wished them well. After some back and forth they asked me to read the resulting paper to make sure they had not mis-used or mis-understood the data from my PhD. I thought this unnecessary but read it anyway. I found nothing to complain about.
All I recall is that I didn’t love that they called the field site I worked at ‘a moonscape’ so I suggested they change it to ‘shrubscape.’ This was partly because I think Mojave desert when I think moonscape and I worked in a coastal habitat with more vegetation (see photo), but also because I was on a total kick of trying to get more folks using shrub as an adjective (I was thrilled when I published about ‘shruboreal arthropods’). This brings me to point 1 I wanted to make about data sharing today.
The idea that we should all contact the folks who produced data to make sure we use it correctly is misguided.
It’s a nice idea and I don’t want anyone to think I don’t enjoy meeting the people who produced data that I use (or really anyone uses, I watch movies to see the CERN people in their hard hats producing data and I would enjoy talking with those folks too) or that I don’t realize a lot of things can make data less straightforward (when I was a student we called these ‘demonic intrusions’). It’s that the idea leads to small (and more often less amazing) science. I think this is especially true in ecology where we spent decades sharing stories with each other of what we each found in our particular system (side shout out to Ryan Thum who will forever live on in my memory of him talking to me during the first year of my PhD and saying: if you truly want to advance ecology, then you should get two wheels, one labeled with different systems and one labeled with different questions. When each PhD student shows up they spin each wheel and that’s what they do their PhD on).
Think about how big the science you can do is if you require everyone who wants to use data produced by someone else to contact them. I definitely am not analyzing tree rings as a topic! (But shout out also to all the researchers who have all replied to our queries about their public data, including showing us photos of their western redcedar tree cores when we were incredulous that they could have 2 cm of growth in one year). No more global analyses of advancing plant leafout for the IPCC! There’s the obvious logistical problems of the effort to do this (including all the dead people who produced data that is now public), but also the issue of what we would ask them exactly. As a researcher who does large-scale analyses using other people’s data it’s on me to make sure I validate the data, understand the data and read all the metadata. Is this a way to skip those steps and just get an emailed approval check-mark? Or is there so often missing information—that I would somehow not find in the process of doing science (visualizations, analyses, etc.) and the researcher who shared the data would have forgotten to mention—that would derail the whole finding? I doubt it and that’s because of:
1) The person most likely to benefit from you posting your data with metadata is you. Future you. Because you will not remember all the details of how it was collected and which day someone turned off the automatic watering on your experiment and dried out your plants in a year, or two or ten.
So what are the odds you will be so good at helping others use your data in the future?
2) I don’t know so many cases where people screw it up. And by that I mean: I don’t know any cases. If you know of a good case tell me. And if you’re someone who is being told to worry about this, ask for the good examples of this problem. There must be a name—beyond the boogeyman—for this tendency by those opposed to data sharing to toss out the risks of data sharing that they themselves have never observed. It’s just out there in the ether, being so scary, like a camp story someone told them, that someone else told them about two campers … who were waiting for a friend … at night … in a dark parking lot…. Boo!
My other point (point 2) sort of relates to this boogeyman problem, which is the belief that data should be protected from ‘the public.’ This was a new one I did not know so many people were into until recently and so I am not (yet?) sure how to combat it (I am open to ideas). It seems pretty obvious to me that data generally thrives in the sunlight.
Open data is a major tenet of the open government movement that many democracies have signed onto (and, must I say this?, open data is not a major policy of autocracies) based on the reality that open data makes discriminatory policies obvious (e.g., redlining). There are realities where public government data has been misused to harm minority groups, but most often the reverse is true. Closed data makes it easier for governments to unequally distribute resources and hide other nefarious actions. So I am nervous about how many people seem quick to invoke minority rights and concerns to back up government policies to keep data closed—with zero evidence to support their supposition (is there a term for this? There should be). Shouldn’t our first reaction be that data should be open without good evidence to the contrary?
There are gradients, slopes, and things to simulate
There are things to model, compute, and estimate
Typing and computing on a laptop
At a hill,
on the top
But when you look around and away from the screen
Gradients and slopes can also be seen
I see flowers, bushes, and many colors including yellow, white, brown, grey, and green
All interspersed with rocks, it’s quite a scene
Perhaps it’s good for something to sometimes look past the screen
If only to better know where you are, and where you have been
I agree that our default position should be that data are open. There are, however, limited cases where a degree of data protection is necessary. In ecology one case where limiting or obscuring data is when it involves geographical coordinates of rare or endangered species, especially if they are species subject to strong collection pressures, e.g., orchids and succulents. Databases (like iNaturalist) that make observations freely available apply some degree of “fuzz” in reporting the location of sensitive species. In human genetics access to data is commonly restricted because it is often possible to reconstruct individual information even if the data have been anonymized. Gaining access requires demonstrating a sound scientific purpose and agreeing to restrictions on how the data can be reported. There’s always the question of whether the authority granting access is legitimate, but I don’t see an alternative.
One interesting thing I’ve noticed in linguistics is that if I ask certain people for their data, they want to become co-authors on any paper that results. Basically, the motivation seems to be that once they created the data, they can keep being co-author on any paper based on that data. A good way to boost publication counts, but not a good way to allow independent investigation of published data. Or maybe the motivation is to prevent the authors from debunking a claim made based on the data by the original author(s).
In any case, I have less than 25% success in getting data from people in psycholinguistics. One author even wrote in their paper that they were willing to release their data, but only to “competent” researchers; I think the authors get to decide who is competent enough to get the data :)
In healthcare, it’s rare to get IRB clearance to publish your data. Ethics committees are generally trying to balance the risk of the data being misused (usually low) with the benefit to it being available (amorphously positive) and lean towards hedging on the former.
What I find interesting, or maybe depressing, is that rather than needing to worry about the data being used for corporate reasons unrelated to benefitting science, I’m now worried about open data being used to actively harm patients through bad faith publications or political reports. It’s a strange time to be in science.
Lizzie said “I don’t know so many cases where people screw it up. And by that I mean: I don’t know any cases. If you know of a good case tell me.”
I’m not sure what your criteria are for a “good case” (one imagines there is a sliding scale of error magnitude, and that error magnitude must interact with the tolerance for error relative to the scientific inferential target of the research), but it is pretty common to see folk misinterpret spatial data on GBIF. One common error is to ignore that uncertainty relative to a centroid point estimate; another is to assume that the uncertainty is the radius of a circle when originally it was a grid side length.
I’m not arguing that data shouldn’t be open by default by the way, I’m just saying that misinterpreting others’ data is commonplace in ecology.
Another is not understanding others’ taxonomic concepts adequately.
> misinterpreting others’ data is commonplace
I’d say this is true not just in ecology, but in general. However, I agree with Lizzie that this is not a good reason to restrict access to data. Just the opposite: making data openly available makes it easier to detect and correct misinterpretations of data, particularly when the original authors are the ones who did the misinterpreting.
To Robin’s point, I share the concern that data is often used in bad faith. However, as we saw with last night’s blathering about autism and Tylenol, people who will act in bad faith will do so with or without data they can misrepresent. So again, I don’t think this is a reason to restrict access to data.
While I fully endorse Lizzie’s position on data availability, I do have an example of a potentially very consequential misinterpretation of ecological data. The story comes from the irrepressible Jim Bouldin here:
https://ecologicallyoriented.wordpress.com/2024/04/25/experiences-with-article-retraction-attempts-at-esa-journals/
The paper in question, Williams and Baker 2011, has not been retracted and only has a minor correction notice (although the basis of the correction – oops we published a bunch of equations that were different from the ones we used – is ugly in its own right).
While Jim’s critiques unfortunately do not appear in the literature, here is a paper debunking the Williams and Baker paper:
https://www.fs.usda.gov/psw/publications/north/psw_2017_north003_levine.pdf
I mentioned that the study was consequential because it suggests an entirely different strategy for management of forests in the eastern Sierra Nevada of California, based on bungled data analysis of publicly available data.
On the flip side, had the data never been available, there would be one less tool for estimating historical forest coverage, and the ongoing debate about GLO tree density data would never have happened.
That is somewhat of “just the tip of the iceberg” related to heterodox papers regarding fire and fire impacts in western frequent low-mixed severity regimes. Their appears a cabal (of which Bill Baker is a member) routinely contesting the impacts of fire exclusion on forest structure in fire adapted forest of the west. A notable example, coming out of the recent spate of fires killing old-growth sequoia exposed equally weak methods (in this case wrt seedling density).
https://www.usgs.gov/publications/effects-recent-wildfires-giant-sequoia-groves-were-anomalous-millennial-timescales-a
Methods in field ecology can be maddening, but I appreciate this thread for its content and as it brought me back to great memories of kneeling in the sagebrush steppe of eastern-Oregon and measuring bunchgrass dimensions.