Aki, Richard, Lizzie, and I put together a special issue on Statistical Workflow for the Philosophical Transactions of the Royal Society. I guess “royal” isn’t as impressive as it used to be, but still.
Statistics and data analytics play an increasingly important role in and across science and policy. But much of what is done by the best practitioners–their “workflow”–is tacit knowledge only glanced over in textbooks and research articles. In this new collection covering a wide range of disciplines, leading statisticians and researchers discuss the motivations and details for their workflows.
The four of us did this project because we were all interested in Bayesian workflow, and we wanted to learn more about statistical workflow in general, not just the Bayesian part.
Here’s what’s in the issue:
- Statistical workflow, by Andrew Gelman, Aki Vehtari & Richard McElreath
- Unsupervised machine learning for scientific discovery: workflow
and best practices, by Andersen Chang, Tiffany M Tang, Tarek M Zikry & Geneva I Allen - PCS workflow for veridical data science in the age of AI, by Zachary T Rewolinski & Bin Yu
- Simulations in statistical workflows, by Paul-Christian Bürkner, Marvin Schmitt & Stefan T Radev
- An automatic finite-sample robustness metric: when can dropping a little data change conclusions? Part I: definitions and experiments, by Ryan Giordano, Rachael Meager & Tamara Broderick
- An automatic finite-sample robustness metric: when can dropping a little data change conclusions? Part II: theory and intuition, by Ryan Giordano, Rachael Meager & Tamara Broderick
- Building a Backdrop of Meaning in Magnitude (BoMM) as part of research workflow, by Megan Dailey Higgs
- A preliminary data analysis workflow for meta-analysis of dependent effect sizes, by Elizabeth Tipton, James Pustejovsky & Jingru Zhang
- A four-step simulation-based workflow for ecological analysis and science, by EM Wolkovich, T Jonathan Davies, William D Pearse & Michael Betancourt
- Scientific workflow in experimental economics, by Anna Dreber & Séverine Toussaert
- Hidden processes of workflow in cognitive developmental psychology, by Lauren N. Girouard & Susan A. Gelman
- Reproducible workflow for online AI in digital health, by Susobhan Ghosh et al.
- Model checks for Bayesian estimation and forecasting of health coverage indicators in low- and middle-income countries, by Leontine Alkema et al.
- Closing the gap between statistical and scientific workflows for improved forecasts in ecology, by Victor Van der Meersch, James Regetz, T Jonathan Davies & EM Wolkovich
- Machine learning workflows in climate modeling: design patterns and insights from case studies, by Tian Zheng et al.
Lots of good stuff here, and lots of different perspectives. Thanks to all the authors. The issue is here, and all the papers should be freely available.
If you have any thoughts on the articles in the volume, or on any other statistical workflow topics, just let us know right here in the comments box.
Thanks for making these papers freely available.
It would be very important and useful to develop a workflow for measurement. Generalising from indicators to a latent construct is also an iterative process that would be better served with a proper workflow.
The emergent consensus around model-based measurement is probably the best starting point but applications remain rare. For example, poverty measurement is a mess but there are some promising studies connecting relative deprivation theory with latent variable methods to compute random and systematic errors. But it is still underdeveloped. I’m sure there are plenty of fields that would benefit from workflow development with examples in Stan using IRT, CFA, Invariance analyses, etc.
Good luck modelling that crazy bird in the cover photo! But the impala and zebra look very countable. Impala counts have been going up since the introduction of man-made watering holes in many African parks (like the one shown here); impala are less drought-tolerant so they benefit, but the more drought tolerant species have declined.
I think that crazy bird might be a bird that was on the back of the jumping impala and had to quickly decide to fly off which is why it kind of looks crazy.
A little search just now made me aware of the fact, if I am understanding correctly, that the impala has a symbiotic relationship with a so-called oxpecker. But please check and verify, I just read some stuff just now about this all.
I think I can see another bird, possibly an oxpecker, on the back of the fourth impala counting from the one jumping at the front of the line.
Good call on the bird on the fourth impala back, and the symbiotic relationship. There’s a yellow-billed and a red-billed oxpecker that pick off parasites, but the yellow-billed is much rarer.
My colleague who worked on the camera trap project that this came from does not think the flying bird is an oxpecker…. I just reached out to the person who took the photo in case he has guess.
I saw pictures of the yellow and red-billed oxpeckers when I looked it up and noticed I couldn’t see either color on the bird in the picture. I conjectured that there could be more types of oxpeckers, or some other bird species that hang around on the back of the impalas.
I zoomed in on the photo of the crazy bird just now as a result of your comment, and it looks to me like it has its feet tucked in to its body and both of the wings are positioned similarly (out to the back?). I conjecture that could fit with my hypothesis that the bird had to get of the impalas back because the impala jumped. Perhaps all it needs to do when the impala suddenly jumps is to jump slightly or simply tuck in its feet (which could fit with the tucked in feet in the picture), and position the wings to keep in the air (which could fit with the position of the wings in the picture).
Alternatively, some bored animal which is out of shot decided to practice their field-goal attempts using the poor bird when spotting the impala’s horns. If this is an option, a juvenile meerkat would be my best guess as being the culprit.
Just checked out some more photos of oxpeckers, and it looks like there are some with less colorful beaks. Perhaps a male-female thing, or certain varation. I also read a description, and saw pictures, that indicate that the underbelly is lighter in color. I zoomed in on the crazy bird, and the one on the other down the line, and both seem to me to have a lighter underbelly. It seems to fit with the hypothesis that the crazy bird is an oxpecker, who possibly jumped of the back of the impala that’s jumping in the front of the picture.
If it’s not a bird, it might be some other lifeform that may come from the UFO that can be seen in the sky above the impala’s horn on the right side above the zebras.
I also read just now that some hosts don’t like it when the oxpecker lands or is trying to land. Perhaps that explains the impala jumping. I don’t know anything about statistical modeling, but given your earlier comment about modeling the crazy bird, there must be some formula or odds-calculation or whatever term is appropriate that can be made somehow concerning the odds that the bird is an oxpecker given the information that came from the searches and results mentioned:
-oxpeckers hang around impalas
-oxpeckers have a ligher underbelly
-some hosts don’t like it when the oxpecker lands or is trying to land on its body
-the birds close to the impalas in the picture seem to have a lighter underbelly
-the impala in the picture is jumping
-the bird in the picture is close to the back of the impala and has its feet tucked in and its wings to the back
I’m a juvenile oxpecker, that’s why there’s no color on my beak
It may also help explain why my behavior might have been less than sleek
I thought pecking the impala’s back was the thing to do, that it was “tres chic”
But I’ve now learned when they are drinking to best keep still and to not peck with my beak
For some reason, the article by Megan Higgs is not freely available.
I’ve only just found out about this and am enormously pleased that (1) this collection is out there, and (2) this is the kind of blog where we not only learn about the crazy bird (of course I thought at first I was only one to notice it) but also hear from the person who took the photo itself.