Feedback on the blog—this is your chance!

Posted on by

Thanks to all of you, we get lots of great discussion in comments on specific posts.

Here I wanted to give youall a chance to share any thoughts you had on the blog more generally. No need here to say how wonderful it is—please just jump right in with your (polite) complaints, criticisms, suggestions, etc.

Go for it! We want to hear what you have to say.

P.S. Please focus your comments on the content of the blog (you’d like more posts of type X, you’d prefer some Y, could we cover Z, etc.), not on issues on which I have less control such as problems with logging in, RSS feeds, etc. Thanks.

74 thoughts on “Feedback on the blog—this is your chance!”

  1. WordPress has no preview function (there are plugins for this though) and it can be hard to put up code or math.

    Also there’s no edit function.

    One real annoyance is that the blog doesn’t remember who I am, so I have to type my name and email every damn time.

    I enjoy discourse more than I do WordPress because it solves all of these issues. But it doesn’t handle threaded discussions. Everything is in one long list. Still the quoting feature seems to be enough to keep people discussing without confusion.

    • Just saw your PS put up after I posted, thanks for the clarification. Here’s some thoughts on content:

      I’d like to see us call out more well-done science and discuss what makes it well done. You did a call for this at one point and I submitted that article where they studied “ultraprocessed foods” including putting people in a calorimeter room and such. There was later some followup about whether the data was fudged, and we came to the conclusion that there were different instruments used to weigh certain things and it was all real data but with differing roundoff (if I remember correctly).

      There’s gotta be more than one really well done study in the last decade though, so maybe we could call out some of them?

      I would like more on topics that have potential to affect large groups of people in real-world ways. To me that means medical, economic, environment, climate, food safety, things like that. Also if there’s something in this category of real-world consequences I’d like to see the blog delay be something more like 1 month than 6. It seems good to give things a little time to shake out in the news, but 6 months out the people who might engage the criticism or concerns or supply relevant data and etc may be kinda off the topic. 1 month ish seems like a good compromise lag.

      • Just a +1 to talking more about good examples of well-done work, particularly in the social sciences, and particularly when study isn’t just well done but well-communicated about (maybe I’ve described a unicorn?).

        And I am acknowledging your P.S., but it would make a huge difference if you had a different commenting system (even a Disqus plug-in, which is compatible with WordPress and just essentially plug and play). For more active discussions, threaded conversations are almost impossible to follow (and people often put things in the wrong thread because of that), and there’s no easy way to search comments (which is too bad! A lot of the exceptional content on this blog is in the discussion, and I’m often trying to find a reference someone pointed to from months before).

        • Regarding upvoting: -1000 in my view. Same for downvoting. That is one “feature” of Microsoft Teams that is the most irritating (and ironic since it is presumably software for businesses to use). I don’t want my students or colleagues providing up or down votes – it turns content into a popularity contest and a diversion from the content. If you want an example, look at what it did to Marginal Revolution. It took a blog that already had issues with trolls, uncivil discourse, and way too many links to Twitter (and now X), and added the totally worthless voting procedure. There are then even discussions about why there are so many up or down votes on a particular comment – usually a comment that was worthless to begin with.

        • @Dale Lehman I’m somewhat sympathetic to your point since not many people actually comment here, so it can be easy to read them all. But it’s useful on other substacks and blogs to sort by the best comments as measured by some function of upvotes or downvotes. It lets you to quickly see if someone already made the point you wanted to make so you don’t have to repeat yourself. It also lets you skip over the comments with fewer net votes.

        • Nuanced comments get few votes – extreme and hyped statements attract attention, but up and down. We have too much of that already in life, I value this blog for having relatively little. If we had voting (thumbs up or down), I’d be inclined to look at the comments that don’t get attention! I prefer Andrew’s annual tally of the posts that got the most attention. That provides some interesting feedback on the interest that various types of content attracts rather than focusing on the individual comments.

        • I have to agree with Dale on this one. Upvoting and downvoting with respect to what criteria? Also consider objective comments that don’t necessarily induce a sentiment may not get a thumbs up or down. Naturally this would put them at ‘0’, whereas a comment that does induce a sentiment will ‘score’ one way or the other. So within the notion of what the ‘best’ comments are – what does this even mean? Do we need referees for content to avoid a popularity slog? Last I checked, the need for referees was imbued in a published journal – and that’s hardly a popularity contest ……. oh wait ….

    • @Daniel Lakeland:
      My browser (Firefox) remembers who I am. I type “R” in the name field and it suggests my handle. A small change in your browser’s configuration could save you some typing.

      • Yeah, Firefox on Linux desktop does that too, but on mobile it doesn’t. Annoying.

        Back in the day when I clicked comment it was just filled in automatically, I think via a cookie. I suspect the cookie thing got removed due to GDPR or some such thing. It might be nice if there’s a plugin for remembering your login.

    • Some time ago (I forget when), Bob Carpenter mentioned to me that while markdown wasn’t explicitly supported, one of the flavors of inline LaTeX was (MathJax, whatever – it was pretty minimal at that time). That may have been before the blog received a facelift however, he may be able to answer that one better. I do agree that if someone intends to supply an argument with symbolic algebra, then enabling/retaining/documenting this feature is one that supports content. Statistics is naturally more than about the maths, but it is certainly ground in it and sometimes arguments using symbologic structure can be quite useful.

  2. I found the “Future of Statistical Modelling” Substack blog annoying and confusing.
    When I originally saw it advertised, I admit I skimmed the post, but it was advertised as an “alternative to RSS feeds”.
    But RSS feeds tell you what you can read *now*.
    The Substack tells you what you *can’t* read now – what’s being unnecessarily withheld from you for another week.
    It’s like going to a restaurant where you get to smell the food but not taste it – there’s no way for that to be a satisfying experience.

    I would, instead, have really loved it if the blog posts mentioned there went to *early drafts* of the blog posts. So if you *want* to read something unpolished, unfinished, and subject to change, you can go there if you’re really curious, or you can wait for the finished product.
    Bonus points would be if the constructive comments on the early-draft blog posts actually improved the finished product meaningfully.

    But as it is, I find the “Future of Statistical Modelling” Substack incredibly annoying and don’t open those emails anymore.

  3. More content, generated via recruitment of guest posts and, after a trial period, new bloggers, mostly with well-defined “beats.”

    The main thing I want from this blog is “more of the same,” but I recognize that there are only so many hours in the day and that there are many demands on Andrew’s time. But, although no one could do posts like this or this or this as well as Andrew, there are plenty of people who could do 80% as well, and would be eager to write for this blog.

  4. Despite your request, I’ll note that the blog is wonderful! There’s no need to change anything related to content, and you should feel free to write about whatever you like. I really like the posts about books, though they tend not to get many comments.

    That said, some suggestions:

    Squash more of the ranting comments — repetitive, redundant, or off-topic. It helps that commenters uses consistent screen names, since there are several I’ve learned to immediately skip, but still it fills the page and makes reading and constructively commenting harder. This is my main suggestion.

    I like Daniel’s idea about discussions of well done studies.

    It helps a lot that the writers who are not you, Andrew, are consistently starting the posts with “This is X.” There are occasional very infrequent writers who could use some feedback that their posts are rather incomprehensible and unclear, especially lacking context for a general audience. (This doesn’t apply to Jessica Hullman and Bob Carpenter!)

    “And also, you should win things by watching!”

    • “Squash more of the ranting comments — repetitive, redundant, or off-topic. It helps that commenters uses consistent screen names, since there are several I’ve learned to immediately skip, but still it fills the page and makes reading and constructively commenting harder. This is my main suggestion.”

      rep(‘+1’, 20)

      • I (trepidatiously) agree with the above folks about squashing ranting comments. I feel particularly turned off by comments that basically consist of personal feuding/bickering. (‘Joshua’ is a frequent offender.) You might consider deleting these comments or even publicly replying to these people asking them to stop. I suspect the latter strategy would actually be effective and might help set a tone for other observers…

        • You’ve demonstrated here exactly why squashing ranting comments is easier in theory than in practice. Often I find Joshua’s comments interesting and informative, so what counts as ranting is obviously quite subjective. As Andrew’s blog then, sure, he can be the arbiter, but, even so I can see it getting messy, quickly. (And sometimes Andrew does ask some commenters to cease and desist).

  5. I drafted this before I saw your P.S., so I will just send it off.
    Occasionally some of my comments get swallowed up by the spam filter. I suppose if it contains more than two links, it goes straight to moderation? I understand that this is a good policy to keep the spammers at bay. Perhaps there is a way to whitelist links to your own blog? True story: I have written comments linking to two of your posts, only to have them devoured by moderation, never to be seen again. If there is indeed a whitelist for links, you might think about adding a few trusted sites, starting with your own site, and then you could think about adding academic publishers, arXiv, PubMed, etc.

    • This is a good idea. I also suggest links to common data sources, I like to link graphs at fred.stlousfed.org, ourworldindata.org, data.worldbank.org, the census, pew foundation, whatever. Also software links, such as to the stan discourse, the julia discourse, Python forums, R forums, Jupyter forums, github…

  6. I’m ready for another one of those brackets if that’s on the table. Still proud I represented Beverly Cleary well that one time on the internet. Engaging content

  7. Love the blog, although I can’t say that I love every blog post. Hard to communicate which posts I’m less enthused about, but it’s usually the longer ones (but not all long ones).

  8. I would like to see more content on the specific strengths, problems, limitations of the different social science disciplines in the context of the purpose of this blog. My specific interest is sociology.

    About 6 months ago, I searched under topic option ‘sociology’ and read every post. I found few that were directly in the discipline’s wheelhouse or addressed debates specific to causal modeling in that field. Several statisticians (e.g. David Freedman) and former sociologists (now teaching in other fields) have argued sociology is less serious than other fields.

    I looked outside this blog for Andrew’s views on this topic and could not find it. I read his 2011 review essay “Causality and Statistical Learning” published in the American Journal of Sociology. The examples were only drawn from psychology, economics, epidemiology and political science (red state, blue state book) and statistics. Of his 20 references, only one came from a sociology journal – economist James Heckman’s article in ‘Sociological Metholodogy’. Why he published there is a mystery to me. Probably 15 sociologists in the US follow his math and half those are Ph.D physicists teaching in sociology (which is another annoying trend I hope you address – at least James Coleman got a Ph.D in sociology and didn’t have the hubris to barge in.)

    • I second this. As a PhD student in a business discipline which draws most of its methodology from economics (for archival/observational studies) and from the social sciences (for experimental studies), I would enjoy seeing more posts that discuss the strengths and weaknesses of the social sciences and economics.

      In this vein, I’d also like to see more coverage of heterodox/unorthodox thinkers in these areas. For instance, I’m curious about the blog authors’ thoughts on work done by Jay Forrester and John Blatt in the realm of economics — they seem to reject much of economic orthodoxy in favor of more complex models, inspired by their prior work in engineering and physics, respectively.

      Finally, easy-to-find resources on how to do social science *right* would be greatly appreciated. Like others have said, keyword searches of past blog posts can yield some good fruit in this area, but some sort of compendium would be very nice to have.

      • It’d be good to see a series of posts highlighting heterodox economics. Maybe not starting with people who are unfortunately already dead though. What’s the active heterodox economic research? I know Steve Keen and pretty much no-one else.

  9. In a similar vein as Daniel (re: well-done science), I would enjoy a greater emphasis on instances where statistical models were used as motivation to uncover or inform a mechanistic model of the world. Uncertainty obviously abounds, but surely there must be some examples of people using statistics to inform deterministic models that can then be tested. If they happen to use Stan so much the better!

  10. I also agree with Daniel and others that it would be interesting to hear about some examples of great research, particularly research that has stood the test of time, has been verified through more than one project, and has been built on. Not that you should go dig it up but if it comes your way that would be an interesting thing to prioritize.

  11. This blog has strayed too far from its historical mission to preserve and promote the great culinary tradition of Jamaican patties.

  12. I come here to develop the statistical intuition I never learned in school. (Maybe because the one statistics course I took in college was taught by an economist.) Entertaining blog posts are a side benefit.

  13. I don’t have much. This blog has done so much for me.

    I like it when Jessica opens her posts with “This is Jessica.” I wish everyone besides Andrew would do that.

    I’d like to see more perhaps on measurement issues of all kinds–scaling, censoring, error modeling.

    I’d like to see more “how to do it right” examples, especially with the pretty limited data available in social science. E.g., the GSS.

  14. Don’t know if it is recency bias, but one thing that I have seen more of that I really enjoy is the book reviews. Similar to how there was the “favorite papers of the year” post for some contributors, it would be fun to have a favorite books of the year post. Or perhaps a “currently reading” spot under the authors where we could see different books or even lengthy papers that people reading.

  16. I love this blog and read regularly.

    I would love to see some posts explaining controversies in statistics, or arguments about statistics in the news, in layman’s terms. Although I am a regular reader of the blog, and I think a pretty smart & educated person, I do not understand most of the posts. I trust you, so I would love to see some more “dumbed down” breakdowns from your perspective, not instead of the regular posts, but as a sideline.

  19. the selection mechanism for papers that get written up on this blog seems to be (1) someone reads a paper they don’t like, (2) they email it to andrew with some comments, (3) andrew writes a post

    there’s nothing wrong with this mechanism* and it generates the entertaining and informative blog we’ve all come to know and love. but by design, it tends to select for crummier papers on average, and we get less of your views on more well-done papers, or even just representative papers

    i would be happy to read your thoughts on a more positively (or at least neutrally) selected set of papers. for example, pick a few applied microeconomics papers recently published in top general interest economics journals such as the American Economic Review or Quarterly Journal of Economics that seem interesting to you and give thoughts on the statistical analysis in each

    i think this approach could lead to discussion of a different set of issues than those that are currently emphasized on the blog

    *though i have wondered if anyone has ever sent you their academic rivals/enemies papers in hopes that you will skewer them

  20. > not on issues on which I have less control such as problems with logging in, RSS feeds, etc. Thanks.

    I just wish I got emails of people replying to my comments because I usually forget to check back. This might not be a feature of your blogging system but you might have some amount of control in using other systems. I’ve seen some other .edu blogs move to Substack, as an example, and since your blog is a subdomain, I think it might be possible to retain the subdomain and just point it to Substack (using a CNAME in DNS).

  21. I enjoy this blog (and your books) and what I get the most from are the articles which have a focus on ‘how’ science is done. Over the last few years the regular articles on reproducibility have provided a running textbook on how to assess science – both in literature and in everyday use (I’m an industrial scientist); articles and books have helped in understanding how to take statistical principles and apply them, how to think about problems in a different way. There is a level of translation from theoretical to applied statistics that is difficult to find elsewhere. I’m a little less sure about providing a request for a given type of article in the future – it would be a bit like asking a chef to write a recipe book but saying I wanted it to fit within the groceries I normally buy rather than buying the book in order to explore new options. I enjoy the other parts of the blog, in all their diverse rangings but the ‘how’ part is what I find most valuable.

  22. I saw some comments about rants (I am guilty) and staying on topic.
    Is a critique of sociology on topic? I can share an anecdote.
    I met the statistician David Freedman around the time he wrote his famous shoe leather article. I knocked on his door at UC Berkeley and introduced myself. I said I completed an MS in Demography from Wisconsin but found it intellectually unfulfilling. For lack of something better to do, I spent a semester in the Ph.D department in sociology at UC Santa Barbara (clown show) and then transferred to UCLA where Richard Berk and Howard Freeman had established a center for applied social research. We spoke for an hour. David had a special contempt for sociology. “Sociology… you start with the conclusion” he said.
    Of course, there are individual sociologists who do worthwhile studies, but not using regression in my opinion. Freedman’s criteria for statistical models were: good design AND relevant data AND testing against reality in a variety of settings. Please show me some studies where these three are met. Since 2018, I have worked half time at Marshalls store in Yonkers, and can assure you that the studies by Harvard sociologist Daniel Schneider do not (https://shift.hks.harvard.edu/).

  23. I’d like to see a policy that commenters should use their real name and have contact details available. The Datamethods discussion site does that and it’s really helpful. You get to know who you might be able to have productive conversations/collaborations with, and who the assholes are. (Can I say that?)

    • Argument from authority/consensus is very convenient, until it breaks down.

      It would be nice if we could be so lazy. Like having a functioning legal system/currency when doing business. Everything is so much easier when when we can use trust.

      Unfortunately, when it comes to stats these heuristics broke down long ago (as anyone with a passing knowledge of Bayesian stats knows). I suspect personal experience with thats why Andrew is more tolerant than others.

      Things become much more robust if people need to show their evidence and reasoning rather than rely on the credibility crutch.

    • While a lot of people do post under their real names and provide contact info at Datamethods, as far as I’m aware it’s not an actual requirement.

  24. I would love to have more answers than questions. Many posts include posting a question or a critique of research or a controversial topic followed by “I dunno the answer”. I think if questions are posted or difficult topics are brought up, it’s really helpful to have at least some clear answer, even if it’s not perfect, or even just a framework for getting to the answer. An example of a post with a good answer was the log(A+x) post. With that one it would have been great to see even more examples and explanation. An example of a post with only a question and no answer was the piranhas/omics post recently, where the answer to the tough question was “I’m not sure!”.

    • The ‘I am not sure’ part is what makes this blog so authentic. Let us face it, in most situations the evidence is not entirely clear, especially to an outside analyst who does not know all the ins and outs of the data. ‘I am not sure’ should be everyone’s answer when asked about something they are not super familiar with, in my opinion. The log(A+x) post was an exception in that it was purely methodological and our host argued that log(A+x) should be the starting point instead of log(1+x). Even in most methodological discussions, the answer is not as clear-cut as in this case.

  25. One of the things I find best about this blog is that many of the issues raised are relevant for my undergraduate teaching: I run a course for upper-year undergraduates that is basically a “critical thinking in science” course, which involves trying to teach students what to look out for when reading published research. The majority of the students are from a social sciences background rather than a hard sciences one, so the subject matter of many of the posts here is strongly related to the sorts of things that they are interested in (or at least, which I think they should be interested in!). Sometimes I would like to tell the students to just read the blog rather than listening to me ramble on about similar issues.

    In this context, one area where I think value could be added would be by providing a more detailed explanation of some of the issues raised in posts—sort of like “explain XKCD” for the blog. (Perhaps this is something that a LLM could do?) This would have the potential to expand the audience and accessibility of the material covered.

  26. How about a very gentle report card that accumulates, per journal, on publishing, retracting, and tracking errors …?

    The idea is to hold them accountable in a low-key, but fact-based way.

    The blog would keep the report card, and surface it whenever new information emerges.

    For example, I’d like to see if prestigious journals – Nature, Science, etc. – do better than more obscure ones.

    I’m reminded of this as I am a co-author on a complex, messy paper that has been rejected by more prestigious journals, and we just received thorough, useful, thoughtful reviews, with revision suggestions, from a journal I’d never heard of.

    — Mark

  27. Do you do a census of your readers? What I want is a function of who I am, and may not generalize. I’m an economist and b-school prof, and I read this for the gossip (its fun to see you complain about bad research, respectfully) and for the advice on how to present material to my students. The research advice is helpful, but I’m pretty stuck in my ways, and I incorporate new stuff as best I can (from you among many sources), but I’m always looking for ways to improve my teaching of students and my reviewing of others’ research.

  28. Frankly, I’ve always enjoyed where this blog has gone, whether I agree or not with posts or comments – that’s the nature of discourse. And I like the periodic ‘fantasy X’ (personalities, songs, books, etc) brackets that appear from time to time as a respite from everything else. And I’m fine with rants also. I come by that one honestly – genetic presdisposition :).

    If I were to add anything else to the above recommendations (not up/down votes – I can’t support that – sorry folks):

    1) The politics surrounding ‘positive outcome’ research (whether in academia, government, or industry) for which no independent study has been developed to reproduce the original experiment(s). I find that commenters here often find some really juicySome would say there’s not much to be said here, but examining methods to disincentivize bad (or at least poorly thought out, or even pedestrian) acting could be something useful here.

    2) The fact vs fiction of ‘good’ vs ‘poor’ use (and even design – I think of python’s ‘prophet’ package here – which isn’t at this point a strong performer in time series analysis) of machine learning methods (given those who generally use them are looking for a ‘push button’ solution to elevate their careers. This is, versus understanding what’s happening under the hood and making sound decisions throughout an analysis … this is still a rampant problem.

    3) The rise of language models (and the tech hype that attracts people like flypaper to it) that are given access to stat/ML tools, and are being tested (and used) for statistical modeling. What’s good, what’s not-so-good, what’s downright ugly about them? What does the notion of ‘data scarcity’, a problem we’ve dealt with for many years now, and of LM generated ‘synthetic’ data mean for both human and machine generated data for modeling respectively (some say, in the LM context, this may generate ‘data collapse’ and subsequent regression of this tech to do what’s intended – kind of like ‘data inbreeding’. The LMs just happen to get there ‘faster’).

    4) The role of inference using numerical methods to explain complex and nonlinear systems (such as social and economic) vs methods by analysis. There’s not a ‘one size fits all’ answer to this, and there’s developing discourse on the idea of using numerical methods inside of Bayesian frameworks.

    5) The science and analysis behind decision making in. well, pick your area of interest (but social science in business, economics, and dare I say, political science are reasonable points to ponder). Yeah, yeah, I know this one’s a real pita to be honest, as it involves not only statistical inference, but components of game theory, optimization, combinatorics, nonlinear dynamical systems (i.e. with respect to social constructs impacting decisions), aspects of psychology, economics etc etc etc. I know this one can feel intractable at times, and there has been plenty of failure to ‘neatly’ model this domain. But I do feel there’s value in its discussion from an point of view that might be some amalgam of the above.

  30. 1. Continue highlighting bad science. It’s why I love it here.
    2. More posts highlighting good analyses, still with a focus on specific critiques and improvements
    3. Anoneuoid guest posts

  31. Topical suggestions: I’d be interested in a bit more meta stuff on scientific/statistical philosophy and communication. Take this suggestion with a grain of salt because it might lead to meaningless rambling …as my poorly formed examples may suggest:

    Example 1: How do we talk about overall strength of evidence for something like climate change, where multiple threads of evidence must be stitched together, in context, to form and support a scientific theory. If people don’t like the climate change example because it’s so settled, then how about climate change theory 50 years ago (or more).

    Example 2: How should we handicap our “credibility priors” when a study finds its way to our field of attention through a channel that is heavily influenced by media-genic factors that aren’t directly related to scientific usefulness or strength of evidence?

    Example 3: I generally favor non-technical (even colloquial) terms in characterizing statistical evidence (this is both a personal proclivity and a professional necessity for me). My goal is usually to craft plain-English expressions that (hopefully) point most people away from common misconceptions (I’d like to think this is a universal goal!). However, I’ve run into surprising resistance in the comment threads when I’ve tried it (maybe because I was incorrect and too stubborn to see it, but maybe not). Anyway, I think serious attention to wide-audience language could be worthwhile (as a sort of digestive next step, separate from clarifying our thinking via careful technical discussion).

  32. I agree that populist up-votes strongly tend to degrade discussions. But it might be interesting to deputize some power users (Dale, Raghu, Daniel, …) with some kind of up-or-down-vote power (and clear rules for what those votes are meant to mean).

    The time I get to spend on the blog is sadly limited, and there are times when it would be helpful to quickly identify scientifically important discussion elements (or at least avoid rants that aren’t really trying to be informative).

  33. I’d love to see some backward-looking efforts to shore up the usefulness of the blog as a reference archive.

    Example 1: I’ve sometimes had trouble locating older posts. The search bar is helpful for this, but I’d find it further-helpful if I could navigate to a full list of in-use tags post somewhere (“zombies”, “miscellaneous science”, etc).

    Example 2: I don’t think the search bar covers comments. I think it would be helpful to have a separate search bar that covers both original posts and comments (and which outputs a list of clickable text snippets that include the search term). For example, a commenter once posted a long explanation about nonstandard analysis (logician stuff, Jerry Keisler, et al). I didn’t have time to give it a focused read at the time, and I wasn’t able to locate it later. I’ve had a few experiences like this…

  34. With regard to positive views on social science research, today Cowen referenced this paper, linked on X, at Marginal Revolution:

    https://x.com/_Tiagoventura/status/1830702879256256791

    Without going beyond the abstract, I think it’s an excellent abstract in that it covers the basic problem succinctly; reports numbers the fraction of people at given sites who are “Survey Professionals”; and offers a sensible (sounding, at least) conclusion: “While concerns are warranted, we conclude that survey professionals do not, by and large, distort inferences of research based on on-line panels”.

    The shortcoming is that, while they report that survey professionals show some different behaviors compared to “ordinaries” (my term) and report that this doesn’t seem to generate a notable difference in the survey results, they do not give the actual numbers regarding the difference in responses between survey professionals and oridinaries. I guess they don’t bother with this because they’re not making an outstanding claim, but it seems like they should provide some results from the paper to directly support their major conclusion, even if it’s just one example out of many variables.

  35. I read this blog since about 10 years and I love it. Nevertheless, I feel the topics and general feel of the blog did not really developed and it is time for something new.

    – I love the technical discussions like log(a + x) instead of log(1 + x)
    – The “shooting down” of bad research starts to annoy me. When I began reading this blog I gained a lot from Andrew and others pointing out these errors; nowadays not so much. Furthermore, there seems to be a tension between shooting down really bad research and a common theme on this blog: The way research is done is fundamentally broken. If the problem is binary thinking, focus on hypothesis testing, researchers degree of freedom, publication bias and “honesty and transparency is not enough” then why not discuss papers considered good?
    – Where is all this successful Bayesian “big-complex model” social science? I’d like very much discussion of papers that qualify as good for Andrew and other contributors
    – Moving beyond problem solving: I feel good research for people on this blog is “solving a problem” which means predicting an outcome with an interpretable Bayesian model. But how does this increase our scientific understanding in general? This brings me to my last point:
    – Often Andrew emphasizes the importance of theory. But what exactly is a good theory? How to develop one? How does fitting a complex Bayesian model lead to better theories?

    So all in all: Moving beyond criticism of bad research and towards the development of good practices, from tackling the right questions, answering them, and developing better theories.

  36. I’ll echo the calls for more coverage of well-done papers. If it makes the blogging more enjoyable, you could even cover them like book reviews. It doesn’t have to be all methods; it could be “I liked this turn of phrase” and whatnot.

  37. Thank you very much to all the contributors and commenters on this blog! The posts and their discussions are always illuminating, and they remind me that I’m not the only one that gets irritated when confronted with shoddy statistical work. I’ll be very happy to keep reading this blog (and its comments) even if nothing changes.

    Having said that, and since you asked, I echo other requests to see more cases of statistics done well. I’m also curious to hear about more cases where applied statistics helped improve estimates of something that was already reasonably well understood. Election forecasts are one example of this, but there are probably other examples of scientists (or people in industry?) that already had good estimates (or at least not terrible ones) but got better ones by thinking more carefully about their data. I’m reminded of this post by Phil on improving electricity prices forecasts to reduce the costs of getting surprised by “crazy big” refrigeration costs:

    https://statmodeling.stat.columbia.edu/2022/11/23/time-series-forecasting-futile-but-necessary-an-example-using-electricity-prices/

    Perhaps I’ve just been missing these types of posts and if so, someone please give me some links to them.

  38. I have been aware of this blog for the past year or so, but it was only this year that I started to read of the posts and comments. I check this blog about once a day. Sometimes I skim the posts if it doesn’t capture my interest. That said, there have been times I wasn’t expecting much, but I found the reading the comments much more engaging. Overall, I am contend with the blog and I do have a few suggestions that I hope are helpful.

    1) In some ways, I consider this blog one of many sources that has changed my thinking on methodology and social sciences. As someone soon finishing their graduate program, I wish these issues were talked about more as part of my training because the actual research is much more messy. Knowing all these statistical techniques doesn’t amount to much if you don’t think about the data and the underlying methodology. I would like to see more resources to good textbooks, papers, etc. for junior researchers. I think the previous posts that linked the two textbooks (Regression and Other Stories; Active Statistics) are good starting points.

    2) I don’t have an issue with covering poor research articles, and I do think the blog could give more examples of research done well. What I would like to see more of isn’t so much “this is ‘bad/good’ research”, but more best practices researchers should think or implement about going forward. I say this because I often encounter that researchers/students are sometimes aware of the problems within social sciences/psychology, but they don’t know what else to do other than what they have been taught as part of their graduate or undergraduate training. In the past year, I have a read a lot about the problems of social sciences/psychology from p-values, p-hacking, forking paths, confidence intervals, effect sizes, data fraud, nonsensical findings, etc. Reading all these problems, it’s hard to not develop a cynical/nihilistic view about the landscape of the field, but ruminating or lamenting about these issues is not always conducive to our well-being. I would like to see more about steps researchers can take to better improve the field.

    3) As someone more new to following this blog, I find it difficult to know if a particular topic has been covered in previous posts. Sometimes I can find through search, but I have to go through a lot of posts at times. Updating the tags more often or having some table of contents would be helpful to navigate when searching a topic.

    4) I echo sentiments of having other folks guest post on this blog. I also would love to see more measurement issues covered since I tend to read those the most.

  40. This is probably against the postscript you added here Andrew, but:

    I enjoy reading the blog but I often find the discussions in the comments quite enlightening. However there are times as others have mentioned that I find a particular thread uninteresting or irrelevant, but extremely long! Could we have a way of collapsing threads to the parent comment, a la Reddit (the old layout) and other blogs?

    Otherwise I enjoy the range of content offered here, and I rather guiltily admit to enjoying discussions related to current politics and academic controversies more than statistical methods (speaking as a non-statistician). But I think I would not want you to do much more of the former as it would probably be a bit too unseemly.

Comments are closed.