13 thoughts on “Maybe they should just write some papers about their priors and not mess around with actual data!”

1. But then they couldn’t “bask in the glorious objectivity of science”! (to quote IJ Good)

   Reply ↓

2. So the posterior is a restatement of the prior. And in between goes…what?

   Perhaps a couple paragraphs about how other smart people previously stated something than can be construed as support. And then five paragraphs of self-aggrandizing, since the only real goal of the paper is to establish the first author as an autodidact polymath whose viewpoints are indistinguishable from fact.

   Reply ↓

3. I am currently trying to see how far I can come with writing a paper and concerning that I am reading stuff about psychopathy. I came across a classic book about psychopathy by Cleckley titled The Mask of Sanity, which if I understand things correctly still holds value concerning the topic of psychopathy. It’s considered a “classic” if I understood things correctly, so perhaps it already has had value.

   On page 20 he mentions how he will follow a procedure of 1) the observation and recording of facts, 2) the grouping of these facts with proper correlation and with proper distinction from other facts, and 3) the effort to devise some summarizing or, if possible, explanatory statement which will enable one to grasp conveniently their significance.

   Perhaps this is in line with the post above in the sense that there are perhaps many ways in which one can contribute to science, and many of these ways do not have to involve complicated data collection and analysis using statistical software and computers, etc.

   Reply ↓

4. Check out Robin Hanson’s paper on alien civilizations. One of the purest forms of the assumptions-based research genre.

   Reply ↓

   • Mark:

     Without commenting on that particular paper, which I have not seen, let me emphasize that I have no problem at all with scientific papers that are purely speculative, as long as they are labeled as such. My problem is when people claim evidential support that isn’t there. One way to see that a paper’s claimed evidential support is illusory is when, after the evidence is shown to be wrong, the author states that their conclusion has not changed.

     Reply ↓

     • The Oxford dictionary defines “science” as: “the systematic study of the structure and behavior of the physical and natural world through observation, experimentation, and the testing of theories against the evidence obtained.”

       A paper based entirely upon speculation may have value, but it is not science by definition.

       Reply ↓

       • Guessing at an explanation for something is the first step of science.

         But the next steps of deducing a prediction from your explanation, then testing it, are also required to make it science.

       • My recollection is that Hanson’s paper is based on observations such as the speed of light being a hard limit. Einstein did a lot of work via thought experiments founded on the same thing. Was Einstein not doing science then?

       • I assume its about this paper:
         https://iopscience.iop.org/article/10.3847/1538-4357/ac2369

         That is speculation, which is fine, but then you need to derive some kind of prediction from it.

         Einstein assumed constant speed of light for special relativity, which was kind of an early toy model that omitted gravity. Then in the presence of gravity, he said the speed would slow down as a function of the gravitational potential. Thus you would get apparent deflection of light from distant stars during an eclipse, so it was testable:

         In the first place, it can be compared with the reality. Although a detailed examination of the question shows that the curvature of light rays required by the general theory of relativity is only exceedingly small for the gravitational fields at our disposal in practice, its estimated magnitude for light rays passing the sun at grazing incidence is nevertheless 1.7 seconds of arc. This ought to manifest itself in the following way. As seen from the earth, certain fixed stars appear to be in the neighbourhood of the sun, and are thus capable of observation during a total eclipse of the sun. At such times, these stars ought to appear to be displaced outwards from the sun by an amount indicated above, as compared with their apparent position in the sky when the sun is situated at another part of the heavens. The examination of the correctness or otherwise of this deduction is a problem of the greatest importance, the early solution of which is to be expected of astronomers.1

         In the second place our result shows that, according to the general theory of relativity, the law of the constancy of the velocity of light in vacuo, which constitutes one of the two fundamental assumptions in the special theory of relativity and to which we have already frequently referred, cannot claim any unlimited validity. A curvature of rays of light can only take place when the velocity of propagation of light varies with position. Now we might think that as a consequence of this, the special theory of relativity and with it the whole theory of relativity would be laid in the dust. But in reality this is not the case. We can only conclude that the special theory of relativity cannot claim an unlimited domain of validity; its results hold only so long as we are able to disregard the influences of gravitational fields on the phenomena (e.g. of light).

         Since it has often been contended by opponents of the theory of relativity that the special theory of relativity is overthrown by the general theory of relativity, it is perhaps advisable to make the facts of the case clearer by means of an appropriate comparison. Before the development of electrodynamics the laws of electrostatics were looked upon as the laws of electricity. At the present time we know that electric fields can be derived correctly from electrostatic considerations only for the case, which is never strictly realised, in which the electrical masses are quite at rest relatively to each other, and to the co-ordinate system. Should we be justified in saying that for this reason electrostatics is overthrown by the field-equations of Maxwell in electrodynamics? Not in the least. Electrostatics is contained in electrodynamics as a limiting case; the laws of the latter lead directly to those of the former for the case in which the fields are invariable with regard to time. No fairer destiny could be allotted to any physical theory, than that it should of itself point out the way to the introduction of a more comprehensive theory, in which it lives on as a limiting case.

         […]

         1. By means of the star photographs of two expeditions equipped by a Joint Committee of the Royal and Royal Astronomical Societies, the existence of the deflection of light demanded by theory was first confirmed during the solar eclipse of 29th May, 1919. (Cf. Appendix III.)

         https://gutenberg.org/cache/epub/5001/pg5001-images.html#xd29e1904src

         Btw, afaict if you look around the internet there are many people claiming GR predicts a constant speed of light, but this was *not* Einstein’s interpretation. It was that of his assistant, who basically restarted GR research in the 1940s:

         Bergmann contacted Einstein again in 1935 and arrived in the United States in 1936.[2] He worked with Einstein, as his research assistant, at the Institute for Advanced Study from October 1936 to June 1941.[4]

         […]

         In 1947, no physics department in the United States had a center for research in general relativity. At Syracuse, Bergmann established one of the first research centers devoted to studying the general theory of relativity to reconcile it with quantum theory.

         […]

         Bergmann and his students were the primary contributors to the literature of general relativity until the mid-1950s. Twenty years later, there were more than a dozen general relativity research centers and now it is in the mainstream of current physics research.[3][5][10]

         […]

         In 1942, Bergmann published the first textbook on general relativity, Introduction to the Theory of Relativity, with a foreword by Einstein.[13] The second edition of this book was published by Dover Publications in 1976.[14]

         https://en.wikipedia.org/wiki/Peter_Bergmann

         Thus it seems what gets called GR these days comes from Bergmann, not Einstein per se. And the constant speed of light appears to be a point they disagreed about.

5. With the number of studies where major errors were found to “not change the findings of the study”, perhaps we can estimate the proportion of science (or scientists) who actually do just promote their priors as “science”. Perhaps the retraction watch database has some information on the frequency of this excuse.

   Reply ↓

   • Aren’t there calculations which are used to support the conclusion that there are too many statistically significant results in science using NHST based on what is to be expected given the statistical power of tests used (or something like that).

     Perhaps you could take the differences between the percentages of observed and expected significant findings as an indication of the proportion of science (or scientists) who actually just promote their priors as “science”…

     Reply ↓

6. Maybe they should just write some papers about their priors and not mess around with actual data!

   I don’t think it needs to be one or the other but I do think that it would be good if people in general spent more time thinking about what their priors are, and how their research might evolve differently if they had different priors.

   I know there’s been some research projects where “teams” of people with different viewpoints work together to address a research question. But there the focus isn’t on priors but on conclusions. More useful, imo, is for teams with different conclusions to first examine whether their priors are different (which would then, logically, lead to different conclusions).

   This is, imo, somewhat analogous to the conflict resolution paradigm of differentiating between positions and interests, where the positions are like conclusions and interests are like priors. Establishing where priors are similar or different can be very instructive, imo, for different answers to research questions just as can identifying where interests are shared (or at odds) can help explain different answers to policy questions.

   We are all subject to cognitive biases based on our priors. But very few of us are very skilled in evaluating our priors can predict our conclusions.

   Reply ↓

7. But then aren’t there plenty of such papers in the (soft) social sciences? They tend to call that “theory”…

   Reply ↓