Andreas Stang points us to the above delightful image, which comes from an otherwise obscure paper in the Journal of Circulating Biomarkers. That’s awesome that they got p=0.0001. And it was multiple-comparisons corrected! I think we finally have hard evidence that people over 40 are older, on average, than people under 40. It’s a good thing it wasn’t p=0.06 or something, or we might still be in the dark regarding the relationship between the “Age” and “AGE>40” variables.
I’m reminded of the immortal line, “Participants reported being hungrier when they walked into the café (mean = 7.38, SD = 2.20) than when they walked out [mean = 1.53, SD = 2.70, F(1, 75) = 107.68, P < 0.001]." Research!
But what about people of age=40? are they significantly in-between? can we even get a test, with a null variance in that subgroup?
The hunger study is much worse. This particular figure seems to be taken a bit out of context (I didn’t access the whole paper, but I looked at the figures – this is one of a set of 4 pairwise comparisons). It looks like the intent was to look at various biological markers and their ability to distinguish between ages 40 (I guess nobody is interested in the =40 year olds). So, it is of some relevance to look at the distribution of actual ages – the ability to distinguish between those slightly less than 40 and slightly greater than 40 would seem to be more impressive than distinguishing between a 20 year old and an 80 year old (I might be able to do that from a blurry image). Of course, that raises issues of much more meaningful ways to look at this data – and it remains silly to test the hypothesis of equal mean ages (but I guess the computer key creates that display automatically). So, I am not offering any defense for what they did, but at least I can imagine some relevance of looking at that data, unlike the hunger before and after eating.
But, what is it about the 40 year old demarcation? Why not 41 or 42 (the latter being the answer to everything in life)?
Agree, but the y-axis should then say ‘Predicted age’
I remember seeing this a few weeks ago and being appalled.
About Thomas’ comment: Note that there’s a gap in data points between about 32 and 42 years old. I suppose it’s possible that the graph is even dumber than it seems, with people in this age range excluded from the study so that the “difference” between the sets was significant! If this was the case, I don’t want to know…
To be fair, the authors do not mention the ‘result’ on age differences in the text, and the figure does give useful data on the age distribution of the sample groups. Figures 2 through 5 in the paper all use a very similar layout which facilitates visual comparison (“small multiples”); they could have left out the p-value for the age panel, while still preserving the otherwise useful design.
The paper may be using the acronym “AGE” = “Advanced glycation end-product”. It’s a common acronym in some nutrition-related circles. This could be a reasonable bio-marker for something or other. The other similar graphs put some biomarker in the graph title at this location, and why should this one graph be the exception?. If only they had capitalized it.
Interesting. When I looked that up, I see references to 15,000 kilounits/day for AGE, so we can only imagine what scale they converted it to for the numbers they are reporting. I still think they’d be better off sticking to 42, the answer to everything.
The full text is available (free, no login required) at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5548330/
I think Gregory Mayer has it right in his comment above: the point of the plot is to show the ages of the individuals in the “above 40” and “below 40” groups. The p-value is presumably auto-generated by whatever plotting program they’re using. Tom Passin’s plausible-sounding explanation of why AGE is used on one axis and Age is used on the other is incorrect, it’s really age on both axes.
The plot is funny, but really only because of the p-value thing. Otherwise it’s kinda-sorta OK as a way of showing the ages of the people in each group. But only kinda-sorta. Why not simply show a dot chart of the ages, with a dividing line at 40? Or there are several other displays that would be better than what they did. I would guess it’s just a matter of convenience, they have this plotting function that they’re using to make all these other comparison plots, and it performs adequately, though imperfectly, at this task too, so they use it lazily. I’m pretty lazy myself, and can sympathize. So I give them a pass for this plot, ridiculous though it looks.
But…why on earth are they dividing the world into ‘over 40’ and ‘under 40’, especially since, as Raghu points out, they have no subjects between about 32 and 42 years old. They could just as easily divide them into ‘over 33’ and ‘under 33’. But also, and more to the point: it’s not like they think there’s some strict threshold, with the biomarkers suddenly leaping up (or down) when someone reaches a specific numerical age. Wouldn’t some sort of continuous model be much more appropriate? But, I grant you, it would be harder to summarize such a model in terms of ‘statistical significance’, and presumably that is very important to them (perhaps because it is very important to the referees).
First off, Epidemiologists (and related disciplines, especially “clinical” ones) just LOVE dichotomous “exposure” variables. I mean really, deep down love. Like an addict loves nicotine.
Given that they’re going to dichotomous age, 40 is a round number and it may well be that other prior studies have dichotomized at 40. There may even be some clinical guidelines out there which reify “over 40” as a risk factor.
In my experience, Epi people are fine with running a continuous model to see if it offers additional information. As long as you also run the dichotomous model and treat it as the definitive statement of your findings.
In my 20s I considered a career in epidemiology, I’m so glad I didn’t do that. I’d probably have been brainwashed.
I now have my first homework problem for my upcoming stats course:
Consider a population with age distribution representative of the United States. Suppose you randomly draw individuals and place them into one of two bins according to age: below United States median age or above United States median age. How large must your sample size be before you can expect to be reasonably confident that the average age of the “below” bin is less than that of the “above” bin?
The more I think about it, this problem is a veritable mother lode of conceptual statistical issues. I currently think that the correct answer is “3”.
Given that around 53% of the US population is less than 40 years old, how can the answer be 3? All 3 would be older than 40 around 15% of the time. You need 5 people before the probability that all will be older than 40 drops below the magical 5%. Are you thinking about this differently?
You need 6 before the probability of all either older or younger drops below the magical 5%. But 5% is the magic number associated with “reasonably confident”. The key word here is “expect”, which doesn’t have a well-established magic number. If the probability of an event is .75, would you “expect” it to happen? I would, and I’d be willing to bet on it. With three samples, the probability that both averages will exist is .75, and if both averages exist, I will be certain that the “below” average is less than the “above” average.
3 people has a ~1-in-8 chance of having nobody in one of the bins. You’ll need another one to get 95% confidence.
(And how do we count exactly-at-median age people? Do they get assigned randomly or thrown out – if they get assigned randomly you could end up with a tie)
Actually, make that 5 people.
The correct answer is zero. You can mathematically prove that the average age of the below group is less than the avg age of the above group. Probability in the Bayesian conception is the unique extension of truth values to uncertainty, and is compatible with 0/1 for false/true. Since you have an a priori proof, the probability is 1 a priori.
But the average age of any group of n=0 persons is undefined. The average age of the below group is not less than the above group until both groups have nonzero members.
If you’re talking about the sample then I see what you mean, at that point it’s just the probability that you’ve gotten at least one head and one tail in a sequence of fair coin flips. The ages don’t enter into it. The sample avg will always be separated since one group has a strict upper bound equal to the other groups strict lower bound.
It would be interesting to replicate this research with dating websites. Does “people under 40 = people under 40” there?
I often use as an example of such tomfoolery a table from a paper long ago reporting the results of a clinical study of gastric bypass surgery for weight loss. They first report an (unpaired) t-test on the participants weights before and 2 years after surgery; bad, but not the worst. Then, they do the same thing on height (spoiler, it didn’t change!), although there was a sizable SEM b/c the comparison was unpaired. Then, they did the same thing on age, which magically was 2 years higher in the post group than in the pre, albeit again with a sizable SEM, even though everyone had aged exactly 2 years.
Even though analyses like that should be considered a form of scientific comedy, a dismaying number of students (and probably clinical researchers) don’t immediately see the ludicrousness of either the implied hypotheses or the improper analyses until they are pointed out.