As a former University of Maryland student, I’m incensed by this story. From Retraction Watch, this story about a professor of Biochemistry and Molecular Biology:
The ORI finding stated Eckert “engaged in research misconduct in research supported by” every NIH grant on which he served as principal investigator, totaling more than $19 million. The finding also lists multiple “Center Core Grants” worth hundreds of millions for shared resources and facilities at research centers. . . .
According to ORI’s findings, Eckert erased a band in one of the paper’s figures “to falsely show a favorable result.”
In the 13 papers and two grant applications, Eckert used and reused images “representing unrelated experiments, with or without manipulating them, and falsely relabeling them as data representing different proteins and/or experimental results,” ORI found.
I don’t really know what to think about the Center Core Grants, but the 19 million dollars he should have to pay back! OK, maybe he doesn’t have $19 million cash on hand, but there’s gotta be a system for this, right? Maybe they start by repoing his car, freezing his bank account, and selling his house, then he’s allowed some reasonable amount to live on . . . hmmm, it appears that the NIH predoc salary level is 28,224 per year, so they could let him keep that much . . . then he could try to come up with a payment plan. I dunno. They throw drug dealers in prison, but that would just cost the taxpayer more money. Better to just put an ankle monitor on him and take away his internet access. . . .
Ummm, let’s read on:
Eckert agreed to forgo contracting with the federal government or receiving government funding for eight years . . . Eckert also agreed not to serve on any advisory or peer review committees for the U.S. Public Health Service, which includes the NIH, for eight years.
OK, maybe after eight years if he shows some evidence of rehabilitation, maybe. But what about the $19 million? Paying that back should be the absolute minimum requirement, no?
This is certainly a good example of how confused our society is about crime and punishment. The impact of different crimes are evaluated on a very skewed scale. Someone caught for repeatedly stealing items worth far, far less than $19 million can receive a long imprisonment. There’s obviously a class bias in play and it reflects structural inequalities that we mostly ignore.
Joshua:
I think the big problem here is way too much prison and way not enough restitution. I don’t want this guy thrown in prison for 5 years making license plates. I’d prefer he do something more useful to society such as working at McDonalds or something, and then he could take some of the money he earns (along with all of his existing ill-gotten gains) and pay it back to the government.
I really don’t like the attitude, “This is a bad guy so he should be punished.” Or, worse, “This is a bad guy so he should go to jail.” I’m much more like, “This guy did something bad and he should not benefit from his crimes,” and “This guy should pay back his victims,” and “This guy should be put in a position where he is a benefit to society.”
The idea that this guy fraudulently misappropriated $19 million and doesn’t have to pay any of it back . . . that seems really wrong to me. And it would still seem wrong if he were sitting in a prison cell. Putting him behind bars doesn’t help anybody. (Yes, I get the idea that there’s a deterrent effect, but you can get the deterrence plus the restitution by just having him pay back the money, or as much of it as he can, after repossessing his house, car, garnishing his salary, etc.)
Andrew’s the one who brought up prison.
Did you write that because you think they should throw drug dealers in prison but not professors who cheat? That’s how I read it.
Taking a “yes, and” improv approach, I’d like to add that (1) they also throw thieves (as Joshua pointed out as I was writing this), embezzlers, and those who commit wire fraud in prison, and (2) throwing drug dealers in prison isn’t cost free (as Andrew notes).
I can’t make any sense of who gets prosecuted and sentenced for what and why. There is a huge amount of variance due to prosecutors and judge, which are nested in cross-cutting jurisdictions in the U.S. It feels like professors get federal judge-level cover for their crimes, be it sexual harassment or embezzlement. Hmm, both those jobs have a tenure system. When I was tenured as a professor, I was told I could be fired for “gross negligence”, which sounded to me like being handed a license to engage in petty negligence. That clause made such a deep impression on me, I still talk about it.
When I worked for the James Beard Foundation (as a volunteer food photographer), the foundation president was thrown in prison for 1–3 years for embezzlement to the tune of $300K a year. The twist is that he didn’t take a salary and was only pretending to be independently wealthy. Next, the former Red Lobster CEO took over after nearly bankrupting Red Lobster with a crab special. The former CEO negotiated an exit package from Red Lobster, saying “you owe it [being rewarded for failure] to yourself.”
Bob:
No, I don’t think they should generally throw drug dealers in prison! As I wrote, “that would just cost the taxpayer more money.”
I think that, with rare exceptions, lawbreakers should not be put in prison. They should pay restitution, be put under house arrest, various restrictions on their freedom as appropriate, etc. Prison is just terrible. I recognize there are some settings where prison is appropriate (and this is setting aside questions about when it is that drug dealing should be a crime at all); in general, though, prison seems barbaric, wasteful, counterproductive . . . the usual story. Given our legal system, I have no idea how one would get from point A to point B, so I’m not offering any useful policy prescriptions here, just giving my take on the situation.
So, good to have a chance to clarify that point. Just to repeat: No, I do not think that drug dealers should be in prison (again, with some exceptions as I guess there are some really violent people who should be kept off the street, and the justice system as it currently stands doesn’t have good alternatives to prison for such people).
Worse(?), others who applied for that money and didn’t get it may have paid in terms of their career. Of course he didn’t personally get all the money but he should pay back at least some portion of it.
Just to clarify, I’m not defending throwing someone in prison for repeatedly stealing items worth far less than $19 million. Neither do I think throwing this guy in prison would serve any real purpose.
Despite the enormous amount of money we spend on preventing crime, our criminal justice system could reasonably be judged as not working based on outcomes.
Part of the problem is in the distorted way that we view “safety.” other forms of crime are more readily viewed as threatening, yet fraud in research does impact our safety. In some direct ways (participants harmed in studies conducted under fraudulent premises), but more indirectly, it makes us less safe as a society when researchers commit outright fraud because it undermines the public confidence in important institutions.
It’s a complicated perception problem. Some crimes are more readily viewed as threatening. A researcher committing fraud doesn’t put that $19 million into his pocket. But it is all part of the cycle. Crimes like his undermine social institutions which leads to distrust of government which leads to ineffective government which leads to more poverty and less housing and worse healthcare which all lead to more obviously threatening forms of crime.
Supporting housing and healthcare and infrastructure are seen as too expensive yet wasting huge amounts of money in an ineffective criminal justice system isn’t. It’s pretty crazy. That’s not to put it all at the feet of fraudulent researchers, but again, it provides a clear window into seeing how incredibly skewed our system is in direct line with class and structural inequalities.
I don’t have to think that throwing this guy in jail the beneficialnin order to recognize how his situation reflects his skewed our criminal justice is. Neither throwing him in jail nor throwing more shoplifters in jail will get us anywhere, but why does that schism in our system exist?
https://en.wikipedia.org/wiki/Kalief_Browder
Kalief received 3 years in prison (800 days in SOLITARY) **pending trial** and then after release committed suicide for basically “allegedly looking like a guy who stole a backpack with something like $1000 worth of stuff”. On the basis that someone in the back seat of a police car said “I think it was him who I saw 2 weeks earlier”.
Tales like this always sadden my heart. But they are important to read. Thanks for sharing.
From the retraction:
From the paper (emphasis added):
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0036941
No one really trusts these representative images anyway. If the guy (or his tech/student) just fiddled around long enough they could really get such an image. So there is really no change to the information content.
And why not share the similar results? Why is it considered normal for data to be generated at rather great expense, then just thrown away like that?
The root problem is an anti-replication culture, that then fosters fraud. Fix that and you fix the fraud for free.
I have been wondering for a while why defrauded funders do so little to recover their money. I am sure that in such cases (not referring to the Eckart case specifically, but to cases of fraudulent research generall) there are grounds for demanding *some* money back. Probably not all of it; there would have to be an assessment of what part of the contract(s) the researcher(s) failed to fulfil. Criminal liability would be another matter entirely, although in certain scenarios criminal prosecution might be justified (e.g. an obviously flawed ‘miracle’ procedure kills all its patients).
However, the amount of money to be repaid is likely to be a matter for litigation: Researchers who commit fraud will not just roll over and give up. There is even a chance that the researchers will be bankrupted by this kind of litigation, so that trying to get their money back would be a net loss for the funders.
Raphael:
Yes, I think the legal system is screwed up. It’s set up so it’s just a lot easier to throw people in jail than to get them to do restitution.
About “why defrauded funders do so little to recover their money.” I think in part because it’s not “their” money — i.e. not the NIH directors’ or program officers’ money — but rather the taxpayers’ money, and the taxpayers are a diffuse blob who aren’t going to make specific demands. In addition there’s no NIH analog of a public prosecutor whose job it is to go after these sorts of things (even if there were a clear way to do so), and time spent by the directors or program officers to address this is time not spent on other things. Should it be like this? No.
I’d question whether this behaviour is contrary to the interests of the funding agencies.
Individual humans mostly would like cures and such, but bureaucracies are a different organism. Is there a single example of people/governments cutting funding to such an agency due to “a few bad apples”?
Ultimately it is people paying for this stuff via taxes/donations. If there is no negative feedback from them, why would the funding agency care?
“I have been wondering for a while why defrauded funders do so little to recover their money.” I guess it’s hard because the fraudster probably already spent the money. Or, they have lawyers – look at Giuliani! So you have to dig yourself a bigger hole hoping to claw back the money.
It’s also because grants are made to institutions, not individuals. So the institution would have to admit to being part of the fraud.
I don’t think that it’s a system that should be replicated, but the healthcare system does claw back payments that were made but shouldn’t have been. When Medicare pays for services, the doctor (really, their office or hospital) submits the claim along with a diagnosis and their assurance that the service occurred and was properly documented. Periodic audits happen. And if the audits find that the documentation doesn’t match what was claimed (or wasn’t indicated), then they *will* go after the money and take it back.
There are *huge* problems with this system, but it is the government that’s clawing back the money (or actually 3rd party contractors that manage Medicare for different regions of the US.)
I’d hate to add an additional level of indirection between researcher institution contractor funding agency, but it “could” be a model to make it easier to audit grant contracts for compliance, honesty, and take back misapplied or fradulently-obtained funds.
Healthcare is a much, much larger part of the federal budget than research, so maybe it’s the scale that matters?
The justice system, as it’s called, should prioritize rehabilitation over punishment or revenge. Our goal should be to prevent future harm, not simply to inflict retribution. In the case of the professor, corrective action could involve reforming the NIH’s grant process to reduce reliance on reputation and implement rigorous blind reviews.
To further enhance the effectiveness of research funding, we could consider a randomized controlled trial. For instance, 10% of grants could be randomly assigned to researchers without established reputations. After a decade, we could evaluate whether this approach leads to better outcomes compared to traditional funding methods. By adopting such scientific rigor, agencies like the NSF and NIH can optimize their impact and ensure that taxpayer dollars are used wisely.
Thats a good sentiment, but somewhat ahistorical.
“Formal” peer review and RCTs are recent techniques (~1950). The vast, vast majority of benefit we derive from science was accomplished without either.
In fact, those techniques are attempts to use (lower quality) heuristics in place of the gold standards of independent replication and predictive skill.
If anything, use of peer review and RCTs is highly correlated with poor/unreliable research. So they are neither necessary nor sufficient, and quite possibly counter-productuve.
“If anything, use of peer review and RCTs is highly correlated with poor/unreliable research. So they are neither necessary nor sufficient, and quite possibly counter-productuve.”
I’ve been getting into AI and machine learning, and have been wondering why many of the major papers are published in ArXiv, and (relative to other fields) rarely anywhere “official”. I suspect this may be one of the reasons.. (the other being many of these writers work at Google/meta/whatever, so perhaps have less of an “publish or perish” incentive.)
Yep, it is all about independent replication. Eg, see this paper with an error in the theoretical justification:
https://math.stackexchange.com/questions/4271692/a-possible-bug-in-a-highly-cited-paper-adam-gradient-descent
Cited ~100k times with the error. A very low percentage of people citing it ever critically read the paper. But they tried it and the ADAM optimizer works for whatever reason anyway.
I’ve always thought of the reason as being one of benefits vs cost. What’s the benefit of a peer-reviewed, subscription-gated publication? The main thing is reputation and currency for promotions in an academic center. What’s the cost though? Lot of processing fees if you want it to be accessible, word limits that cut out so much it’s impossible to include enough details to be reproducible if you wanted, and weeks to months of delay while “peers” review some of it (but still miss the glaring errors because no peers is an experts in everything.)
If I had my choice, I’d never publish to a for-profit journal with page/word limits where important details have to be moved to the supplemental information and the content goes behind a paywall. But I need to for my job …
Anoneuoid –
If anything, use of peer review and RCTs is highly correlated with poor/unreliable research. So they are neither necessary nor sufficient, and quite possibly counter-productuve.
I think there’s a problem with your logic.
I take a new medication (Camzyos, a treatment for obstructive hypertrophic cardiomyopathy) that almost certainly improves my quality of life and probably will extend my life beyond how long it would have lasted otherwise. I previously took another medication (disopyramide) which helped to treat the same condition (with some problematic attributes that recommended switching to Camzyos).
No doubt, there are many, many other people who have improved quality and/or length of life because of other recently developed therapeutics and treatments like medications for hypertension or joint replacements. Consider that not long ago many knee injuries that now are easily repaired would have been lifelong crippling injuries.
I’d say that in those types of situations, RCTs and peer review were likely integral parts of the development process. Asking if they were necessary or sufficient are the wrong questions imo, because they hold demonstrated benefit hostage to an unrealistic standard. Perhaps more or better achievements would have been made in their absence but you certainly don’t know that, so asking if they were counterproductive is basically sophistry.
The world isn’t binary. Just because peer review and RCTs are far from perfect isn’t a reason to conclude we’d be better off without them. Overall measures like life expectancy or certain measures of cancer survival rates jumble together clear benefits with myriad confounders that don’t negate those benefits but obscures them by “averaging them out.”
So…
The vast, vast majority of benefit we derive from science was accomplished without either.
Seems to me a logically flawed construction. First, it divides up “benefit” into arbitrary bins where some benefit is dismissed and some lack of benefit is elevated. Second, it suggests that the growth in benefit (which I suspect you’re measuring very loosely) would have maintained or accelerated absent RCTs and peer review. That seems to me to be a very complicated counterfactual. It also suggests that we already can measure the benefit going forward all of the scientific output that has occurred over the past 70 years. That seems simplistic to me. Just take how revolutionary AI may prove to be, or the future benefits from many other areas of research done during the last 70 years, which might not yet be manifest. Perhaps some important future discovery will necessarily hinge on research which hasn’t produced something of value yet, but which represents a necessary step before future knowledge.
You can critique current research practices without being so dramatic.
Yes there are lots of effective therapies developed in the last 30-40 years associated with publications in the peer-reviewed literature and clinical trials to assess efficacies and safety. I know quite a few people (my mum included) who’s quality of life has hugely benefited from development of anti-VEGF treatment for wet macular degeneration (scientific publications in the 1990’s; clinical trials in the late 1990’s/early 2000’s; drug approvals starting in 2000’s).
Quality of life and life expectancies have also hugely improved for individuals with the dominant form of cystic fibrosis (CF) through recent CFTR correctors/potentiators (basic science published in peer-reviewed journals from late 1980’s and continuing to the present; clinical trials from around early 2010’s; drug approvals in late 2010’s). Life expectancies for CF patients in the 1970’s was around 10 years; in the early 2000’s around 35 years; now an individual with the dominant mutation can expect to live into their 60’s – maybe further, since research into improved treatments continues.
There are quite a few examples one could choose – very effective HIV-AIDS treatments through (eventually) development of triple combination therapies (research in peer-reviewed journals from early 1980’s; clinical trials from mid-1980’s and continuing; drug approvals for HIV drugs from late 1980’s and continuing).
Actually, the Cystic Fibrosis (CF) story illustrates some of the difficulties with understanding and treating diseases that are more complex than the “easy” “replacement” therapies for (for example) insulin or growth hormone or calcitonin or thyroid hormone deficiencies. Successful treatments for CF required development of methods for identifying gene loci and gene sequencing (late 1980’s), understanding the nature of the disorder (the “mutated” CFTR protein misfolds), developing a high throughput screen to test the effects of drug libraries on CFTR folding and function, and then clinical trials from around 2016. This was not easy and unfortunately, most of the “easy” disease treatments have probably already been discovered (of course, if people stopped smoking and ate moderately and healthily and exercised and didn’t drink too much alcohol nor take drugs and shoot each other and didn’t live in hugely unequal societies etc. – those would be “easy” solutions to enhanced well-being and life expectancies but sadly these are not politically-palatable).
The last time someone brought their own treatment here it was “unanon” for an allergy medication where the RCTs showed ~90% decrease in symptoms for the placebo, and ~95% for treatment.
Ie, most people get treated during a “flare-up” because it finally triggers them to seek care.
Anyway, is this the study you are referring to?
https://pubmed.ncbi.nlm.nih.gov/32871100/
Essentially the RCT reports about 20% of people feel better (less shortness of breath during minor exercise, etc) after placebo, but 40% after the treatment. There were no severe outcomes (eg, mortality) in either group.
That *is* a very good result for modern medicine, in the short term at least.
@Chris (and Joshua)
Even accepting these (non-quantitative) claims as accurate, they are all rounding errors compared to discovering something like electricity, cheap computations, or eradicating disease (scurvy, etc).
Anoneuoid –
Essentially the RCT reports about 20% of people feel better (less shortness of breath during minor exercise, etc) after placebo, but 40% after the treatment. There were no severe outcomes (eg, mortality) in either group.
The effect of my medication is assessed by an “objective measure,” being the left ventricular outflow tract pressure “gradient,” as measured by echos, over a sustained period of time (both before and after starting the medication). The increased ability for sustained moderate–to vigorous-exercise isn’t the measure of the effect but the manifestation of the improvement caused by a measured effect of the medication. The results are quite dramatic. An N = 1 doesn’t mean anything, but my results have clearly been “replicated” widely (it’s clear through my interactions with my providers, through their expectations and their predictions and caveats).
And the point is how that improvement affects my quality of life (not terribly dramatic as compared to the previous medication, but the improvement with that medication was dramatic, and this medication doesn’t have some of the important complications, for me, the previous one had).
And I have relatively little doubt that this improvement, meaning I have/will be able to engage vigorous exercise for many years, with my heart needing to do much less work) will have a benefit in lifespan all else being equal.
Ie, most people get treated during a “flare-up” because it finally triggers them to seek care.
Lol. This has nothing to do with a “flare up.” I had this condition diagnosed some 15 years ago. Many people have had it diagnosed prior to this recently developed medication. It’s not just a medication someone goes on when there’s a “flare up.” This is a new treatment that is (1) at least sometimes more effective than older medications and (2) doesn’t have electrical complications my older medication had. I doubt that few patients, if any, are given this medication because of a “flare up.”
Even accepting these (non-quantitative) claims.
Always intersting to see non-(meaningfully) quantitative claims made in response to non-quantitative claims that were in response to non-(meaningfully) quantitative claims.
According to the RCT ~20% of patients who got placebo had improved quality of life.
You appear to reject the most obvious explanation. So why then?
Anoneuoid –
Last contribution to “garbage time.”
According to the RCT ~20% of patients who got placebo had improved quality of life.
You appear to reject the most obvious explanation. So why then?
I already explained that. I had a dramatic improvement (and counting) in an objective measure. I’m not only going on “feeling better” or improved quality of life a measure of the efficacy. There’s temporality, there’s a dose effect, and it’s longitudinal.
So then there’s a question of why you need the obvious explanation repeated?
Yes, I mentioned RCTs. Then a drug was brought up by you (with no reference to any RCT). When I then introduced the RCT for that drug into the discussion, you ignore, even belittle (as subjective), the primary outcome of said RCT.
I am unable to identify any relevance of your comments to the discussion (or science in general), garbage indeed.
“The justice system….should prioritize rehabilitation over punishment or revenge.”
Except that doesn’t work. Dreamy dreamy i-wish i-wish policy. You can’t seriously believe that this man doesn’t know right from wrong. That’s what got us here in the first place: no credible *punishment* for stealing the public’s money for bogus research.
He should be behind bars, right along with the bogus research floggers.
“corrective action could involve reforming the NIH’s grant process”
Totally unacceptable!. Piling on the red tape is another way of harming the public by increasing the cost of research and thereby decreasing its effectiveness – it means the entire burden of the misconduct is then borne by the public rather than the perpetrator! Absolutely unacceptable. The public has a right to expect both honesty and competence from such highly trained and well-positioned people. It is not incumbent on the public to jump through hoops to keep researchers from committing crimes. When researchers are dishonest, they should be punished consistently, as severely as possible and made a spectacle of to set an example for others.
Here’s an idea: this guy and others like him could be used to amp up the deterrent effect on street criminals: he should be deployed in prisons to teach biology courses 10 hours a day (with no pay, just food and lodging). Instead of hanging out in the prison yard all day, the street criminals should be forced to take every biology course in the undergrad curriculum repeatedly until they pass it!! Now, that would be a serious deterrent to crime! Maybe he could even teach algebra and trig. Whoa, talk about scared straight!
Anon:
The current justice system doesn’t work either! It’s “dreamy dreamy i-wish i-wish policy” all the way down.
You’re right! Medieval methods of punishment, such as the brutal practice of limb amputation, proved ineffective. Similarly, the concept of eternal damnation in hell failed to deter crime. While I don’t have a definitive solution, I agree with Andrew’s assessment that our current system is also not achieving the desired results.
I wonder what percentage of the $19m went to the University of Maryland for indirect costs? When I worked in Sponsored Research (as it was known) the rate was something over 40%.
I think this would be a big part of the practical issue. Eckert didn’t receive $19 million so much as a whole bunch of people were given $19 million, a portion of which he was in charge of spending. Paying back $19 million would involve taking money from a bunch of post-docs, grad researchers, U Maryland folks who do maintenance, etc etc.
Alex:
I kinda know what you mean, but (a) there must be many hundreds of thousands that he personally pocketed, so he could start by paying that back, and (b) suppose someone stole, or fraudulently obtained, $19 million from you, and then you asked for the money back, and they said, Hey, sorry, there’s no money here anymore, I gave it all to my friends. The thief would still be responsible for paying you back, right?
To put it another way, I think it’s that researcher’s responsibility to come up with the $19 million. he doesn’t have to take it from a bunch of postdocs, etc. He stole it, he should give it back. If someone steals my wallet which has $300 in it and then gives that $300 to charity, I’m not asking the charity to give the money back to me. I’m asking the thief to make me whole–he’s the one who stole the money.
P.S. I agree that there is a practical issue that this dude probably does not have $19 million in cash lying around. For one thing, if he really does have $19 million in personal assets, or anything like that, he can surely afford a legal team that has some skill in hiding it. That’s where white-collar enforcement can come in. They can repo his house and car, empty his bank accounts, liquidate his stocks and retirement funds, garnish his salary . . . the whole deal. If they can’t get all $19 million back, they can try their best. And if he wants to go hat in hand to former postdocs etc. and ask him to help get him off the hook, he would be free to do so. Who knows? Maybe he’s such a wonderful scientist that all these former associates would willingly help him out.
Andrew, in my opinion it’s the university that’s the real thief here, and they certainly have $19M. I mean, you don’t go after the call center employee on the phone who scams you out of $1000, you go after the guys running the call center.
That’s a fair point. In the background of thinking about my comment was the idea that he didn’t pocket all $19 million and so he almost certainly can’t pay it back (there are no yachts to sell, and so on). But you’re right that he could still be held responsible for it even if there’s no chance of it all being recovered.
Alex:
Yeah, and in real life I expect they’re asking him to pay exactly $0. For all we know, the dude bought a new Lexus last week.
I mean, it seems fine to me to claw back money from the university. If they were hit hard enough with claw backs they might have to actually consider doing their jobs.
As one who has personally known two fraudulent scientists, this is always a complicated issue. Lots of years ago, I was contracted to write a chapter for a yearly review book along with a colleague. The writing was pretty straight forward until he started giving me a lot of unpublished data. I told him that we could not use this in a review which covers only published literature. He grumbled a bit and then agreed. 2-3 years later he was caught fabricating crystallography data while in the US on sabbatical. He ended up losing his academic position in Europe.
The second was a fellow post-doc at an Ivy school. He was caught fabricating a research grant as well as some papers. He sued the university but did not prevail. While his academic career was over, he went on to found some healthcare companies, but I don’t think any were successful.
Perhaps the ignominious end to their research careers is punishment enough. The real cautionary tale is the damage that Efraim Racker received because of the Mark Spector incident back in the 1980s.
Speaking of Mark Spector:
https://www.newspapers.com/article/the-des-moines-register-dr-spector-frau/25029308/
Something about leopards and spots……
Kind of an interesting paper on male/female brain differences:
https://www.psychologytoday.com/us/blog/sax-on-sex/202405/ai-finds-astonishing-malefemale-differences-in-human-brain
https://www.pnas.org/doi/10.1073/pnas.2310012121
I’m still parsing the real paper and lack a lot of expertise in brain imaging or statistics. But if you look at the clusters on the PCA-looking graph, it looks like two very different clusters. Like you could maybe even make high probability bets on images and say what sex they were from. But I’m not an expert on PCA (let alone ML/AI), so not sure how much the training affects the algorithm.
Also, not clear how the differences affect expressed behavior in socially significant (fighting, nurturing) or even stereotypical joke areas, like refusing to ask for directions or not having a sense of humor.
It’s a tSNE plot so it means basically nothing
With a little critical thinking, you might notice that the image with exactly zero overlap obviously contradicts their main conclusion of around 90% accuracy (considerable overlap).
Here before I get told off for engaging with trolls
This keeps bugging me. Here is the general idea regarding progress in cystic fibrosis: https://cystic-fibrosis.com/wp-content/uploads/2021/06/CF-life-expectancy-IAI.png
What I see is a disease originally diagnosed according to extreme pathology ~1940, then a sweat-salinity test ~1960 that both identified milder cases *and* filtered out similar pathologies due to other causes. Then genetic testing introduced ~1990 that allowed identification of even milder cases (that didn’t even have the abnormal sweat).
On top of that, there is a spectrum of mutations, and prenatal genetic testing allows selective abortion of the worst ones.
How can you attribute the increase in life expectancy only to the treatments rather than accounting for the evolving testing? The testing effect here easily looks like it is measured in decades.
And also, if life expectancy is 60 years now, isn’t that mostly due to whatever happened 60 years ago (ie, the patient surviving to adulthood)? How can we know the effect on life expectancy of a drug only released in he last few years?
@Chris
Found this:
https://www.jpeds.com/article/S0022-3476(16)31048-4/fulltext
Back of the napkin using (normal) ~76 yrs life-expectancy in 64% diagnosed CF patients, then ~1 yr life-expectancy in 36% percent: 76*0.64 + 1*0.36 ~ 49 yrs.
That 1 yr life-expectancy approximates using the original diagnosis. The 76 yr one is assuming 64% of CF patients are average in terms of life-expectancy.
Obviously those are very crude assumptions that don’t even bother with how the life-expectancy would evolve over time. Is there any interest within the CF community for actually working out the details of this type of model?
I don’t think anyone here would argue the improved life expectancies are due entirely to one drug only. There has been a variety of treatment improvements that resulted from flawed, yet still productive research practices.
I suppose we could respond to you by saying: “How can you attribute the increase in life expectancy only to the evolving testing rather than accounting for the improved treatments?”
Except that would be rhetorical gamesmanship (gamespersonship?).
Yea, you need account for everything that went into generating the numbers (the “data-generating process”). One of my favorite science examples is about the pioneer anomaly, see table II here: https://arxiv.org/abs/1204.2507
Its so beautiful how they keep finding sources of error with the goal of *not* getting significance.
Lets treat medical data the same way. Some sacred cows might need to get sacrificed but a golden age will ensue.