Seth Roberts

I met Seth back in the early 1990s when we were both professors at the University of California. He sometimes came to the statistics department seminar and we got to talking about various things; in particular we shared an interest in statistical graphics. Much of my work in this direction eventually went toward the use of graphical displays to understand fitted models. Seth went in another direction and got interested in the role of exploratory data analysis in science, the idea that we could use graphs not just to test or even understand a model but also as the source of new hypotheses. We continued to discuss these issues over the years; see here, for example.

At some point when we were at Berkeley the administration was encouraging the faculty to teach freshman seminars, and I had the idea of teaching a course on left-handedness. I’d just read the book by Stanley Coren and thought it would be fun to go through it with a class, chapter by chapter. But my knowledge of psychology was minimal so I contacted the one person I knew in the psychology department and asked him if he had any suggestions of someone who’d like to teach the course with me. Seth responded that he’d be interested in doing it himself, and we did it.

After we taught the class together we got together regularly for lunch and Seth told me about his efforts in self-experimentation involving sleeping hours and mood. One of his ideas was to look at large faces in the morning (he used tapes of late-night comedy monologues). This all seemed a bit sad to me, as Seth lived alone and thus did not have anyone to talk with in the morning directly. On the other hand, even those of us who live in large families do not always spend the time to sit down and talk over the breakfast table.

Seth’s self-experimentation went slowly, with lots of dead-ends and restarts, which makes sense given the difficulty of his projects. I was always impressed by Seth’s dedication in this, putting in the effort day after day for years. Or maybe it did not represent a huge amount of labor for him, perhaps it was something like a diary or blog which is pleasurable to create, even if it seems from the outside to be a lot of work. In any case, from my perspective, the sustained focus was impressive.

Seth’s academic career was unusual. He shot through college and graduate school to a tenure-track job at a top university, then continued to do publication-quality research for several years until receiving tenure. At that point he was not a superstar but I think he was still considered a respected member of the mainstream academic community. But during the years that followed, Seth lost interest in that thread of research (you can see this by looking at the dates of most of his highly-cited papers). He told me once that his shift was motivated by teaching introductory undergraduate psychology: the students, he said, were interested in things that would affect their lives, and, compared to that, the kind of research that leads to a productive academic career did not seem so appealing.

I suppose that Seth could’ve tried to do research in clinical psychology (Berkeley’s department actually has a strong clinical program) but instead he moved in a different direction and tried different things to improve his sleep and then, later, his skin, his mood, and his diet. In this work, Seth applied what he later called his “insider/outsider perspective”: he was an insider in that he applied what he’d learned from years of research on animal behavior, an outsider in that he was not working within the existing paradigm of research in physiology and nutrition.

At the same time he was working on a book project, which I believe started as a new introductory psychology course focused on science and self-improvement but ultimately morphed into a trade book on ways in which our adaptations to Stone Age life were not serving us well in the modern era. I liked the book but I don’t think he found a publisher. In the years since, this general concept has been widely advanced and many books have been published on the topic.

When Seth came up with the connection between morning faces and depression, this seemed potentially hugely important. Were the faces were really doing anything? I have no idea. On one hand, Seth was measuring his own happiness and doing his own treatments on his own hypothesis so the potential for expectation effects are huge. On the other hand, he said the effect he discovered was a surprise to him and he also reported that the treatment worked with others. Neither he nor, as far as I know, anyone else, has attempted a controlled trial of this idea.

Seth’s next success was losing 40 pounds on his unusual diet, in which you can eat whatever you want as long as each day you drink a cup of unflavored sugar water, at least an hour before or after a meal. To be more precise, it’s not that you can eat whatever you want—obviously, if you live a sedentary lifestyle and you eat a bunch of big macs and an extra-large coke each day, you’ll get fat. The way he theorized that his diet worked (for him, and for the many people who wrote in to him, thanking him for changing their lives) was that the carefully-timed sugar water had the effect of reducing the association between calories and flavor, thus lowering your weight set-point and making you uninterested in eating lots of food. I asked Seth once if he thought I’d lose weight if I were to try his diet in a passive way, drinking the sugar water at the recommended time but not actively trying to reduce my caloric intake. He said he supposed not, that the diet would make it easier to lose weight but you’d probably still have to consciously eat less.

In his self-experimentation, Seth lived the contradiction between the two tenets of evidence-based medicine:
1. Try everything, measure everything, record everything.
2. Make general recommendations based on statistical evidence rather than anecdotes.
Seth’s ideas were extremely evidence-based in that they were based on data that he gathered himself or that people personally sent in to him, and he did use the statistical evidence of his self-measurements, but he did not put in much effort to reduce, control, or adjust for biases in his measurements, nor did he systematically gather data on multiple people.

I described Seth’s diet to one of my psychologist colleagues at Columbia and asked what he thought of it. My colleague said he thought it was ridiculous. And, as with the depression treatment, Seth never had an interest in running a controlled trial, even for the purpose of convincing the skeptics. Seth and I ended up discussing this and related issues in an article for Chance, the same statistics magazine where in 2001 Seth had published his first article on self-experimentation.

Seth’s breakout success happened gradually, starting with a 2005 article on self-experimentation in Behavioral and Brain Sciences, a journal that publishes long articles followed by short discussions from many experts. Some of his findings from the ten of his experiments discussed in the article:

Seeing faces in the morning on television decreased mood in the evening and improved mood the next day . . . Standing 8 hours per day reduced early awakening and made sleep more restorative . . . Drinking unflavored fructose water caused a large weight loss that has lasted more than 1 year . . .

As Seth described it, self-experimentation generates new hypotheses and is also an inexpensive way to test and modify them. One of the commenters, Sigrid Glenn, pointed out that this is particularly true with long-term series of measurements that it might be difficult to do on experimental volunteers.

About half of the published commenters loved Seth’s paper and about half hated it. The article does not seem to have had a huge effect within research psychology (Google Scholar gives it 30 cites) but two of its contributions—the idea of systematic self-experimentation and the weight-loss method—have spread throughout the popular culture in various ways. Seth’s work was featured in a series of increasingly prominent blogs, which led to a newspaper article by the authors of Freakonomics and ultimately a successful diet book (not enough to make Seth rich, I think, but Seth had simple tastes and no desire to be rich, as far as I know). Meanwhile, Seth started a blog of his own which led to a message board for his diet that he told me had thousands of participants.

On his blog and elsewhere Seth reported success with various self-experiments, most recently a claim of improved brain function after eating half a stick of butter a day. I’m skeptical, partly because of his increasing rate of claimed successes. It took Seth close to 10 years of sustained experimentation to fix his sleep problems, but in recent years it seemed that all sorts of different things he tried were effective. His apparent success rate was implausibly high. What was going on? One problem is that sleep hours and weight can be measured fairly objectively, whereas if you measure brain function by giving yourself little quizzes, it doesn’t seem hard at all for a bit of unconscious bias to drive all your results. I also wonder if Seth’s blog audience was a problem: if you have people cheering on your every move, it can be that much easier to fool yourself.

Seth had a lot of problems with academia and a lot of legitimate gripes. He was concerned about fraud (and the way that established researchers often would rather cover up an ethical violation than expose it) and also about run-of-the-mill shabby research, the sort of random and unimportant non-findings that fill up our scientific journals. But Seth’s disconnect from the academic research world was unfortunate, for two reasons. First, others were not getting the advantages of his perspective; second, he was not engaging with the sort of serious criticism that can make one’s work better. I certainly don’t think an academic connection is necessary for someone to engage with criticism, but it provides many opportunities, for those of us lucky enough to be so situated.

Seth was a complete outsider in the psychology department at Berkeley for decades and eventually took early retirement while barely in his fifties. I was surprised: as I told Seth, being a university professor is such a cushy job, they pay you and you don’t have to do anything. He responded dryly that retirement works the same way. As things turned out, though, he did take a new job, teaching at a university in China. That worked out for him, partly because he enjoyed undergraduate teaching—as he put it, the key is to work with the students’ unique strengths, rather than to spend your time trying to mold the students into miniature versions of yourself.

In the late 1990s, my friend Rajeev and I would sometimes go to poetry slams. Our most successful effort was once when I read Rajeev’s poem aloud and he read mine. It turns out that it’s easier to be expressive with somebody else’s words. Another thing I learned at that time was that people have a lot of problem pronouncing “Rajeev.” It’s pronounced just as it’s spelled, but people kept trying to say something like “Raheev” but in a hesitant voice as if it was this super-unusual name.

Anyway, one I idea I had, but which I never carried out, was to write down a bunch of facts about Seth and read them off, one at a time, completely deadpan. The gimmick would be that I’d come up on the stage, pull out a deck of index cards and ask a volunteer in the front row to shuffle them. I’d then read them off in whatever order they came up. The idea was that Seth was such an unusual person that his facts would be interesting however they came out.

In order to preserve some anonymity, my plan was to refer to Seth as “Josh.” (I think of “Josh” as an equivalent name to Seth, just as “Samir” is an alternative-world name for Rajeev.) I’m not sure what happened to my list of Josh sentences—I never actually put them into an act—but here a few:

Josh rents rats.

Josh stares at Jay Leno in the morning.

Josh works on a treadmill.

Josh lives in the basement.

There are a bunch more that I just can’t remember. Seth used to stay with me when he’d visit NY (I moved to Columbia because my Berkeley colleagues didn’t want me around; Seth and I joked that we should trade jobs because he liked NY but Columbia would never hire him), but it’s been several years since we’ve hung out and it’s hard for me to remember some of the unusual things he’d do. OK, I do remember one thing: he would often strike up conversations with perfect strangers, for example asking someone at the next table at a restaurant what they were eating or asking someone on the subway what they were reading. Most of the time this didn’t bother me—actually, I found it interesting, as I could get some information without suffering the difficulty of talking with a stranger. One time, though, he blocked me: I don’t think he realized what was going on at the time, but I was chatting up someone at some event—I can’t recall any of the details here (it was something like 15 years ago) but I do remember that he joined in the conversation and started yakking his head off until she drifted away with no chance that I could see for recovery on my part. Afterward, I berated Seth: What were you thinking? You ruined my chances with her, etc. . . . but he’d just been oblivious, just starting one more conversation with a stranger. Seth was always interested in what people had to say. His conversational style was to ask question after question after question after question. This didn’t really show up in his online persona. It’s interesting how our patterns of writing can differ from how we speak, and how our interactions from a distance can differ from our face-to-face contacts.

One of Seth’s and my shared interests was Spy magazine, that classic artifact of the late 1980s. When I found out that Seth had actually written for Spy, I was so impressed!

Here was our last contact, which won’t be of interest to anyone except to me because it happened to be the last time I heard from him:

Screen Shot 2014-04-29 at 10.39.08 PM

My friend is gone. I miss him.

37 thoughts on “Seth Roberts

  1. I still can’t believe Seth is gone. Sad, very sad news.
    Andrew would you like to be part of his scientific memorial? Sometimes in August in SF?

    • Nassim, please get in touch with me. Seth was a good friend of mine. It seems his family will not be doing a memorial, but I and other friends want to do one. It occurred to me that Seth’s friends may be able to be a part of the scientific memorial in August. We really want to participate and help. I look foward to hearing from you.

  2. Pingback: Assorted links

  3. Pingback: Andrew Gelman on Seth Roberts | Entertaining Research

  4. Seth was part of the life of this blog, at least to me, who has been following it since 2005/06. I remember your discussions with Seth here on the blog, and I was always amused about his ideas. He surely seemed to be an amazing person.

    I’m sorry for your loss.

  5. Seth was a very interesting person and is what Andrew would call “An Object of the Class ‘Seth Roberts'”: I found him frustrating and sometimes irritating but I always got something out of speaking with him, so even though I wouldn’t call him a good friend I looked forward to having lunch with him a few times a year for conversations that were always interesting, ranging over a wide range of topics and lasting a couple of hours. He always had interesting things to say about his experiences in China, and of course we discussed items of long-standing interest to him such as methods of scientific inquiry in general, and research on weight loss, sleep, effects of different diets, and so on.

    Seth had a lot of bad ideas but I came to realize that he exemplified Linus Pauling’s observation that “the best way to have a great idea is to have a lot of ideas.” If you have a lot of ideas most of them are going to be bad but you will occasionally have a good ones, and that was Seth’s approach. I don’t think he was as discriminating as he should have been about distinguishing the bad ideas from the good ones — I thought he was far too quick to be convinced by small amounts of evidence — but he did have genuine insights and I came to agree with him on a lot of points.

    To give an example: Seth was scornful of the conduct of mainstream medicine, especially as relates to behavior (as opposed to, say, drug efficacy, where he didn’t strongly object to standard double-blind trials as far as I know). Someone would want to test whether eating such-and-such a food or such-and-such a behavior would lead to weight loss, and would enroll a bunch of people into a program in which they would eat the food or do the behavior a certain number of times per week, and compare them to a set of controls who don’t eat the food…Seth thought these sorts of trials were useless: “They’ve been doing these for fifty years and they haven’t found anything that makes people lose weight,” he said. I don’t think he had a theoretical objection to the approach, it was more empirical: this approach has been tried literally thousands of times with no success, and sure, maybe someday you will stumble on something if you are very lucky, but this is a ludicrously inefficient way to do it. I used to pretty much accept the idea that big controlled studies are the right way, maybe the only way, to test these food or behavior-related hypotheses, but Seth largely convinced me that it’s far more efficient to track individuals (including oneself) over time, and find what works that way. Seth had the courage, or the I-don’t-give-a-damn-ness, to say “I don’t care if everybody disagrees with me, I don’t think large controlled trials of this stuff are useful at all and I do not agree that they should be the gold standard for determining whether a weight loss approach works.” I think he went too far — I think such trials _are_ the gold standard — but I agree with him that other approaches are more efficient for screening for things that _might_ work well, and throwing out things that almost certainly don’t. Basically I agree with him on this issue more than I disagree.

    I’m conflicted about whether to say some negative things too, and not just because one “shouldn’t speak ill of the dead” — I’d be hesitant if he were alive, too! But I’m writing this mostly to document some of my thoughts about Seth and to share them with other people who knew him, and for both of those purposes I feel like I should be honest about his flaws as well as his virtues, so I’ll say a few things that aren’t complimentary.

    Seth’s disdain for much of mainstream scientific method made him a sucker for the scientific underdog, and, in my opinion, overly skeptical of scientific consensus. We all know examples like the Continental Drift and heliobacter pylori, in which scientists “knew” something that was later proven completely wrong. Seth thought lots and lots of scientific knowledge was like that. I don’t. And he was over-fond (I think) of ‘Just So’ stories as regards human evolution, with an example being his explanation about why seeing faces in the morning helps one sleep at night (I won’t go into the explanation here). This made conversations with him sometimes perplexing and/or frustrating. For example, he was a global warming skeptic to the extent of nearly being in the denialist camp, but he couldn’t really articulate why; he just felt that all of these scientists who are so damn sure couldn’t really be so damn sure, so they were fairly likely to be wrong. I think it’s perfectly reasonable to suspect that scientists are over-certain about the estimated effects of greenhouse gases, but it’s far too big a jump to say that therefore they’re fairly likely to be wrong. What made this especially frustrating is that, unlike diet and weight loss where Seth knew quite a bit, he didn’t even pretend to know anything about the climate or how to predict it; his skepticism was based entirely on sociological grounds, just feeling like the scientific establishment was probably wrong about this as they are wrong about so much else.

    As Andrew pointed out in his post, Seth was unusual in interesting ways. He’s the only person I know who had a treadmill in front of his desk so he could (and did) walk while he worked; who treated himself as an experimental subject for years; and (as Andrew mentioned too) he’s the first person I ever heard of who thought about what life was like for most of human evolutionary history and tried to use that information to figure out what would be good for him.

    I was looking forward to another couple of decades of interesting/frustrating/thought-provoking lunches with Seth and I’m very sorry that I will miss out.

      • Bruce:

        Phil can respond better to this than I, but just very briefly: I don’t think that anyone thinks that “the absence of global warming is evidence that it really exists.” My impression is that there is a lot of evidence of global warming from many different sources, and the argument that the scientists are making is that recent temperature behavior is consistent with a greenhouse warming effect. There are many data sources; it is not required or even expected that every data source will, by itself, provide evidence for a particular hypothesis, but it is appropriate for proponents of a hypothesis (in this case, global warming) to explain how each piece of data fits into their theory. Finally, I doubt it’s true that you’re “not smart enough to understand their reasons”; after all, their reasons are explained clearly enough in the news article you link to. But, again, these are reasons that the recent temperature pattern can be consistent with global warming, they’re not reasons that the pattern is, all by itself, evidence for warming.

        • Andrew,

          With all due respect to you, I’m a big fan of Occam’s razor. Consider two hypotheses. (A) the globe is not warming and the temperature is not increasing, vs. (B) the globe is warming and the temperature is not increasing but here’s all the reasons it should be increasing but it just doesn’t happen to be increasing now but it will increase in the future.

          A large part of my scepticism derives from witnessing the multiple abuses that Mann and his coterie has inflicted on the data, beginning with the so-called “principal components” method that Mann programmed in fortran (rather ineptly), which will take a zero mean noisy series and produce a hockey stick. And then conveniently smoothing over the Medieval Warm Period and the Little Ice Age (both of which appear in the 1990 IPCC report, but somehow disappear as if they never existed in the 2001 IPCC report).

          But enough of global warming in this thread. I offer a tribute to Seth, whom I never met but with whom I traded several emails over the years after discovering him through your blog: his work and his advice improved my life dramatically. Every morning when I awake well-rested and without an aching back, I thank him.



      • There are two separate issues: what was the basis of Seth’s beliefs about global warming, and what is a reasonable basis for thinking about global warming. My point about Seth — and the reason it constitutes a criticism (which, unfortunately, Seth is not here to reply to…but I did discuss this issue with him several times — is that the answer to those questions is not the same. Seth believed what he believed, not because he followed the science or even followed the scientific debate, but merely because he thinks scientists can’t know what they think they know and are therefore probably wrong. I just don’t think that’s reasonable.

    • Phil:
      > some negative things too
      Trying to understanding weaknesses and strengths seems fine to me.

      > scientists who are so damn sure couldn’t really be so damn sure, so they were fairly likely to be wrong
      Yes, I believe that’s true, but not wrong about everything – though very likely what the true uncertainty is.

      • “very likely [wrong about] what the true uncertainty is.”

        AGW is similar to the HIV-AIDS hypothesis. There are tons of indirect data and theory backing the hypothesis, but also some glaring red flags. In the case of AGW it is that the temperatures on Venus at 1 atm pressure are ~1.176 those on earth, which is what is predicted solely due to distance from the sun. In the case of HIV causes AIDS, no one has ever been able to purify the virus directly from an AIDS patient. The proponents have not been able to explain why these anomalies exist, but (for better or worse) think there is enough other evidence backing their theory to chalk them up to “something unknown”. They can’t explain it but don’t want to admit they don’t know so you have endless argument as they make excuses to avoid appearing uncertain. This is “unscientific” behaviour, so it raises doubts about credibility of the other evidence. Like Feynman said, the secret sauce of science is utter honesty.

    • I remember Seth even arguing that the common story about H. Pylori is wrong. He thinks everyone has it, the real question is whether your immune system is compromised enough to give you ulcers or whatnot.

      • It is now standard to look at H. Pylori as a common part of our stomach system, and its overgrowth as the problem. There may also be factors like its acid-reducing properties and its prompting of reflux that turn it from normal to unhealthy.

        Funny thing is I find many doctors who suspect ulcers will automatically simply put someone on antacids, or even suggest old fashioned diet changes. It’s like we’ve moved on to think of H. Pylori as sometimes ulcer causing and sometimes not while many GPs still haven’t gotten the memo that it’s involved in ulcer creation at all.

    • Thank you for your candor. I totally agree. Seth was a catalog of interesting ideas, but I feel he often chose them simply because they were interesting. I found some of the stuff he wrote on vaccines, ultrasounds, and global warming to be pretty woo woo. It was frustrating at time to talk to him because of that.

      And I can’t stop thinking about his flax consumption– he introduced me to it and I personally discontinued because I developed clotting problems. In a young woman it’s much easier to ID abnormal clotting because you bleed every month, but in a man it might not be obvious soon enough. Abnormal clotting can cause strokes.

      That said, it’s a tragic loss and he introduced me to a lot of great ideas as well.

  6. One possibility for his increased rate of success in self experimentation could be that he gave people the confidence to do their own self experiments and then took those results seriously.

    I believe he derived enjoyment out of generating new ideas to test and proving to the world that the effect he found was indeed real is not enjoyable.

  7. I corresponded with Seth at around 1998-2000, spoke with him on the phone, and read several of his papers. I encountered nearly everything that Andrew said about him. Phil’s comments are also right.

    Seth was a great person, interesting, friendly, and smart.

  8. I never met Seth in person, but he and I corresponded by email (and occasionally spoke over Skype) for about four years.

    I liked and admired Seth. He encouraged me in my self-experimentation, and he treated my work seriously. Seth had a low-key but infectious enthusiasm, and his interest kept me motivated. I also admit that I found the attention flattering. Here I was, just some random schmoe on the Internet, and I was collaborating with a tenured psychology professor. How cool is that?

    I tried Seth’s Shangri-La diet and ended up losing a fair amount of weight. I had been keeping careful track of my weight for ten years, and I shared my dataset with Seth. He blogged about it and encouraged me to pursue other types of self-experimentation. More recently, I had been doing some reaction-time experiments, one of which ended up contradicting Seth’s theory that eating soy products is detrimental to brain function. To his credit, Seth didn’t bury the results – he published a blog post with a link to my soy-related experiment.

    Seth also had a tremendously positive effect on my wife, although she never communicated with him directly. My wife had long-standing digestive problems of unknown cause. Conventional medicine didn’t help. Seth often blogged about the benefits of eating fermented foods, so my wife and I experimented with soy yogurt, miso soup, kim chee, etc. Nothing worked very well, until we tried kombucha. Drinking kombucha daily has essentially cured her issues (although the symptoms return if she stops drinking the tea). Even if that were Seth’s only contribution, my wife and I would be always grateful to him.

    Seth’s iconoclastic approach to science really resonated with me. I used to work for a pharmaceutical company, and I was dismayed by the underhanded marketing tactics used in the industry. If the science didn’t support the intended marketing message, well… so much the worse for science. Seth’s whole approach was to empower the individual to do his own research, to think for herself, and to reject established beliefs if those beliefs didn’t make sense.

    I really miss Seth. Even though it’s only been a few days, I’ve already seen a couple of articles that made me think, “Oh, Seth would love this. I should send him a link.”

  9. Thank you for these very recognizable memories. I felt Seth’s influence quite a lot through his involvement with Quantified Self. He was constantly generous with his ideas and techniques. He always wanted to tell you how you could improve your experiments. If you didn’t want to hear it, he might keep telling you anyway, but most people in our meetings _did_ want to hear it, and his ideas were often very helpful. Seth wasn’t above showing a newcomer learn how to run a program in R, or remake a relevant chart, or transform some data (for instance, to compensate for practice effects). He gave a talk a couple of years ago at the Quantified Self conference in Europe about “personal science” in which he argued that non-professionals are destined to make more important contributions to understanding health than are clinicians and medical researchers. His reasoned that accurate measurements are becoming easier to make, analytical techniques are becoming easier to learn and apply (both due to software and to the capacity to find expert advice online), and, most importantly, people working to solve their own problems are more motivated than professionals to look for true answers.

    I’m in some shock that he’s suddenly gone, he was such a constant presence and contributor. If there is to be a public memorial, I hope some of us can find out and be there.

    Here are a couple of links. The first is to a 1981 paper by Seth’s undergraduate advisor at Reed College, Allen Neuringer, on Self Experiment. It’s interesting to think about the proximate origin of Seth’s approach, and his almost single handed devotion to continuing it. (Of course, there is a tradition here that goes back much further, but I think the Neuringer paper gives a sense of where Seth started.)

    The second is a video of Seth talking a few years ago to a Quantified Self meetup about designing self-experiments:

    • Gary:

      The Neuringer article is excellent. I’d never seen it before and now I wonder why Seth did not mention it more prominently in his own article on self-experimentation, given that it shares so many themes with Seth’s work, including the topics being studied, the use of simple graphs to assess outcomes, and the interactions with undergraduate students. Seth did cite the Neuringer paper in his 2005 Behavioral and Brain Sciences paper but only in passing, to refer to one of the student experiments that were reported there. In 1998, Seth published a book chapter with Neuringer on self-experimentation, but based on his later writing it was not so clear to me that Seth’s approach was basically all a continuation of Neuinger’s 1981 paper.

      • I hope my citation to the Neuringer paper wasn’t taken as a claim that Seth’s work was derivative. Given his extreme originality that would be a mistake worth correcting right away. My point was that in his pursuit of self-experiment he almost single-handedly carried forward a scientific research program that had its roots in his own academic training. Although Seth sometimes appeared to be a person carrying on his own completely idiosyncratic agenda, I saw him as advocating for the importance of a neglected and almost unnoticed branch of science. He was often insulted with terms like “woo woo” and I think the way he worked invited these insults, but I hope that his influence lasts.

  10. I was friends with Seth for 17 years and had many conversations with him — he helped me a lot. What the world needs is controlled double blind experiments which confirm or disconfirm his many “findings”/claims/hypotheses. Over a 14 year period I have lost an average of 20% of my body weight by consuming substantial amounts of calories without flavor, so I don’t need it confirmed that calories without flavor reduce hunger — in fact, one can transiently confirm this for one’s self in the course of a day or two’s self experimentation.

    But the world needs it. Seth was right that we should not be an evidence snob; we should not let the perfect (controlled double blind experiments) be the enemy of the good (rigorous self experimentation). But I agree that he was too quick to assert general causal relations (e.g., that butter improves brain function) based on his own self experimentation. The great value of self experimentation lies in the plausible hypotheses it generates.

    His strong claims — ones where we have strong anecdotal confirmation — need rigorous science. We need to know if a teaspoon of honey just before sleep (and a piece of fruit 3 hours before) improves sleep; we need to know whether one-legged standing improves sleep (I find that spending some time carrying heavy boxes when helping a friend move improves my sleep); we need to know whether random stimulation improves work speed and efficiency; we need to know whether exposure to faces in the morning improves mood the following day (he presented a powerful case study of a person with bipolar disorder apparently being helped by it).

    He was the one person who insisted to me in 1997 that if I stopped being a night person (going to bed at 3:30 a.m.) my mood would improve. Three clinical psychologists and three psychiatrists I’d seen made no mention of this. Now, as I understand it, good circadian rhythms are considered important for treating depression.

    We can’t currently know how many of Seth’s “successes” were “illusory” a function of “placebo” or something else. But if we try them out and find that they help us, and continue to help us, that counts for a lot, even if their success is somehow a function of our expectations.

  11. Thank your for this.

    Here’s a methodological suggestion. I strongly believe that echinacea tea helps me head off colds — if I get run down and develop a sore throat, which for me is always the precursor of a ten-day long cold, several glasses of echinacea tea will make the sore throat go away over 50% of the time. But that doesn’t seem to work for most other people, even my sons. I’m not surprised — immune systems are highly variable from person to person.

    Standard studies of echinacea have come up with mixed results. That’s hardly surprising. If echinacea works for, say, 2% of the population, you need a massive sample to see a statistically significant result. And maybe it makes 2% of the people worse off, so there is no net effect on the population at all. But, even in that case, it would be good if echinacea were used by the 2% if benefits and avoided by everybody else.

    But why not do a traditional experimental study, but only on people who believe echinacea benefits them? I’d sign up for such a study.

    • Yes, why not?

      One issue is, why do the people who believe echinacea benefits them believe that? Perhaps they ‘always’ have long colds, but a few times they try echinacea and the cold quickly goes away. (I would guess that this is the usual mechanism for why people thing echinacea helps them). Maybe the echinacea did in fact help, or maybe that cold was going to go away more quickly anyway; some people were unlucky to have long-lasting colds in the past, or were lucky to happy to get some short ones that coincide with taking echinacea: immune systems aren’t just highly variable between people, and between viruses. So, even with your suggested study, and even if some people are helped by echinacea, there will still be some people in the study who aren’t actually helped with it. But — your point — if echinacea does help then the proportion of such people will probably be much higher in your proposed study than in a random selection of people.

      So I agree, a study such as the one you propose could be useful. The paragraph above isn’t meant to contradict that at all, only to point out that you will still have a signal:noise problem, it’s just that it should be a smaller problem. How much smaller is impossible to know beforehand, because it depends on what fraction of people echinacea actually helps, compared to those who think it helps. But, as you say, why not?

      More broadly, your point about interpersonal variability is a very good one (and was also a major theme in Seth’s thinking). Different things work for different people. The holy grail for a drug company or doctor is to find something that helps everybody, and it’s still worth searching for things like that, but there are surely lots of things that will only help certain people and it’s worth looking for those too. As Seth said in the talk in the video that Gary posted, if all you want to do is find something that works for you, and you find something that does work, it doesn’t matter if you’re one in a million, it’s a success.

  12. Pingback: Faces | nydwracu niþgrim, nihtbealwa mæst

  13. Thank you for sharing your memories of Seth. I never met him in person, but he had a profound effect on my thinking. I have a son with classic autism. I was driving myself crazy reading reams of research to decide what was worth trying. After I’d been reading Seth’s blog for a while it dawned on me that I could do my own experiments. I have learned a great deal about my son and made significant improvements as a result. The best is when I tried giving him honey at bedtime, and that allowed him to finally sleep through the night consistently. Until you’ve gone five years without three consecutive nights’ sleep, you can’t imagine what a gift that was.

    I emailed with Seth for the past couple of years. I still find myself thinking “I should send this to Seth” when I come across an interesting item. I will always be grateful for his interesting ideas and insights. His untimely death has been heartbreaking.

  14. My only personal interaction with Seth was at JSM a few years ago. I had read his blog a few times but was unprepared for his passion and his powerful testimonials for his unconventional methods. He got the biostatisticians up in a lather with his dismissal of randomized trials dogma, and despite my training and belief in the dogma, the biostat crowd came across as belligerent and unyielding, while Seth came across as a pioneer in a new scientific paradigm. It was electrifying. It is the rare talk (at JSM!) that literally changes the way you think about science.

    Thank you Andrew for this wonderful and honest tribute.

  15. Pingback: Links to Remembrances on Other Sites | Seth Roberts Memorial

  16. Thank you Andrew Gellman. Your comments capture much of Seth as I knew him too. My connection with him was more tangential but felt strong. The only period in which I had some regular interaction with him was more than forty years ago when we were both high school students. He was a few years younger, but we somehow touched base with one another in the various spheres of the high school world. He was already as you describe him, minus the advanced degrees, but known as the most brilliant mathematics student ever to cross the grassy slopes and cement stairways of Tam High. He had insights and ideas that would startle me, always put in his somewhat blunt and often socially ill-timed way. I doubt he was questioning strangers in restaurants yet, but he had no sense of the ins and outs of social interaction. He seemed lonely and yet able to make his way. I always felt protective toward him, but did not quite know how to be his friend. The rest of his career–including his research interests, forays beyond academia, and lack of conformity with the academic norm–did not surprise me in the least. When I returned to the Bay Area after my own adventures in academia we would get in touch from time to time. It was actually quite hard to see him face to face. He had discovered that exposure to human company (he usually described it in terms only of the human face) was a harmful practice if carried on in the evening. I was practicing medicine at the time, and could rarely get away for any sort of lunch. We met for extremely early dinners twice, spoke on the phone a few times, and exchanged a few emails. I felt protective towards him, and admired him deeply, but always left our interactions with a sense of amazement and consternation as well. His preoccupation with self-experimentation seems revealing and moving to me. I think he may have somehow felt more objectified and separate than most of us do, all the while nurturing a living, original, and full sense of self. I think he turned his mind to finding a fitting and generous role in life and among the specimens of humanity whose social worlds he inhabited. He was wonderful, creative, and brave.

    • Ellen:

      Thank you. It is interesting how many of us who lag in social maturity in high school manage to fit in later in life, but Seth never did. It is good that he found an online community of people who valued him.

  17. Pingback: The insider-outsider perspective (Jim Bouton example) « Statistical Modeling, Causal Inference, and Social Science

Leave a Reply

Your email address will not be published.