Is it really true that only 8% of people who buy Herbalife products are Herbalife distributors?

A reporter emailed me the other day with a question about a case I’d never heard of before, a company called Herbalife that is being accused of being a pyramid scheme. The reporter pointed me to this document which describes a survey conducted by “a third party firm called Lieberman Research”:

Two independent studies took place using real time (aka “river”) sampling, in which respondents
were intercepted across a wide array of websites

Sample size of 2,000 adults 18+ matched to U.S. census on age, gender, income, region and ethnicity

“River sampling” in this case appears to mean, according to the reporter, that “people were invited into it through online ads.” The survey found that 5% of U.S. households had purchased Herbalife products during the past three months (with a “0.8% margin of error,” ha ha ha).

They they did a multiplication and a division to estimate that only 8% of households who bought these products were Herbalife distributors: 480,000 active distributorships / (0.05 * 114 million U.S. households) = 0.08.

The reporter asked what I thought about that 8%. Here are some of my reactions (again, recall that I had not heard about this case before, and these reactions are based on the information provided to me):

I think there are some serious problems here.

1. Generalizing from 2000 to the whole population is not a problem—as long as the sample is a representative sample, and as long as the survey responses are correct. I did not look at the details of the survey so I don’t know about the issue of representativeness. I wonder, though, exactly how they “fished” for respondents in their river sampling. It could be that they were more likely to get the sort of person who was interested in dietary supplements. Or maybe not, I don’t know.

2. However the sampling was done, I worry about the survey responses. It’s a well-known problem that rate events get overestimated. For example (from an article by David Hemenway from 1997): “The National Rifle reports 3 million dues-paying members, or about 1.5% of American adults. In national random telephone surveys, however, 4-10% of respondents claim that they are dues-paying NRA members. Similarly, although Sports Illustrated reports that fewer than 3% of American households purchase the magazine, in national surveys 15% of respondents claim that they are current subscribers.” The mathematics is that if there is a small rate of error, it can show up as a large error in the estimate of a small population. So, just because 5% of respondents say they used Herbalife products in the previous three months, that doesn’t mean that 5% of respondents actually used Herbalife products in the previous three months. I’d guess it’s a lot less.

3. I assume that “distributors” buy a lot more Herbalife products than non-distributors. Hence, even if 8% of the Herbalife consumers are distributors, I’d guess that a lot more than 8% of Herbalife products are bought by distributors. This doesn’t address Herbalife’s claims (they talk about the number of households, not the amount of products) but it seems relevant to the discussion.

P.S. Amusingly, one of Herbalife’s points is “Fact: Majority of Former Distributors Would Recommend Herbalife to Friends and Family.” But that’s exactly what you’d expect of a still-active pyramid scheme, no? Existing members want new people below them on the pyramid. I’m not saying this means it is a pyramid scheme, but it doesn’t seem like evidence against the hypothesis!

P.P.S. I tried to choose the most appropriate category for this one.

15 thoughts on “Is it really true that only 8% of people who buy Herbalife products are Herbalife distributors?

  1. This has been a hot topic for a while now. Basically, a prominent finance guy put out a few hundred pages of research saying that Herbalife was a pyramid scheme and consequently their stock price was going to zero. This finance guy has made a very public billion dollar bet that this will happen.

    Others however believe that Herbalife is a profitable company and although it’s shady in some respects, there is more than enough legitimate activity to the company to keep it in existence. For a good take on this subject check out Bronte Capital (brontecapital.blogspot.com). Bronte Capital spends a lot of their time identifying fraudulent companies, shorting their stock and exposing the fraud. Interestingly, in this case they’ve taken the bull side of the argument and have gone long on Herbalifes stock.

    The reporter was probably doing a story on the Herbalife debate and wanted to know if the 8% was a legit number since it would support bull case for Herbalife.

    • I think the crux of the problem is whether Herbalife would satisfy some technical legal definition for being a pyramid scheme in which case it could be shut down by the government. Not every multilevel marketing scheme is so eggregous that it gets shut down. I wouldn’t have anything to do with Herbalife, but from what I’ve seen, it’s unlikely the government is going to shut it down.

  2. The ration estimate is bogus. There might be some value to the ratio estimate if there was a two part question: 1. Have you purchased HL products? 2. Are you now, or have you been, a HL distributor? That at least gets the same base for the numbers.

    This is particularly true in the vitamin and supplement area, where all the names sound similar.

  3. A typical pyramid scheme pattern is that the family members of “distributors” (marks) end up buying a lot of whatever crap in question, to save their relatives from excessive losses, or maybe just to stop them constantly bothering them with sales pitches.

  4. Statistics was always my hardest subject. I am pleased that the effort was taken to parse some of these claims, given that Herbalife is a $4 billion/yr company and, if they are a pyramid, a lot of people are being damaged. On the statistics/math side of things, I recommend The Myth of MLM Income, as it breaks down reported corporate data into actual income results – which I think means that about 99% of all victim/participants fail. See

    http://pyramidschemealert.org/PSAMain/news/MythofIncomeReport.html a full PDF copy is available, ck w/me if you want a copy, or you can order it.

    If you’d like to get some idea about the legal elements of this from a former consumer fraud attorney, take a look at

    http://seekingalpha.com/article/1090771-ackman-s-short-of-herbalife-highlights-lack-of-adequate-information-from-federal-agencies

  5. We can look at this from the financial perspective.

    Herbalife is a public company and it is required to file financial disclosures with the SEC. The most recent one reports $700M in annual sales in North America. (That’s the revenue received by the mother company; distributors add extra costs on top of that.) If 5% of households are regular consumers, that works out to $120 per household per year.

    This is improbably low. Google search informs us that Herbalife is quite expensive and provides a range of figures in the general vicinity of $100 to $150 for one month’s supply. You don’t have much room for customer turnover if you’re claiming 5% of total population.

    Without digging any further, it seems to me that an estimate of 1% to 2% of all households should be closer to reality.

    • Maybe we should just reformulate this in terms of months of supply shipped. If distributors add 50% markup (which leaves them with an average of $700/year/distributor), and one month of supply retails for $120, an average distributor sells only 18 months of supply of Herbalife per year.

      I’d call that a pyramid scheme.

  6. Andrew:

    This is off topic but you may be interested in the Edge.org’s “What should we be worried about article” where they ask 155 Super Geniuses what they think the human race should be worried about. http://edge.org/responses/q2013

    In amongst all the worries about human extinction, there was Bart Kosko’s contribution. Kosko’s big worry was the overuse of the Normal, Poisson, Exponential, Uniform, and Binomial distributions and their conjugate priors by Bayesians. Final quote:

    “Most Bayesian applications assume some form of one of these three conjugacy relations—and almost always for ease of computation or to comply with common practice.

    That is no way to run a revolution in probabilistic computing.”

    • Wow—that Kosko guy is really confused! At one point he seems to be criticizing the use of the binomial distribution for coin flipping. Getting back to his larger point about the problems with normal distributions etc., he doesn’t seem to realize that these distributions can be used as building blocks, e.g., the t distribution is a mixture of normals.

  7. Pingback: Wouldn’t it be cool if Glenn Hubbard were consulting for Herbalife and I were on the other side? « Statistical Modeling, Causal Inference, and Social Science

Comments are closed.