Survey Statistics: Fat Bear Week

Posted on by

Happy Fat Bear Week to all who celebrate. Last year I made a cartoon called Basu’s Bears, adapted from Basu’s (1971) elephants example, a lesson on the use of auxiliary information in survey statistics.

I was inspired by Brendan Leonard, who wrote Bears Don’t Care About Your Problems.

Bears Don't Care about Your Problems bookcover

This bear creates my problems:

Here is R code to generate your own estimates and plots like the below:

In my code, I played around with distributions of pre-feast weights to better see how SRS vs PPS sampling differs. I was curious because using the pre-feast weights in the sampling design (PPS) with an unbiased estimator is the same estimator as using the pre-feast weights in the analysis via calibration. But their distributions differ with different sampling designs.

library(tidyverse)
options(scipen = 999)
set.seed(1)

N = 50 # number of bears on stage

# Assume a bimodal distribution of pre-feasts weights
# (this helps see differences between SRS and PPS sampling):
mu_x_small = 500 # average weight of small bears before feasts
mu_x_big = 3000 # average weight of large bears before feasts
sigma_x = 100 # variability in size around small or large averages

# now assume all the bears feast on about 3000 pounds of salmon:
alpha = 3000
beta = 1
sigma_e = 300 # variability in feasting ability
X = c(rnorm(n = N/2, mean = mu_x_small, sd = sigma_x),
rnorm(n = N/2, mean = mu_x_big, sd = sigma_x))
Y = rnorm(n = N, mean = alpha + beta*X, sd = sigma_e)
sum(Y<0)
sum(X<0)
sd(X)
sd(Y)
cor(X,Y)
sd(Y/X) # how proportional are pre and post weights ?

Sambo = which.min(abs(X - mean(X)))

ranger_design = rep(0, N)
ranger_design[Sambo] = 1
sum(ranger_design)

# reduce the extremeness of the compromise design so the plot is readable:
compromise_design = rep(1/490, N)
compromise_design[Sambo] = 9/10
sum(compromise_design)

SRS_design = rep(1/N, N)
sum(SRS_design)

PPS_design = X/sum(X)
sum(PPS_design)

num_sim = 10000000

That_ranger = rep(NA, num_sim)
That_compromise_unbiased = rep(NA, num_sim)
That_SRS_cal_X = rep(NA, num_sim)
That_PPS_unbiased = rep(NA, num_sim)

for (s in 1:num_sim) {
i_ranger = sample(x = 1:N, size = 1, prob = ranger_design)
i_compromise = sample(x = 1:N, size = 1, prob = compromise_design)
i_SRS = sample(x = 1:N, size = 1, prob = SRS_design)
i_PPS = sample(x = 1:N, size = 1, prob = PPS_design)

# this estimator is best if the ranger is correct in 
# choosing Sambo as the average-sized post-feast bear:
That_ranger[s] = N * Y[Sambo]

# this estimator is bad no matter which bear you choose
# but correct on average (i.e. unbiased):
That_compromise_unbiased[s] = 1/compromise_design[i_compromise] * Y[i_compromise]

# These two are the same estimators, but differ in sampling design,
# they are best if you choose an average-expanding Y_i/X_i bear:
That_SRS_cal_X[s] = sum(X) * Y[i_SRS]/X[i_SRS]
That_PPS_unbiased[s] = 1/PPS_design[i_PPS] * Y[i_PPS]
}

T_y = sum(Y)

groups <- c("That_compromise_unbiased","That_ranger","That_SRS_cal_X","That_PPS_unbiased")
values <- c(That_compromise_unbiased, That_ranger, That_SRS_cal_X, That_PPS_unbiased)

df <- data.frame(
value = values,
group = factor(rep(groups, times = c(length(That_compromise_unbiased),
length(That_ranger),
length(That_SRS_cal_X),
length(That_PPS_unbiased))),
levels = groups)
)

lines_df <- data.frame(
xint = c(Y[Sambo], T_y),
which = c("Sambo Value","T_y")
)

ggplot(df, aes(x = value, fill = group)) +
geom_histogram(aes(y = after_stat(density)),
position = "identity", alpha = 0.6, bins = 200) +
geom_vline(data = lines_df,
aes(xintercept = xint, color = which),
linetype = "dashed", linewidth = 1) +
scale_fill_manual(
values = c(That_compromise_unbiased = "orange",
That_ranger = "green",
That_SRS_cal_X = "red",
That_PPS_unbiased = "blue"),
breaks = groups,
labels = c(
expression(hat(T)[y]^{compromise~unbiased}),
expression(hat(T)[y]^{ranger}),
expression(hat(T)[y]^{SRS~cal:~T[x]}),
expression(hat(T)[y]^{PPS~unbiased})
)
) +
scale_color_manual(
values = c("Sambo Value" = "gray", "T_y" = "black"),
name = "Values",
breaks = c("Sambo Value","T_y"),
labels = c(expression(Y[Sambo]), expression(T[y]))
) +
labs(
title = expression("4 Estimators of Basu's Bears' Total Weight"~T[y]),
x = expression(hat(T)[y]),
y = NULL,
fill = "Estimators"
) +
theme_minimal() +
theme(axis.text.y = element_blank(),
axis.ticks.y = element_blank()) + 
theme(panel.background = element_rect(fill = "white", color = NA),
plot.background = element_rect(fill = "white", color = NA))


ggsave("basus_bears.png", width = 8, height = 6, dpi = 300)

11 thoughts on “Survey Statistics: Fat Bear Week”

    • Thanks, Gaurav ! I missed your write-up, it’s great ! (I will forgive that you did not replace elephants with bears. Elephants are great too.)

      What makes the estimator wobble is not HT’s algebra but the sampling plan.

      But if instead of HT algebra you use calibration to T_X you get T_X/X_i * Y_i, which doesn’t involve inverse sampling probabilities. The sampling plan affects which i is chosen, and indeed will be Sambo with 99/100 probability, and Jumbo with 1/4900 probability, but maybe there isn’t much wobble between X_Sambo/Y_Sambo vs X_Jumbo/Y_Jumbo. Folks can change my code from:

      sum(X) * Y[i_SRS]/X[i_SRS]

      to

      sum(X) * Y[i_compromise]/X[i_compromise]

      to see how this changes the calibrated estimator.

      As I write in the code, the calibrated estimator is algebraically the same as the HT estimator with PPS sampling. The best sampling plan for the calibrated estimator would choose i such that Y_i/X_i is average, but that isn’t information available up front.

      Reply ↓

  2. Off topic, but I wonder if you have read at all about the Mar Saba Secret Gospel of Mark forgery (or not) mystery. I came onto discussion of it after reading (quite late to the game) the whole Jesus Wife Gospel kerfuffle.

    In addition to the general issues of research fraud/hoax that I think interest you, there is sort of a (casual usage of the term) “Bayesian” aspect. As there is no smoking gun, just a lot of circumstantial evidence.

    In addition, there is even sort of a micro statistics aspect, with a 2009 stats journal article. Supposedly vocabulary analysis shows the document is “too” much using the expected vocabulary. E.g. it would be the most distinctively “Clement” (ancient author) article of all the Clement articles (as if someone overdid imitation of quirks).

    There’s also a Columbia connection. The forger (or discover) was a brilliant Columbia professor–if that is not redundant! ;)

    Reply ↓

    • Hi Anonymous, I haven’t read about that ! Without knowing any details about this case, I think you’re right to look to Bayesian statistics when evidence isn’t a “smoking gun” (when we have that kind of data, priors won’t matter).

      Reply ↓

      • See https://en.wikipedia.org/wiki/Mar_Saba_letter for a starter (not a perfect overview, I’ve Googled a lot now and even ordered some books).

        I’m just a casual reader, not a mathematical statistician. But have a functioning “smells like BS” detector.

        My bias is that “if it seems to good to be true, it probably is”. And there’s a lot of things to give pause on the Secret Mark story. All that said, it would be nice to find a smoking gun (probably only possible in the “prove it a forgery” direction).

        Things that give me pause (big list, not priority order):

        1. “Discovery” was by the “interpreter”, Morton Smith, not by some shepherd (Dead Sea Scrolls) or in a building excavation. I.e. the provenance starts at him.

        2. Discoverer did not have a large history of finding relics. Almost all his work was more classics stuff (interpreting words and translating known documents, not archeology). [Maybe similar to point above.]

        3. The discovery was a “banger”, not minor.

        4. The topics (magic, homosexuality) fit interests of Smith’s (some of it alluded to in his thesis and previous publications) I.e. right up his alley. A little like the GJW story, although there it was more the fraud identifying her as a mark and her not wanting to disprove something that helped her…but KK was not fabricating GJW!

        4.1. The construction of the supposed letter seems to have a huge amount of easily explained content, rather than the confusion you’d expect jumping into correspondence between two people dealing with a known issue. E.g. the “secret” Gospel of Mark is quoted rather than just referred to. The Carpatians are referred to. Etc. It just seems all laid out, rather than confusing and leading one to want other letters to understand what was going on.

        4.2. And it’s not just an issue of the secret gospel cited, but that the author says Clement was telling someone to lie. And also was saying the (somewhat homoerotic and definitely non canonical) Mark Gospel was correct (just being misinterpreted). I would think if Clement was tamping down on that gospel, he’d just call it a heresy. Whole thing is rather Byzantine (pun intended), not just in the homo Mark part but in the Church factions.

        5. The Morton Smith 1973 book has a dedication to ‘the one who knows”, which seems a little wink, wink.

        6. There was a 1940 mystery novel about a heretical forgery placed AT MAR SABA monastery! [This is a big one for me.]

        7. The 17th century volume used for writing in, was not listed in the collection of the monastery as of 1923. Which would fit with the general idea that Smith (who visited the place in 41, 44 and 58) smuggled it in.

        8. The 17th century printing, even now, is readily available on the Internet for a few hundred bucks.

        9. The title page of the 17th century book was missing. This could be consistent with cutting out a note of ownership by a person or library or a price from a used book dealer. (This is not a strong piece of circumstantial evidence as a good forger would have just found a volume without the missing title page.)

        10. The last page of the 17th century printing deals with text forgery, in Latin (and even faces the start of the inscribed Greek text). Of course could be coincidence, but just feels like more “wink, wink”.

        11. Smith did not go back to secure the volume, to do additional photographs, or to do physical testing…or even to look at the volume some more in the flesh. Instead, he spent 15 years working on the topic, from just his first set of B&W photos. (Granted, we have seen from KK and GJW that the historian types are not that great at physical archeology.)

        12. Smith was an erudite scholar of all the Bible topics involved and a skilled linguist.

        13. The most thorough examination of the handwriting and grammar (by a native Greek paleographer with over 150 publications about different era Greek writing) found several grammar errors that seem to implicate a forger that was strong at period Greek…for an American. Like “smart…for a Marine”. I.e. traces of non-native.

        14. The same paleographer also found some issues with the handwriting (not as flowing as normal) as well as some letter formation. Not smoking gun, another small weight on the scale.

        15. The monks suspected him of fabricating the document, thought Smith was an odd duck, not likeable, and a glory hound. Granted…they could be biased against new heretical apocrypha. Still…they saw the guy in the flesh and that was their take. (Small weight.)

        16. The word usage in the document was “hyper Clementine”. I.e. if you look at every Clementine document, the Mar Saba document was the one that was most Clementine of all! And interesting result for a new document, not just in the middle of the pack. Also, the significance testing of rare Clementine words showed like a .009 p or chi or whatever result in terms of frequency. I.e. the hypothesis of overusage is 99%. [I are not a statistician, so forgive me if I don’t translate the math right…but you get the gist.] Note also that there was an already published list of rare words of the Clementine corpos and it’s the sort of thing a wicked smart Columbia prof would know about and how to use.

        17. [Not a factor for forgery per se, but important for evaluating the claims and counterclaims] I find the “it was not a forgery” advocates to be sophistic in picking weaker arguments of the forgery advocates (like hesitancy in the writing style) and arguing against these and assuming this disproves all the stronger points also or the totality of circumstantial evidence. I just get the impression they are more of a tendency to expert shop and try to support their thinking vice testing it.

        18. No other examples at the monastery of inscribed books or paperwork with the “inscriber” handwriting exist. (Note in ME monasteries, it’s common to be able to find/identify multiple works by individual scribes/copyists.

        19. A 1923 inventory did not list the Latin 17th century printed book, at the monastery.

        20. There is an en passant reference to pure salt that seems out of place (may be wink wink on Morton). Also, the concept of pure and smooth flowing salt is not consistent with pre-industrial salt.

        21. There is an en passant reference to “seven veils”, which is now commonly associated with Biblical Salome, but was not called “seven veils” (specifically) dance until Oscar Wilde in the 19th century.

        22. The format of the discovery (a transcribed into end pages of a 17th century printed book) seems like it could be calculated to try to get around carbon testing of papyrus fragments. It also opens the question where this thing was floating around for 1500 years before the copying. However the skill (hyper Clementine-ishness) would seem to rule out a 17th or 18th century forger. Implying it was a genuine document, transcribed c. 1700…or a 1950 forgery, mimicking 1700 Greek grammar and handwriting.

        On the pro real side:

        1. His students almost all (except one disaffected one) think he wasn’t the sort of person to screw with scholarship. Some of his peers disagree. And I personally think people are capable of crap…but overall, I still think you have to give the character point to Smith. He wasn’t a Fritz GJW conman.

        2. The Greek is pretty decent. The Clementine content is decent. (Yeah, perhaps not perfect…see contra points…but still…pretty damned good…way better than GJW Coptic and maybe the contra points are nitpicking.)

        ——-

        My net/net is like 80-90%+ a fraud. I’m an atheist, so this is not a religious bias. Although…I just may be biased against liberals and revisionists! But, net, net. I’d take 1:4 or maybe even 1:9+ odds on a bet.

        Reply ↓

        • This (recent) article does some more primary research. Just read it a few minutes ago.

          https://archive.is/lDlnL

          Fleshes out the gay suspicion. Which can be uncomfortable. But Sabar is a bulldog. He’ll turn over rocks. He looked at the “hotwife” Fritz porn. He asked in Germany if Fritz had reported priest abuse. He’ll just check…he won’t back down.

          Also, in addition to adding weight onto the gay aspect (which is not 100% confirmed, but raised in probability and some said “no evidence” in the past), there’s also a tidbit in that Smith had a particular early fascination about the Gospel of Mark and wanted to find new variants of it, supporting his ideas for variation from canon. And what does he find at Mar Saba? A new Mark Gospel variant. Kinda sooper suspicious.

          One detective idea I had was that it would be interesting to check library records (Brown, Harvard, etc. as well as the city public library) to see if Smith had checked out any of the three books associated with the possible hoax (Ignatious collection, Clementine’s writings, or especially the Mar Saba Mystery novel, prior to 1958. Not a smoking gun, but might be clues.

Leave a Reply

Your email address will not be published. Required fields are marked *