John Cook writes:
In the process of writing my latest post I stumbled on the observation that the skewness of linear correlation is proportional to the correlation. Have you seen this before? I assume it’s well known, if it’s true. Apparently it’s at least approximately true.
The above plot, from Cook’s post, comes from a simulation with n=100.
I replied that this result seems plausible. I guess one way to get some intuition about this is to consider the extreme case when rho = 1 – epsilon for a very tiny epsilon. There it should be possible to work out the distribution analytically. It’s funny–we talk about skewness but I don’t usually think about the numerical value of skewness.
Also, when considering transformations I’m usually working in a regression context and my goal is to transform the relationship to be closer to additivity and linearity, not to obtain a symmetric or normal distribution of raw data or residuals. So, while I find the result to be intriguing from a mathematical and theoretical perspective, I think that concerns of skewness are often overrated when they come up in statistical analysis!
John responded:
Agreed. For me it’s a yellow flag when I hear someone talk about skewness and kurtosis.
The result isn’t important, but I find it curious that it’s so simple. it’s well known that the distribution becomes less normal as rho increases, but apparently it becomes less normal in a way that’s trivial to describe, at least approximately.
I was amused that he referred to this as “a yellow flag.” That’s so charmingly precise! The usual expression is “red flag,” but John’s right, talking about skewness and correlation isn’t really a red flag; it’s a more mild concern than that, hence “yellow flag.” As a statistician, I appreciate that sort of precision in communication.
But back to the research question. My gut is that the relationship is not exactly linear, even in the limit of large N, but who knows? I think a starting point would be to evaluate the skewness of the sample correlation of a sample of size N from a bivariate normal distribution with true correlation rho, under three conditions:
• rho = 0: the skewness is then zero, by symmetry
• rho = 0.5: evaluate this by simulation
• rho = 1 – epsilon: figure this out analytically in the limit of epsilon approaching 0.
This isn’t trivial but it shouldn’t be too hard to do. It also should be possible to express the asymptotic skewness of the distribution in the limit of large N as a low-dimensional integral, which can then be evaluated numerically–this should be more stable than trying to compute things using brute-force simulation, also you can differentiate twice with respect to rho inside the integral sign and compare the result to zero, which is what it would be if the function is exactly linear. Which, again, I’m guessing it isn’t–it just seems that too many things would have to cancel out for this to work out–but, again, my intuition could be wrong on this.
So, go to it, probability and statistics students! It’s a small, well-defined research project.
And if and when you make some progress, please let us know in the comment section.
P.S. Apparently this isn’t a research project as the result is already known, at least if the underlying distribution is bivariate normal. See this comment below.
I have run across a situation where skewness and kurtosis were important – and it make me think these are worth thinking about. The basic use (perhaps misuse is a better term) of the Central Limit Theorem to establish sample sizes for inference run afoul when skewness or kurtosis are large. I’m thinking of the usual guidance about sufficient sample sizes for using a normal distribution for the sampling distribution of a mean. I had a consulting project where a government audit had used a sample size of 100, usually thought to be sufficiently large to approximate the sampling distribution using the CLT. But the data had extremely high kurtosis and skewness and I found one published attempt to determine sufficient sample sizes under such conditions (via simulations) with the resulting sample size an order of magnitude larger.
Of course, the whole inferential approach (close to, if not equivalent to, NHST) is questionable. But such “rules” about sample sizes are used, and generally do not account for skewness and kurtosis. So, I will respectfully take down the yellow and red flags.
Of course it’s skewed. It’s bound on one side.
With respect to skewness or kurtosis, I don’t view them as problems per se, but if it’s an important feature of your data, then you should try to model that too. For instance, stock returns typically have fat tails (particularly on the downside). Models assuming t-distributed residuals are a good way to take that into account.
John:
Yes, the skewness is obvious. The amusing math pattern here is that the skewness is close to a linear function of the correlation, and the math question is, how close is this to exactly linear.
Does equation (6) from https://mathworld.wolfram.com/CorrelationCoefficientBivariateNormalDistribution.html help?
Anon:
Yes, this seems to be it. It’s approximately but not exactly linear.
At a simpler level, I wonder if it follows from the approximate Normality of arctanh(rho), Fisher’s z-transformation. That is, suppose Z~N(t, 1/(n-3)), does tanh(Z) have skewness approximately linear in tanh(t) as n increases.