Someone wrote in with a complicated question about some model he’d fit, he was trying to test for endogeneity in the residuals, a bunch of things. It was a short email, just two paragraphs long, but with lots of details, and then he asked me a bunch of questions of what to do.
I replied:
Hi, sorry, I don’t know. I will just give you the general advice to write a small computer program to simulate fake data from your model and then you can apply your statistical procedures to your simulated data and repeat the process to understand the statistical properties of your methods. Best of luck.
A few days later he replied:
Many thanks for your advice, it was very helpful.
That’s satisfying. I’m posting this, partly because it’s a pleasant story and partly out of the possibility that others out there with difficult questions will see this helpful (I hope) advice.
I still think you’re asking to be blamed for a Wansink-like character taking this the wrong way:
“Just generate fake data and apply your stats to that instead”.
Andrew, This the best advice for statistical modeling. I will add: “once you understand the model, simulate data with strong deviations (outliers, leverage points,etc.) to assess the robustness of your conclusions”