Parthsarthi Joshi writes:

There is a stark problem that I have observed with how data science is taught in most books and tutorials – the concepts are taught on an individual level but fail to give a complete idea. The so-called “real-world” projects are carefully orchestrated like an “act” already setup by the instructor. The students are not given the taste of the real world – where data is messy, models do not continuously improve, and assumptions are almost always broken. We teach them the concepts of statistics and ML but not the art of data science, which includes curiosity and hit-and-trial experiments.

I am writing a one-of-its-kind book on Data Science that would be completely based on projects. The data will be messy, not everything we do will improve the performance. The aim is to give a feel of uncertainty and challenges that promotes learning from failure and develops an analytical bent of mind.

I agree with Joshi’s concerns, and it’s something that’s bothered me for a long time. Textbooks and research articles are full of examples that work, they’re full of estimates that are a bit more than 2 standard errors from zero, everything’s a bit too clean.

I’ve tried to push against this in my own teaching and writing. Here are some examples:

1. Decision analysis textbooks often use examples that are trivial and artificial (a college student is trying to choose an apartment or dorm room, balancing the factors of cost, amenities, and commute time) or important but vague (a company is deciding where or whether to build a new plant, it’s an interesting problem of decision making under uncertainty but the details are not given). A bit reason I wanted to do the full decision analysis with Phil for the home radon problem was that I wanted a real problem, with real data, going from beginning to end.

2. A few years ago there was a heart-stent study reported in the news. The experiment yielded results that were not statistically significant (the p-value was 0.20) and it was inappropriately reported as a null effect. We looked at the paper carefully, and a more reasonable analysis gave a p-value of 0.09–still not reaching conventional levels of significance. We wrote this up, and it’s a good example where the result is not clean. Again, something you don’t usually see much discussion of in textbooks, applied research papers, or articles on research methods.

3. Our book Active Statistics is full of stories with real-world complexity.

4. Appendix A of Regression and Other Stories, on computing in R, has examples where you have to download and clean the data. One problem with a lot of computer-friendly textbooks is that they’ll have clean data already loaded into packages so you can just click and do the analysis. I like there to be a bit of a struggle, to give a sense of what data analysis really feels like.

We try also to convey a sense of real-world complexity in the worked examples and homework problems in our textbooks.

The flip side of this is that, when you’re using live problems in your teaching, it’s good to come to class prepared! It’s fine to show students the steps of data cleaning and the frustration of messy data and models that don’t fit–but it’s best for you, the teacher, to encounter and resolve those problems on your own first, so that you don’t confuse the students even further. In class it’s good to have a sense of what’s coming next, so that when something doesn’t work, you’re not surprised and you can contextualize it for the students right away.