The Map Is Not The Territory

This post is by Phil Price, not Andrew.

My wife and I are building a new house, or, rather, paying trained professionals to build one for us. We are trying to make the house as environmentally benign as we reasonably can: ducted mini-split heating, heat pump water heater, solar panels, heat-recovery ventilator, sustainably harvested lumber, yada yada. But there’s one exception: we insisted on a wide expanse of north-facing windows that look out on the creek that runs through the backyard.


All those windows cause a problem because even low-emissivity windows are much less insulating than the insulated walls they replace. We have compensated by making the walls extra-thick, and the ceiling too — indeed we have to extend all of the windowsills and other window trim because the walls are thicker than the thickest pre-fab windows; ditto for door jambs etc. And yet, even with all that extra insulation, we were still not quite compliant with the California Title 24 energy requirements. This is entirely because there is a large penalty for using a heat pump water heater rather than a gas water heater, which makes no sense whatsoever given California’s energy mix and which everyone expects will be revised in the next set of Title 24 requirements, but you go to war with the regulations you have, not the regulations you want.

How does one determine whether one is in compliance with Title 24? One runs approved software that evaluates a computer model of the building: the orientation, the insulation values of the walls, how big are the windows and what kind are they and which direction do they face, and so on. The model said that our original design was just barely non-compliant, even with its heavily insulated walls and ceiling. We could take out some of those precious windows, or reduce them in size, but what other options did we have (besides hooking up to the gas pipeline and using a gas water heater)? The architect noted that we were compliant in all seasons except winter, so even a little bit more solar gain would help. Why not increase the size of the south-facing windows (which face the street), which would get us credit for enough winter heat gain that we would be in compliance? I pointed out that since the south side of the house is in the shade in winter, due to the large house on a hill immediately to the south, increasing the size of the south-facing windows would make the winter energy performance worse, not better. The architect agreed that yes, it would make the energy performance worse, but it would make the predicted energy performance better, and that’s what we need in order to get the permit. The suggestion bothered me some, although I know I shouldn’t find it surprising: most people think of regulations as hurdles to be cleared, and if it’s cheaper or more convenient to honor the letter of the law rather than the spirit, that’s what they’ll do.

I don’t really know what alternative to suggest for enforcing energy efficiency requirements (whose goals I support). Classical and neoclassical economists will say you can just charge more for energy and if you get the cost right people will automatically make the right energy choices, but those economists make their evaluations based on a model world that differs from the real world just as the model of our house differs from the real house. Anyway the whole experience just made me wish — not for the first time, or the last — that people could be counted on to do the right thing so we wouldn’t have to have all these damn rules, especially when they are actually counterproductive in some cases. But they can’t, so we do.

I was going to give two more examples of people confusing the model with reality that have come up in my work as an energy industry consultant, but I think this post is long enough and anyway it’s basically the same story with slightly different characters. The point is that optimizing the performance of the model of your building, or of your electric grid, or of your demand response program, will not optimize the actual performance of your building, or your electric grid, or your demand response program. Sometimes the difference doesn’t matter much but sometimes it matters a whole whole lot. Never forget it.

This post is by Phil Price, not Andrew.

Let’s try this again: It is nonsense to say that we don’t know whether a specific weather event was affected by climate change. It’s not just wrong, it’s nonsensical.

This post is by Phil Price, not Andrew.

If you write something and a substantial number of well-intentioned readers misses your point, the problem is yours. Too many people misunderstood what I was sayinga few days ago in the post “There is no way to prove that [an extreme weather event] either was, or was not, affected by global warming” and that’s my fault.  Let me see if I can do better.

Forget about climate and weather for a moment. I want to talk about bike riding.

You go for a ride with a friend. You come to a steep, winding climb and you ride up side by side. You are at the right side of the road, with your friend to your left, so when you come to a hairpin turn to the right you have a much steeper (but shorter) path than your friend for a few dozen feet. Later you come to a hairpin to the left, but the situation isn’t quite reversed because you are both still in the right lane so your friend isn’t way over where the hairpin is sharpest and the slope is steepest. You ride to the top of the hill and get to a flat section where you are riding side-by-side.  There is some very minor way in which you can be said to have experienced a ‘different’ climb, because even though you were right next to each other you experienced different slopes at different times, and rode slightly different speeds in order to stay next to each other as the road curved, and in fact you didn’t even end up at exactly the same place because your friend is a few feet from you.  You haven’t done literally the same climb, in the sense that a man can’t literally step twice in the same river (because at the time of the second step the river is not exactly the same, and neither is the man) but if someone said ‘how was your climb affected by your decision to ride on the right side of the lane rather than the middle of the lane’ we would all know what you mean; no reasonable person would say ‘if I had done the climb in the middle rather than the right it would have been a totally different climb.’

You continue your ride together and discuss what route to take where the road splits ahead. One road will take you to a series of hills to the north, the other will take you to a series of hills to the south. You decide to go south. You ride over some hills, along some flat stretches, and over more hills. Three hours into the ride you are climbing another hill, the toughest one yet — long, with some very steep stretches and lots of hairpin turns. As you approach the top, your riding companion says “how would this climb have been different if we had gone north instead of south?”  What is the right answer to this question? Here are some possibilities: (1) “There is no way to prove that this climb either was, or was not, affected by our decision to go south instead of north.” (2) “The question doesn’t make sense: we wouldn’t have encountered this climb at all if we had decided to go north.” (3) “This climb was definitely affected by our decision to go south instead of north, but unless we knew exactly what route we would have taken to the north we can’t know exactly how it was affected.”

1 is just wrong (*).  If you had gone north instead of south you might still had a steep climb  around hour 3, maybe it would have even been a steeper climb the one you are on now, but there is no way it could have been the same climb…and the difference is not a trivial one like the “twice in the same river” example.

2 is the right answer.

3 is not the right answer to the question that was asked, but maybe it’s the right answer to what the questioner had in mind. Maybe when they said “how would this climb have been different” they really meant something like, if you had gone the other way, “what would the biggest climb have been like”, or “what sort of hill would be climbing just about now”?

I think you see where I’m going with this (since I doubt you really forgot all about climate and weather like I asked you to).  On a bike ride you are on a path through physical space, but suppose we were talking about paths through parameter space instead. In this parameterization, long steep climbs correspond to hurricane conditions, and going south instead of north corresponds to experiencing a world with global warming instead of one without. In the global warming world, we don’t experience ‘the same’ weather events that we would have otherwise, but in a slightly different way — like climbing the same hill in the middle of the lane rather than at the side of the lane — we experience entirely different weather events — like climbing different hills.

The specific quote that I cited in my previous post was about Hurricane Katrina. It makes no sense to say we don’t know whether Hurricane Katrina was affected by global warming, just as it would make no sense to say we don’t know whether our hill climb was affected by our decision to go south instead of north. In the counterfactual world New Orleans might have still experienced a hurricane, maybe even on the same day, but it would not have been the same hurricane, just as we might encounter a hill climb on our bike trip at around the three-hour mark whether we went south or north, but it would not have been the same climb.

No analogy is perfect, so please don’t focus on ways in which the analogy isn’t ‘right’. The point is that we are long past the point where global warming is a ‘butterfly effect’ and we can reasonably talk about how individual weather events are affected by it. We aren’t riding up the same road but in a slightly different place, we are in a different part of the territory.

(*) I’m aware that if you had ridden north instead of south you could have circled back and climbed this same climb. Also, it’s possible in principle that some billionaire could have paid to duplicate ‘the same’ climb somewhere to the north — grade the side of a mountain to make this possible, shape the land and the road to duplicate the southern climb, etc.  But get real. And although these are possible for a bike ride, at least in principle, they are not possible for the parameter space of weather and climate that is the real subject of this post.

This post is by Phil, not Andrew.

“There is no way to prove that [an extreme weather event] either was, or was not, affected by global warming.”

This post is by Phil, not Andrew.

It’s hurricane season, which means it’s time to see the routine disclaimer that no single weather event can be attributed to global warming. There’s a sense in which that is true, and a sense in which it is very wrong.

I’ll start by going way back to 2005. Remember Hurricane Katrina? A month afterwards some prominent climatologists (Rahmstorf, Mann, Benestad, Schmidt, and Connolley) wrote “Could New Orleans be the first major U.S. city ravaged by human-caused climate change? The correct answer–the one we have indeed provided in previous posts (Storms & Global Warming II, Some recent updates and Storms and Climate Change) –is that there is no way to prove that Katrina either was, or was not, affected by global warming. For a single event, regardless of how extreme, such attribution is fundamentally impossible.”

Well, that’s just nonsense. How on earth could Katrina not have been affected by global warming? There’s no way. You can argue that a major hurricane might have struck New Orleans on August 29, 2005 with or without global warming — sure, could be. Or maybe it would have happened a day earlier or a week earlier or a year earlier or a decade earlier. But sure, OK, maybe it would have happened on August 29, 2005. It’s extremely unlikely but not impossible. But there’s no way, literally no way, that it could have been the same storm. Katrina was definitely affected by global warming.

Does it matter? “We all know what they meant”? Well, I don’t know what they meant! And I’ve seen similar statements hundreds of times.

The weather is different than it would have been without global warming, every day and in every location. In some places and at some times the differences are large and in some places they are small. On some days there are fewer tropical cyclones in the Atlantic than there would have been, and on some days there are more; on other days there are exactly the same number of tropical cyclones but they are not in exactly the same places with exactly the same winds.

To say we don’t know whether a given city would have been destroyed by a hurricane on such-and-such a date in the absence of global warming, OK, fine, coincidences happen. But to say that we can’t say whether the storm was affected by global warming, that’s just wrong.  That goes for Hurricane Dorian, too.

I’ve been waiting 14 years to get this off my chest. I feel better.

This post is by Phil Price

Book Review: Good to Go, by Christie Aschwanden

This is a book review. It is by Phil Price. It is not by Andrew.

The book is Good To Go: What the athlete in all of us can learn from the strange science of recovery. By Christie Aschwanden, published by W.W. Norton and Company. The publisher offered a copy to Andrew to review, and Andrew offered it to me as this blog’s unofficial sports correspondent.

tldr: This book argues persuasively that when it comes to optimizing the recovery portion of the exercise-recover-exercise cycle, nobody knows nuthin’ and most people who claim to know sumthin’ are wrong. It’s easy to read and has some nice anecdotes. Worth reading if you have a special interest in the subject, otherwise not. Full review follows.

The book is about ‘recovery’. In the context of the book, recovery is what you do between bouts of exercise; or, if you prefer, exercise is what you do between periods of recovery. The book has great blurbs. “A tour de force of great science journalism”, writes Nate Silver (!). “…a definitive tour through a bewildering jungle of scientific and pseudoscientific claims…”, writes David Epstein. “…Aschwanden makes the mid-boggling world of sports recovery a hilarious adventure”, says Olympic gold medal skier Jessie Diggins. With blurbs like these I was expecting a lot…although once I realized Aschwanden works at FiveThirtyEight, I downweighted the Silver blurb appropriately. Even so, I expected too much: the book is fine but ultimately rather unsatisfying. It is fairly interesting and sometimes amusing, but there’s only so much any author can do with the subject given the current state of knowledge, which is this: other than getting enough sleep and eating enough calories, nobody knows for sure what helps athletes recover between events or training sessions better than just living a normal life. The book is mostly just 300 pages of elucidating and amplifying that disappointing state of knowledge.

Continue reading

Why, oh why, do so many people embrace the Pacific Garbage Cleanup nonsense? (I have a theory).

This post is by Phil, not Andrew.

Over the couple of months I have seen quite a few people celebrating the long-awaited launch of a big device that will remove plastic garbage from the Pacific ocean. I find this frustrating because this project makes no sense even if the device works as intended: at best it will turn out to be a good piece of technology that is deployed in a stupid location where it will cost a lot of money to maintain while removing much less plastic than it could otherwise.

Every now and then I hear similar enthusiasm being expressed for devices that will remove carbon dioxide from the atmosphere…just build these things all over the place and you can solve or at least reduce the problem of excessive atmospheric carbon dioxide. As with the ocean cleanup, if you have such a technology you’d be a fool to deploy it this way.

As many readers of this blog will have already recognized, if you are trying to remove plastic from the ocean, or to remove carbon dioxide from the atmosphere, the place to do it is where the concentration is highest.  If you have a device that can separate plastic from water, put it in or near the outflow of a river that is bringing the plastic into the ocean; don’t wait until the garbage-filled water has been diluted by a factor of many thousand. The device can only remove plastic from the water that it interacts with, so it can only process a certain volume of water per day; much better if that water has a high concentration of the plastic you are trying to remove. Similarly, if you have a technology that can separate carbon dioxide from other gases, you should put it in or near smokestacks from power plants and cement plants…that is, places where the concentration is much higher than you find if you wait for the gases to be fully mixed into the rest of the atmosphere.

The situation with the Pacific Garbage Patch cleanup is especially bad because, in addition to the (very large) inefficiency that is due to putting the devices in places with unnecessarily low concentrations of floating plastic, there are farther inefficiences associated with putting the devices far, far out at sea, where they are more costly to maintain than if they were closer to land.

All of the above seems pretty obvious, and I daresay it is pretty obvious to most readers of this blog. Why, then, are so many people excited about the idea of putting a bunch of devices way out in the Pacific, or sprinkling carbon dioxide removal devices all around the globe? I don’t know but I have a theory: I think people make a cognitive error, or perhaps experience a cognitive illusion, in which they don’t count the input stream as part of the system in a logical way. I had some interesting and somewhat perplexing conversations with friends who are enthusiastic about the carbon dioxide removal idea, and I think that’s a nice clean example of the cognitive error that I’m talking about so I’ll focus on that one.

I’ve discussed the carbon dioxide removal approach with several friends at various times, and said that if there is a good technology for separating carbon dioxide from other gases it should be used at major carbon dioxide sources, not in the general atmosphere. All of them express some variation of this sentiment: We need to stop emitting carbon dioxide, but we also need to remove carbon dioxide from the atmosphere because carbon dioxide concentrations are already too high. Putting the devices in places where they remove carbon dioxide from ‘the atmosphere’ seems like it is actually solving the problem, whereas decreasing the amount of carbon dioxide that is emitted is merely a way of stopping things from getting worse.

But of course, taking N tons of carbon dioxide out of the atmosphere and taking N tons of carbon dioxide out of a stream of combustion gases that is entering the atmosphere have the same effect on atmospheric carbon dioxide concentrations. And if, by putting your device at the smokestack, you can remove 3N tons in the same time and for the same cost, you are much better off putting the device in the smokestack. But somehow this doesn’t appeal to people because it doesn’t “reduce the carbon dioxide concentration in the atmosphere.” It’s almost like they would prefer to add 3N tons to the atmosphere and then remove N tons, than to add zero tons in the first place; after all, in the former case you are removing N tons of carbon dioxide from the atmosphere, and in the latter you aren’t removing anything! They love the idea of removing the pollutant; decreasing the rate at which it is added just doesn’t seem the same somehow, even though it is.

I think this error, or something close to it, clouds people’s thinking about ocean plastics too.

The situation is more complicated with ocean plastics: 97% of plastic that enters the ocean does not end up in the “Great Pacific Garbage Patch” so if you want to remove plastic specifically from the ‘patch’ then maybe you do want to put your device there. But I’ve talked to people about this and they seem to agree that they do want to decrease the amount of plastics in the oceans in general, not just in the middle of the Pacific Ocean. But still, they seem to think that a device that removes plastic from the Pacific Ocean is needed, and they’re much more excited about that than about preventing plastic from entering the Pacific Ocean. One of my friends said “we need to do both.” Well, no: if it’s more effective to just do one of them, then that’s what we should do.

The Ocean Cleanup project is probably going to collect many tons of garbage from the Pacific Ocean (at great expense) and I’m sure some people will declare it a success…and that’s a crying shame because they could do much, much better for a lot less money.

This post is by Phil, not Andrew.

How dumb do you have to be…

I (Phil) just read an article about Apple. Here’s the last sentence: “Apple has beaten earnings expectations in every quarter but one since March 2013.”

[Note added a week later: on July 31 Apple reported earnings for the fiscal third quarter.  Earnings per share was $2.34 vs. the ‘consensus estimate’ of $2.18, according to Thomson Reuters.]

 

Exercise and weight loss: long-term follow-up

This post is by Phil Price, not Andrew.

Waaaay back in 2010, I wrote a blog entry entitled “Exercise and Weight Loss.” I had added high-intensity interval training back into my exercise regime, and had lost 12 pounds in about 12 weeks; but around the same time, some highly publicized studies were released that claimed that exercise does not lead to weight loss in overweight people. I suggested that that claim was too strong: at best they had demonstrated that moderate-intensity exercise does not lead to weight loss in most overweight people. I am completely convinced that when I am slightly overweight, I lose weight when I do occasional high-intensity workouts.

Well, I’m back with another data point. After spending a month in hell earlier this year, during which I got no exercise, I had not only failed to lose my winter weight but had added a few pounds. When I was finally able to get to my usual spring activities, which include road biking — sometimes with high-intensity intervals — I quickly lost a couple of pounds. But then I crashed, nothing serious but enough to keep me off the bike and mostly sedentary for more than a month, and I put on some more weight, topping out at about 203 or 204 pounds, the heaviest I had been since I wrote that “Exercise and weight loss” blog post back in 2010. Already this experience would seem to contradict the suggestion that exercise doesn’t control weight: if I wasn’t gaining weight due to lack of exercise, why was I gaining it?

I was able to resume exercise in early May, and in the next six weeks I lost about six pounds. In the past few weeks I’ve lost a few more. Yesterday and today, I’ve weighed in at 193 pounds, ten pounds lighter than I was two months ago. Given past experience, I expect to remain somewhere in the 192- to 195-pound range until November, when I will start edging upwards.

So I’m reiterating the point of that eight-year-old blog post I mentioned at the top: maybe moderate-intensity exercise doesn’t lead to weight loss in most overweight people, but high-intensity exercise does lead to weight loss in me when I am somewhat overweight, and as long as I regularly do some high-intensity exercise I don’t tend to gain weight.

The broader point here is that I think researchers (and journalists) tend to over-generalize. If you do a test that subjects one group of people to one set of conditions, don’t assume the results will extend to a different set of people and/or a different set of conditions, even if the people and the conditions have some similarity to those used in the experiment. The differences can matter.

Reminder: this post is by Phil, not Andrew.

Lessons learned in Hell

This post is by Phil. It is not by Andrew.

I’m halfway through my third year as a consultant, after 25 years at a government research lab, and I just had a miserable five weeks finishing a project. The end product was fine — actually really good — but the process was horrible and I put in much more time than I had anticipated. I definitely do not want to experience anything like that again, so I’ve been thinking about what went wrong and what I should do differently in the future. It occurred to me that other people might also learn from my mistakes, so here’s my story.

Continue reading

An obvious fact about constrained systems.

 

This post is not by Andrew. This post is by Phil.

This post is prompted by Andrew’s recent post about the book “Everything is obvious once you know the answer,” together with a recent discussion I’ve been involved in. I’m going to say something obvious.

True story: earlier this year I was walking around in my backyard and I noticed a big hump in the ground next to a tree. “This hump wasn’t here before”, I thought. I looked up and saw that the tree, which had always been tilting slightly, was now tilting a lot more than slightly. It was now tilting very substantially, straight north towards our neighbor’s house! The hump in the ground was the roots on the other side of the tree being pulled up from the ground.

It was a Sunday but I immediately called our tree guy and left a number on his emergency line. (Did you know tree guys have emergency lines? They do. They’re like plumbers: a significant fraction of their calls are urgent). Then I called our neighbors.

The tree guy came out and said something I already knew: this tree is well on its way to falling down. He immediately had his crew come out with heavy ropes to anchor the tree to the trunks of two other trees as a temporary measure. See the diagram above (north is diagonally to the left). A week or so later, his crew came out and cut down the tree piece by piece. They started from the top ;)

Continue reading

NIMBYs and economic theories: Sorry / Not Sorry

This post is not by Andrew. This post is by Phil.

A few days ago I posted What’s the deal with the YIMBYs?  In the rest of this post, I assume you have read that one. I plan to post a follow-up in a month or two when I have had time to learn more, but there are a couple of things I can say right now.

I. Sorry

  1. I apologize unreservedly to YIMBY supporters who know, or think they know, that buiding more housing in San Francisco will decrease rents there or at least will greatly reduce the rate at which they rise. I characterized the entire YIMBY movement as being at least partly motivated by a desire to stick a thumb in the eye of the smug slam-the-door-now-that-I’m-inside NIMBY crowd, rather than by a genuine belief that loosening land use restrictions in SF will decrease rents there. This was simply wrong of me.
  2. There might be something else I need to apologize for too, I’m not sure. (I don’t remember whether or not I said this in the comments; I’ve taken a quick skim through and didn’t see where I said it, but maybe I did). I have seen a few articles that touch on Bay Area housing, and the YIMBY movement, in which people who characterize themselves as YIMBYs say things like “we aren’t talking about turning San Francisco into Manhattan, we are talking about building some more housing to take the pressure off so rents come down.” [That is not a quote, it’s a paraphrase.] I think that in the current economic environment you would need to build an enormous amount of housing in SF to get the price to come down, so I’ve felt that people who say things like that are being disengenuous. It’s possible that I characterized the entire movement incorrectly, based on those examples. If I did, then I apologize to people like Sonja Trauss, who is a leader in the YIMBY movement: Sonja is absolutely up-front about wanting to build however much a free- or nearly-free market would allow, even if this does indeed lead to the Manhattanization of San Francisco. Trauss is not being disengenous at all, there are others in the movement that are also straightforward about the fact that their desired policies might completely transform the city.
  3.  I’m also sorry that I wrote authoritatively rather than speculatively. What I should have said is “Here’s what I think is happening, and I’d welcome comments” rather than, in essence, “Here’s what is happening, and I welcome comments.”

II. Not Sorry.
I proposed a model for San Francisco housing prices that makes sense to me. Quite a few people posted that my economic model is nonsense and I’m an arrogant fool, or worse, for having proposed it…that I should be embarrassed… etc.  But not only do I not think I’m arrogant, I think some of the people who accused me of being arrogant are themselves arrogant. (As for whether I’m a fool, that’s possible but I am not convinced).

Continue reading

What’s the deal with the YIMBYs?

This post is not by Andrew. It is by Phil.

There’s at least one thing people in San Francisco seem to agree on: the rent is too damn high. The median rent is between about $3000 and $3500 per month…for a one-bedroom apartment. High-tech workers and upper-echelon businesspeople can afford a place, but baristas and hair salon workers and teachers and shop clerks etc. etc. have real trouble.

Of course there is plenty of development pressure, and new high-rise apartments are going in that have hundreds of apartments each, typically with a rent of $4000 – $8000 per month. If you let a developer build “market rate” apartments, that’s what they’ll build.

Suppose San Francisco adds 10,000 market-rate units. Some will be one-bedrooms, some two- or three-bedrooms, and some will be occupied by singles, others by couples, etc. But for simplicity let’s just assume that on average each of these new 10,000 units is occupied by a household that earns $100K per year after taxes. Total, the occupants of these apartments have $1,000,000,000 in disposable income. That’s a lot! They’re going to spend some of that money — indeed, a lot of that money — in San Francisco: they will go to coffee shops, get their hair cut, buy things in shops, etc. Serving those extra 10,000 high-income households will require tens of thousands more waiters and shop clerks and car mechanics and plumbers etc etc etc….that is, there will be more jobs for the kinds of people who already have trouble affording a place in San Francisco. Those additional people will need to live somewhere, so there will be increased competition at the lower end of the market, which means higher rents. Most of these people will end up commuting from other cities.

Continue reading

How can time series information be used to choose a control group?

This post is by Phil Price, not Andrew.

Before I get to my question, you need some background.

The amount of electricity that is provided by an electric utility at a given time is called the “electric load”, and the time series of electric load is called the “load shape.” Figure 1 (which is labeled Figure 2 and is taken from a report by Scottmadden Management Consultants) shows the load shape for all of California for one March day from each of the past six years (in this case, the day with the lowest peak electric load). Note that the y-axis does not start at zero.

Duck Curve

Figure 1: Electric load (the amount of electricity provided by the electric grid) in the middle of the day has been decreasing year by year in California as alternative energy sources (mostly solar) are added.

In March in California, the peak demand is in the evening, when people are at home with their lights on, watching television and cooking dinner and so on.

An important feature of Figure 1 is that the electric load around midnight (far left and far right of the plot) is rather stable from year to year, and from day to day within a month, but the load in the middle of the day has been decreasing every year. The resulting figure is called the “duck curve”: see the duck’s tail at the left, body in the middle, and head/bill at the right?

The decrease in the middle of the day is due in part to photovoltaic (PV) generation, which has been increasing yearly and is expected to continue to increase in the future: when the sun is out, the PV panels on my house provide most of the electricity my house uses, so the load that has to be met by the utility is lower now than before we got PV.

Continue reading

Numbers too good to be true? Or: Thanks, Obama!?

This post is by Phil.

Flat until April 2010; steady drop until October 2012; flat since then.

Readmissions to hospital within 30 days, by quarter

The “Affordable Care Act” a.k.a. “Obamacare” was passed in 2010, with its various pieces coming into play over the following few years. One of those pieces is penalties for hospitals that see high readmission rates. The theory here, or at least one of the theories here, was that hospitals could reduce readmission rates if they wanted to, but they didn’t have a strong incentive to do so, and indeed there was a moral hazard: if a hospital sends a patient home for good, they’re done collecting money from them, but if the patient has to come back for more treatment…cha-ching.

I have to admit I didn’t think this was going to be a big deal. I know doctors, I’ve seen doctors, some of my good friends are doctors, and I know they’re not scheming to make more money by providing bad treatment so the patients have to come back for more.

But…well, check out this plot, from the Department of Health and Human Services. The plot does us all a disservice by not starting its y-axis at 0, but still…wow. If the data are real and the plot is real, this is pretty stunning: a 20% reduction (or a 3.5 percentage point reduction) in readmissions for “HRRP”, and a similar scale reduction in all other readmissions.

My first thought was that the hospitals are gaming the system somehow by readmitting patients but not reporting them, but I’m not the first person to suggest this and supposedly “The new research shows that this isn’t the case. The number of observation stays are very small compared to readmissions and have increased steadily since at least 2008, with no acceleration after the Affordable Care Act was enacted.”

Of course, there are other possibilities, like maybe the hospitals are refusing to readmit patients even if they really should, or maybe they put them off a bit and readmit them on day 31 instead of day 28 or 29 or 30. But something like this would make their fatality numbers go up, and I assume someone tracks those.

One of the interesting things about this is that you really don’t need a statistician: the signal is so clear that the only questions are related to the definitions of things like “readmission.” A sixth-grader can look at the numbers and come up with a good estimate of the effect of the law on readmissions.

Le Menu Dit : a translation app

This post is by Phil Price.

“Le Menu Dit” is an iPhone app that some friends and I wrote, which translates restaurant menus from English into French. (The name is French for “The Menu Says.”) The friends are Nathan Addy and another excellent programmer who would like to remain nameless for now.

Here’s how the app works from the user perspective: You take a photo of a printed (as opposed to handwritten) English-language menu, and the app translates it into French. C’est formidable! We don’t yet have a version that goes the other way, which might be of much more interest to readers of this blog, but the whole problem is interesting and has enough of a statistical component to it that it seems fine for this blog. Of course if you or your French-speaking friends want to buy the app, that would be good too! And we will soon be releasing versions that go from English into Italian and from English into Chinese; and a bit farther down the road, probably around May, we will finally have one that goes from French into English. See how we do it, below the fold.

Lines of a menu are shown along with the translation into French

Screenshot of Le Menu Dit results

Continue reading

Cancer statistics: WTF?

This post is by Phil.

I know someone who was recently diagnosed with lung cancer and is trying to decide whether to get chemo or just let it run its course. What does she have to go on? A bunch of statistics that are barely useful. For example, its easy to find the average survival time for someone with her stage of this particular cancer. Let’s say that’s 12 months. Fine, but that’s an average over all ages, both sexes, and includes people who did and didn’t opt for chemo! Maybe the average time is 6 months if you eschew chemo and 18 months if you get it, and half the people do and half the people don’t? Or maybe 80% of victims get chemo, and those that don’t only average about 2 months.

The only other stat that seems to be widely available is “5-year survival rate.” Let’s say that’s 6%…that’s not so good. But if the victim is 80 years old they’re not so likely to live another 5 years anyway! Plus, again, the information that would be useful is the information that might change your behavior. If you have a 1% chance of living two years if you eschew chemo, and a 20% chance if you get it, then maybe you’d choose to get it even if you know you won’t make it 5 years.

What people should care about is conditional probabilities, not summary statistics. You don’t care what the average survival time is, averaged over all patients, instead you want to know it for people like you: your stage of cancer, your sex, your age, your physical condition.

I’m kind of shocked, and very disappointed, that there’s no website somewhere that lets you put in your age, sex, cancer type and stage, and maybe a few other relevant details, and get the statistical distribution of survival times if you do or don’t get chemo (or surgery, or radiation, or whatever). This lack was understandable in 1996 or even 2006 but it’s 2016 for crying out loud!

This post is by Phil.

Guys, we need to talk. (Houston, we have a problem).

This post is by Phil Price. I’m posting it on Andrew’s blog without knowing exactly where he stands on this so it’s especially important for readers to note that this post is NOT BY ANDREW!

Last week a prominent scientist, representing his entire team of researchers, appeared in widely distributed television interviews wearing a shirt covered with drawings of scantily clad women with futuristic weapons (I believe the term of art is “space vixens.”) In that interview, he said about the comet that his team is studying, “she’s sexy, but she’s not easy.” Here’s a photo of the shirt (sorry about the fuzziness):
Continue reading

Statistical distribution of incomes in different countries, and a great plot

This post is by Phil Price.

This article in the New York Times is pretty good, and the graphics are excellent…especially the interactive graphic halfway down, entitled “American Incomes Are Losing Their Edge, Except at the Top” (try mousing over the gray lines and see what happens).

The plot attempts to display the statistical distribution of incomes in about 10 different countries. That alone is not so easy; one natural idea is to display a bunch of histograms in a small multiples plot. But the plot also tries to show how each of the distributions has changed since 1980. I can think of other approaches to this plot that might be worth trying, but I’m not sure any of them would be better. Nicely done, NYT graphics team.

If I wanted to make interactive graphics like this myself, I could presumably figure it out. But suppose I want to do it routinely, as part of exploratory data analysis. I don’t necessarily need polish, just the basics. I’d like to work within R but am open to other possibilities. What are my options? GGobi? iPlots? Anything else worth considering?

This post is by Phil Price

Shamer shaming

This post is by Phil Price.

I can’t recall when I first saw “shaming” used in its currently popular sense. I remember noting “slut shaming” and “fat shaming” but did they first become popular two years ago? Three? At any rate, “shaming” is now everywhere…and evidently it’s a very bad thing.

When I first saw the term, I agreed with the message it was trying to convey: it is bad to try to make people feel ashamed of being fat, or of wanting to have sex. Indeed, I’d say it’s bad to try to make people feel ashamed of anything that isn’t unethical or morally wrong or at least irritating. Down with slut shaming! Down with fat shaming! Down with gay shaming!

But somehow all criticism seems to have become “shaming.” A few days ago I posted a message to my neighborhood listserv, reminding people that (1) we are in a severe drought (I live in California), (2) washing one’s car with a hose uses a lot of water, and indeed is a fineable offense if you don’t use a nozzle that shuts off the water when you release it, (3) all commercial car washes in our area recycle their water, and (4) our storm drains empty directly into a creek. The next day I got an angry email from a neighbor: how dare I shame him for washing his car on the street?

On this blog, Andrew has frequently posted about researchers doing shameful things, such as plagiarizing, and refusing to admit to major mistakes in their published work. (There’s nothing shameful about making a mistake, at least not if you’ve tried hard to get it right, but it is shameful to refuse to admit it). And, sure enough, some people have complained that Andrew is “shaming” these people.

Plagiarist-shaming, academic fraud-shaming, hack journalist-shaming, all of those are evidently in the same unacceptable category as fat-shaming and slut-shaming. There is nothing shameful in the world, except trying to make somebody feel ashamed. Shamer-shaming is the only kind of shaming that is OK.

This post is by Phil Price

One of the worst infographics ever, but people don’t care?

This post is by Phil Price.

Perhaps prompted by the ALS Ice Bucket Challenge, this infographic has been making the rounds:
Infographic: disease deaths and dollars spent

I think this is one of the worst I have ever seen. I don’t know where it came from, so I can’t give credit/blame where it’s due.

Let’s put aside the numbers themselves – I haven’t checked them, for one thing, and I’d also say that for this comparison one would be most interested in (government money plus donations) rather than just donations — and just look at this as an information display. What are some things I don’t like about it? Jeez, I hardly know where to begin.

1. It takes a lot of work to figure it out. (a) You have to realize that each color is associated with a different cause — my initial thought was that the top circles represent deaths and dollars for the first cause, the second circles are for the second cause, etc. (b) Even once you’ve realized what is being displayed, and how, you pretty much have to go disease by disease to see what is going on; there’s no way to grok the whole pattern at once. (b) Other than pink for breast cancer and maybe red for AIDS none of the color mappings are standardized in any sense, so you have to keep referring back to the legend at the top. (c) It’s not obvious (and I still don’t know) if the amount of “money raised” for a given cause refers only to the specific fundraising vehicle mentioned in the legend for each disease. It’s hard to believe they would do it that way, but maybe they do.
2. Good luck if you’re colorblind.
3. Maybe I buried the lede by putting this last: did you catch the fact that the area of the circle isn’t the relevant parameter? Take a look at the top two circles on the left. The upper one should be less than twice the size of the second one. It looks like they made the diameter of the circle proportional to the quantity, rather than the area; a classic way to mislead with a graphic.

At a bare minimum, this graphic could be improved by (a) fixing the terrible mistake with the sizes of the circles, (b) putting both columns in the same order (that is, first row is one disease, second row is another, etc)., (c) taking advantage of the new ordering to label each row so you don’t need the legend. This would also make it much easier to see the point the display is supposed to make.

As a professional data analyst I’d rather just see a scatterplot of money vs deaths, but I know a lot of people don’t understand scatterplots. I can see the value of using circle sizes for a general audience. But I can’t see how anyone could like this graphic. Yet three of my friends (so far) have posted it on Facebook, with nary a criticism of the display.

[Note added the next day:
The graphic is even worse than I thought. As several people have pointed out, my suspicion is true: the numbers do not show the total donations to fight the diseases listed, they show only the donations to a single organization. For instance, according to the legend the pink color represents donations to fight breast cancer, but the number is not for breast cancer as a whole, it’s only for Komen Race for the Cure.

If they think people are interested in contributions to only a single charity in each category — which seems strange, but let’s assume that’s what they want and just look at the display — then they need a title that is much less ambiguous, and the labels need to emphasize the charity and not the disease.]

This post is by Phil Price.

As if we needed another example of lying with statistics and not issuing a correction: bike-share injuries

This post is by Phil Price

A Washington Post article says “In the first study of its kind, researchers from Washington State University and elsewhere found  a 14 percent greater risk of head injuries to cyclists associated with cities that have bike share programs. In fact, when they compared raw head injury data for cyclists in five cities before and after they added bike share programs, the researchers found a 7.8 percent increase in the number of head injuries to cyclists.”

Actually that’s not even an example of “how to lie with statistics”, it’s simply an example of “how to lie”: As noted on StreetsBlog, data published in the study show that “In the cities that implemented bike-share…all injuries declined 28 percent, from 757 to 545. Head injuries declined 14 percent, from 319 to 273 per year. And moderate to severe head injuries also declined from 162 to 119. Meanwhile, in the control cities that do not have bike-share, all injuries increased slightly from 932 to 953 per year — 6 percent.”  There’s a nice table on Streetsblog, taken from the study(make sure you read the caption).

So the number of head injuries declined by 14 percent, and the Washington Post reporter — Lenny Bernstein, for those of you keeping score at home — says they went up 7.8%.  That’s a pretty big mistake! How did it happen?  Well, the number of head injuries went down, but the number of injuries that were not head injuries went down even more, so the proportion of head injuries that were head injuries went up.
According to StreetsBlog, University of British Columbia public health professor Kay Ann Teschke “attempted to notify Bernstein of the problem with the article in the comments of the story, and he was initially dismissive. He has since admitted in the comments that she is right, but had not adjusted his piece substantially at the time we published this post.” (I don’t see that exchange in the comments, although I do see that other commenters have pointed out the error).

To be fair to Bernstein, it looks like he may have gotten his bad information straight from the researchers who did the study: The University of Washington’s Health Sciences NewsBeat also says “Risk of head injury among cyclists increased 14 percent after implementation of bike-share programs in several major cities”. It’s hard to fault Bernstein for getting the story wrong if he was just repeating errors that were in a press release approved by one of the study’s authors!

But how do Bernstein, the Washington Post, the study’s author at University of Washington (Janessa Graves), and the University of Washington justify their failure to correct this misinformation?  It’s a major error, and it’s not that hard to edit a web page to insert a correction or retraction.

[Note added June 18: When I posted this I also emailed Bernstein and the UW Health Sciences Newsbeat to give them a heads-up and invite comment. Newsbeat has changed the story to make it clear that the proportion of injuries that are head injuries increased in the bike share cities. They do not note that the number of head injuries decreased, and it looks like they forgot to correct the headline so it’s still wrong. At least they acknowledged the problem and did something, although I daresay most readers of that page will still be misled. But it’s no longer flat wrong. Except the headline.]

Of course, even simply retracting the story is a missed opportunity: the real story here is that injuries went down in bike share cities in spite of the fact that there were more people riding. That’s a surprise!  As a bike commuter, I know that it has long been argued that biking becomes safer per biker-mile as more people ride, because drivers become more alert to the likely presence of bikes. But I would not have expected that the decrease in risk per mile would more than counteract the number of miles ridden, such that the number of injuries would go down. Or, of course, maybe that’s not what happened, maybe there were other changes that were coincident with the introduction of bike share programs, that decreased risk in the bike share cities but not the control cities.

This sort of thing — by which I mean mis-reporting of scientific results in general — is just so, so frustrating and demoralizing to me. If people think bike share programs substantially increase the risk of injury, that belief has consequences. It affects the amount of public support for such programs (and for biking in general) as well as affecting individuals’ decisions about whether or not to use those programs themselves. To see these stories get twisted around, and to see journalists refuse to correct them…grrrrr.

This post is by Phil Price
[Andrew, please add “Ethics” and “Journalism” categories to this blog]