David Madigan pointed me to this interesting analysis of internet weather forecasts. I think the person who wrote the article was pretty annoyed. Key quote:
The hail, rain and lightning eventually subsided, but the most alarming news was waiting on cell phone voicemail. A friend who lived in the area had called frantically, knowing we were at the park, as the local news was reporting multiple people had been by struck by lightning at Schlitterbahn during the storm.
“So much for the 0% chance of rain,” I repeated.
Anyway, the post continues with analysis of temperature forecasts, but maybe they’ll go back and look at precipitation too. They also have to work on their graphs–but that’s the trouble with using Excel, I suppose. Here’s an example:
Cool. I've been using the weather channel forecasts for a couple of years now, ever since I saw a talk from the head of the national center for atmospheric research (NCAR) where he mentioned that their models consistently beat the NWS . . . Nice to see the stats back him up.
Interesting data, terrible chart. I'd like to see what Kaiser of Junk Charts could do with this.
It's interesting that high temperatures can be forecast so much more accurately than low temperatures, at least for the near term. The low temp. error of about +/- 5 for predictions made THAT DAY is surprising to me. I would have thought that just predicting today's low to be the same as yesterday's low would have gotten you that close.
I would have liked to see the temperature forecasts compared to the sort of 0th-order prediction that the weather N days from now will be the same as it was today; it seems to me that that is the real test of a forecast.
In the movie L.A. Story, Steve Martin's character is a weather forecaster who films all of his forecasts a week in advance so he doesn't have to go into the studio so often.
The author didn't take into account the fact that not everyone agrees on what "tomorrow's high temperature" means. The BBC, despite their poor showing in the essay, at least tells: the maximum "forecast between 0600 and 1800." Even the same source may consider "tomorrow's high temperature" to be the afternoon high yet report a 24-hour maximum as "yesterday's high temperature."
For high temps in Houston, this isn't too big a deal. Especially in the summer, the Houston 0600-1800 high and 24-hour high are rarely different, because the sun rules the temperature. In the winter, however, the sun is lower, and the 24-hour maximum temperature can occur at night. The author's December-January sample, then, is likely to exaggerate the effect of differing definitions.
For low temperatures, it's even more important to consider how "tomorrow's forecast low" is defined. It may mean "tomorrow morning's low," and it may mean the "overnight low for tomorrow [into the day after tomorrow]". The reported low, if not the forecast low, may be a 24-hour minimum.
I compared the reported extremes at Houston on weather.com (the author's source for actuals) with those from noaa.gov. The mean absolute difference between the highs was 1.3 degrees, with a standard deviation of 1.4. From the author's point of view then, the National Weather Service is 1.3 degrees off, on average, when predicting the past. (The NWS readings are from the international airport, and not everyone may agree that this is where Houston weather is.)
The National Weather Service was even "worse" at "predicting how cold it was." Their low temperature reports were 3.3 degrees off, on average, with a standard deviation of 3.7. Especially big difference occurred on specific days last month with large midnight-to-midnight temperature changes, strongly suggesting that different definitions are in use.
Another confounding factor is that temperatures are given as integers. Are they truncated or rounded? Towards zero or away? (A Javascript web page might even take temperatures measured in Centigrade and rounded to an integer, convert them to Fahrenheit, and round them again!) Differences of a half degree or so aren't too meaningful if the data isn't more precise than that.