Author Archives: Phil

Golems, auditors, and AI

Posted on by
14

This post is by Phil.

Some time ago I wrote some thoughts about “Neuromancer” ( https://statmodeling.stat.columbia.edu/2025/06/12/what-does-neuromancer-have-to-teach-us-about-the-role-of-ai-is-society/ ), which features two kinds of artificial intelligence, one of which seems like it could be realized with a Large Language Model, i.e. we could pretty much make it today. The other is something more powerful, an artificial general intelligence that not only has computational power but also imagination and desires. I think it’s an open question whether an LLM can have genuine desires (and even a genuine imagination) as opposed to being able to pretend that it does. Also an open question whether that distinction even makes sense to talk about.

I’ve read some other fiction within the past few months that has also given me things to think about, AI-wise.

First there was Feet of Clay, by Terry Pratchett. Pratchett writes lightweight, fun, but generally forgettable fantasy novels. I mentioned that book in an earlier post, https://statmodeling.stat.columbia.edu/2026/01/21/what-a-coincidence-what-a-coincidence/ , because it uses a rare plot device that happened to crop up in the very next book that I read. But I mention it now for a different reason: in the book there are golems (an animated, artificial humanoid in Jewish folklore created entirely from inanimate matter, such as clay or mud) that are treated pretty much like robots. A golem’s operating system is written on a piece of paper contained in its head. In the book, Golems are treated like we treat industrial robots or Roombas or similar: they are given simple, repetitive tasks at which they work, sometimes day and night. Nobody feels bad about using them however they want, because the golems have no emotions. Or do they? In the book some golems get together and create a golem of their own, and give it instructions that are…well, basically they are trying to create something more human. Of course, the fact that they desire to do such a thing suggests that they are not in fact emotionless objects.

Well, I just read another Pratchett book, “Thief of Time”. (Spoilers follow. Stop reading here if you want to read this book and be surprised.) This book has beings called ‘auditors’ who are responsible for maintaining order in the universe. They are described as being nearly emotionless except for hating disorder. To them, humans pretty much personify disorder so I think they could be said to hate humans. To better understand humans so they can learn to control us better, some of the auditors create human bodies for themselves and occupy them…and, uh oh, with the bodies come emotions. They get hungry, they can feel pain, things taste good or taste bad, etc. As they strive to satisfy their bodies’ desires, they start to act more and more like humans. They want things.

I mention this here because it touches on something I wonder about AIs, or at least LLMs: can they have desires? Certainly they can be told to _pretend_ they do — one could prompt an LLM to pretend that it wishes to take over the world, for example — but would it _really_ “want” to take over the world? Would it want anything at all?

Thinking about those kinds of questions, I realized that I don’t understand human emotions and sensations either. I don’t see how a bunch of computer circuits can be made to feel pain, but I also don’t understand how a bunch of nerves and neurons can feel pain either. I can understand how either one can respond to stimuli — if the temperature at this point exceeds such-and-such a temperature, fire these muscles — but I’m talking about the _sensation_ of pain. How does that arise? And is there something about a computer that works with voltages on a chip that prevents it from being able to have that sensation? Do nerves and brains somehow allow a sensation that literally cannot be duplicated in silico?

Sadly, Thief of Time did not answer any of those questions for me. But it did get me thinking about them, so I guess that’s something.

This post is by Phil

Ted Williams and Me

Posted on by
10
This post is by Phil Price, not Andrew.

Every spring, my friend Sam and I work on a consulting project together. The project is a program evaluation: the State of California requires the company (our client) to hire a disinterested firm (that’s us) to evaluate the effectiveness of one of the company’s programs.

In previous years, the program has been a big part of the client’s revenue and they’ve cared a lot about the answers. But the client has expanded nationwide over the past year, they’ve got contracts with a lot more companies in a lot of states, and this particular program is suddenly (really suddenly!) only a small part of their business… so small, as a percentage, that they pretty much don’t care about the numbers we get this year, as they said (politely) in a phone call. The analysis has a lot of required elements that are kind of a hassle for the client and it wouldn’t surprise me if they don’t participate in this program at all next year.

Still, we were on the hook for the report so Sam and I worked on it just as we always do.

With a couple of days to go before our deadline for submitting the report, we were on Slack on a Saturday afternoon hashing out a few things. We had this exchange:

Me: I get it that they don’t want or need to mess with this nonsense anymore, I’m happy for them and that’s good news for the industry in general. But it’s not a great feeling that the client doesn’t care about our work!   There’s a moderately famous John Updike piece about the retirement of Ted Williams, in which he says “For me, Williams is the classic ballplayer of the game on a hot August weekday, before a small crowd, when the only thing at stake is the tissue-thin difference between a thing done well and a thing done ill.” 
  
Sam: So you are Ted Williams in this scenario?

Me: You and me both. I mean, why are we both doing our best (or something approximating our best) when nobody else cares?

Sam: Gotta have standards and as an independent consultant, standards/reputation are about all that matters. I think it comes from internal compass, but that in turn means we can do this type of work successfully. To answer your question, like Ted Williams, I think we are both worried about personally being associated with deficient work.


There’s no particular message here, I just have the impression that there are a few people who follow this blog who appreciate hearing stories from the world of statistical consulting every now and then.

This post is by Phil

What, a coincidence? What a coincidence!

Posted on by
16

This post is going to have spoilers for “Feet of Clay” by Terry Pratchett, and “As a Thief in the Night” by R Austin Freeman.

Last week I was browsing in a used bookstore… or, rather, a used-book store… and decided to buy a Terry Pratchett “Discworld” novel: Feet of Clay.  Pratchett wrote many many books set  on Discworld, they’re kinda zany, very lightweight, fun reading. I’ve read three or four of them over the past few decades and I can’t remember the plot or characters in any of them, but I remember enjoying all of them, so I thought eh, why not.  And sure enough, the book was a fun read.  There’s a kind of murder mystery, as well as an attempted-murder mystery, and some lovable rogues and so on. The attempted-murder mystery involves administering poison by making candles with arsenic in them. The victim reads by candlelight at night, even as he gets sicker, and over the course of several nights the poison does its work.

I finished that book, and the next day I was walking to lunch on my own and decided that rather than my usual solo-lunch pastime of playing terrible online chess, I would pick up a book at one of the three (!) Little Free Libraries I can pass on the five-block walk to lunch. I had already passed two of them and didn’t feel like walking back, and the remaining one had rather slim pickings, but I was happy enough with As a Thief in the Night, by R Austin Freeman. As you have no doubt guessed by now, in the first third of the book someone is discovered to have died of arsenic poisoning and the police are baffled as to where the arsenic came from, but of course it was done via poisoned candles.

As far as I recall, in my entire life I’ve read only one other story involving poison candles: Edgar Allan Poe’s “The Imp of the Perverse” (I had to look up the title).  I assume all subsequent stories with that plot element are ultimately children of that one.  ChatGPT in ‘extended thinking’ mode comes up with four such stories: Imp of the Perverse, Feet of Clay, a Marvel Comic from 1983, and a mention of a shop selling poisonous candles in Harry Potter and the Chamber of Secrets. Of course we know that’s not a complete list, since it doesn’t include As a Thief in the Night, and presumably there are some others missing too. But still, I’m pretty sure there are very few published works that include this plot device.

I read a fair amount — I’ve read a couple of thousand books in my life, I guess — so it’s not so surprising that I’ve read a few stories involving poisoned candles.  But to read Feet of Clay and then, in the very next book that I selected from a very small choice being given away on the street, to pick up another one… when I reached the point in As A Thief in which the cops start puzzling over how the poison was administered, I thought “you’ve gotta be kidding me. What are the odds?”

“What are the odds” is essentially unanswerable for this kind of thing. Yeah, sure, this particular coincidence was extremely unlikely but a person experiences millions of opportunities for coincidences in their life and some unlikely ones are bound to occur.  Also, to mention it just so nobody else has to, almost everything that we experience is ‘unlikely’ just due to statistical mechanics (or something akin to statistical mechanics, just very high entropy): a car passes me on the street, its license place is CNV 1193, wow, isn’t that amazing, what are the odds that that specific car would pass me at that exact location at that exact time, why, it’s unfathomable.

Still, we can acknowledge that there is nothing supernatural about coincidences like the one with the arsenic-laden candles while still enjoying them when they occur.

I have discussed coincidences on this blog a couple of times: https://statmodeling.stat.columbia.edu/2019/12/01/amazing-coincidence-what-are-the-odds/ and https://statmodeling.stat.columbia.edu/2024/11/08/if-you-wanted-to-be-a-top-tennis-player-in-the-late-1930s-there-was-a-huge-benefit-to-being-a-member-of-____-or-to-being-named-____/

I’m not claiming that these are in any way enlightening.  And I apologize if hearing about my coincidence is as boring as me telling you about the dream I had last night.

 

What does “Neuromancer” have to teach us about the role of AI in society?

Posted on by
18

This post is by Phil Price, not Andrew.

From junior high through about sophomore year in college I read a lot of science fiction, went to some science fiction conventions, etc., but then I drifted away from the genre for eight or nine years. What brought me back was “Neuromancer”, by William Gibson. It had come out in 1984 when I was in college but I guess I had already stopped reading science fiction by then, or else I somehow missed that specific book, so I didn’t get around to reading it until about 1992. The book is generally credited as starting the “cyberpunk” sub-genre, of which Neal Stephenson’s “Snow Crash” is another great example, although there are many progenitors with similar DNA; indeed I’m not sure why the much movie Blade Runner isn’t given the credit (or perhaps something even earlier).

I haven’t read Neuromancer in about thirty years, but came across it while browsing a bookstore and thought eh, why not give it another read, the development of artificial intelligence is a major theme and maybe it’ll be interesting in that context, not just for entertainment.

There are some minor spoilers below, nothing that I think would interfere with one’s enjoyment of the book but if you are especially picky about this kind of thing then you might want to stop reading now.

In Neuromancer we encounter two types of artificial intelligence: artificial _general_ intelligence, as personified (machinified?) by AI’s known as Wintermute and Neuromancer; and ‘constructs’ such as the “Dixie Flatline construct”, which, we are told, is not truly intelligent but merely seems intelligent. The Dixie Flatline construct is “just a bunch of ROM” that answers questions the way a guy called “Dixie Flatline” would answer them himself. But then it turns out it’s not just about answering questions, Dixie Flatline can also hack into computer systems, pretty much like the person on whom it is based. And it can’t _just_ be ROM because it can remember things that you tell it.

I recall being somewhat puzzled by the distinction between the real AI’s and the “construct”, back when I read the book, since the construct sure _seems_ intelligent. But now that distinction seems entirely reasonable: the construct behaves very much like an LLM like chatGPT, which…well, I know there are people who think that as LLM’s get more sophisticated they are going to turn into artificial general intelligences, but I don’t think that’s the case. Artificial general intelligence is possible, and an LLM might even be a key component to attaining it, but I don’t think any LLM, no matter how grand, will be enough on its own. My younger self was puzzled by the distinction between the construct and a “real” AI, but now it makes perfect sense to me! Just think of the construct as an LLM that is trained to respond like a specific real person.

Another somewhat-realistic-seeming element of the book is that there’s an organization, colloquially called the “Turing Cops”, that is tasked with preventing AI’s from becoming too powerful. There’s a fear that if an AI becomes powerful enough it could destroy humanity, or at least do terrible things. There’s a lot of current discussion about whether or how AI’s should be regulated, although at least for now I don’t think that discussion is focused on the capabilities so much as who can use them and how, whereas in the book the Turing Cops only care how smart they get.

So…what about the title of this post, what does the book have to teach us about the role of AI in society? Nothing. Or at least, nothing I can think of. It’s a work of fiction written forty years ago by someone who, by his admission, knew nothing about the technologies he was writing about. Nowadays we might say he was “vibe-writing” or something. There’s lots of nutty stuff and some plot holes.

I guess I’ll mention one more thing that is purely on the literary side. The main character of the book, a hacker/cracker named Case, is objectively a horrible person, as is his girlfriend Molly. They lie, cheat, steal, kill, get involved in a scheme that kills dozens of innocent people and show no remorse about it, etc. But I spent the whole book rooting for them! The story is told mostly from Case’s point of view, and I kind of adopted his view of the world. He’s not without a sense of emotion or a sense of morality, but while reading I found that I liked the people he liked, disliked the people he disliked, was appalled by the things he found appalling but unbothered by the things that didn’t bother him. I don’t really have a point here, I just find the phenomenon interesting.

This post is by Phil.

Which AI coding assistant should I be using?

Posted on by
51

This post is by Phil Price, not Andrew.

For more than a year I have been using chatGPT to help write code. Maybe “help write code” is an understatement: often chatGPT writes the code to my instructions. I almost always get much better code in much less time than of I wrote everything myself, but I also frequently run into frustrating problems in which chatGPT acts like a chowderhead. (Here I’m referring to the o4-mini-high flavor of chatGPT.)

One recent experience: I need to do an SQL query: Use Table A to find the ID numbers of all of the customers in a specific group, cross-reference with Table B to find the locations and electric meter numbers associated with those customers, refer to Table C to get the dates and times that the group was selected for some kind of action, and refer to Table D to get the electricity consumption for those meters at those times. The variables have different names in the different tables (e.g. ‘customer_id’ in one table is called ‘customer_number’ in another table). The result is a somewhat involved query, but not an _extremely_ complicated query, you just have to go through one link at a time and be a bit careful. I thought chatGPT would nail this easily but in the end I spent more time coercing chatGPT to do it than it would have taken to do it myself: it would propose a chunk of code; I would try it and it would fail because (for instance) the variable name matching wasn’t done for every one of the steps or some similar issue; I would point out the problem and ask it to try again and it would propose another chunk of code with some different problem; and so on. I even tried the “deep thinking” option, which took much longer but still produced buggy code.

I mentioned this to a friend and he asked if I have tried Claude Code, which is his favorite. This has some features that seem potentially quite useful, including that you can have it scan your whole project code base and use that as context. That sounds great in a way, but also rather scary: what’s to stop it from going off the reservation and reading files I don’t want it reading? To some readers this may seem paranoid: do I really think Anthropic is going to steal my credit card information or something? Other readers will think I’m not nearly cynical enough: of _course_ these companies are going to do all kinds of unethical stuff, maybe not as crude as stealing my credit card information but not necessarily a lot better than that either. OK, yes, chatGPT has significant flaws as a programming assistant, but at least it only knows what I tell it and I have complete control over that.

So…I’m somewhat dissatisfied with ChatGPT o4-mini-high, I’m scared of Claude Code…what else should I consider? There are quite a few coding assistants, are there any that can write good code without the risk that my information will be used in ways I don’t like?

This post is by Phill.


There’s no accounting for taste, whodunnit edition.

Posted on by
20

This post is by Phil Price.

Andrew recently gave a rave review to the whodunnit “Everyone In My Family Has Killed Someone”, by Benjamin Stevenson (and he also loved the sequel, “Everyone On This Train Is A Suspect”).

I read EIMFHKS and, in short, I liked most of it pretty well but hated the ending. Below is an what I said to Andrew in an email. It has a pretty significant spoiler, not about the identity murderer but about a plot point, so I’m putting it below the fold.

[Whoops, for some reason the ‘read more’ tag isn’t working correctly, or at least it isn’t when I preview the post. I’m adding this filler here just to put some more space so you have a chance to avert your eyes before inadvertently reading the next several sentences before you can stop yourself.]

Continue reading →

If you wanted to be a top tennis player in the late 1930s, there was a huge benefit to being a member of ____. Or to being named ____.

Posted on by
14

This post is by Phil. A couple of months ago, this blog had a discussion that was prompted by the fact that 2 of the top 5 female American tennis players are the children of billionaires. One, that could be a coincidence, but with two it seems that there must be something going on. See the discussion.

Well, right around the time Andrew made that blog post, my wife and I joined the Berkeley Tennis Club. On the wall of the club there’s a little display.

Photo showing Helen Wills, Don Budge, and Helen Jacobs (1938 Wimbledon finalists).

Both female Wimbledon singles finalists were named Helen! Isn’t that amazing? What are the odds?

Nah, of course the real news here is that three of the four singles finalists were from the Berkeley Tennis Club. And it’s not like they were honorary members, or had once been members and were kept on the books: they lived within a few miles of the club. To me, this seems even more remarkable than the ‘billionaires’ thing. Even in 1938, tennis was played by a lot of people in a lot of countries. I guess the two Helens might have benefited by being able to play against each other, maybe there weren’t that many other places two elite female players could get practice like that…so that helps a bit, I guess. But then what are the odds that a top male player would happen to play at the same club?

Under any circumstances it seems kind of amazing to have 3/4 of Wimbledon singles finalists come from the same small club, but there’s another fact which increases the level of difficulty: the club had (and has) no grass courts. (Don Budge’s Wikipedia page says “Accustomed to hard-court surfaces in his native California, Budge had difficulty playing on the grass courts in the east.”) Wimbledon has grass courts.

Honestly, I don’t know how to think about this, as far as how remarkable it is. If you think about it too narrowly — what are the chances 3/4 of Wimbledon singles finalists in 1938 would be from Berkeley? — that’s like throwing a dart at a wall and then drawing a bullseye around the dart once you see where it hits. But what’s the right way to think about it? What are the chances that at least 3 finalists at any major tennis tournament would come from the same town, in any year?

As long as I’m here, I’ll mention some off-topic facts that I came across when doing extensive research for this post (by which I mean I skimmed the Wikipedia entry for each of the three).

1. Charlie Chaplin was once asked what he considered to be the most beautiful sight that he had ever seen. He responded that it was “the movement of Helen Wills playing tennis.” (From Helen Wills’s Wikipedia page).

2. In 1938, when Budge was 22, he won Wimbledon without losing a set, and later that year he won the U.S. Open, becoming the first person to win the Grand Slam (winning all four major titles in a row).

3. Here, in its entirety, is what Wikipedia has to say about Helen Jacobs’s WWII experience: “Jacobs served as a commander in the U.S. Navy intelligence during World War II, one of only five women to achieve that rank in the Navy.”

A client tried to stiff me for $5000. I got my money, but should I do something?

Posted on by
34

This post is by Phil Price, not Andrew.

A few months ago I finished a small consulting contract — it would have been less than three weeks, if I worked on it full time — and I find it has given me some things to think about, concerning statistical modeling (no surprise there) but also ethics. There’s no particular reason anyone would be interested in hearing me ramble on about what was involved in the job itself, but I’m going to do that anyway for a few paragraphs. Maybe it will be of interest to others who are considering going into consulting. If you are here for the ethical question then you can skip the next several paragraphs; pick up the story at the line of XXXX, far below.

Continue reading →

ChatGPT4 writes Stan code so I don’t have to.

Posted on by
74

Several months ago I (Phil Price) wrote a Stan model to do some time series forecasting. It took me almost a full day to get it running and debugged. Today I decided to test ChatGPT4, by seeing if it could write a Stan model to give me the same functionality. I have been using ChatGPT to help me with coding, but in the past I’ve used it for little stuff, such as “I have a data frame with column names that all start with mm, and some of the names have ( and ) in them; write a function to remove all of that”. I have just started using it to write entire programs for me, and it looks like it is going to be enormously useful. Below, I present a verbatim transcript of the start of a ChatGPT4 session. At the end I got tired of cutting and pasting — and you would have gotten tired of reading it — but I tell you what happened. And at the very end, I’ll summarize my thoughts on the ChatGPT4 programming experience thus far.


Phil: I have monthly time series data. I want to fit a model to the data (typically from the past five to eight years) and use the model to forecast future values. Let’s call the data y[t]. Here are some things I want to include:
1. Seasonality. If the value in March is high in one year it is probably also high in the following year, and so on for other months. So I want ‘month effects’.
2. Autocorrelation in month effects. The month effect, for a given month of the year, is not iid random. If the March effect is high for several years and then drops a lot, it is more likely to stay low than to jump back up.
3. Local trend. If the value has increased for the past several months it will probably keep increasing, although it might not.
4. Soft constraints. The value of y[t] is very unlikely to fall below a value y_low, or to go above y_high. I don’t want to impose the constraints after the fact. Instead, I want to repel the forecasts from these limits. I want a fairly short-range repulsion, so the forecasts aren’t affected unless they get close to the limits.

Please write a Stan program to do this, and include comments that show where each of those four requirements is implemented.

ChatGPT4: Here’s a Stan program that incorporates seasonality, autocorrelation in month effects, local trend, and soft constraints:

Continue reading →

The cleantech job market: Every modeler is supposed to be a great Python programmer.

Posted on by
70

This post is by Phil Price, not Andrew.

I’ve had a run of luck ever since I left my staff scientist position at Lawrence Berkeley Laboratory to become a freelance consultant doing statistical modeling and forecasting, mostly related to electricity consumption and prices: just as I finished a contract, another one would fall into my lap. A lot of work came my way through my de facto partner Sam, but then my friend Clay brought me into a project, and every now and then my friend Aeneas has something that he needs for his company, and I had a couple of clients who found me through having heard about me without any personal connection.

One lesson is: even in today’s world, with LinkedIn and websites and blogs and other ways of making ourselves known to the world, personal contacts matter a lot in getting consulting work. Or at least that has been the case for me. That’s been good for me because I’ve had good contacts, but it’s not necessarily good for society. If you’re younger and don’t have a lot of work experience, and you don’t have many friends doing the same sort of work you’re doing, you won’t have the advantages I’ve had.

So, for seven years everything was great. But this year has not gone so perfectly: I’m down to two clients at the moment, and one of them only needs a little bit of work from me each month. I’m looking for work but, never having had to do it before, I don’t really know how. But one thing I know is that people use LinkedIn to look for jobs and for people to fill those jobs, so I updated my long-moribund LinkedIn profile and clicked a few buttons to indicate that I’m looking for work. Several recruiters have contacted me about specific jobs, and I’ve also been looking through the job listings, looking for either more consulting work or for a permanent job.

Three things really stand out. Here’s the TLDR version:
1. There’s a lot of demand for time series forecasting of electricity consumption and prices.
2. The modeler has to write the production code to implement the model.
3. It’s gotta be Python.

That’s pretty much it for factual content in this post, but then I have some thoughts about why one aspect of this doesn’t make much sense to me, so read on if this general topic is of interest to you.


I. Modeling and Forecasting of Electricity Supply, Demand, and Price.

There are quite a few jobs for electricity time series modeling, and for optimization based on that modeling. Some companies want to predict regional electricity demand and/or price and use this to decide when to do things like charge electric vehicles or operate water pumps or do other things that need to be done within a fairly narrow time window but not necessarily right now. And then there are other forecasting and optimization problems like whether to buy a giant battery to use when the electricity price is high, and if so how big, and how do you decide when to use it or recharge it. All of this stuff is right up my alley: I’m good at this and I have lots of relevant experience. To give an example of a job in this space, here’s something from a job description I just looked at (for a company called Geli): “Your primary responsibility will be to lead the development of our time series forecasting models for solar and energy consumption using machine learning techniques, but you will also help develop new forecasting models as various needs arise (eg: prototyping forecasting wholesale prices for a new market).” This is extremely similar to work I have been doing off and on for one of my clients for the past eighteen months or so. Sounds great.

And there’s a bullet list for that same job listing:
* Feature engineering
* Prototyping new algorithms
* Benchmarking performance across various load profiles
* Integrating new forecasting algorithms into our production code base with robust test coverage
* Collaborate with the rest of the team to assess how forecasts can be adjusted for various economic objectives.
* Proactively identify opportunities within [our company] that can benefit from data science analysis and present those findings.
* Work collaboratively in a diverse environment. We commit to reaching better decisions by respecting opinions and working through disagreements.
* Gain in depth experience in an exciting industry as you work with storage sizing, energy financial models, energy tariffs, storage controls & monitoring.

Continue reading →

Time Series Forecasting: futile but necessary. An example using electricity prices.

Posted on by
61

This post is by Phil Price, not Andrew.

I have a client company that owns refrigerated warehouses around the world. A refrigerated warehouse is a Costco-sized building that is kept very cold; 0 F is a common setting in the U.S. (I should ask what they do in Europe). As you might expect, they have an enormous electric bill — the company as a whole spends around a billion dollars per year on electricity — so they are very interested in the cost of electricity. One decision they have to make is: how much electricity, if any, should they purchase in advance? The alternative to purchasing in advance is paying the “real-time” electricity price. On average, if you buy in advance you pay a premium…but you greatly reduce the risk of something crazy happening. What do I mean by ‘crazy’? Take a look at the figure below. This is the monthly-average price per Megawatt-hour (MWh) for electricity purchased during the peak period (weekday afternoons and evenings) in the area around Houston, Texas. That big spike in February 2021 is an ice storm that froze a bunch of wind turbines and also froze gas pipelines — and brought down some transmission lines, I think — thus leading to extremely high electricity prices. And this plot understates things, in a way, by averaging over a month: there were a couple of weeks of fairly normal prices that month, and a few days when the price was over $6000/MWh.

Monthly-mean peak-period (weekday afternoon) electricity price, in dollars per megawatt-hour, in Texas.

If you buy a lot of electricity, a month when it costs 20x as much as you expected can cause havoc with your budget and your profits. One way to avoid that is to buy in advance: a year ahead of time, or even a month ahead of time, you could have bought your February 2021 electricity for only a bit more than electricity typically costs in Texas in February. But events that extreme are very rare — indeed I think this is the most extreme spike on record in the whole country in at least the past thirty years — so maybe it’s not worth paying the premium that would be involved if you buy in advance, month after month and year after year, for all of your facilities in the U.S. and Europe. To decide how much electricity to buy in advance (if any) you need at least a general understanding of quite a few issues: how much electricity do you expect to need next month, or in six months, or in a year; how much will it cost to buy in advance; how much is it likely to cost if you just wait and buy it at the time-of-use rate; what’s the chance that something crazy will happen, and, if it does, how crazy will the price be; and so on.

Continue reading →

Chess cheating: how to detect it (other than catching someone with a shoe phone)

Posted on by
67

This post is by Phil Price, not Andrew.

Some of you have surely heard about the cheating scandal that has recently rocked the chess world (or perhaps it’s more correct to say the ‘cheating-accusation scandal.’) The whole kerfuffle started when World Champion Magnus Carlsen withdrew from a tournament after losing a game to a guy named Hans Niemann. Carlsen didn’t say at the time why he resigned, in fact he said “I really prefer not to speak. If I speak, I am in big trouble.” Most people correctly guessed that Carlsen suspected that Niemann had cheated to win. Carlsen later confirmed that suspicion. Perhaps he didn’t say so at the start because he was afraid of being sued for slander.

Carlsen faced Niemann again in a tournament just a week or two after the initial one, and Carlsen resigned on move 2.

In both of those cases, Carlsen and Niemann were playing “over the board” or “OTB”, i.e. sitting across a chess board from each other and moving the pieces by hand. That’s in contrast to “online” chess, in which players compete by moving pieces on a virtual board. Cheating in online chess is very easy: you just run a “chess engine” (a chess-playing program) and enter the moves from your game into the engine as you play, and let it tell you what move to make next. Cheating in OTB chess is not so simple: at high-level tournaments players go through a metal detector before playing and are not allowed to carry a phone or other device. (A chess engine running on a phone can easily beat the best human players. A chess commentator once responded to the claim “my phone can beat the world chess champion” by saying “that’s nothing, my microwave can beat the world chess champion.”). But if the incentives are high enough, some people will take difficult steps in order to win. In at least one tournament it seems that a player was using a chess computer (or perhaps a communication device) concealed in his shoe.

I don’t know if there are specific allegations related to how Niemann might have cheated in OTB games. A shoe device again, which Niemann uses to both enter the moves as they occur and to get the results through vibration? A confederate who enters the moves and signals Niemann somehow (a suppository that vibrates?). I’m not really sure what the options are. It would be very hard to “prove” cheating simply by looking at the moves that are made in a single game: at the highest levels both players can be expected to play almost perfectly, usually making one of the top two or three moves on every move (as evaluated by the computer), so simply playing very very well is not enough to prove anything.

Continue reading →

Praising with Faint Damnation

Posted on by
3

This post is by Phil Price, not Andrew.

A friend and I were discussing a route for a bike ride. I was pretty tired and unmotivated so I said sure but let’s do a really easy ride. I suggested taking Bay Bridge bike path from Berkeley (California) to Treasure Island. My friend had never done that, and said “is that pretty nice?” I replied “No it’s not nice at all…it’s probably the least-pleasant ride I do with any regularity.”

One might think “jeez, why would you want to do that ride, then? But consider: _something_ has to be “the worst ride I do with some regularity.” I do this one every couple of months, when I want a change from my usual rides and when I don’t want to exert myself too hard. If it really sucked I wouldn’t do it at all! It’s sort of the opposite of “damning with faint praise”: I’m praising with faint damnation.

This is peripherally related to the Reebok Principle, which came up here about eighteen months ago in a post that I think is worth re-visiting because of its comment section, which went off into pandemic-related stuff a bit and provides an interesting reminder of what people were thinking at that point.

This post is by Phil.

Is Martha (Smith) still with us?

Posted on by
30

This post is by Phil Price, not Andrew.

It occurred to me a few weeks ago that I haven’t seen a comment by Martha (Smith) in quite a while…several months, possibly many months? She’s a long-time reader and commenter and often had interesting things to say. At times she also alluded to the fact that she was getting on in years. Perhaps she has simply lost interest in the blog, or in commenting on the blog…I hope that’s what it is. Martha, if you’re still out there, please let us know!

The Course of the Pandemic: What’s the story with Excess Deaths?

Posted on by
86

This post is by Phil Price, not Andrew.

A commenter who goes by “Anoneuoid” has pointed out that ‘excess deaths’ in the U.S. have been about as high in the past year as they were in the year before that. If vaccines work, shouldn’t excess deaths decrease?

Well, maybe not. Anoneuoid seems to think vaccines offer protection against COVID but increase the risk of deaths from other causes. Or something. I don’t much care about Anon’s belief system, but I do think it’s interesting to take a look at excess deaths. So let’s do that.

I went to https://stats.oecd.org and searched for ‘excess’ in the search field, which led me to a downloadable table of ‘excess deaths by week’ for OECD countries. “Excess deaths” means the number of deaths above a baseline (which I believe is the average over the previous ten years or something, perhaps adjusted for population; I don’t know the exact definition used for these data). “Excess deaths” over the past couple of years have been dominated by COVID deaths but that’s not the only effect: at least in the first year of the pandemic people were avoiding doctors and hospitals and thus missing out on being diagnosed or treated for cancer and heart disease and so on, suicides and car accident numbers have changed, and so on.

Below is a plot of excess deaths, by week since the beginning of 2020, in nine OECD countries that I selected somewhat haphazardly. You can download the data yourself and make more plots if you like.

“Excess Deaths” by week, as a percent of baseline deaths, in nine OECD countries, including the US.

If you had asked me a year or so ago, “what do you think will happen with US COVID deaths now that we have vaccines” I probably would have guessed something like what has happened in Italy or the UK or Belgium or France: there would be some ups and downs, but at substantially decreased magnitude. Instead, the US really stands out as being the only country that had high excess mortality prior to the vaccines and also has high mortality now.

But then, I also expected that just about everyone in the US would get vaccinated, which isn’t even close to being the case (about 20% of US residents haven’t gotten any COVID vaccination, and about 30% are not ‘fully vaccinated’…a term that is a bit misleading, perhaps, as the effects of the vaccines wears off for those of us who got our booster several months ago).

Also, there are competing factors — competing in the sense that some tend to make excess deaths increase while some make it decrease. Vaccines provide substantial protection, and doctors have gotten better at treating COVID, so those tend to lead to lower COVID deaths. But most people seem to have resumed normal life without many COVID precautions, presumably leading to higher infection rates than there would otherwise be. And of course there are still traffic accidents and suicides and drug overdoses and so on that could either increase or decrease compared to baseline.

I find the figure above really interesting. Here are a few things that stand out to me, in no particular order:

  • Denmark had no excess mortality through early 2021! That’s remarkable, they saved a lot of lives compared to the other countries.
  • Canada looks like the US in temporal pattern, which kinda makes sense, but with mortality at about half the US level.
  • I knew Italy got hit very hard early on, northern Italy especially, but hadn’t realized Belgium had it so bad. Jeez they had a terrible first year.
  • The time series in the U.S. is much smoother than in the other countries. Belgium, France, Sweden, the UK, Italy…they all had a big initial spike and then dropped all the way back to 0 excess deaths for a few weeks before the next spike. The U.S. went up and never came all the way back down, even briefly, until a few weeks ago. The U.S. has a much larger population and much larger geographic area than any single European country; perhaps the data on some small part of the U.S., like just New England or just Florida, it would look more like one of these other countries.
  • If the U.S. had matched the average excess mortality of the rest of the OECD countries, hundreds of thousands of Americans would be alive who are now dead.

I guess I’ll leave it to commenters to provide insights on all of this. Go to it!

This post is by Phil.

High-intensity exercise, some new news

Posted on by
22


This post is by Phil Price, not Andrew.

Several months I noticed something interesting (to me!) about my heart rate, and I thought about blogging about it…but I didn’t feel like it would be interesting (to you!) so I’ve been hesitant. But then the NYT published something that is kinda related and I thought OK, what the hell, maybe it’s time for an update about this stuff. So here I am.

The story starts way back in 2010, when I wrote a blog article called “Exercise and Weight Loss: Shouldn’t Somebody See if there’s a Relationship?” In that article I pointed out that there had been many claims in the medical / physiology literature that claim that exercise doesn’t lead to weight loss in most people, but that those studies seemed to be overwhelmingly looking at low- and medium-intensity exercise, really not much (or at all) above warmup intensity. When I wrote that article I had just lost about twelve pounds in twelve weeks when I started doing high-intensity exercise again after a gap of years, and I was making the point that before claiming that exercise doesn’t lead to weight loss, maybe someone should test whether the claim is actually true, rather that assuming that just because low-intensity exercise doesn’t lead to weight loss, no other type of exercise would either.

Eight years later, four years ago, I wrote a follow-up post along the same lines. I had gained some weight when an injury stopped me from getting exercise. As I wrote at the time, “Already this experience would seem to contradict the suggestion that exercise doesn’t control weight: if I wasn’t gaining weight due to lack of exercise, why was I gaining it?” And then I resumed exercise, in particular exercise that had some maximum short-term efforts as I tried to get in shape for a bike trip in the Alps, and I quickly lost the weight again. Even though I wasn’t conducting a formal experiment, this is still an example of what one can learn through “self-experimentation,” which has a rich history in medical research.

Well, it’s not like I’ve kept up with research on this in the mean time, but I did just see a New York Times article called “Why Does a Hard Workout Make You Less Hungry” that summarizes a study published in Nature that implicates a newly-discovered “molecule — a mix of lactate and the amino acid phenylalanine — [that] was created apparently in response to the high levels of lactate released during exercise. The scientists named it lac-phe.” As described in the article, the evidence seems pretty convincing that high-intensity exercise helps mice lose weight or keep it off, although the evidence is a lot weaker for humans. That said, the humans they tested do generate the same molecule, and a lot more of it after high-intensity exercise than lower-intensity exercise. So maybe lac-phe does help suppress appetite in humans too.

As for the interesting-to-me (but not to you!) thing that I noticed about my heart rate, that’s only tangentially related but here’s the story anyway. For most of the past dozen years a friend and I have done bike trips in the Alps, Pyrenees, or Dolomites. Not wanting a climb up Mont Ventoux or Stelvio to turn into a death march due to under-training, I always train hard for a few months in the spring, before the trip. That training includes some high-intensity intervals, in which I go all-out for twenty or thirty seconds, repeatedly within a few minutes, and my heart rate gets to within a few beats per minute of my maximum. While I’m doing this training I lose the several pounds I gained during the winter. Unfortunately, as you may recall we have had a pandemic since early 2020. My friend and I did not do bike trips. With nothing to train for, I didn’t do my high-intensity intervals. I still did plenty of bike riding, but didn’t get my heart rate up to its maximum. I gained a few pounds, not a big deal. But a few months ago I decided to get back in shape, thinking I might try to do a big ride in the fall if not the summer. My first high-intensity interval, I couldn’t get to within 8 beats per minute of my usual standard, which had been nearly unchanged over the previous 12 years! Prior to 2020, I wouldn’t give myself credit for an interval if my heart rate hadn’t hit at least 180 bpm; now I maxed out at 172. My first thought: blame the equipment. Maybe my heart rate monitor isn’t working right, maybe a software update has changed it to average over a longer time interval, maybe something else is wrong. But trying two monitors, and checking against my self-timed pulse rate, I confirmed that it was working correctly, I really was maxing out at 172 instead of 180. Holy cow. I decided to discuss this with my doctor the next time I have a physical, but in the mean time I kept doing occasional maximum-intensity intervals…and my max heart rate started creeping up. A few days ago I hit 178, so it’s up about 6 bmp in the past four months. And I’ve lost those few extra pounds and now I’m pretty much back to my regular weight for my bike trips. The whole experience has (1) reinforced my already-strong belief that high-intensity exercise makes me lose weight if I’m carrying a few extra pounds, and (2) made me question the conventional wisdom that everyone’s max heart rate decreases with age: maybe if you keep exercising at or very near your maximum heart rate, your maximum heart rate doesn’t decrease, or at least not much? (Of course at some point your maximum heart rate goes to 0 bpm. Whaddyagonnado.)

So, to summarize: (1) Finally someone is taking seriously the possibility that high-intensity exercise might lead to weight loss, and even looking for a mechanism, and (2) when I stopped high-intensity exercise for a couple years, my maximum heart rate dropped…a lot.

Sorry those are not more closely related, but I was already thinking about item 2 when I encountered item 1, so they seem connected to me.

 

An Easy Layup for Stan

Posted on by
12

This post is by Phil Price, not Andrew.

The tldr version of this is: I had a statistical problem that ended up calling for a Bayesian hierarchical model. I decided to implement it in Stan. Even though it’s a pretty simple model and I’ve done some Stan modeling before I thought it would take at least several hours for me to get a model I was happy with, but that wasn’t the case. Right tool for the job. Thanks Stan team!

Longer version follows.

I have a friend who routinely plays me for a chump. Fool me five times, shame on me. The guy is in finance, and every few years he calls me up and says “Phil, I have a problem, I need your help. It’s really easy” — and then he explains it so it really does seem easy — “but I need an answer in just a few days and I don’t want to get way down in the weeds, just something quick and dirty. Can you give me this estimate in…let’s say under five hours of work, by next Monday?” Five hours? I can hardly do anything in five hours. But still, it really does seem like an easy problem. I say OK and quote a slight premium over my regular consulting rate. And then…as always (always!) it ends up being more complicated than it seemed. That’s not a phenomenon that is unique to him: just about every project I’ve ever worked on turns out to be more complicated than it seems. The world is complicated! And people do the easy stuff themselves, so if someone comes to me it’s because it’s not trivial. But I never seem to learn.

Anyway, what my friend does is “valuation”: how much should someone be willing to pay for this thing? The ‘thing’ in this case is a program for improving the treatment of patients being treated for severe kidney disease. Patients do dialysis, they take medications, they’re on special diets, they have health monitoring to do, they have doctor appointments to attend, but many of them fail to do everything. That’s especially true as they get sicker: it gets harder for them to keep track of what they’re supposed to do, and physically and mentally harder to actually do the stuff.

For several years someone ran a trial program to see what happens if these people get a lot more help: what if there’s someone at the dialysis center whose job is to follow up with people and make sure they’re taking their meds, showing up to their appointments, getting their blood tests, and so on? One would hope that the main metrics of interest would involve patient health and wellbeing, and maybe that’s true for somebody, but for my friend (or rather his client) the question is: how much money, if any, does this program save? That is, what happens to the cost per patient per year if you have this program compared to doing stuff the way it has been done in the past?

As is usually the case, the data suck. What you would want is a random selection of pilot clinics where they tried the program, and the ones where they didn’t, and you’d want the cost data from the past ten years or something for every clinic; you could do some sort of difference-in-differences approach, maybe matching cases and controls by relevant parameters like region of the country and urban/rural and whatever else seems important. Unfortunately my friend had none of that. The clinics were semi-haphazardly selected by a few health care providers, probably slightly biased towards the ones where the administrators were most willing to give the program a try. The only clinic-specific data are from the first year of the program onward; other than that all we have is the nationwide average for similar clinics.

The fact that no before/after comparison is possible seemed like a dealbreaker to me, and I said so, but my friend said the experts think the effect of the program wouldn’t likely show up in the form of a step change from before to after, but rather in a lower rate of inflation, at least for the first several years. Relative to business as usual you expect to see a slight decline in cost every year for a while. I don’t understand why but OK, if that’s what people expect then maybe we can look for that: we expect to see costs at the participating clinics increase more slowly than the nationwide average. I told my friend that’s _all_ we can look for, given the data constraints, and he said fine. I gave him all of the other caveats too and he said that’s all fine as well. He needs some kind of estimate, and, well, you go to war with the data you have, not the data you want.

First thing I did is to divide out the nationwide inflation rate for similar clinics that lack the program, in order to standardize on current dollars. Then I fit a linear regression model to the whole dataset of (dollars per patient) as a function of year, giving each clinic its own intercept but giving them all a common slope. And sure enough, there’s a slight decline! The clinics with the program had a slightly lower rate of inflation than the other clinics, and it’s in line with what my friend said the experts consider a plausible rate. All those caveats I mentioned above still apply but so far things look OK.

If that was all my friend needed then hey, job done and it took a lot less than five hours. But no: my friend doesn’t just need to know the average rate of decrease, he needs to know the approximate statistical distribution across clinics. If the average is, say, a 1% decline per year relative to the benchmark, are some clinics at 2% per year? What about 3%? And maybe some don’t decline at all, or maybe the program makes them cost more money instead? (After all, you have to pay someone to help all those patients, so if the help isn’t very successful you are going to lose out). You’d like to just fit a different regression for each clinic and look at the statistical distribution of slopes, but that won’t work: there’s a lot of year-to-year ‘noise’ at any individual clinic. One reason is that you can get unlucky in suddenly having a disproportionate number of patients who need very expensive care, or lucky in not having that happen, but there are other reasons too. And you only have three or four years of data per clinic. Even if all of the clinics had programs of equal effectiveness, you’d get a wide variation in the observed slopes. It’s very much like the “eight schools problem”. It’s really tailor-made for a Bayesian hierarchical model. Observed costs are distributed around “true costs” with a standard error we can estimate; true inflation-adjusted cost at a clinic declines linearly; slopes are distributed around some mean slope, with some standard deviation we are trying to estimate. We even have useful prior estimates of what slope might be plausible. Even simple models usually take me a while to code and to check so I sort of dreaded going that route — it’s not something I would normally mind, but given the time constraints I thought it would be hard — but in fact it was super easy. I coded an initial version that was slightly simpler than I really wanted and it ran fine and generated reasonable parameter values. Then I modified it to turn it into the model I wanted and…well, it mostly worked. The results all looked good and some model-checking turned out fine, but I got an error that there were a lot of “divergent transitions.” I’ve run into that before and I knew the trick for eliminating them, which is described here. (It seems like that method should be described, or at least that page should be linked, in the “divergent transitions” section of the Stan Reference Manual but it isn’t. I suppose I ought to volunteer to help improve the documentation. Hey Stan team, if it’s OK to add that link, and if you give me edit privileges for the manual, I’ll add it.) I made the necessary modification to fix that problem and then everything was hunky dory.

From starting the model to finishing with it — by which I mean I had done some model checks and looked at the outputs and generated the estimates I needed — was only about two hours. I still didn’t quite get the entire project done in the allotted time, but I was very close. Oh, and in spite of the slight overrun I billed for all of my hours, so my chump factor didn’t end up being all that high.

Thank you Stan team!

Phil.

Is “choosing your favorite” an optimization problem?

Posted on by
82

This post is by Phil Price, not Andrew.

A week or so ago, a post involving an economic topic had a comment thread about what is involved in choosing a “favorite” (or a “preferred choice” or “the best”…terms like these were used more or less interchangeably in the comments). The thread got pretty long and, to me, a bit frustrating. Here’s what started it off: I wrote that “you need a consistent dimension to compare things because you need to be able to put choices in order if you want to choose the best one. You need a utility function that puts everything in one dimension.” (I shouldn’t have used the term “utility function”, with its strong connection to economics and rational choice. I think the actual situation is more general. )

Much to my surprise, several thoughtful, intelligent readers disagreed (and still disagree) with that statement.

In order to decide which you prefer among A, B, and C, you need to evaluate your preference for A, B, and C so that you can compare them. You need to be able to put them in order, e.g. pref(B) > pref(A) > pref (C)…or at least, pref(B) > pref(A or B). So far, so tautological.

You need to be able to evaluate the choices even when your preference depends on multiple parameters. If I’m choosing a phone, for example, I have a choice of (speed, cost, battery life) and of course many other parameters too. So you might have a choice between:

A: (fast, expensive, long-lived)
B: (fast, cheap, short-lived)
C: (slow, cheap, long-lived)

How can you choose among these? Well, go back to the tautology above: to decide which one you prefer, you need to be able to evaluate your preference for each. The tautology is still a tautology.

You get nowhere by saying “I prefer the speed of A, the expense of, B, and the battery life of either B or C”…you need to boil it down to one ‘preference function’, although you may not think of it that way. As Daniel Lakeland put it: “Phil, here’s a more mathy way to say what I think you’re saying. All complete totally ordered fields are isomorphic to Real Numbers. This is a known mathematical fact.” (And this is also where the Wikipedia article on optimization takes you.)

So what do people object to? Different things. At first I thought the objections were specious, but the discussion changed my mind, I think there’s some stuff worth thinking about. Hence this post.

1. One respondent said “You say “a consistent dimension” but surely you know that it is possible to think more multidimensional than a single one.””

2. A couple of respondents challenged the claim that you _always_ need a single preference function. Suppose, for the cell phone, I prefer fast to slow, cheap to expensive, and long-lived to short-lived. If there were an option D (in addition to A, B, and C above) that is characterized by (fast, cheap, long-lived) then D would be the dominant choice, preferred in every dimension, so there would be no need for a single preference function. You might have one, sure, but you don’t need one.


3. Responding to my claim that to come up with a favorite “ultimately you have to be able to put these in order, or at least to have one of them bubble to the top”, another respondent said “I guess you do, but I certainly don’t. In fact I sometimes specifically avoid ranking them by “value” so as to pretend I don’t have to deal with the consequences of my choices.”


I’ll give my take on these, below, but would also be interested in reading what others have to say.

Continue reading →

Objectively worse, but practically better: an example from the World Chess Championship

Posted on by
22
A position from Game 2 of the 2021 World Chess Championship match. White has just played e4.

This post is by Phil Price, not Andrew.

The World Chess Championship is going on right now. There have been some really good games and some really lousy ones — the challenger, Ian Nepomniachtchi (universally known as ‘Nepo’) has played far below his capabilities in a few games. The reigning champ, Magnus Carlsen, is almost certain to retain his title (I’ll offer 12:1 if anyone is interested!).

It would take some real commitment to watch the games in real time in their entirety, but if you choose to do so there is excellent coverage in which strong grandmasters discuss the positions and speculate on what might be played next. They are aided in this by computers that can evaluate the positions “objectively”, and occasionally they will indeed mention what the computer suggests, but much of the time the commenters ignore the computer and discuss their own evaluations.

I suppose it’s worth mentioning that computers are by far the strongest chess-playing entities, easily capable of beating the best human players even if the computer is given a significant disadvantage at the start (such as being down a pawn). Even the best computer programs don’t play perfect chess, but for practical purposes the evaluation of a position by a top computer program can be thought of as the objective truth.

I watched a fair amount of live commentary on Game 2, commented by Judit Polgar and Anish Giri…just sort of got caught up in it and spent way more time watching than I had intended. At the point in the commentary shown in the image (1:21 into the YouTube video), the computer evaluation says the players are dead even, but both Polgar and Giri felt that in practice White has a significant advantage. As Giri put it, “disharmonious positions [like the one black is in] require weird solutions…Ian has great pattern recognition, but where has he seen a pattern of pawns [on] f6, e6, c6, queen [on] d7? Which pattern is he trying to recognize? The pattern recognition is, like, it’s broken… I don’t know how I’m playing with black, I’ve never seen such a position before. Fortunately. I don’t want to see it anymore, either.”

In the end, Nepo — the player with Black — managed to draw the position, but I don’t think anyone (including Nepo) would disagree with their assessment that at this point in the game it is much easier to play for White than for Black.

Interestingly, this position was reached following a decision much earlier in the game in which Carlsen played a line a move that, according to the computer, gave Nepo a slight edge. This was quite early in the game, when both players were still “in their preparation”, meaning that they were playing moves that they had memorized. (At this level, each player knows the types of openings that the other likes to play, so they can anticipate that they will likely play one of a manageable number of sequences, or ‘lines’, for the first eight to fifteen moves. When I say “manageable number” I mean a few hundred.). At that earlier point in the game, when Carlsen made that “bad” move, Giri pointed out that this might take Nepo out of his preparation, since you don’t usually bother looking into lines that assume the other player is going to deliberately give away his advantage.

So: Carlsen deliberately played in a way that was “objectively” worse than his alternatives, but that gave him better practical chances to win. It’s an interesting phenomenon.







Why are goods stacking up at U.S. ports?

Posted on by
87

This post is by Phil Price, not Andrew.

I keep seeing articles that say U.S. ports are all backed up, hundreds of ships can’t even offload because there’s no place to put their cargo, etc. And then the news articles will quote some people saying ‘this is a global problem’, ‘there is no single solution’, and so on. I find this a bit perplexing, although I feel like my perplexification could be cleared up with some simple data. How many containers per day typically arrived at U.S. ports pre-pandemic, and how many are arriving now? How many truck drivers were on the road on a typical day in the U.S. pre-pandemic, and how many are on the road now? How many freight train employees were at work on a typical day pre-pandemic, and how many are at work now?

I understand that there are problems all over the place: various cities and countries go in and out of lockdown, companies have gone out of business, factories have closed, there are shortages of raw materials and machine parts etc. due to previous and current pandemic-related shutdowns…that’s all fine, but it does nothing to explain why goods that are sitting at US ports are not moving. Have all of the U.S. truck drivers died of COVID or something? Inquiring minds want to know!