Regarding the question of what to call x and y in a regression (see comments here), David writes, “The semantics are ugly, and don’t really add much, because we are concerned with the relation of one to the other, not what they themselves are.”
I agree that the semantics don’t really add much, but they can subtract, I think! First off, the words “dependent” and “independent” sound similar and can lead to confusion in conversation. Second, as commenter Infz noted, people confuse “independent variables” with statistical independence, leading to the incorrect view that multiple regression requires the predictors to be independent.
I agree, though, that the term “parameter” can be confusing; sometimes it’s something that you can vary and sometimes it’s something you can estimate. And I’ve already discussed how “marginal” has opposite meanings in statistics and in economics.
A particular problem with 'indepedent variable' (in psychology at least) is that we are great pains to teach students experimental design and usually make them learn the correct definition and then proceed to violate the definition when we teach correlation and regression. I try hard to teach the terms 'predictors' and 'outcome' or 'response'. Actually I'm not that happy with the multiplicity terms for the Y variable in regression and sometimes find it easier to use Y!
I'm glad to know I'm not the only one who struggles with this.
Just to bring more confusion to the mix… Calling X a "predictor" is not always appropriate either. Prediction implies that Y is measured at some point in the future (relative to X). To some ears calling Y an "outcome" has the same problem.
I don't mean to sound too high-horsey about it — I use both of those terms incorrectly myself.
I've always hated the dependent/independent variable thing (which I first encountered helping psych students with their stats, but many more times since). It's … fractally wrong.
Though it's not always perfectly appropriate, I much prefer response/predictors – at least it's right some of the time.