“I coach the jumpers here at Boise State . . .”

Jeff Petersmeyer writes:

I coach the jumpers here at Boise State and as a fan of the book Moneyball by Michael Lewis (the book that got my brain initially wired to look further than just recruiting the “best” jumpers out of high school (as listed by Track and Field News, etc), I have tried to delve a lot deeper. While coaching at the Olympics this summer in London I began reading—a lot. I read close to 30 books while there for six weeks—including, Outliers, Thinking, Fast and Slow (amazing), Judgment in Managerial Decision Making (Bazerman), The Power of Habit, Start with Why, Switch, Talent is Overrated, The Talent Code, Freakonomics, The House Advantage, among others and more recently Nate Silver’s The Signal and the Noise. I have been collecting data from past years of NCAA championships in the long and the triple jump—finding out where the All Americans have come from (not too surprising: Texas, Louisiana, N. Carolina, Virginia, California, Florida, etc—warm states = more opportunity to practice the technical nature of the jumps (10,000 rule?)). I’ve collected the information as to what it takes to become an All American (distances) and how tall on average these jumpers were and what their personal bests were coming out of high school.

For example, a future All American might look like this in the men’s long jump: High school best of 24’1″, 6’1″ in height, and he’s got a 67% chance of coming from the states listed above. However, when looking at the Top 10 rankings over the last 13 years, only 12 of those jumpers became All Americans. Jumping over 24′ is the average of those who became All American, but it certainly doesn’t guarantee your success!

After reading Silver’s book and learning of Bayes’ Theorem, (as I’ve seen you discuss it in your blog and in a review of Taleb’s Fooled by Randomness), I started pondering if there were a way for me to make a rudimentary predictive model of high school recruits (long and triple jumpers). I could do what Kahneman prescribed for hiring an employee (pick six attributes and score them up, and always take the person with the highest score—removing any potential bias). I’ve thought of those traits as potentially: Best three jumps, performance at the state championship, speed, test score or GPA, height (not always easy to find), etc). There are several biases coaches fall victim to in recruiting (judgments based on intuition indeed: going to watch an athlete perform in practice or a competition—a year I HAVE to sign a good jumper, let’s say, and we “think” he’s going to be good.. not based on fact but based on our faulty intuition because we NEED him to be good and he’s interested in our program). Also, we get calls from coaches who claim their athlete is going to be good, or has high potential.

I’ve collected over 350 of the best jumps from the 2007 high school class (among tons of other data) to see without hindsight bias (not throwing anything out—Julio Jones plays for the Atlanta Falcons, Jeremy Kerley for the Jets—but keeping them in any potential rankings I devise). So now I’m getting to my question you can already see: Do you think there’s a way using a regression model or Bayes’ Theorem, or any direct or indirect correlations (Bill James is a hero of mine :)) that I could come up with something to weed out one jump wonders or find diamonds in rough?

Perhaps I’m wasting my time but I feel the more we can select student-athletes based on factual information vs. faulty intuition sabotaged by some sort of bias, we will be better served. My background is in political science as an undergrad, but unfortunately they let me escape college without taking statistics!

The jumpers at Boise State . . . cool! I love these sports examples. I don’t know enough about jumping to offer any great ideas right now, but I thought that if I post this, maybe some of you will have useful thoughts? (I’m looking at you, Phil.)

29 thoughts on ““I coach the jumpers here at Boise State . . .”

  1. Without any data, who could say?

    > I’ve collected over 350 of the best jumps from the 2007 high school class (among tons of other data) to see without hindsight bias (not throwing anything out—Julio Jones plays for the Atlanta Falcons, Jeremy Kerley for the Jets—but keeping them in any potential rankings I devise).

    What’s the outcome that lets you judge whether these highschoolers turned out to be champs or mediocrities?

    • Gwern– a generic definition of success for me for a high school long or triple jumper is attaining top 8 at the NCAA Championships. I have accumulated data from 2001 for that meet and have the ave performance from 1st to 8th place. The average 8th place, for example, in the triple jump had a personal best out of high school of 48’9″. 48’9″ would be a top 20 ranking (guesstimate) in the US each year.

  2. Any idea how much measurement error there is in the outcomes? And might that be a large factor in the longest or even three longest jumps in a high school student’s career? And might it vary by the event at which it was measured in a way that could be modeled (e.g., league meets vs. state championships)?

    And I second gwern’s comment: you need something to predict. So if you had a sample of all college jumpers in a year with their high school stats, you could do something like try to predict the college career (longest jump? number of championships?) based on high school attributes such as you describe.

    P.S. I would very highly recommend Jim Albert’s Curve Ball — it’s the best sports stats book I’ve read, and it takes a very Bayesian approach to prediction. There are lots of relevant discussions about things like low count data and rookie of the year regression to the mean effects.

    • Thanks Bob! I bought that book last night :). I love that stuff –like Win Shares by Bill James. I’m not sure what you mean “measurement error”. I’m just a dumb track coach haha.

      • In this case, “measurement error” means literally what it says. Someone has to measure how far the jumper jumped and make sure they started from the right place (assuming I’m remembering the long jump correctly). Failures by judges to disqualify jumps that took off past the line or to judge the appropriate landing places would be measurement errors.

        So the question is whether most people’s longest jumps are in fact not really that far. It’s what you’d expect statistically if you expressed the measurement error as a noise term. That is, you have a model that predicts how far someone can jump, then there’s some variation from form, effort, condition on a given day, etc., and then there’s further measurement error. You’d expect a person’s longest jump to be when the form etc. are good and the measurement error’s in their favor.

        • I see what you mean. I guess there’s always the potential for human error, but the same could be true at the college or international level. Chances are there’d be human error in calling fouls that weren’t fouls as well though.

  3. How many jumps do they get in practice in college? I ask because how you approach this might depend on your goals. If you want the best jumpers, then looking at the warm states you listed might be viable. But if you want the highest return jumper, it might make sense to look at the good jumpers from colder states, or those who have only been jumping for a year or two. Once they get more experience in college, returns are potentially high (with potentially low payout if you don’t have to give them full scholarships).

    Why not average their top 3 jumps in high school, and along with other predictors, use that to predict an average of their best 3 jumps in college (you now have 5 years of college data from the 2007 HS graduating class presumably)? You won’t have results for the people who didn’t jump in college, but that has to be fairly limited, right? Even people who weren’t recruited to big programs might have been on Div. III teams, with results and PRs on the internet.

    • Thanks for all the input. Yes, I have a lot of data (mostly from 2001 on). I chose to go even deeper in 2007 for the reason you cited (5 years has just passed for those who were seniors in 2007) and because their results are still online for the most part –from high school. I was thinking of doing what you said and collect as much data from those jumpers and 2008, 2009, 2010, etc to make better predictions. I could find their average improvement and see how many Black Swans I have :). As per cold weather athletes, being an Ohio native I’d love to see more come from colder states–including Idaho where I am now, but for some reason it just doesn’t happen –looking at the data. Maybe the 10,000 hour rule –with jumpers who can go outside and actually triple jump, aids those in warm states. Maybe the state focuses more on track. I do like the idea of weighing performance at the state championship (not sure how I would do that though). Thanks for all your thoughts!

      • I realized my first comment wasn’t actually as clear as it could have been. If you have the data, the advantage to averaging the top 3 jumps in HS (and the top 3 jumps in college) is that it would HELP reduce the measurement error problem Bob is talking about. I emphasize help, not resolve completely. This is probably more important in high school, where measurement is shoddy, compared to college meets. But even in college you don’t want a one-hit wonder–you want someone who is consistently very good.

        The approach I would take is straightforward, starting with a linear model and then implementing something multi-level like Magnus suggested (you don’t want your only take-away from this analysis to be “sign kids from Texas”). The biggest problem I foresee is that you actually need more predictors in this model. You have height, PR in high school, and state (and maybe more detailed geographic information). I would really want more than that–maybe sprint PRs? I’m not really sure what else goes into being a good jumper.

        If looking for potential value-added, in addition to the cold state/warm state dichotomy, you might also consider a variable indicating whether the athlete competed year round in track or played other sports. Youth track participation could enter in here too. I’d take a 6′ high jumper who played basketball instead of doing indoor track over a 6’1 jumper who did indoor and outdoor and summer meets, , but I assume you already subconsciously factor that kind of thing in when you’re doing recruiting the old-fashioned way too.

        The point here is that a model with just height and HS PRs isn’t going to be as satisfying or useful to you as I would like–you need more data for the predictors, if possible. Still worth tinkering with either way though.

        • I do like the idea you said, “averaging the top 3 jumps in HS (and the top 3 jumps in college).” Also, sprint PR’s would be great but not all jumpers run the 100 meters unfortunately–but for those who do, I’ve been collecting their times. More variables is what I’m looking for… I just haven’t thought of any factual things I haven’t (we haven’t) come up with ..yet.

          Also, I’m not sure what side of the fence I lie as far as if they did track all year around or played basketball. One side is that the guy/girl who did basketball and a fall sport could have more upside, while the guy who did track all year round has less room for improvement. But I could argue that the person who practices a skill like triple jumping year round has an advantage (10,000 hour rule?) and will be a leg up on the competition in college.

          I failed to address your question before: how many jumps do they take in college? It’s hard to say and varies per program. I’d say we have days where we bound and technical days where we jump (mostly from a short approach) (of course we sprint a ton too). We compete on average (I’ll ballpark it) 14-16 times per year but not always will they do their LJ or TJ at each meet so let’s say they jump 12 times x 6 jumps (if they make finals). Seventy-two long jumps (give or take a few) + if they triple jump (less jumps than LJ probably). The high school athlete has meets during the week often times and on the weekend but they compete less than the college athlete (although lots in the north east do indoor track).

          As for height, I think, like Malcolm Gladwell spoke of in Outliers, that at a certain height you’re tall enough (Michael Jordan in basketball). For example: in the women’s LJ at the NCAA level since 2001; if you’re under 5’5″ you probably run sub 12 seconds in the 100 meters out of high school–not my opinion but based on the data I’ve collected in that event. Those who weren’t good at the 100 m ran sub 14 in the 100 meter hurdles or high jumped 5’9″ or something–they did something else special.

          Thanks for the comments! All this feedback has been great.

  4. I have some comments on this interesting approach you are taking. The most important step you have already taken in is the collection of the data. In my opinion, you could now benefit from doing some more exploratory analyses before starting to run prediction models.

    1. First of all, it could be smart for you to see how much of the variance in the observed results your current variables can explain? A low R2-value (which is a measure of this) is not uncommon, but can give an indication on the possibility and accuracy of predictions. Another way of saying this, is to assess how much randomness is involved for the results that each individual gets (i.e. uncorrelated with your measured variables).

    2. As you note yourself, to measure an individual’s talent directly is not possible, since this is a so-called latent trait. Instead you use a set of variables that are supposed to be highly correlated with this trait. Perhaps you could run an exploratory factor analysis on your data and see which of the variables have the strongest relationship with the trait? Also, to me, the inclusion of GPA must be more of an indicator of another trait, motivation (however, I might be wrong here). This is a characteristic which is even harder to measure than talent, but in the long run will often be even more important. Do you have more indicators that can be used to measure this characteristics? (one possible data could for instance be the level of absence from school)

    3. You speak about a possible difference between warm and cold states. To me, this sounds as made for a multi-level approach. With this method, you can specify that the effect of the variables may differ between the different states. For instance, you speak about a 10.000+ effect. Could it be that this effect is different in the north and south due to climate factors?

  5. In this case, “measurement error” means literally what it says. Someone has to measure how far the jumper jumped and there can be error in that process. Presumably there can also be error in making sure they take off from behind the official starting line.

    If there’s substantial measurement error, you’d expect a person’s longest jump to happen when they combine good form and conditions and conditioning with measurement error in their favor.

    Maybe you could find a grad student in stats or a sharp undergrad to help you follow Magnus’s good advice (below this reply in another comment).

  6. Do you have any idea about how variable the jumpers are from the data you collect or is consistency something that you want to coach into them? If you have the three best jumps, then do you know how many attempts it took to make these? How often do they no-jump?

    Even from my (very) limited experience of athletics, I see long jumpers who will record 5 no-jumps, but the single recorded jump will win the competition. So, I guess my question comes round to whether you want someone who can jump out of the pit once in a while or someone who will consistently jump a ‘good’ distance – who is most likely to win most often?

    Obviously, in an ideal world you want someone who can jump out of the pit on a weekly basis but there you go.

    • Certainly we strive for consistency in our run throughs so that we don’t foul 5 of 6 jumps at a meet. I don’t have ALL of the jumps from the athletes I’m collecting data for as usually they only post the best mark from each meet. The state meet will post the series of jumps often times however.

  7. A lot of good comments. My quick thoughts:

    1. It is hard to tell from the description of the data that you have, but if you are only looking at high performers, you may also want to think about getting data on relatively poor performers to make sure you have a good idea about base-rates and to avoid selection effects. It may also allow you to test for less-intuitive proxy variables that could give you a recruiting advantage: e.g., it may be that more 6’1″ men jump well but that a 5’10” man who also jumps well has a higher likelihood of ending up as an all Americans, etc.

    2. For a criterion variable, the ideal is actual performance at the college level (or, more accurately, at BSU in particular). But that has to be a small number of cases. As a work around, you could look into major categories of likely predictors. Operationally, talent, for example, may be something like rate of return. So you would be looking for variables that can put performance into the metric of how much time the athlete has spent on the activity: higher performance for lower investment would be good, assuming that you can make up the difference. Along these lines, if you have multiple jumps for each individual, you can test the relationship between early jumps and later ones. Take the 10,000 rule. The relationship between experience and potential performance is generally not linear. Assuming performance after 6,000 jumps is a reliable predictor of .85% of an athlete’s jumping potential (something you would have to determine), finding high-school athletes that have very high performance for 6,000 jumps might make exceptional recruits because their current distances are shorter than many other top recruits (who presumably have had 10,000+ jumps), while their predicted distances, once you give them the opportunity to make the other 4,000 jumps, are longer. (Proxy variables for fewer number of jumps, aside from asking, might be multi-sport/activity (e.g., debate) athletes, job during high school, low number of meets attended, etc., which may indicate lack of time for jumping.)

    To this, for predicted final performance at BSU, you could add factors that predict a willingness to favor Boise, stick with training generally (e.g., motivation) and your style of training in particular, etc. Throw the parts in a model and see how it does, then tweak and test again as necessary.

    (I am at UOregon, but the rest of my family is in Boise, so on that basis, go Broncos!)

    • Erik,

      I’d definitely love to talk to you further to get your ideas! I like “1. It is hard to tell from the description of the data that you have, but if you are only looking at high performers, you may also want to think about getting data on relatively poor performers to make sure you have a good idea about base-rates and to avoid selection effects. It may also allow you to test for less-intuitive proxy variables that could give you a recruiting advantage: e.g., it may be that more 6’1″ men jump well but that a 5’10″ man who also jumps well has a higher likelihood of ending up as an all Americans, etc.”

      While the data is available for the “not-so-good” jumpers, I’d say there are a LOT more bad jumpers than good ones so do you have an idea of how I would go about that or could I just have a “sample” group of bad jumpers — or average jumpers?

      Tracking the number of jumps would be next to impossible as I don’t think I could track that, even in college (as I listed above, many days are bounding days and some are technical days where we’ll jump–and of course track meets).

      I’d love to hear from you as I think you’re on to something! :)

      Thanks!

  8. Have you considered looking at some free online courses on stats and machine learning? I normally wouldn’t suggest something like that, but, since you mention having read over 30 books, you’re clearly willing to put in the time.

    I’ve done a few courses on http://www.coursera.org (among other places), and I’m working through the neural networks course right now. It assumes no specific background, other than some math, and the ability to program a bit, and it covers state of the art material in the area (deep learning, which is what Google currently uses for voice recognition and transcription), including one method that was published after the course started, in October. I don’t actually suggest that course, since you probably don’t want to apply deep learning to your problem, but I bring it up as an example because it shows how you can go from having pretty much no background to being able to solve significant problems after taking a course or two.

    I’m not sure what your background is, but this course (https://www.coursera.org/course/dataanalysis), which starts in a month, might be a good fit.

    • Dan,

      That’s great! Thanks. I’ll definitely check that out..probably better than the texts I ordered on amazon that will have me scratching my head :)

  9. Adjust for altitude. Bob Beamon broke the world record by almost 2 feet at the Mexico City Olympics in part because he was long jumping at 7300 feet. Adjusting for altitude is particularly relevant to athletes who live with 1000 miles of Idaho, many of whom compete at altitude.

  10. A big issue for long jumping is that a lot of the talent drops out to focus on more remunerative sports, such as football or the 100m dash. That’s one reason that records don’t get broken very often in long jumping. Look for clues about whether a high school athlete will stick with long jumping.

  11. Pocatello, Idaho is at 4400 feet elevation, so your long jumpers should get an extra, say, 3 to 8 inches per jump over sea level jumpers. That may attract athletes looking for an edge in making goals such as the minimum length necessary to qualify for the NCAA tournament or the Olympic trials.

  12. The very best high school athletes aren’t likely to accept a scholarship offer from Idaho State or stick around Pocatello jumping when they could be playing wide receiver for Alabama, so you might want to look in your model to see if there are any features that predict late blooming.

    Another concern I would have is of high school athletes who got excellent coaching and tutoring in exotic specialties when young. For example, when I was a kid, three different Curran brothers won the California state pole vaulting championship. That was because their dad had been a pole vaulter and they had a pole vaulting pit in their backyard. They were good in college, but not as great as they had been in high school where their outstanding training had given them a big advantage over kids who had just taken up pole vaulting.

    • Steve,

      Yes, I always factor in altitude jumps…while the NCAA doesn’t penalize jumpers for jumping at altitude, I will factor that in if someone, say from Denver, has all of their best jumps at altitude. I do deal a lot with jumpers who play football and/or want to play football in college. With our football program being as good as they are, they often times aren’t interested in these guys–which is understandable. On your point about early exposure to good coaching: I wish had more information on those types of kids… we know a lot of programs where that stereotype is often true but there are many unknown unknowns where we don’t.

  13. When I read “I coach the Jumpers here at Boise…” I immediately thought: “If they have a guy on staff whose entire job is to prevent students from jumping off buildings perhaps they should just close down the school and send the students elsewhere…”

    Glad to realize this was simply a misunderstanding.

Comments are closed.