I use just-in-time teaching assignments in all my classes now. Vince helpfully sent along these instructions for setting these up on Google. See below.

I think Jitts are just wonderful, and they’re so easy to set up, you should definitely be doing them in your classes too. I’ve had more difficulty with Peer Instruction (the companion tool to just-in-time teaching) as it requires questions at just the right level for the class. I do have students frequently work in pairs, though, so I think I get some of the benefit of that.

P.S. I’d love to share all the Jitts with you for Bayesian Data Analysis, but I’m afraid this would poison the well and future students would not have the opportunity to be surprised by them. Yes, I know, I should just come up with new ones every year—but I’m not quite ready to do that! Perhaps soon I will.

In the meantime, a commenter asked for some Jitts, so here are the ones for the first and last weeks of class:

Jitt questions for Bayesian Data Analysis

For each class, 3 questions:

1: basic statistics

2: from this week’s reading

3: student feedbackFor Class 1a (to be done during the first class):

1. In a national survey of n people, you estimate the gender gap—the difference in support for Obama among men and women. How large does n have to be so that you can estimate this gender gap to within a standard error of +/- 3 percentage points?

2. I have three cards: one is black on both sides, one is red on both sides, and one is black on one side and red on the other. I pick a card at random out of a hat and look at one side only. It is black. What is the probability that the other side is black?

3. What would you like to hear more about from Chapter 1?

For Class 1b (chapter 1):

1. If x ~ N(0,1) and y ~ N(0,1) are indepdent random variables, calculate the probability that |x| > 2|y|.

2. In the football example, use the normal approximation to estimate the probability that the favorite wins, if the point spread is 7 points.

3. What material in the course so far is new to you?

. . .

For class 12a (chapter 22):

1. It is generally recommended that when you fit a regression model including the interaction between two predictors, you should include the main effects as well. But give a real-world example in which it can make sense to include an interaction without including both main effects.

2. Suppose you have a mixture model with three components. For each data point you want to identify which of the three components it comes from. It would be best to use the full posterior distribution but you need a point estimate. Which of the following would you prefer: the posterior mean, the posterior median, or the posterior mode? Assume each of these is done pointwise (that is, you are getting the marginal mean, median, or mode of the latent component for each data point, not the joint mean, median, or mode for all the data points at once).

3. What was the most important thing you think was missing in this course?

For class 12b (chapter 23):

1. Assume x and y are normally distributed with mean 0, variance 1, and correlation r>0. We draw a large sample from this bivariate distribution and then throw away all (x,y) pairs where either x or y is less than zero. We calculate the sample correlation using the remaining samples. On average, will this sample correlation be higher, lower, or equal to r?

2. Consider a Dirichlet process with the precision parameter alpha converging to 0. The limiting posterior of is sometimes known as ________. What is the limiting posterior as alpha increases to infinity?

3. What aspect of statistics would you most like to learn more about?

And the slides for the classes are here.

OK, here are Vince’s instructions:

1) go to https://drive.google.com/

2) Sign in, if necessary

3) Click create -> Form

4) Bypass the annoying pop-up, if necessary

5) Name the form, probably pick the default theme

6) Repeat until done:

6.0) For the first time through, create a text response with the student’s name

6.1) Name the question, not that important (Question 1, Question 2, …)

6.2) In the help text, add the question body

6.3) Change the question type to “Paragraph text”

6.4) Make sure “Required question” is checked

6.5) Click “Add item” if necessary

7) Under Confirmation Page at the bottom, probably disable “Show link to submit…” and enable “Allow responders to edit”

8) Under “Choose response destination” button on the toolbar, can have it all go into a common spreadsheet. Create a spreadsheet first and then select it

9) Share with students by clicking the “Send form” button at bottom

Probably easiest to copy/paste link into an email client with a group setup

Can create a group on Google, send to them

Am I right that these responses are anonymous?

Mark:

No, the Jitt responses are not intended to be anonymous. We grade the students on their Jitts. But we don’t assign grades on the basis of correctness of the answers, we give full credit to students who try their best.

Very cool. I had a professor once who would create a solutions manual (for assignments) by drawing from excellent student answers. The possibility of being *cited*, even if your name is withheld, seems like a motivator that may be even more powerful than traditional incentives (i.e. points).

The online interface seems great — also perhaps a good opportunity for experimentation?

I have been thinking of teaching my Bayesian stats course “flipped” next time, using things like jitts to ignite responsive class sessions, while making all the lecture content available as recordings. I’m attracted to the flipped/jitts idea, because I find students have a lot of idiosyncratic problems with the material.

If anyone has tried an approach like this, I’d love to hear about it, both what worked and what did not work.

On the topic of education, how do you assess your student’s performance? I teach high school AP Stats and one project that I will have my students work on is analyzing different grading systems. They usually find this very interesting. Often they end up wondering why their grades are usually based on averages of many assignments as opposed to other methods. I then have them create their own methods to measure their overall performance in the course.

Do you do anything statistically interesting to measure your students performance and learning in class? If you do, I would love to share your method with my students during our discussion.

Michael:

Psychometric principles suggest that it makes sense to have several independent short questions rather than a few long ones. Also, it can’t be hard to show that, if the questions are well-behaved, that summing up the points is a reasonable approximation to fitting a full item-response model.

The hard part, though, is deciding what goes on the test. I think it would be good for college teachers such as myself to have more standardized testing material and less of us just making up the exams on our own.

“I think it would be good for college teachers such as myself to have more standardized testing material and less of us just making up the exams on our own.”

-Kyrie eleison to this. I’m teaching sections of a college class for the first time this semester, and the idiosyncrasy of the professor’s assessments and points of emphasis drive me crazy. You chose the textbook, why not assess some of the stuff in the textbook?

Otherwise, it often becomes an assessment of who is clever enough to understand the professor’s idiosyncrasies, rather than of who (struggled and) mastered the material.

I was very annoyed with a self-congratulatory article from psych professors at Dartmouth who announced that they wouldn’t be offering credit to those who scored a “5” on the AP Psychology exam any more, because when they gave them the Dartmouth Psych 101 Final on the first day, they didn’t do well. Well, surprise, surprise! It may be that what they’re teaching at Dartmouth is more important, or it may be that what the College Board has selected is more important, but the mere fact of their lack of alignment isn’t dispositive of anything.

[…] Meg’s talked in the past about how she uses clickers in her lectures. If your university doesn’t have a clicker system, or you don’t like the one it has, here’s how to use Google Docs as a free clicker system. […]

[…] posted a 15-step process for using Google Forms* (now part of Google Drive, of course) to set up web-based […]

Just one or two sample jitts would be illustrative, for BDA or for anything else.

OK, I put in a few. See revised P.S. above.

Andrew, the links to the best lines in BDA and the student lecture notes seem broken.

Links fixed. But, just to warn you, the 75 best lines might be a bit of a letdown if you didn’t actually hear them live. I’ve included them so that anyone teaching with the slides will have some funny lines to use throughout the semester.

[…] Gelman‘s post got me interested, I wrote my own first post on JiTT and ordered this book. I’ve now read the […]

[…] Alex Dow, Lada Adamic, and Adrien Friggeri. A post written in September shared how to set up just-in-time teaching assignments. I liked the way Andrew introduced Vince and his approach to setting up on-line just-in-time […]