Homework and treatment levels

Interesting discussion here by Mark Palko on the difficulty of comparing charter schools to regular schools, even if the slots in the charter schools have been assigned by lottery. Beyond the direct importance of the topic, I found the discussion interesting because I always face a challenge in my own teaching to assign the right amount of homework, given that if I assign too much, students will simply rebel and not do it.

To get back to the school-choice issue . . . Mark discussed selection effects: if a charter school is popular, it can require parents to sign a contract agreeing they will supervise their students to do lots of homework. Mark points out that there is a selection issue here, that the sort of parents who would sign that form are different from parents in general. But it seems to me there’s one more twist: These charter schools are popular, right? So that would imply that there is some reservoir of parents who would like to sign the form but don’t have the opportunity to do so in a regular school. So, even if the charter school is no more effective, conditional on the level of homework assigned, the spread of charter schools could increase the level of homework and thus be a good thing in general (assuming, of course, that you want your kid to do more homework). Or maybe I’m missing something here.

P.S. More here (from commenter ceolaf).

13 thoughts on “Homework and treatment levels

  1. One of the things you're missing is the psychology research that says that assigning more than a very small amount of homework to kids is actually detrimental, this is up through about high school I think. Don't have time to look up the references but I have seen several studies on this for grade school age children in the past.

  2. When I was putting together my final for my intro stat class, I cam across a rather interesting (if a bit old) study reported in Larson & Marx (1986) which documents the placebo effect of expectations.

    Rosenthal and Jacobson (1968) conducted a study of teacher expectations using the following method. The children in the study were all given a general intelligence test, but the teachers were told that this would be predictive of their ability to grow intellectually. Instead of reporting the results to the teachers the teachers were told one of three things: (a) that the student would have difficulty in class (low expectations), (b) that the student would develop at a moderate rate, or (c) that the student would experience sudden spurts of intellectual growth (high expectations). The “expectations” condition was randomly assigned and the true pretest scores were unknown to the teachers. A second test was given at the end of the academic year.

    Larson & Marx provide simulated data based on the study, but at least in the simulation results, students in the highest expectation group did significantly better than those in the lower too.

    In addition to the placebo effect, you are missing the differential attrition. Typically, if student violated the homework policy, the charter school could "council them out" and they would wind up back in the public school. I'm not sure to what extent these evaluations are doing intent-to-treat rather than treatment-on-treated analyses.

    Finally, the previous comments mention of gold standards made me think: I'm awfully glad that Mr. Bernanke doesn't use the gold standard, but instead uses a much more flexible monetary policy, otherwise we would be in a bigger mess than we are currently in.

    Larson, Richard J. and Marx, Morris L. (1986). An Introduction to Mathematical Statistics and Its Applications. Prentice-Hall.

    Rosenthal, Robert and Jacobson, Lenore F. (1968) Teacher Expectaions for the Disadvantaged. Scientific American, 218, 19—23.

  3. Andrew,

    Those are good points. I'm not sure you're 'missing' anything here but there are some issues we need to keep in mind.

    First of all we need to remember that overall charter schools tend to underperform comparable public schools according to test scores. This isn't necessarily as damning as it sounds — many of these school do well by other metrics — but it does raise concerns.

    But let's limit ourselves to schools like KIPP that focus on long hours and heavy workload and let's say, for the sake of argument, that these schools do well. Would making more of these schools available be a good thing or a bad thing? That's a tough question.

    Many of the concerns here also apply to tracking. There are definite pros and cons to segregating students based on ability. I'd say there are even greater concerns when the students are being segregated based on parents' time and interest level. I generally approach tracking on a case by case basis and I'm inclined to do the same here.

    The bigger question though involves aliasing. We aren't giving just one treatment here. Different charter schools incorporate different pedagogical and management theories (for example, many reformers advocate whole language instruction, an approach that, though interesting in theory, in practice often ends up flaky as your grandma's best biscuits). Since successful charter schools are often used to argue for major changes in the education system, I worry about confounded data.

    Finally there's the question of resources. We often hear demands that we should give more money to schools that work. The assumption there is level playing field, but what if there's an imbalance? What if the school that doing worse simply faces tougher conditions? Then diverting the money simply confounds the problem and runs the risk of setting up a nasty cycle where the school that needs more money gets less which makes it do worse which makes it get still less which…

    None of this means that we shouldn't offer this option to more parents. I just want to know why we;re seeing what we're seeing.

  4. Couple of points:

    1. We're talking about homework and longer school days and extended school years;

    2. I share your concerns about piling on HW, but I suspect heavy loads probably do help the kind of short term learning measured by standardized tests.

  5. But let's limit ourselves to schools like KIPP that focus on long hours and heavy workload and let's say, for the sake of argument, that these schools do well. Would making more of these schools available be a good thing or a bad thing? That's a tough question.

    I have battled with this one too…
    Should kids be allowed to be kids i.e. play with their friends and play sports, without "out of school" work taking all their time?
    But if a KIPP school is what it takes to get a disadvantaged kid into college maybe missing out on being a child doesn't matter?

  6. Excellent statisitcal points ceolaf.

    To be honest, I many times hate giving "craming type" homework assignments. If a student misunderstands a math problem, the problem gets much worse to solve when they return to class later. Homework works best for me when I give a smaller amount that is more challenging. Students come back with good questions that foster better understanding.

  7. In my one experience with a charter school lottery, the charter school made it quite difficult to enter the lottery; and to find out if your kid was selected, you had to go there and ask them. And, it appeared, it wasn't a random lottery at all. My son was known to the founders of the school as a star student, so when I showed up and nervously asked if his name had been picked, I was told, Don't worry about it, of course he's in.

  8. My daughter goes to a charter school, we also signed one of these agreements. I have huge concerns about the HW levels assigned in traditional HS, but I've found the charter school in my area also does this as well.

    Our education system needs a complete overhaul to benefit our children and our future. It doesn't take 12 years to go from basic addition to calculus. It takes half that time.

    It also doesn't take most children 12 years of english to learn to write and speak eloquently.

    150 years ago the average 4th grader had the today's equivalent of a 12th grade classical education.

    Is it because we've overwhelmed our children with information overload?

  9. This has come up at least once before on this blog (though ceolaf's piece was a pleasant surprise – nicely written)

    In 2007 someone presented a comprehensive meta-analysis of studies on the effect of the amount of assigned homework (tentative weak speculative conclusion was that too much was bad)

    There was one and only one RCT located and it was done by a Phd student who wrote up how difficult it was for them to find a statistically significant result that their committee insisting on having before they were passed (i.e. the reported results should be taken with a grain of salt).

    Maybe by 2070 there might be a few good ones done and reported …


  10. "150 years ago the average 4th grader had the today's equivalent of a 12th grade classical education."

    Do you have documentation on this assertion, because frankly I find it HIGHLY unlikely. I agree that the pace of our schools is set to the lowest common denominator, but 150 years ago (1861 – just prior to the start of the civil war) students were learning to read and write, do some basic math and then shuffled off to apprenticeship programs or back to the farm. Mandatory attendance was less than 10 years old and in very few states, and even where it existed it was rarely enforced.

    With respect to the original discussion on homework, I am in the first semester of using an online homework system with my community college statistics class. It is actually a larger amount of homework than I had previously assigned, but it has many advantages in terms of teaching. The students can try each question 3 times, it randomizes the numbers so they can't cheat off one another, and after they tell it to grade the problem it gives them feedback on what is wrong. I'll let you know at the end of the semester whether this additional teaching helps them truly learn the material better.

Comments are closed.