New estimates of the effects of public preschool

Tom Daula writes:

You blogged about Heckman and the two 1970s preschool studies a year ago here and here.

Apparently there are two papers on a long-term study of Tennessee’s preschool program. In case you had an independent interest in the topic, a summary of the most recent paper is here, and the paywalled paper is here.

The research paper, by Mark Lipsey, Dale Farran, and Kelley Durkin, concludes:

This study of the Tennessee Voluntary Pre-K Program (VPK) is the first randomized control trial of a state pre-k program.

Positive achievement effects at the end of pre-k reversed and began favoring the control children by 2nd and 3rd grade. . .

It’s a long paper and I have not read it in detail so I won’t attempt to evaluate this empirical claim. At first it might sound surprising that preschool did not show positive effects, but the authors in section 6 of their paper do give some reasons why this might be the case.

26 thoughts on “New estimates of the effects of public preschool

  1. My wife teaches pre-school. First, much of pre-school is not about learning but about identifying what ways children can’t learn. This ranges from gross and fine motor skills – stair climbing coordination to using scissors – to memory, language, etc. Much time is spent evaluating kids for what they need, thus either reassuring parents or arranging for special services, and in many cases trying to get through to the parents that this is the actuality of their child and this kid needs help. Many kids actually go to pre-school because their parents see issues or worry in some way about what’s going on with the kid. The idea that it’s about ‘learning’ is academic world nonsense. Second, most of a kid’s learning is in the family and then among peers, so much of pre-school is about trying to help a child establish peer social abilities that help. This includes overcoming the problems the parents cause. There’s an absurd fiction that parents are all fine. They aren’t. Many are downright terrible. Many beggar description. My wife teaches in an upscale area filled with professionals affiliated with places like Harvard.

    Third, teachers know that children learn at substantially different paces through their early years. Actual gifted children are very rare. Highly talented children are rare. Verbally talented children are somewhat more common than artistically talented. Most eventually come up to some measure of grade level when grade levels become important. In most cases, that takes up to 3rd grade, so why anyone would think that measuring pre-school academic benefits … again, the main reasons for pre-school are to identify issues, to assist the kids in learning how to work through things socially and skillwise (and somewhat ‘academically, but not really) in a positive way, and to assist the families by giving them a framework for seeing how their children can progress and what they can do to help (if they’re willing to put in the effort, which is often not the case). One can say that maybe this entire thing is ‘unnecessary’ since kids will eventually grow up if they don’t die along the way, but then you could say the same thing about all school for large numbers of people. You could say the same thing about vaccinations for most people: they won’t die, probably, and the odds of getting polio really weren’t that high so screw off Jonas Salk.

  2. The authors’ comments in the press release about the “vitriol” they encountered in trying to get the paper past reviewers were interesting. (I honestly can’t tell if Jonathan’s comment above is an example of that sort of vitriol, or what.)

    I follow two areas of policy research pretty closely as an educated layperson: nutrition/weight and early anti-poverty interventions. In both areas, the open scorn, motive-questioning, straw-manning, and quick memory-holing among established “players” toward research that questions received frameworks is striking. This paper is probably like a paper that suggests that people can safely eat more fat or get a bit fatter than current guidelines prescribe; insiders tried to stop it from getting published, and once someone puts out a quick and dirty “refutation,” most people in the field will probably go on thinking and working on what they do now, with barely a ripple.

    Point #2: If I worked in poverty research, I would always ask whether an intervention would work better than just giving the target population the money that the intervention would cost, at scale.

    • Point #2: If I worked in poverty research, I would always ask whether an intervention would work better than just giving the target population the money that the intervention would cost, at scale.

      Seriously. It is almost always going to be better to give out the cash. First of all it has almost no overhead cost, so the quantity of resources going to the target population is almost 100% of the total resources in use, second of all, the resources will be used by people who know the most about what they need. The only cases where that’s not true are when trying to treat mental illness.

      • Although this (Daniel’s response to Kyle’s point #2) sounds plausible I’m not sure it’s right, at least unless we broaden the definition of ‘mental illness’ to include everyone with substance abuse issues and certain kinds of impulse-control issues. If you give a random selection of poor people a bunch of money, _some_ of that is going to be spent on lottery tickets, liquor, methamphetamines, etc. etc. (Of course that is also true if you give a random selection of rich people a bunch of money!). In an antipoverty program the leakage of money into those presumably undesirable directions constitutes a kind of ‘overhead’ to the money you are giving out.

        The question becomes, is this kind of overhead worse or better than the overhead that is skimmed off in running a different sort of antipoverty program: if the program’s targets would spend 15% of their money on stuff you don’t want them to spend it on, then you might be better off with some other kind of program even if it has 15% overhead. (I assume this is the idea behind giving people food stamps instead of money to buy food). And of course, it could even be that some of the expenditures are worse than simply loss of money: if some of the money is spent on drugs, that is presumably worse than having it simply disappear.

        I definitely disagree with Kyle’s point #2, but I’m not sure about Daniel’s statement that “it is almost always going to be better to give out the cash.”

        • Phil, substance abuse is already a mental illness issue (in that for example it’s treated by my sister who is a psych NP at the VA) and impulse control issues are also very plausibly mental illness, and often result in people winding up in the criminal justice system. Their outcomes are usually better if they receive psych services. So basically yes to your caveat.

          The worst thing though is that standard programs can easily have greater than 100% overhead, by trapping people into poverty traps they can never get out of we destroy the wealth they would have created for themselves and their families.

        • To expand on that idea. Phil, you’ve taken the first step towards understanding the economic idea, which is to say that money spent on “bad outcomes” is a kind of overhead and should be compared to the “overhead” of spending money on say civil servants salaries who sit behind a desk and serve long lines of impoverished people who stand in lines instead of doing something productive… so yes it’s true.

          Let’s just skip right to the end though. The *actual* cost of a program *is* the difference between the net societal rate of wealth creation with the program and the net societal rate of wealth creation with the best program we can think of. This is the opportunity cost and all economic costs are opportunity costs (this is a hard lesson for people to learn, it seems like plenty of people who are employed as “economists” forget it even)

          I will happily bet $10,000 that if we implement a flat tax at a rate of about 30% and Universal Basic Income at the rate of around $500/mo per adult and $250/mo per child (rates that I back-of-the-envelope calculate would keep the debt growing at constant rate relative to now), plus ramp up the UBI with age after age 55 until it’s basically the same as current Social Security payments by age 65, plus we provide a stop-loss Medicare insurance (Medicare pays 100% of all individual medical costs exceeding say 15% of GDP/capita in a year) and then eliminate all the individualized programs like housing subsidies and food stamps and etc that the rate of growth of median wealth, of total GDP, and the mean real income of people currently in the bottom half of income distribution will all increase relative to what we’re doing now. (Bet to be decided after 10 years of data collection)

          If you can make that happen I will gladly place $10,000 in an escrow account to be paid in 10 years time ;-)

        • Daniel, you say “I will happily bet $10,000 that if we implement a flat tax at a rate of about 30% and Universal Basic Income at the rate of around $500/mo per adult and $250/mo per child (rates that I back-of-the-envelope calculate would keep the debt growing at constant rate relative to now), plus ramp up the UBI with age after age 55 until it’s basically the same as current Social Security payments by age 65, plus we provide a stop-loss Medicare insurance (Medicare pays 100% of all individual medical costs exceeding say 15% of GDP/capita in a year) and then eliminate all the individualized programs like housing subsidies and food stamps and etc that the rate of growth of median wealth, of total GDP, and the mean real income of people currently in the bottom half of income distribution will all increase relative to what we’re doing now.”

          Talk about an untestable hypothesis!

        • :-) if I could get you to make that happen for $10,000 It’d be a steal worth more than buying Manhattan for a few beads.

          Seriously though, I am convinced that bad economic policy including bad tax policy, bad welfare policy, bad education policy (student loans), bad “intellectual property” policy, and bad monetary policy (“quantitative easing” to banks instead of print money and use it to pay the UBI) are together sucking trillions of dollars per year of potential productivity out of the US economy in large part because of *the things we aren’t doing* given the presence of those policies.

        • Also Phil the current rate of death by drug overdose and suicide under our current poverty treatment system is alarmingly high: something like 80000 people a year die of drug overdoses and 45000 people die from suicides. These are at all time highs in my lifetime measured as a rate.

          https://en.wikipedia.org/wiki/Drug_overdose

          https://en.wikipedia.org/wiki/Suicide_in_the_United_States

          So if you want to count bad outcomes as overhead you have to compare bad outcomes giving people cash to bad outcomes currently. It’d be hard to imagine it would get worse than our current poverty trapping system that gives poor people essentially zero chance of making it over the “potential energy barrier” to a middle class life.

          example explanation here:

          https://www.illinoispolicy.org/illinois-warped-welfare-system-traps-families-in-poverty/

      • Phil has the politics right! Most Americans probably think like him. Still, where is the research arguing, “Give Teachers Rather Than Money To Poor Families, Because They Will Blow the Money on Drugs”? Is this taught in textbooks? This is all assumptions.

        • Yeah, in fact almost all the tiny quantity of research that’s allowed to be done shows that Universal Basic Income helps people a lot more than the shitty means-tested constantly proving to bureaucrats that you “deserve” the aid, poverty-trapping bullshit we actually do. The biggest problem is people think like accountants (follow the visible dollars) and not economists (follow the real wealth creation/destruction).

        • Daniel, I’ve been pretty intrigued by UBI for a while now. I wonder though if your payout, $500/month is sufficient. IF we retain separate disability and unemployment benefits at current levels, then sure. BUT, if a major UBI selling point is eliminating the admin overhead of all that, people are gonna need WAY more than $6K/year to get by.

        • I think the big mistake people make in how they think about the UBI is that it’s supposed to by itself make it so you can kind of live in an ok way without working or some such thing. That’s not the point of UBI.

          In my opinion there are three things that need to go together:

          1) A flat tax on income to avoid tax complexity, tax perversity, and high marginal rates that prevent people from working, and to reduce the overhead of running small to medium businesses.

          2) A UBI that is independent of income to avoid the problems that flat taxes have at small incomes (ie. that you’re taxing away essential dollars from low income people). You can’t take the UBI away as people earn more because this increases the effective marginal tax rate and creates the poverty trap we have now. If you graph Consumption vs Income a UBI + flat tax is a straight line with a positive y intercept and a positive slope.

          3) Rules that adjust the UBI *based on factors other than income* such as age, educational participation, health and disability status, mental health status, even local cost of housing or food.

          4) Nice to have but not essential would be elimination of minimum wage, the problem minimum wage is supposed to solve is far better solved by the UBI itself without distortion of labor prices.

          The idea that “because $500/mo isn’t enough for someone to lounge on the couch and live safely UBI is useless” is very wrongheaded but it’s common enough. UBI isn’t supposed to by itself solve everyone’s problems, UBI is supposed to enable policy changes that taken together eliminate situations that currently keep people in poverty that they’re perfectly capable of getting out of if the policies didn’t make it so hard.

          Let’s look at the effect for a family of 2 adults and 2 kids: $1500/mo in income. Suppose further that both parents work at a low paying job, say $10/hr (here in CA min wage is $12/hr). Total after tax income would be:

          10*40*4*2*.7 + 1500 = $3740/mo and cost of tax compliance is basically “employer writes a check to the govt for 10*40*4*2*.3 every month”

          I plug married, earning $20/hr and working 160 hours per month into this: https://smartasset.com/taxes/california-paycheck-calculator#09DH8xJVgT

          and get estimate of $3200/mo So UBI is basically making every one of these families better off, and cost of compliance is… all the bullshit you have to go through as a small business to be tax compliant (and it’s kind of a full time job for one person if you have a business employing say 10 people).

          Similarly, we can do a calculation for a single person:

          10*40*4*.7 + 500 = 1620 whereas the calculator website says current system would have take home of $1429

          When you look at the distribution of incomes in the American Community Survey, there is a HUGE spike at zero income. There are a lot of people who don’t work for various reasons, but one reason is that *doing so hurts them* because they lose eligibility for housing subsidies or daycare subsidies or whatever.

          Do some people need more than the around $500/mo I’m proposing? Yes, certainly. Can we find rules that don’t have to do with their income to identify them? Almost certainly. Particularly for those with physical or mental disabilities. Those without physical or mental disabilities, if they aren’t working now, it’s highly likely that it’s due to work hurting them given the way benefit eligibility works, and this is particularly true for those with children.

          Would a UBI with bonus for participating in a first bachelors degree program be better than our current predatory school loan system? Certainly.

          I don’t claim to have the right rules written down, I just claim that it seems obvious that studying and designing such rules to make a UBI far far better than our current system is well within our power, the only reason it doesn’t happen is political and often pseudo-moralistic. The back of the envelope calculation suggests it’s not out of the question we could replace everything we do now with a UBI + rules for adjustments and *create a lot more real wealth* while essentially solving poverty entirely.

  3. It seems like the adverse effect is driven by the higher rate of placement of the children in preschool into special needs services. It could be that selection bias drives this result. However, is it really likely that parents sense that their toddler is behind in some way and think that a good idea is to put them in a more stressful environment where they will fall behind their peers. I think that this study could just as easily be taken as evidence that diagnosing toddlers, and providing them special services may often be counter-productive. All such services may do is label and stigmatize a child without really providing a benefit. The earlier studies that found a positive effect from preschool were conducted during a period when educators did not as readily believe that they could diagnose special needs at ages 3 and 4.

    • “It seems like the adverse effect is driven by the higher rate of placement of the children in preschool into special needs services. It could be that selection bias drives this result.”

      It may be the case that some preschool teachers are prone to classify children as special needs. I recall one colleague whose two-year-old son’s preschool teacher told her that the child had autism. The mother was flabbergasted, since her child seemed well within the normal range to her, so she talked with other parents of children in the preschool. She found that the same teacher had told the parents of several other boys in the class that their child had autism.

    • The “scores” and “gains” referred to in that headline are ratings of teachers. A cynic might think that the local government searched for a secondary metric that it believed it could improve. I would not say that, of course.

      • You might not, but I would.

        I remember being part of a big effort to improve quality in the company, about 20 years ago. We set up error metrics. In the first months we made decent progress in the metrics. Then the CEO announced that his bonus would heavily depend on further improvement in the metrics during the second year.

        The quality metrics jumped drastically higher in the second year. But why? Basically the definition of the metrics was “improved” subtly. If one root error causes 4 products to be faulty, is that 4 errors (old metric) or 1 (new metric)? Either metric can be defended, but the change in the metric over time cannot.

  4. Thanks for posting the two links to the preschool studies. I am quite sure that this same paper was referenced in a Brookings panel several years ago. Senator Lamar Alexander was at the Brookings event too. He had been a strong advocate of School CHOICE.

    I concur with many of Jonathan’s points. Parents can stress a kid out. We all have done that at some point or another if we are honest about it. I learned through my mistakes. It’s liberating to admit that. And now I am relaxed with my grandchildren.

    A teacher’s emotional makeup can make the difference in a child learning or not learning.

  5. The fadeout of cognitive effects from intervention programs is a standard finding, e.g.:

    We meta-analyze the evidence for the fadeout effect of IQ, determining whether interventions that raise IQ have sustained effects after they end. We analyze 7584 participants across 39 randomized controlled trials, using a mixed-effects analysis with growth curve modeling. We confirm that after an intervention raises intelligence the effects fade away. We further show this is because children in the experimental group lose their IQ advantage and not because those in the control groups catchup.

    Link: http://teknologipartiet.dk/wp-content/uploads/2016/03/The-environment-in-raising-early-intelligence-A-meta-analysis-of-the-fadeout-effect.pdf

    Some non-cognitive effects may persist, but that’s quite fragile as well.

Leave a Reply

Your email address will not be published. Required fields are marked *