Skip to content
 

What is expected of a consultant

Robin Hanson writes on paid expert consulting (of the sort that I do sometime, and is common among economists and statisticians). Hanson agrees with Keith Yost, who says:

Fellow consultants and associates . . . [said] fifty percent of the job is nodding your head at whatever’s being said, thirty percent of it is just sort of looking good, and the other twenty percent is raising an objection but then if you meet resistance, then dropping it.

On the other side is Steven Levitt, who Hanson quotes as saying:

My own experience has been that even though I know nothing about an industry, if you give me a week, and you get a bunch of really smart people to explain the industry to me, and to tell me what they do, a lot of times what I’ve learned in economics, what I’ve learned in other places can actually be really helpful in changing the way that they see the world.

Perhaps unsurprisingly given my Bayesian attitudes and my preference for continuity, I’m inclined to split the difference, along the lines of two other people quoted by Hanson. Christopher McKenna divides “the [consulting] roles into two parts”:

The first part is . . . they bring advice to a firm that doesn’t otherwise have it. . . . The second thing that they provide is legitimacy . . . you hire the consultants to confirm what you already thought.

Similarly, Nick Bloom says:

There are really two types of consulting. There’s operational consulting, you know, down on the factory floor . . . those guys are very much like seasoned, gnarly, ex-manufacturing managers that have spent twenty years working in Ford and are real experts, and are now getting paid as consultants to hand out advice. . . . And then there’s the very small elite end, strategy consulting, about five percent. And that’s much more helping CEOs make big decisions.

I just thought I’d add to the discussion here by sharing my own experiences. I’ve done some consulting of the “operational” sort, where I provide advice so that people can better solve their problems. Other times I’ve been paid to give a short course on Bayesian statistics or multilevel modeling (I’ve done that at Procter & Gamble, Merck, and Google), and I hope it will be useful to the people there, but I really have no idea. Still other times I’ve been paid to do really easy stuff—I’m thinking here of some legal consulting involving simple random sampling of documents from files. Really anyone could’ve done that; on the other hand I know enough not to get distracted from the main goal in such problems.

Only once have I been hired simply to be a yes-man as in the scenario described by Yost in the first quote above. This was in 2000 when someone from the exit-poll consortium called me and asked if I wanted to consult for them. That sounded fun. They sent me some material and I came to the meeting. I was expecting that I’d sit quietly for a few hours listening to everything and then I’d have the chance to give my suggestions. Instead, when I got to the meeting they seemed to want me to stand up and talk. So I did so for awhile. I told them about multilevel regression and poststratification and how great it was, also I gave some suggestions about improving the exit polls. Eventually I stopped. Somehow I’d talked for about 2 hours. There was silence. I said, Do you want me to leave now? Someone said yes. And that was it. They did pay me. But I never quite figured out what they wanted from me. Maybe just a rubber stamp? I have no idea.

There have also been lots of consulting meetings that have gone nowhere, sometimes (annoyingly) after I’ve been assured I’d be paid. But usually I think they really want my expertise. Not a lot of Rick Mishkin jobs, is what I’m saying. But this is when I’m consulting directly. Maybe things are different when companies hire McKinsey-type consultants, rather than hiring a statistician or economist directly.

16 Comments

  1. Robin Hanson says:

    I thought my position was less extreme than you portray it. I am mainly skeptical about the more limited set of strategy consulting, and I admit they can give useful advice. I just think they could probably get that advice cheaper, if the weren’t trying so hard to provide legitimacy.

  2. Entsophy says:

    Also a huge portion of consulting is there just to satisfy regulatory requirements. In most of those cases they aren’t interested in having a prestigious consultant tell them something they already know, there more interested in having any old consultant satisfy a regulatory check-in-the-box. The real decision makers generally couldn’t care less what the consultants do as long as it satisfies the requirement (except in the rare example where the consultants conclusions are particularly extreme).

  3. Douglas Knight says:

    This is about management consulting, which is not, I think what you mean by “expert consulting.” This is not about bringing in a statistician to a company that can’t afford one full time. This is for firms that have managers but could manage better. Here is the experiment by Bloom, et al. The experiment involved non-experts providing basic book-learning to small firms, though maybe the list of advice was compiled by an expert. Bloom’s description of operational consulting is of experts at managing factory lines, which doesn’t quite match, but might reflect firm size.

    I agree with Robin that your quote is not representative of his post, though I think his position is clearer in his earlier post.

  4. K? O'Rourke says:

    Douglas:

    Robin’s second post rings true with my unusual summer job while an MBA student in the early 80s.

    The new CEO of a major corporation who was supplanting a former CEO who had just been “kicked upstairs”, but with palpable resentfulness. They had obtained a quote from one of those famous upscale management consulting firms help blast through the coalitions and then he had an idea. Why spend all that money when he could hire available MBA students for just the summer and get the same result. Even with complete naiveté and no industry knowledge at all, I think we (20+) were able to do that, at least a reasonable benefit/cost. It was about giving an air of a possible alternative new way to run the business, divide and inquire, shake skeletons out of the closet and write long involved (albeit largely baseless) strategy plans. All the calculations were down on hand calculators except for mine, which at the very end of the summer I had learned how to use SAS for doing lots of sales analysis and forecasting. Even though 90% of the time was spent just trying to get the calculations done on hand calculators, enough of an air of expertise was somehow generated. One student was very good at this http://www.cbc.ca/metromorning/columnists/business/ and perhaps has even improved.

    Now consulting probably very a lot by subject and client’s wants – in clinical research it was almost all about getting the paper by the journal reviewer or a grant by the statistical reviewer volunteering at a funding agency.

  5. I’ve done a fair bit of consulting for my small natural language processing company over the past ten years. Like statistics, natural language processing is something may companies think they want, but have no idea how to do themselves.

    We almost always handed out “free” consulting. Usually on the phone to people who called us out of the blue. Our blog and tutorials Google ranking was pretty much our only approach to marketing other than occassionally going to business-oriented conferences.

    Our goal was to sell software licenses (because consulting doesn’t scale nor does it provide continuing royalty income), but since so few people knew how to use toolkits like ours, we had to help them along the way. We even provided “free” consulting with our startup license package.

    We were brutally honest with customers, both about our goals and their goals. Their goals were often incompatible with ours (use company X’s software to do Y — we didn’t take that kind of job, but would send work to other people we trusted). More often, their goals were unrealistic, even if we had the big-brain count and computer power of Google, much less for a two-person company. Sometimes we had a hunch about how we could do what they were asking, but weren’t certain enough to just sell it. We found honesty up front often led to the company funding us to do some research or proof-of-concept studies (the advantage of supplying something with very little competition and desperate customers).

    When we signed contracts with people and did consulting, it was a combination of technical and strategic consulting. Not that we provided business strategy consulting, but we had to work with customers from their vaguely specified goals and needs (and often over-specific preconceptions about how they wanted to do it) toward a feasible project that could actually help with their business needs.

    In my experience, the downside to hiring academics to do consulting is that they tend to be fixated on 2nd or even 3rd order details of problems that are fairly simple, while ignoring the grungy details needed to make something work in the field.

    In the end, we built lots of cool stuff with lots of different customers and even got some of them to fund some open research and software development.

    • Joe Newkib says:

      When you say unrealistic is that due to lack of abstract conceptual collaboration between you and a client or more so lack of resources budget constraint

  6. zbicyclist says:

    I suspect Levitt is deluding himself, and if he went back to those companies 3 years later he would find they might remember him fondly, but would not have changed anything basis his input.

  7. K? O'Rourke says:

    zbicyclist: If you change Levitt’s words industry to topic and economics to thinking, its almost word for word what Ian Hacking http://en.wikipedia.org/wiki/Ian_Hacking said to me once.

  8. Steve Sailer says:

    “Only once have I been hired simply to be a yes-man as in the scenario described by Yost in the first quote above.”

    But, Prof. Gelman, you’re an honest man, so you don’t get hired when the “consulting” gig is just to apply some outside prestige to whatever the CEO wants to do anyway.

    • Andrew says:

      Steve:

      They could still do this. They just need a project where they’re pretty sure ahead of time that I’d say yes. Or they could just ask me and use my advice if it’s in accord with what they want anyway. I think the problem with the exit poll consortium is that there was poor communication: they never made it clear to me what they wanted. Or maybe they were divided internally about whether or why to hire me.

      • Jonathan (a different one) says:

        Exactly. Those who complain that experts are whores generally misunderstand the process. Your opinion on these matters is pretty well understood before they ever call you, and the initial meetings cement understandings, and occasionally lead to parting company. There is still the possibility, of course, that the data won’t support the opinion, but this is generally pretty unlikely; in my experience expert opinions differ on assumptions underlying competing methodologies, not the inferences from the data given the methodology.

  9. […] wrote this long comment that I think is worth posting: I [Bob] have done a fair bit of consulting for my small […]

  10. Logit says:

    I do market research consulting and the older I get I see this pattern here: managers in a particular industry and product area are often very experienced; they have some sense of the right decision to a marketing or business problem. But these managers are not 100% certain of their decision. And above all, they can’t take their uncertain decision to their CEO and say, “Well, in my experience this looks like a situation where we should do X.” These managers just don’t have enough “data” to back-up their instincts. Or, to put it in statistical sense, the “prior mean” of such managers may be correct but their “prior variance” is much too broad.

    Enter the consultant who then goes thrashing about for data on the problem and updates the manager’s prior with more information. Oftentimes we confirm what the manager expected. Again, the manager is often already smart without us. In the 20-20 vision of hindsight, such outcomes can and do look like rubber stamping and ass-covering. …But that’s in hindsight. And sometimes we do reverse a manager’s expectations or, even more interesting–we reveal heretofore unseen conditions necessary to make their decision the correct one.

    What I haven’t seen mentioned in the comments yet, however, is the value consulting brings well BEFORE an actual recommendation gets made. An unacknowledged value in this type of consulting is in the process of getting a manager to articulate his business question and transform it into researchable hypotheses. There’s a huge discipline imposed when one is forced to answer the following questions: 1) What is the risk in the decision I’m trying to make? 2) What do I need to know from consulting to help me make this decision? (and can that information even be accurately obtained?) 3) What are my research objectives and specific hypotheses? And finally, 4) Am I prepared to be surprised if the results contradict my hypotheses? Am I prepared to do something differently based on the consulting outcome? If the client can’t answer those questions, neither the client nor the consultant should pursue the gig.

  11. Guy says:

    As a current consultant (and Columbia student) I don’t like the dichotomy that consultants are either there as operation or strategy consultants. I’m paid to consult on study designs and interpret results. I say yes, I say no, I concede, I fight, and I also crunch numbers. If I restricted my work to either of the roles described, or even to just those two described, I would be out of work.

    • Andrew says:

      Guy:

      One difficulty is that different clients want different things. And clients don’t always know what they want ahead of times. And the “client” can be many different people with different goals. The clients who annoy me the most are the ones who say they will pay me and then don’t.

  12. […] What is expected of a consultant […]