David Fox writes:

As a ‘classically’ trained statistician who works on ‘real’ problems (mainly environmental ones) I have come to appreciate the utility and benefits of working within a Bayesian framework. I would not classify myself as a ‘convert’ but prefer to have an array of statistical tools from which I can select the most appropriate one for the job at hand. As they say – if all you’ve got is a hammer, then the whole world’s a nail! On the issue of choice of priors, I believe this is an absolute strength in the evaluation and setting of environmental regulatory limits. In situations characterized by high levels of data paucity but rich with expert knowledge (albeit diverse), why would you choose to ignore the latter?

However, I should get to the real purpose of this email. A rather fierce debate has been taking place among academics in our departments of Botany and Mathematics and Statistics about the use of a ‘new’ form of decision-making under extreme uncertainty. It is called Info-Gap (short for information gap) Theory and owes its existence to Prof. Yakov Ben-Haim at Technion in Israel (Ben-Haim 2006). Yakov is well known to the aforementioned academics – he visits here regularly and has done a remarkably good job at ‘selling’ his product – to the extent that some staff and students in our Botany department and The Australian Centre of Excellence in Risk Analysis (http://www.acera.unimelb.edu.au) have enthusiastically (and some would say, blindly) embraced this ‘new’ paradigm for decision-making under extreme uncertainty. I must plead mea culpa, having been swept up in the initial enthusiasm and published a couple of papers which use info-gap. However, I have a growing unease that IG is not ‘new’ but in fact a variant of existing methodologies.” While not wishing to draw you into our local debate, I was wondering if you have ever heard of info-gap theory and if you have, do you have an opinion? Prof. Ben-Haim has recently launched his own web site (http://www.info-gap.com) presumably in response to the ‘hi-jacking’ of the Wikepedia entry (http://en.wikipedia.org/wiki/Info-gap_decision_theory) by IG’s most strident local critic, Moshe Sniedovich. Sniedovich has also established a web site (http://info-gap.moshe-online.com/) and a quick look will demonstrate the ferocity of the debate.

Just today, the following paragraph in a paper I was reading [Hickey, G.L., Craig, P.S., and Hart, A. (2009) On the application of loss functions in determining assessment factors for ecological risk. Ecotoxicology and Environmental Safety, 72, 293-300] caught my attention:

“There do exist other forms of risk measurement. However, by a very well-known theorem of Wald (1950), any admissible decision rule is a Bayes rule with respect to some prior distribution (possibly an improper prior distribution), whereby admissibility is defined to mean that no other decision rule dominates it in terms of risk. It is therefore argued by many, for example, Bernardo and Smith (2000) that it is pointless to work in decision-theory outside the Bayesian framework”.

This accords with my own gut feeling that IG Theory is in fact a Bayes Rule with a non-informative prior.

My reply: I had never heard about Dr. Ben-Haim or his methods before receiving this email. I checked out the links but couldn’t really see the point in this approach. The mathematics looked complicated and appeared to be a distraction from the more important goals of modeling the decision problems directly.

For some of my thoughts on Bayesian decision analysis, see chapter 22 of Bayesian Data Analysis (second edition). Bayesian decision analysis is a lot more flexible than people realize, I think, especially when used in the context of hierarchical modeling. See here for a brief discussion of my idea of “institutional decision analysis” and here for an example of Bayesian decision analysis in action.

In my article on the boxer, the wrestler, and the coin flip, I discuss some fundamental difficulties with Bayesian robusness and similar approaches.

Finally, I don’t know that I’d agree with the statement that it’s “pointless” to work in non-Bayesian decision theory. For me, I’ve found the Bayesian approach to do the job, but I can imagine there are settings where other methods can be useful. I’m not, however, a fan of those 1950’s-style alternatives such as “minimax regret” and all the reat. I offer no comment on Info-Gap since I didn’t put in the effort to try to understand exactly what it is.

I'd never heard of info-gap theory either. Looking at the info-gap.com website, I read the "What is Info-Gap Theory" page, which has a general description, but couldn't find anything that revealed the hard details. It looks, however, like a theory based on ordinal preferences, which can sometimes tell you something (eg, if decision A produces less-preferred outcomes in all possible situations than decision B, then do B), but in general isn't going to be powerful enough.

I wonder whether the theory provides any checks? A great strength of probability theory is that you can be wrong. Eg, you can specify P(A), P(B|A), P(B), and P(A|B), and find that these are inconsistent. In contrast, systems like fuzzy logic (and maybe info-gap?) seem to just let you specify things in one way, providing no feedback as to whether you made a mistake.

There seem to be interesting sociological questions about how such theories come to be dominant in certain narrow fields…

Radford:

Good point about the advantage of the consistency requirement.

Regarding the sociological question, I have a theory, which I believe I mentioned in the rejoinder to my recent Bayesian Analysis article. The theory is that (a) there are a lot of ways to get a good solution to any particular statistical problem, and (b) people will often attribute the success to the method rather than to the analyst. The result is that, first, people in applied fields can become easily convinced of the efficacy of any particular method, if applied by a charismatic practitioner; and, conversely, said practitioner will become even more confident of the virtues of his or her method, once it is endorsed by practical researchers in applied fields.

"I was wondering if you have ever heard of info-gap theory" — No.

I dont know even what it is.

From the brief discussion of my [Andrew's] idea of "institutional decision analysis" — the "objective" or "institutional" Bayesian approach

Names are important, "objective" is a crummy name Jim Berger has argued we are stuck with, "institutional" may be less crummy but we are unlikely to be stuck with it yet.

Your central idea here seemed to be a forceful imperative to argue for and justify "to others" all the ingredients in a decision analysis and one could do this for a personal as opposed to an institutional decision – if one wished to.

An old MBA professor used the term "management by perception" to refer to an awareness of the reasons that should drive the decisions and the term perceptive Bayesian approach might be even a little bit less crummy …

Keith

Some clarifications concerning my Info-Gap campaign.

1. The fundamental flaws of Info-Gap decision theory are profoundly grave but at the same time, are easy to identify and describe ("prove") formally. This I have done at the end of 2006.

Hence, the objective of my campaign is not — as some would have it — to debunk Info-Gap decision theory. This relatively easy task has already been accomplished long ago.

Rather, the main objective of this campaign — and this has proved more difficult — is to dissuade adherents of Info-Gap from using/promoting what is an obviously profoundly flawed decision-making theory.

Since most users and promoters of Info-Gap decision theory in Australia (and elsewhere) are academics, this campaign is very much "academic" in nature. In other words, it concentrates on expounding why the use and promotion of Info-Gap decision theory is bad for science in general and for the science of decision-making in particular.

2. Info-Gap proponents do not refute my criticism. They can't because my criticism is rock-solid and formal in nature: I proved a number of theorems whose validity is indisputable and can be easily verified (almost by "inspection").

3. I did not "hi-jack" the WIKIPEDIA entry on Info-Gap. Rather, through formal analysis and effective illustrative examples I convinced WIKIPEDIA editors that my criticism is valid.

In subsequent comments I explain the major flaws in Info-Gap decision theory and its relationship to Bayesian decision theory.

I welcome you all to my website

where you can find information about Info-Gap decision theory and related topics such as:

Warning: I am a reformed Bayesian addict.

The two main flaws of Info-Gap decision theories are as follows:

The Maximin Theorem dispels the claim/myth that Info-Gap is a new theory that is radically different from all current theories for decision-making under uncertainty.

The Invariance Theorem dispels the claim/myth that Info-Gap decision theory seeks robust decisions under severe uncertainty.

Proofs of these theorems are available on line and in a number of articles on my website.

Bayes vs Wald

It should be noted that Bayes Rule and Wald's Maximin Rule do not compete with each other: rather, they complement each other.

In particular, the Maximin rule is

nota degenerate instance of Bayes rule. This is so because in the framework of Bayes rule thesameprobability distribution over the state space is used to evaluate each of the decisions. In contrast, in the framework of the Maximin rule each decision may have its very own worst case (state). Thus, it is easy to generate examples where no Bayes rule can simulate the Maximin rule.Since Info-Gap's robustness rule is a Maximin rule, the situation is the same. There are cases where no Bayesian rule can simulate Info-Gap's robustness rule.

See an illustrative example in the discussion on Bayes vs Wald on my website.

The moral of the story is that

you should not leave home without them (both), even if you are on your way to Australia to commence your best job in the world!