Monte Carlo is the ubiquitous little beast of burden in Bayesian statistics. Val points to the article by Nick Metropolis “The Beginning of the Monte Carlo Method.” Los Alamos Science, No. 15, p. 125, 1987 about his years at Los Alamos (1943-1999) with Stanislaw Ulam, Dick Feynman, Enrico Fermi and others. Some excerpts:
Among the attendees was Stan Ulam, who had rejoined the Laboratory after a brief time on the mathematics faculty at the University of Southern California. Ulam’s personality would stand out in any community, even where “characters” abounded. His was an informal nature; he would drop in casually, without the usual amenities. He preferred to chat, more or less at leisure, rather than to dissertate. Topics would range over mathematics, physics, world events, local news, games of chance, quotes from the classics—all treated somewhat episodically but always with a meaningful point. His was a mind ready to provide a critical link.
During his wartime stint at the Laboratory, Stan had become aware of the electromechanical computers used for implosion studies, so he was duly impressed, along with many other scientists, by the speed and versatility of the ENIAC. In addition, however, Stan’s extensive mathematical background made him aware that statistical sampling techniques had fallen into desuetude because of the length and tediousness of the calculations. But with this miraculous development of the ENIAC—along with the applications Stan must have been pondering—it occurred to him that statistical techniques should be resuscitated, and he discussed this idea with von Neumann. Thus was triggered the spark that led to the Monte Carlo method.
The spirit of this method was consistent with Stan’s interest in random processes—from the simple to the sublime. He relaxed playing solitaire; he was stimulated by playing poker; he would cite the times he drove into a filled parking lot at the same moment someone was accommodatingly leaving. More seriously, he created the concept of “lucky numbers,” whose distribution was much like that of prime numbers; he was intrigued by the theory of branching processes and contributed much to its development, including its application during the war to neutron multiplication in fission devices. For a long time his collection of research interests included pattern development in two-dimensional games played according to very simple rules. Such work has lately emerged as a cottage industry known as cellular automata.
John von Neumann saw the relevance of Ulam’s suggestion and, on March 11, 1947, sent a handwritten letter to Robert Richtmyer, the Theoretical Division leader (see “Stan Ulam, John von Neumann, and the Monte Carlo Method”). His letter included a detailed outline of a possible statistical approach to solving the problem of neutron diffusion in fissionable material.
Johnny’s interest in the method was contagious and inspiring. His seemingly relaxed attitude belied an intense interest and a well-disguised impatient drive. His talents were so obvious and his cooperative spirit so stimulating that he garnered the interest of many of us. It was at that time that I suggested an obvious name for the statistical method—a suggestion not unrelated to the fact that Stan had an uncle who would borrow money from relatives because he “just had to go to Monte Carlo.” The name seems to have endured.
[….]
Enrico Fermi helped create modern physics. Here, we focus on his interest in neutron diffusion during those exciting times in Rome in the early thirties. According to Emilio Segre, Fermi’s student and collaborator, “Fermi had invented, but of course not named, the present Monte Carlo method when he was studying the moderation of neutrons in Rome. He did not publish anything on the subject, but he used the method to solve many problems with whatever calculating facilities he had, chiefly a small mechanical adding machine.”
In a recent conversation with Segre, I learned that Fermi took great delight in astonishing his Roman colleagues with his remarkably accurate, “too-good-to-believe” predictions of experimental results. After indulging himself, he revealed that his “guesses” were really derived from the statistical sampling techniques that he used to calculate with whenever insomnia struck in the wee morning hours! And so it was that nearly fifteen years earlier, Fermi had independently developed the Monte Carlo method.