From the proceedings of the December 4-6, 1962, fall joint computer conference, two researchers from General Electric Company’s Missile and Space Division write:

In general, there are two distinct modes of simulation; mathematical and physical. Mathematical simulation utilizes a mathematical model of the physical system under study. . . .

Physical simulation requires the excitation of the system under conditions which are representative of those encountered in actual system operation. This testing can involve anything from an inclined plane to large multi-million dollar ventures like the Space Environmental Simulator located at General Electric’s Valley Forge, Penna., Space Technology Center. These two types of simulation can be combined by mating physical hardware with a mathematical model. The general purpose computers available today are primarily designed for mathematical simulation. . . .

An electronic analog computer is an array of computational building blocks, or modules, each being able to perform a particular mathematical operation on an input voltage signal and provide a specific output response. These building blocks normally provide the functions of summation, integration with respect to time, multiplication by a constant, multiplication and division of variables, function generation, generation of trigonometric functions, and representation of system discontinuities. All quantities are represented on the analog by continuously varying voltages, restricted on almost all analog computers to the range between -100 and +100 volts. . . .

Data are fed into the analog computer in the form of parameter settings, which are usually associated with the coefficients that exist in the mathematical equations. Data are extracted from the computer in the form of voltages, either as steady-state values which can be read out on a voltmeter, or as varying values which can be recorded on a strip chart recorder or a plotting table. Some of the analog characteristics pertinent to our discussion are:

1. The analog is a parallel machine. All the variables are computed simultaneously and continuously. Thus, the speed with which the calculations are made is completely independent of the size or complexity of the problem.

2. The bigger a problem is, the more equipment is needed, as each piece of equipment works on one part of the problem.

3. Numbers on the analog are fixed point. Every variable must be scaled. The scaling will greatly affect the accuracy of the results.

4. The analog is best suited for solving systems of ordinary linear differential equations, although it can handle many other types of problem in a very satisfactory way.

5. There is no such thing as a computational cycle with the analog, because of characteristic No. 1. The analog can be set to calculate at any rate desired, but in practice there is an optimum time base associated with any particular problem, and attempts to run the problem much faster or slower will severely degrade the accuracy. The analog, generally speaking, is faster than the digital.

6. Analog outputs are almost always accurate to within 1%, but seldom better than 0.1%.

7. It is very easy, with most problems, to introduce extensive changes in the simulation in a matter of minutes.

Although the analog computer was designed primarily for the solution of problems in the aircraft field, its area of application has broadened considerably over the years. . . .

Many of these concerns still arise today, albeit in different form: scalability of computation (items 1 and 2), scalability of workflow (item 7), putting parameters on a natural scale (item 3), precision (item 6), and the idea that the method runs at some natural speed (item 5), which comes up with HMC and, before that, efficient Metropolis jumping rules.

They then move on to a discussion of digital computing:

The digital computer works by a counting technique and obeys logic rules exactly. The solutions are at discrete points dependent on the size of the time increment used. The smaller the mesh size, the more we approach the continuous solution. In contrast to the analog computer, which uses continuous variables in the form of voltages, the digital computer uses discrete variables, and operates with numbers as opposed to voltages. The digital computer is essentially a very fast calculating machine. . . .

There are a number of digital computer characteristics that are of particular interest in connection with hybrid simulation. These are:

1. It will deal only with numbers. Any problem must be reduced to a series of numerical operations before it can be handled by the computer. This is not to say that every step must actually be written each time. All sorts of aids to compiling programs are available. A program is nothing more than the entire sequence of instructions given to the computer to solve a problem. In actual practice, the machine itself will write most of its own instructions.

2. It will do exactly what it is told. All changes involve writing new instructions. The easier it is to make a change, the more complicated the original instructions have to be to include the option.

3. The results are exactly repeatable, but their accuracy is dependent on the numerical methods used to solve the problem.

4. The computer will perform only one operation at a time. That is, if the instruction reads, “Move number N from location A to location B,” the machine will, for a given period of time, be doing nothing but that.

5. The computer works with increments. None of the variables are calculated continuously. Generally speaking, the larger the calculation increment of the digital computer, the faster and the less accurate is the computation. There is absolutely no drift with a digital computer.

6. Compared with an analog, the digital is very much better equipped to make decisions. These can be made on the basis of comparison, time, reaching a point in the program, or almost any other criterion chosen by the programmer.

7. The digital can store very much more information than the analog. It can store tables, functions of several variables, whole programs, and many other things.

It is almost impossible to list the areas of application of the computer because of the diversity involved. We can say, however, that the digital computer lays sole claim to those problems which store a lot of information, use much logic, or require extreme accuracy. It will calculate trajectories, solve problems in astronomy, simulate mental processes such as learning and memory, analyze games, do translations, help design new computers, and do untold numbers of other tasks. The major effort to discover new computer applications is devoted to the digital area, with the analog a poor second, and the hybrid far behind.

They were right about that! Digital computers really did take over. Again, I find it interesting how much of the discussion turns on workflow, which we can roughly define as a process of exploration requiring science-like exploration by fitting multiple models.

They continue with some thoughts on the precision of computation which remain relevant over sixty years later:

The subject of accuracy is so complicated, and dependent on so many factors, that it just didn’t seem possible to summarize it by a mark in a box. While this is to some extent true of all the other characteristics listed, we believe considerations of accuracy fall into a special case.

On an analog computer, the result is usually within 0.1% and 1% of the value inherent in the equations. Whether this is excellent or poor depends on the nature of the problem. In many engineering investigations, this is much more precise than the data upon which the problem is based. The use to which the answer will be put also affects the accuracy required. Determination of the region of stability of a control system to within a millionth of the control range would be valueless, as the nature of the input could affect it much more than that. On a digital computer, the ultimate limit of accuracy is the number of bits in a word. This accuracy is seldom attained by the output variables of a problem, due to the approximations involved in almost any mathematical model, the idiosyncrasies of programming, and the practical necessity of taking reasonably large computing steps. The question concerning accuracy is more often, “How much cost and effort is needed to obtain the required accuracy?”, than “What accuracy is obtainable?” The answer has to be determined separately for each individual problem.

Next they move on to “hybrid” setups that combine analog and digital computing, sharing their own experiences:

The advantages of a hybrid that we felt to be of most value to the work of the department were in the area of increasing the size and variety of the problems we could solve. The things a hybrid can do to help in that endeavor are:

1. Assign different sections of a problem to each computer. For instance, in simulating a missile, the trajectory calculations can be assigned to the digital, because of the available precision, and the control simulation put on the analog because of its flexibility.

2. Assign different functions to each computer. For instance, all integrations might be assigned to the analog computer, in order to save time and get a continuous output. Or, all function generation might be assigned to the digital computer (where it is known as table look-up).

3. Provide analog plots of digital variables. This is particularly useful in observing the behavior of selected variables while the simulation is in progress. In one case, a stop was put on a 7090 after the first 15 seconds of what would otherwise have been a 10 minute run because it was easy to tell from the behavior of a continuous analog output that a key variable was not behaving quite as desired.

4. Let the digital provide logic for the analog. Things such as switching, scale changing, ending the program, choosing tables to examine, can be readily programmed into the digital and can greatly simplify and possibly even speed up an analog simulation.

5. Allow real hardware to be part of a simulation. Most hardware can readily be connected into the analog, and hybrid operation would allow it to connect to the digital just as easily. Similarly, digital devices can be included in analog operation the same way. Real hardware could also be considered to include people, as part of a control loop.

6. Provide accurate digital printouts of analog variables. Normally, the accuracy with which the analog variables are plotted is less than the accuracy that actually exists in the equipment. Hybrid operation enables selected variables to be converted to digital form and printed out from a digital tape.

The details of this sort hybrid computing don’t really matter anymore, but the general idea of looking at leaks in the modeling pipeline, that still is important.

I was also struck by the larger framework of simulation. Of course this makes sense: a missile test is expensive so you want to understand as much as you can using simulation before going out and launching something. In addition to being cost- and time-effective, simulation also makes the live test more effective. The real-world launch gives real-world data which you can compare to your expectations. The better your simulations, the better will be your expectations, and the more you will learn from discrepancies in the live data.

I’ve thought about these issues for awhile in the context of model checking and exploratory data analysis (see BDA starting from the first edition in 1995, and my 2003 article, A Bayesian formulation of exploratory data analysis and goodness-of-fit testing, but it was only just now that I realized the connection to workflow and simulated-data experimentation.

If only someone had given me this article to read 40 years ago, back when I was first doing simulations of physical systems. I blame the author of that 1962 article, who easily could have shared it with me at the time. The trouble was that he was too self-effacing.

P.S. The diagram at the top of this post comes from this 1963 article, “Corrected inputs: A method for improved hybrid simulation,” which begins:

Makes sense to me, to use some feedback to reduce transmission errors.

They were doing cool stuff back then, 60 years ago. Just regular guys, no Ph.D. or anything. Kinda like Steven Spielberg’s dad. Maybe that’s one reason I liked that movie so much.