Jouni writes,

Here is a link (see also here) to a patent on Bayesian linear regression. Yes, they call their algorithm an “invention.”

A simple yet powerful Bayesian model of linear regression is disclosed for methods and systems of machine learning. Unlike previous treatments that have either considered finding hyperparameters through maximum likelihood or have used a simple prior that makes the computation tractable but can lead to overfitting in high dimensions, the disclosed methods use a combination of linear algebra and numerical integration to work a full posterior over hyperparameters in a model with a prior that naturally avoids overfitting. The resulting algorithm is efficient enough to be practically useful. The approach can be viewed as a fully Bayesian version of the discriminative regularized least squares algorithm.

Now, hurry up and patent Bayesian nonlinear regression before they do it.

Jouni continues:

Maybe we all should be submitting our papers to the patent office instead of journals? Perhaps they would probably be more easily accepted?

It’s all fun and games until they sue your a$$. . . .

This is not a patent.

This is a provisional application for a patent, also known as a priority document. The applicants have (had) one year to make a more formal application that would include claims as well as an embodiment of the invention.

Moreover, even if they were claiming one tiny novelty in this algorithm they would be required to describe the entire preferred embodiment in order to support their claim.

I am not a lawyer, but here are some rules of thumb in reading patents:

a) a patent is only a patent after it issues

b) a patent only grants rights as described in the claims

c) the claims can only be interpreted precisely by reference to the litigation history

d) the description can (and must) describe many things that are well known as well as the new innovation.

e) the provisional application has to describe the invention well enough so that the ultimate claims are supported, but otherwise need have no direct connection with the claims.

You often make comments about people saying silly things outside of their field of expertise.

You just did that.

The mere

possibilitythat this might become a real patent is already patently absurd.Ted,

Thanks for the clarification.

In my defense, this is a blog entry, not a scientific publication, a magazine article, or an op-ed in a major newspaper. I'm more careful when writing for publication than when forwarding an amusing note for the blog.

Bell Labs beat you to the punch on the submission model. We concurrently sent papers to journals and patent lawyers. If the lawyers liked the paper, they'd put a provisional application in. Later, the diagrams would get renumbered and a patent would pop out the other end with a bonus check.

The patent lawyers explained to us researchers that they wanted us to create an "intellectual property minefield". That is, anyone stepping into the speech recognition area would likely get near one of our patents.

It might help to clarify what rights you get with a patent (Ted's point b). Specifically, patents do not grant any rights to embody the patent. They only let you stop other people from doing so. Suppose company A owns patent X (say X=linear regression). Then company B can patent a Y that depends on X (say Y=linear regression with priors). Company B's patent on Y does not allow company B to build and sell a Y. Rather, it allows company B to prevent company A from building a Y, or more likely, license patent Y to company A for royalties, or pay royalties to company A to license patent X.

In practice, the large companies tend to share patents. Luckily they don't often go after little guys like us until we're making enough money to make it worthwhile. We couldn't even afford to defend ourselves.

I'm afraid Microsoft beat you to the punch on non-linear models. Logistic regression anyway.

US Patent 7184929 – <a href="http://www.patentstorm.us/patents/7184929.html

" rel="nofollow">Exponential priors for maximum entropy models

US Patent 7266492 – Training machine learning by sequential conditional generalized iterative scaling

US Patent 7324927 – <a href="http://www.patentstorm.us/patents/7324927/claims.html

" rel="nofollow">Fast feature selection method and system for maximum entropy modeling

You could be annoyed by this (provisional) patent for two reasons:

–It doesn't strike us as particularly original

–It is a mathematical algorithm

Ted's points (b)–(e) only apply to the first concern, and he's probably right that after it goes through prosecution, it may be limited to something relatively innocuous.

But the odds are good that some sort of pure mathematical algorithm will pass examination (albeit with purely boilerplate wording to make it sound techy or tangible), so the second point certainly holds.

There are a lot of reasons to oppose patents on pure math—I wrote an entire book enumerating them. [_Math You Can't Use: Patents, Copyright, and Software_, Brookings Press, 2005] It's certainly something of a leap to say that a mathematical result can be treated as physical property, and that the first inventor(*) can be uniquely determined and has the right to sue other users of a theorem. I think most mathematicians bristle at such thoughts. However, it hasn't been a huge priority among mathematicians (e.g., the AMS has been keeping on the sidelines), partly because math patents still aren't as common as software patents.

So I think you're right to post this, and the faux pas about provisional applications doesn't change the fact that these sorts of applications exist, and patent attorneys are increasingly pressuring mathematicians to treat mathematical results as physical property. It's the sort of game where once one person is playing, everybody has to play, so even scattered examples may be very important.

(*) Inventor versus discoverer: U.S. courts have ruled that math is the discovery of a truth only when it is not useful. When it is useful it is an invention. ["State Street v Signature"; see section on the mathematical algorithm exception.]

Regarding Mr. Dunning's comment, there is a distinction between "fields of expertise" which impinge on my freedom of action, and those which do not.

It's one thing to butt into a mostly academic discussion without knowledge of the facts, as the objective is after all to discern the facts. It is another thing to be concerned by apparent abuses of a system which is primarily intended to restrict, through government intervention, the free exercise of ideas. Anyone has the right and obligation to protest an

implementedeconomic policy which seems to be doing harm, even while their theories may not be totally sound. Or, to paraphrase Pericles, "although the crafting of policy is restricted to the informed, everyone should criticize it." If only the former clause were true, we'd have less need of the latter.This patent application (yes, I read it) does not seem to cover a specific real-world application, having only auxiliary (and general) benefit, however clever or not it may be. Thus it seems to be "one of those" patents whose primary purpose is to opportunistically snipe at other people working on hard problems; either you'll only make a few "free" dollars off government subsidy, or even better you'll weaken a competitor!