Patenting Hinton’s ‘greedy training algorithm’ under US Law

Original article can be found here (source): Artificial Intelligence on Medium

Patenting Hinton’s ‘greedy training algorithm’ under US Law

In this post, we will explore whether Geoffrey Hinton’s ‘greedy training algorithm’, which was a huge breakthrough, would be given patent rights protection under US Law.

PART I: Hinton’s fast learning algorithm in nutshell.

PART II: Approach of U.S Law to Patentable subject matter.

PART III: How would U.S Courts decide on the patentability of Hinton’s algorithm?

PART IV: What would be the correct approach?

I first came across Geoffrey Hinton’s name while I was binge-watching on Youtube. There was this video about him called ‘ the guy who revolutionized A.I’. This video made an impression on me because he was an academic who believed and worked on neural networks while the investment in this field was drying and the interest in A.I was fading because of disappointment in AI’s capabilities. Yet, he kept working on these difficult problems despite everybody telling him that he was wasting his time.

Combining my fascination with his dedication with my patent law background, I decided to analyze if his ‘greedy training algorithm’ would be patentable under US Law. Patent Law exists to encourage innovations and reward innovators so that the humanity can move forward by solving its difficult problems. Therefore, I believe that rewarding the inventions of hard-working and dedicated inventors is crucial.

This post will consist of three parts: a) explaining Hinton’s algorithm, b) US Law’s position on patentable subject matter, c) applying the existing legal rules to the Hinton’s algorithm.

The main question this post will try to answer is: Would such a breakthrough algorithm be considered patent-worthy in US Law.

I. Hinton’s fast learning algorithm in nutshell, what did it achieve exactly?

Deep Belief Networks (DBNs) is a type of Artificial Neural Networks(ANNs) and the ANNs were trained with back-propagation algorithm which adjusts weights by propagating the output error backwards through the network. However, the problem with the ANNs was that as the depth was increased by adding more layers, the error ‘vanished to zero’ and this severely affected the overall performance.

In regard to the DBNs specifically, there were so many hidden layers that the back -propagation method was not able to perform well and the learning was very difficult. In DBNs, “Learning is difficult in densely connected, directed belief nets that have many hidden layers because it is difficult to infer the conditional distribution of the hidden activities”. This phenomenon is called “explaining away” because the inference process was further complicated.[1]

From the beginning of 2000s, there has been a resurgence in the field of ANNs owing to two major developments:

“increased processing power” and

The ground-breaking algorithm which enabled the further development of ANNs in general and DBNs in particular was Hinton’s “greedy training algorithm”.[2] It functioned by “training the layers sequentially and greedily by tying the weights of unlearned layers”.[3] Thanks to this new algorithm, DBNs has been applicable to solve a variety of problems such as “image processing natural language processing automatic speech recognition and feature extraction and reduction.”[4]

Thanks to this breakthrough, the recognition rates in tasks done by computers has gone up by almost %30 percent.[5]

As can be seen, the Hinton’s ‘fast learning algorithm’ revolutionized the field of machine learning because it made the learning easier and as a result, DBNs have been used to solve many problems across different fields such as image processing and speech recognition.

II. Approach of U.S Law to Patentable subject matter

a) Patent Law in nutshell

According to article 1(8) of US Constitution, the main purpose of giving exclusive rights to inventors is to promote progress of science and useful arts. Patent rights have the function of encouraging innovation and rewarding innovators.

Thanks to patent rights over an invention, the patent owner gains a huge competitive advantage and certain economic benefits because it gives the patent owner the exclusive rights to use, import and sell its invention and moreover, preventing others from reaping benefits from his/her invention.

Under US Law, for an invention to be patented, the invention must meet following conditions cumulatively:

-must be patentable subject matter,

In this post, I will be completely focusing on patentable subject matter criteria and not touching upon other criteria because being ‘patent eligible subject matter’ is the main obstacle before the granting of patent rights to innovations in A.I. field. To illustrate this, 84% of A.I. related patent application was rejected by U.S patent office based on patent-eligibility.[6] Therefore, other requirements will not be analyzed.

b) Patentable-subject matter-applying pre-emption test

When U.S courts analyze if an invention is worthy of patents, they start off with the so-called pre-emption test to decide if an invention is patentable subject matter.

Pre-emption test simply means that basic tools of technology, science and nature should be free for everyone to use so that everybody has the basic tools to innovate; resulting in more innovation overall and higher societal benefits. If courts decide that an applicant tries to patent such a basic tool, such as algebra formula or basic rules of engineering, the patent application is deemed to be unpatentable subject matter.

Imagine for example a scenario where a company comes up with the ‘binary code’ concept and have patent rights over it. In such scenario, all other people cannot use this binary-code system and they cannot innovate based on computers.

As you can see, the pre-emption test is solely about preventing an individual having monopoly over basic tolls of innovation.

Now, to better understand how pre-emption test is applied in US courts, let’s take a look at the McRO[7] case as our main guidance.

In this case, the inventor tried to patent an automated lip-synchronization of 3-D characters. Federal Circuit did not invalidate the patent by saying that it did not pre-empt use of all the rules for this automation process.

What does this reasoning tell us? It tells us that while applying the pre-emption test, so long as there are alternative ways to achieve the same technical result of patent claim, the pre-emption concerns are unlikely to arise.

Our question then becomes: If we reward Geoff Hinton’s algorithm with patent right, will this patent right monopolize a basic tool of technology, namely the base algorithm.

III. How would U.S Courts decide on the patentability of Hinton’s algorithm?

In a scenario where Hinton’s algorithm is patented and challenged before US courts, it will likely be invalidated considering McRO precedent.

In McRO case, the court reasoned that the algorithm should not be invalidated because the use of set of rules within the algorithm is not a ‘must’ and other methods can be developed and used. Hilton’s algorithm will inevitably pre-empt some A.I. developers from engaging with further development of DBNs technologies because this algorithm is a base algorithm which made the DBNs plausible to implement so that it may be considered as a ‘must’.

Hilton’s algorithm enabled the implementation of image recognition technologies and some may argue based on McRO case, that Hilton’s algorithm patent would be pre-empting because it is impossible to implement current image recognition technologies without using this algorithm. As a result, the Hilton’s patent will likely be invalidated for pre-empting other processes in consideration of McRO precedent.

IV. What would be the correct approach? Key takeaways

1. Not allowing the patenting of such algorithms which contributed a lot to A.I field would be an undesirable result because even if an algorithm is a must-use for a technology, there is no reason to exclude it from patent protection.

2. The fact that an algorithm is a must-use, should not lead to the conclusion that it cannot be patented. Patent rights are granted for processes which have “primary and even sole utility in research.”[8]

3. For example, a microscope is a “basic tool” for scientific work, but surely no one would assert that a new type of microscope lay beyond the scope of the patent system. Even if such a microscope is used widely and it is indispensable, it can still be given patent protection. Therefore, even if an algorithm is widely used and indispensable, it would be unjust to leave it outside patent protection.

3. A.I is a field where there is a huge potential for great economic benefits. In case an individual or a company invents a new technology which is highly profitable and they are not granted patent protection, they may be deprived of the chance to reap the benefits of their hard-work. This is an undesirable result because patent law should be rewarding our innovators and geniuses with exclusivity rights so that they can enjoy great benefits.

[2] Geoffrey E. Hinton (n. 97) 1527.

[6] Kate Gaudry and Samuel Hayim, ‘Artificial Intelligence Technologies Facing Heavy Scrutiny at the USPTO’ [IPWatchdog, 8.10.2018]

[7] McRO Inc. v. Bandai Namco Games Am. Inc. 837 F.3d 1299, 1308 [Fed. Cir.2016]

[8] Donald S Chisum, ‘The Patentability of Algorithms’ (1986) 47 U PITT L REV 959, 983.