Let's Vote on It
Principal Skinner wonders if he’s out of touch. Lawyers occasionally wonder why engineers think software patents are worthless. But to both, it’s the children who are wrong.
Last week I wrote about section 112 of the patent statute and the need for source code in software patents. This is about the solution.
Until Alice v. CLS Bank was finally decided in the Supreme Court (2014), the great hope was that SCOTUS would put an end to software patents. That failed. I’m going to show you what’s happened since Alice.
I’m afraid the answer was in Aliens.
Ripley: I say we take off and nuke the entire site from orbit. It's the only way to be sure.
“Nuke it from orbit” in this context means an Act of Congress.
What’s Happened since Alice
Alice v. CLS Bank was granted certiorari at the Supreme Court in part because the Court was tired of the Court of Appeals for the Federal Circuit (CAFC) ignoring its clear skepticism on software patents. CAFC, the appellate court for patent cases before SCOTUS, is a prime example of regulatory capture: a judicial authority which serves its powerful clients rather than the public.
The Alice decision was a step in the right direction, but unfortunately it left a loophole, and the pro-patent people have crawled right through it.
As this doc explains, Alice laid out a two-prong test for determining the patentability of a software claim:
When applying the test to a claim, one must first determine whether the claim at hand is directed to a judicially-excluded law of nature, a natural phenomenon, or an abstract idea. If so, then one must further determine whether any element, or combination of elements, in the claim is sufficient to ensure that the claim amounts to significantly more than the judicial exception. [emphasis mine] Notably, generic computer implementation of an otherwise abstract process does not qualify as "significantly more."
“significantly more” — oops! Loophole there.
The Lawyers Run Through the Loophole
Here is an analysis from the Chicago-Kent Journal of Intellectual Property from 2016, affirming that software is still patentable, but you can’t just add “on a machine” and make it a hardware claim. That rule pretty much got rid of business method patents, which nearly everyone hated.
Now we’ll consider some cases post-Alice where software patents were (mostly) upheld by the CAFC or the PTO.
Note: for the last of these, the Facebook patent, I actually show you the back-and-forth with the PTO that led to this patent being issued. Compiling this was time-consuming, so I haven’t done it for all of them.
Enfish, LLC v. Microsoft Corp. (Fed. Cir. 2016)
Wikipedia’s take on it.
This is one of the first post-Alice decisions by the CAFC that upheld a software patent. Here’s the patent’s Abstract, which summarizes it fairly well in non-legalese:
The information management and database system of the present invention comprises a flexible, self-referential table that stores data. The table of the present invention may store any type of data, both structured and unstructured, and provides an interface to other application programs. The table of the present invention comprises a plurality of rows and columns. Each row has an object identification number (OID) and each column also has an OID. A row corresponds to a record and a column corresponds to a field such that the intersection of a row and a column comprises a cell that may contain data for a particular record related to a particular field, a cell may also point to another record. To enhance searching and to provide for synchronization between columns, columns are entered as rows in the table and the record corresponding to a column contains various information about the column. The table includes an index structure for extended queries.
It’s a database with one table, and no fixed schema like RDBMS’s use.
language guide: the word “plurality” is patent-ese for “two or more,” usually, and for “said” you can just read “the”. When it says “each said logical row” it’s just telling the examiner “this is the same object as the ‘logical row’ I just mentioned in ‘a plurality of logical rows’. ” “Said” is absolutely unnecessary; “the” is perfectly OK.
That having been said (pun intended), here’s the first claim:
1. A data storage and retrieval system for a computer memory, comprising:
means for configuring said memory according to a logical table, said logical table including:
a plurality of logical rows, each said logical row having an object identification number (OID) to identify each said logical row, each said logical row corresponding to a record of information;
a plurality of logical columns intersecting said plurality of logical rows to define a plurality of logical cells, each said logical column having an OID to identify each said logical column; and wherein
at least one of said logical rows has an OID equal to the OID of a corresponding one of said logical columns, and at least one of said logical rows includes logical column information defining each of said logical columns.
Let’s ignore for the moment whether anyone’s created that fairly obvious data structure before. Does this sound like a software claim to you? The District Court thought so:
Given these observations, the Court determines that the claims are addressed to the abstract purpose of storing, organizing, and retrieving memory in a logical table. This abstract purpose does not become tangible because it is necessarily limited to the technological environment of computers. . . . When a claim recites a computer generically, the Court should ignore this element in defining the claim's purpose.
In other words, the District Court correctly said that just because you add the phrase “on a computer,” that doesn’t make it a hardware claim.
However, the CAFC disagreed. It’s worth quoting a part of the doc I referenced:
the Court found that "describing the claims at such a high level of abstraction and untethered from the language of the claims all but ensures that the exceptions to § 101 swallow the rule." The Court further observed that, based on the District Court's construction, claim 17 recited a specific implementation of a self-referential table rather than the storing of tabular data in general. Thus, "the district court oversimplified the self-referential component of the claims and downplayed the invention's benefits."
Whatever that means. They upheld a patent on a data structure.
DDR Holdings v. Hotels.com
This is about patent 7,818,399 and its related patents. Oversimplifying, this is a patent on having links to other sites on your web page but pre-crawling them, so that you can “show” them without losing control and letting the user actually go there.
1. An e-commerce outsourcing process comprising:
a) capturing a look and feel description associated with a host website and storing HTML code corresponding to the look and feel description at a second website;
b) providing the host website with a link for inclusion within a page on the host website for serving to a visitor computer, wherein the provided link correlates the host website with a selected commerce object; and
c) upon receiving an activation of the provided link from the visitor computer, serving to the visitor computer from the second website page with a look and feel corresponding to the captured look and feel description of the host website associated with the provided link and with content based on the commerce object associated with the provided link;
A panel of the CAFC ruled that this patent was valid under Alice. You can read the entire analysis in the Wikipedia article. Here is an excerpt:
the court concludes that, even stipulating any of the characterizations of the alleged abstract idea put forth by defense counsel and the dissent, the '399 claims still contain an inventive concept sufficient to render them patent-eligible under step two of the Alice analysis.
In re TLI Communications LLC Patent Litigation (Fed. Cir. 2016)
I think I had this case when I was in Google Patent Litigation! I don’t remember if it got settled or what. Anyhow, here is the Patent Docs summary of it.
1. A communication system for recording and administering digital images, comprising:
at least one telephone unit including:
a telephone portion for making telephone call,
a digital pick up unit for recording images,
a memory for storing digital images recorded by the digital image pick up unit,
means for allocating classification information prescribed by a user of said at least one telephone unit to characterize digital images obtained by said digital pick up unit,
a processor for processing the digital images recorded by the digital image pick up unit;
a server including the following components:
a receiving unit for receiving data sent from said at least one telephone unit,
an analysis unit for analyzing the data received by the receiving unit from the telephone unit,
the data including classification information to characterize the digital images,
a memory in which at least the digital images are archived, the archiving taken into consideration the classifying information; and
a transmission system coupled to said at least one telephone unit and to the said server to provide for transmission of data from said at least one telephone unit and to the said server, the data including at least the digital images recorded by the digital image pick up unit and classification information.
Essentially, they had a patent on a photo server. CAFC ruled that this patent was invalid. Some sanity on this one.
The Facebook Patent
This is the patent I wrote about last week. It has not been the subject of litigation, as far as I know, but the PTO did allow it, and that was after Alice. It provides a way of choosing a “mask” for your photo, based on the emotions displayed in your friends’ photos.
Maybe you wonder sometimes why your patent takes four or five years to finally get issued. In the Appendix, I show you what happened with this one.
Engineers Hate Software Patents
In my 20 or so years dealing with patents, nearly every engineer I’ve ever met hates software patents. For a recent example, check the reactions on Hacker News to last week’s article. Yet these are the “persons of ordinary skill in the art, or POSITA’s” (maybe some are more than ordinary) who are the purported clients of a patent! A patent is supposed to teach a POSITA how to make and practice the invention. I’ve literally never met a software engineer who reads patents to learn anything. Check out this article on Techdirt. Does the opinion of the entire profession count for nothing?
I also know a few people in the biotechnology field, as well as electrical engineers. They are not nearly as unanimous in their condemnation of patents. That’s because the patent system evolved to handle things, not abstractions. Those POSITA’s produce things.
I often cite this three-step set of responses from anyone who wants to dismiss unpleasant news:
It’s not true
It’s true, but it’s not important
It’s true and it’s important, but we already knew it
Now look at what a patent lawyer on Reddit said when I brought this up:
I can’t speak to what the software folks feel because I haven’t asked any. What I will say is that I’m inclined to believe most POSITA’s in a given technical field are pretty clueless about what’s patentable in their field and what isn’t. So I think there’s bound to be a disconnect between the technical community and the patent-procuring community.
I think we’d call that #1: “it’s not true”. Possibly #2 as well. Maybe he’s ready to fall back on #3. We could call this the Principal Skinner effect: all of you are wrong. “Even if there isn’t a single one of you who believes these things are worthwhile, we know better and we’re keeping them.”
We need to definitively remove #1 as an answer: so that you know it’s true that they hate software patents. But is that important? That’s for Congress to decide.
We can help it do that. But we can’t repeat the mistake of waiting years for one more court case to wind its way up to the Supreme Court, like Alice v. CLS Bank. Congress can make laws. Let’s make it do its job. Let’s make them vote.
How Engineers Would “Vote”
What would it actually mean for software engineers to vote? Would it be like the Brexit vote, or something?
No, people in the UK who voted on Brexit were told that it would be binding. There is no precedent for a binding national referendum in the United States. Maybe there should be, but that is a very, very heavy lift.
What’s the Point?
Your obvious question is, “Why do this? We know what the answer’s going to be!”
I’d call this “the #3 dismissive response” : “It’s true and it’s important, but we already knew it.”
There is a difference between “everyone already knew it” and “here’s the proof.” If you’re a politician, it’s a lot easier to ignore the first one. Elected officials live and die by polls and elections, plus campaign contributions.
Polls
My suggestion is:
Only a US-resident person who would have been classified as a POSITA in any software patent-related litigation is eligible. No lawyers, no executives, no investors, no “stakeholders” of any kind. There could be a poll for them as well, but POSITA’s need their own.
The litigation I’ve seen defines a POSITA as someone with a Bachelor’s degree in Computer Science or equivalent, plus two years of industry experience. Some require more education or experience; some less.
Why “US-resident”? Primarily because the purpose of the poll is to inform Congress, and a poll that includes foreigners is less likely to do that. It is true that non-citizens can apply for patents.
We would have a poll of a representative sample of eligible voters. There are organizations who do this sort of thing every day. They would expect to be paid. We’d run a GoFundMe or other public contribution fund, so that there’d be no question of a vested interest funding the poll to get the results they want (not that anyone ever does that).
Who Gets Polled?
Clearly you have to avoid sampling bias, so we can’t just have a Twitter poll. That’s why part of the cost of the poll is identifying the sample to be polled. You cannot volunteer for the poll.
Who Designs the Poll Question?
The poll question should be along the lines of “should Congress outlaw software patents?” In a California initiative, at least, the actual law is quoted in the Voter’s Guide. I’m not a lawyer, much less a legislative draftsman, so I won’t attempt it here.
This: “2106.04 Eligibility Step 2A: Whether a Claim is Directed to a Judicial Exception [R-07.2022]” section of the Manual of Patent Examining Procedure cites the “judicially recognized exceptions” to section 101:
claims directed to nothing more than abstract ideas (such as a mathematical formula or equation), natural phenomena, and laws of nature are not eligible for patent protection.
These are judicially created rather than being written into the statute. In any case, we want to add “computer software” to this list, with no exceptions. I’ll leave the wording to the lawyers.
We all know that you can ask the same question two different ways and get two different results. The question, “Should Ukraine be defended from Russia’s aggression?” will get different responses from, “Should the United States send your tax dollars to Ukraine?”
Therefore, the poll should be designed by a committee where the pro-patent side (The American Intellectual Property Law Association maybe?) is fairly represented. The anti-patent side might be The Electronic Frontier Foundation. These are up for debate.
What About Other Disciplines?
I’m not as familiar with the professional associations for POSITA’s in chemistry, biology, medical devices, etc., but it would be advantageous to run a similar poll in those disciplines as well. The hypothesis is that software people are much more anti-patent than practitioners in the more traditional disciplines, but it would be good to check if that’s true. Recall that the lawyer on Reddit suggested it’s the same everywhere. I think he’s wrong.
Appendix: The Facebook Patent
All of this happened after the Alice decision, so clearly that did not abolish all software patents.
Warning: This can get mind-numbingly boring. I’m going to do my best to distill it down to a human-readable discussion, and if that simplifies it too much for some lawyers, sorry. The original documents are all public. I’m going to concentrate on Claim 1, which gives us more than enough to discuss.
If you find this intolerably tedious, you can skip down to “The Post-Game Analysis”
The First Iteration
(this is 91230_16809868_2020-03-05_CLM)
The first version of Claim 1 was :
1. A method comprising, by one or more computing devices:
identifying a first user in an input image;
accessing social data of the first user in the input image, wherein the social data comprises information from a social graph of an online social network;
selecting, based on the social data of the first user, a mask from a plurality of masks, wherein the plurality of masks comprise masks previously selected by friends of the first user within the online social network, and wherein the mask specifies one or more mask effects; and
applying the mask to the first user in the input image.
(The PDF file I’m drawing from is images, so I can’t copy & paste. I ran them through an online OCR service, and then formatted it to look like the images. If there are errors, please report them in the Comments and I’ll fix them.)
A “mask effect” might be, the patent tells us, “larger eyes, a hat, a face painting, or the like, to be shown on or near the object from the image or video.”
The First Rejection
Generally, the examiner takes each element of the claim (the part between semi-colons), finds some reference from before the filing date, and says “it would have been obvious to a POSITA to combine these.” That’s what happened here.
(This is 91230_16809868_2020-09-25_CTNF).
3. Claims 1, 3, 4 and 7-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tarquini et al., U.S. Patent Publication Number 2015/0070351 Al, in view of Arrasvuori et al., U.S. Patent Publication Number 2011/0252320 Al, further in view of Liba, Face and Photograph Augmentation Based on a Custom Theme, pages 1-6.
This is followed by 4 pages of explanation on how the three references together disclose everything in Claim 1. You can look up those references if you’re masochistic enough: just go to patents.google.com and enter “US 15/0070351” for Targuini, etc.
Note here that the derisive comment about patent examiners, that “they only know what’s been patented” is wrong. The Liba reference is not a patent.
The Second Iteration
Sometimes an attorney, when confronted with an Obviousness rejection, will counter with boilerplate language, e.g. “There is no teaching, suggestion, or motivation to combine Tarquini with Arrasvuori and Liba in the manner described.” Meaning: there was no reason a POSITA would have done this.
However, in this case they amended the claim to add some more conditions:
1. (Currently Amended) A method comprising, by one or more computing devices:
identifying a first user in an input image;
identifying a facial expression of the first user in the input image;
accessing social data of the first user in the input image, wherein the social data comprises information from a social graph of an online social network;
determining a theme associated with masks previously used by the user;
selecting a mask from a plurality of masks, wherein the plurality of masks comprise masks indicated by the social data as being previously selected by friends of the first user within the online social network, and wherein selecting a mask from the plurality of masks is based on the social data of the first user and the determined theme associated with the masks previously used by the user and the identified facial expression of the first user in the input image, wherein the mask specifies one or more mask effects; and
applying the selected mask to the first user in the input image.
(The parts that in bold are the additions (Substack doesn’t have underlining, but in the image they’re underlined).
So they added language about how they figure out what facial expression the first user has, and use that to determine the mask he should have. The hope is that the examiner won’t be able to find any references that do that.
The Second Rejection
However, the examiner did manage to find all those words, in a Final rejection. “Final” does not really mean final; it means you have to pay again.
(91230_16809868_2021-02-08_CTFR)
Claims 1, 2, 4-14 and 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Konoplev, U.S. Patent Publication Number 10,062,216 B2, in view of Isberg et al., U.S. Patent Publication Number 2016/0048943 Al, in view of Liba, 0.. "Face and Photograph Augmentation Based on a Custom Theme EE 368 Project, Autumn 2015." (2015) pages 1- 6, in view of Tang et al., U.S. Patent Publication Number 2016/0092551 Al, further in view of Imoto et al., U.C. Patent Publication Number 2018/0182145 Al.
The Third Iteration
You get “two bites of the apple” in a patent prosecution. After that, you have to pay again for a Request for Continued Examination. So it wasn’t really Final. But they did have a phone interview, which failed to lead to an allowance:
STATEMENT OF SUBSTANCE OF INTERVIEW
Matthew Avery and Katherine Burgess had a telephonic interview on 27 October 2021 with the Examiner regarding the rejections to the claims. No agreement was reached during the interview. The general thrust of the Applicant's argument was that the proposed Konoplev-Isberg-Liba-Tang combination fails to disclose, teach, or suggest each and every limitation of independent Claim 1. The Examiner suggested that Applicant amend the claims to differentiate independent Claim 1 from the proposed Konoplev-Isberg-Liba-Tang combination.
In Remarks later, they argued that the “facial expression” as opposed to a “linguistic expression” was what was unique and patent-worthy:
As an example, the proposed Konoplev-Isberg-Liba-Tang combination fails to disclose, teach, or suggest identifying a facial expression of the first user in the input image, as independent Claim 1 recites. As discussed above, Isberg merely discloses processing and modifying digital photographs, such as by applying photographic filters inclusive of changes to color, lighting, and composition. (Isberg at Para. 0045.) Isberg further discloses processing communications to determine a "linguistic expression in the communication," but does not teach or suggest identifying a facial expression in an image. (Isberg at Paras. 0005-06.) Konoplev, , Liba, and Tang do not make up for the deficiencies of Isberg, either alone or in combination, and the Examiner does not assert otherwise.
This is what patent prosecution is: finding some word that does not appear in the prior art. You would think the examiner would say “Who cares if it’s ‘facial’ or ‘linguistic’? It’s obvious, case closed” but that rarely happens.
Their new claim was:
1. A method comprising, by one or more computing devices:
identifying a first user in an input image;
accessing social data of the first user in the input image, wherein the social data comprises information from a social graph of an online social network;
determining a theme associated with masks previously used by the user;
selecting a mask from a plurality of masks, wherein the plurality of masks comprise masks indicated by the social data as being previously selected by friends of the first user within the online social network, and wherein selecting a mask from the plurality of masks is based on the social data of the first user and the determined theme associated with the masks previously used by the user, wherein the mask specifies one or more mask effects; and
applying the selected mask to the first user in the input image.
There are quite a few more words there. Now “the theme associated with the masks previously used by the user” is one of the bases of the mask selection, and the mask “specifies one or more mask effects.”
The Third Rejection
The examiner still didn’t buy it:
3. Claims 1, 2, 4-14 and 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Konoplev, U.S. Patent Publication Number 10,062,216 B2, in view of Isberg et al., U.S. Patent Publication Number 2016/0048943 Al, in view of Liba, 0.. "Face and Photograph Augmentation Based on a Custom Theme EE 368 Project, Autumn 2015." (2015) pages 1- 6, in view of Tang et al., U.S. Patent Publication Number 2016/0092551 Al, further in view of Imoto et al., U.C. Patent Publication Number 2018/0182145 Al.
The Fourth Iteration
So they added another element, where you have to look at the facial expression of the first user, and use that to determine the mask:
1. A method comprising, by one or more computing devices:
identifying a first user in an input image;
accessing social data of the first user in the input image, wherein the social data comprises information from a social graph of an online social network;
determining a theme associated with masks previously used by the user;
selecting a mask from a plurality of masks, wherein the plurality of masks comprise masks indicated by the social data as being previously selected by friends of the first user within the online social network, and wherein selecting a mask from the plurality of masks is based on the social data of the first user and the determined theme associated with the masks previously used by the user, wherein the mask specifies one or more mask effects; and
applying the selected mask to the first user in the input image.
The Fourth Rejection
This is the second Final Rejection:
3. Claims 1, 2, 4-14 and 16-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Konoplev, U.S. Patent Publication Number 10,062,216 B2, in view of Isberg et al., U.S. Patent Publication Number 2016/0048943 Al, in view of Liba, 0.. "Face and Photograph Augmentation Based on a Custom Theme EE 368 Project, Autumn 2015." (2015) pages 1-6, in view of Tang et al., U.S. Patent Publication Number 2016/0092551 Al, further in view of Imoto et al., U.C. Patent Publication Number 2018/0182145 Al.
The Fifth Iteration
We know now that “Final” doesn’t really mean final, so they bought another two bites of the apple. Now they’ve added “identifying an emotion” to the identification of the first user’s facial expression, and that emotion is used to look up the mask (the bold parts are the changes):
1. (Currently Amended) A method comprising, by one or more computing devices:
identifying a first user in an input image;
identifying a facial expression of the first user in the input image, wherein identifying a facial expression comprises identifying an emotion;
accessing social data of the first user in the input image, wherein the social data comprises information from a social graph of an online social network;
determining a theme associated with masks previously used by the user;
selecting a mask from a plurality of masks, wherein the plurality of masks comprise masks stored in a database and corresponding to a mapping table, the mapping table associating the identified emotion with one or more masks of the plurality of masks, the plurality of masks being indicated by the social data as being previously selected by friends of the first user within the online social network, and wherein selecting a mask from the plurality of masks is based on the social data of the first user and the determined theme associated with the masks previously used by the user and the identified
facial expressionemotion of the first user in the input image, wherein the mask specifies one or more mask effects; andapplying the selected mask to the first user in the input image.
And that got past the examiner, and the patent was issued! They finally (no pun intended) added enough words to distinguish it over the prior art.
The Post-Game Analysis
When you file a patent and it seems like nothing is happening for four or five years, chances are this is what’s happening. The lawyers don’t tell you about it, because you probably (a) wouldn’t understand it, and (b) would be appalled.
The game during patent prosecution (those five iterations we just covered) is often just adding words to your claims until the examiner can’t find prior art with those words, or else cancelling the claims, or writing new ones.
So What Does It All Mean?
If you stayed with me for all that (or even if you didn’t), and you’re an engineer, you probably think this is all ridiculous. A lawyer would smile indulgently and patiently explain how you just don’t understand the power and majesty of the law. Not me. I think you’re right; it is ridiculous.
In my SSRN article, I laid out a rationale within existing law where this could all be rejected. In a nutshell, the examiner would say that this whole “invention” is Obvious to Try; it’s a tool in the programmer’s toolbox to try to find a mask matching the expression on the user’s face. It’s a social network, after all; that’s what you’d expect it to do. I will be plumbing the depths of the legal reasoning governing that in another article.
It turns out that Obvious to Try does not usually mean, in legal circles, what I thought it meant; it’s more for chemistry, where there are a fixed number of techniques or chemicals that a scientist could try, and the question would be “what is the probability of success?”
Ripley had the right idea.