USPTO seems to be of two minds about PDF layers and it wastes everybody’s time

When Adobe started to create the Acrobat PDF standard in 1992, it began as a fairly simple standard.  As the years passed, the PDF standard grew and grew, so that all manner of weird and poorly understood things could be tucked away inside a PDF file.  By now in 2021, a PDF file might contain multimedia content.  It might have a hyperlink to launch a web browser or email client or a VOIP telephone calling client.  It might be “locked” with a password to prevent editing.  It might contain metadata indicating who created or edited the PDF, or the type of software that generated the PDF.  I used to joke that Adobe had a defined datatype for embedding scents in PDF files and then was astonished to learn that this had actually been proposed some years ago in some PDF working group.  I gather that scent embedding in PDFs did not actually get implemented.

In the world of USPTO patent e-filing, the chief way that the inner workings of the PDF standard become challenging is that the programmers at the USPTO profess to be unable to handle certain kinds of “fonts”.  (In a separate blog article I will elaborate on this “font” challenge which is a completely artificial challenge that exists only because the USPTO programmers cause it to exist.)

The other way that the inner workings of the PDF standard are challenging in the world of USPTO patent e-filing is that the programmers at the USPTO (along with their colleagues who draft policy documents and their colleagues who carry out day-to-day work) cannot quite make up their minds about whether “layers” in a PDF file are a problem or not.  And that is the focus of this blog article.  This lack of clarity in the minds of USPTO people leads to wasted time for people at the USPTO and wasted time for patent practitioners and applicants, as I will describe.

The Adobe people defined something called “layers” that might or might not exist within a particular PDF file.  In a simple PDF file there are no layers.  Or, you might say, there is only one layer, in which case there is no reason to bother to call it a layer.  But it is possible for a PDF file to have “layers”.  If it does have “layers”, then each layer can be marked as being “visible” or “invisible” at any particular moment.  When you view the PDF file on a computer screen, the layers that are marked to be visible will show up on the screen, and the layers that are marked to be invisible will not show up on the screen.  Depending on what kind of PDF editing software you happen to be using, you might find that you are able to change the visibility settings of the layers.  For example you might be able to change some layer to be visible that was previously invisible.  If so, then that layer would become visible on your computer screen.  Later if you like you could make the layer invisible again and it would cease to be visible on your screen.

There is a thing called “flattening” that it turns out everybody knows how to do.  By this I mean that all computer programmers who work with PDF files know how to do this thing called “flattening”, using free open-source software tools.  One part of “flattening” is that if there are any layers that are marked as “invisible”, they get discarded.  If after this discarding process there is only one layer left, then that is the end of the process.  That layer is the PDF file.  What you see when you look at that layer is what you see when you are looking at the PDF file.  

But if, after discarding any “invisible” layers, there are still two or more “visible” layers, then there is a bit more work to do in the “flattening” process.  What remains to be done is that all of the visual information from all of the visible layers is combined into a single image.  A good way to think of this is as if each of the “visible” layers were printed onto clear plastic, and then the sheets of clear plastic could be laid one on top of the next, lined up correctly.  Then you could snap a picture of what you get when the plastic sheets are stacked up, or you could put them onto the glass of a photocopier to make a photocopy.  The resulting image would be what we call a “flattened” image.  

The combining of the visible layers might sound a bit complicated when I describe it in the 134 words of the previous paragraph, but from a software point of view it is quite straightforward and unambiguous.  As I say, all of the computer programmers who do this soft of stuff for a living know exactly how to do it, using freely available software tools that everybody knows how to use.  The outcome of the flattening process is not the subject of any uncertainty.   If you give this task to any two (or any ten) computer programmers, they will all get exactly the same answer — exactly the same result.

What I just said in the previous paragraph is the important part about layers and flattening, as it relates to patent e-filing at the USPTO.  If a filer files a PDF that happens to contain layers, this absolutely does not introduce any ambiguity or uncertainty into the e-filing process.  Assuming for sake of discussion that the USPTO were to choose to do the “flattening” then the outcome of the flattening is what it is.  It is not the case, for example, that there are two or three or twenty possible outcomes to the flattening process and thus that the USPTO might profess not to know which outcome was the outcome intended by the filer.  

There must be at least one person at the USPTO who understands this stuff about layers and flattening.  The way that we know this is that there is a document on the USPTO web site called PDF Guidelines.  The document has a paragraph that says this:

Layers

PDF documents with multiple layers must be flattened prior to submission to ensure that the complete document is available to the examiner. If a document contains layers that are marked as “invisible”, the invisible layers will be lost when the received document is processed within the USPTO. Documents submitted with multiple layers will be flattened by the USPTO when converted to an image.

The three sentences of this paragraph don’t actually fit together.  It is necessary to take the sentences one by one to try to make sense of them.  The best place to start is the third sentence:

Documents submitted with multiple layers will be flattened by the USPTO when converted to an image.

This sentence has the advantage of actually being a true statement by the USPTO.  The actual observed behavior of the USPTO systems is that both EFS-Web and Patentcenter do actually carry out “flattening” on each and every PDF file that happens to have been e-filed that happens to contain two or more layers.  Another way to say this is that the USPTO programmers who programmed EFS-Web did somehow educate themselves about the well-known software tools that are available for use in carrying out the work of flattening a multi-layered PDF file into a single-layer image file, and they did make use of one of those tools.  What ends up in IFW is indeed a flattened single-layer image PDF file.  (And, if the filer went to the trouble to index the PDF as a non-black-and-white drawing, the original multilayer PDF will have gotten preserved in SCORE.)  

Likewise, the USPTO programmers who programmed Patentcenter did also somehow educate themselves about the well-known software tools that are available for use in carrying out the work of flattening a multi-layered PDF file into a single-layer image file, and they did make use of one of those tools.   Again if the filer in Patentcenter happens to upload a multi-layer PDF, the Patentcenter software will load into IFW a flattened single-layer image PDF file.  (And again if the filer went to the trouble to index the PDF as a non-black-and-white drawing, the original multilayer PDF will have gotten preserved in SCORE.)

As we try to make sense of the paragraph from the PDF Guidelines, the best place to go next is the second sentence.  It says:

If a document contains layers that are marked as “invisible”, the invisible layers will be lost when the received document is processed within the USPTO.

This sentence also has the advantage of actually being a true statement by the USPTO.  The actual observed behavior of the USPTO systems is that in the flattening process, the software in both EFS-Web and Patentcenter discards any layer that has been marked as “invisible”.  This is absolutely common sense.  No sensible programmer would ever do anything else when carrying out such flattening.  The standard off-the-shelf software tools for flattening of multilayer PDF files do this automatically and it is not up to the discretion of the programmer to do otherwise.  

So again, for emphasis, we should point out a couple of things.  First, the filer who happens to e-file a multilayer PDF, perhaps with one or more layers that are marked “invisible”, is not introducing any ambiguity or uncertainty into the e-filing process.  Anybody who carries out a “flattening” process will always get exactly the same outcome.  The outcome will be an image that is just like what you see on your screen if you were to view that PDF file with any PDF viewer.  The outcome will be an image that is just like what you see on the paper that comes out of the printer if you were to print that PDF file on any printer. 

So now there is no choice but to try as best we can to make sense of the first sentence in the paragraph.  It says:

PDF documents with multiple layers must be flattened prior to submission to ensure that the complete document is available to the examiner.

This sentence is flatly false.  Or it is nonsense.  No matter how one interprets this sentence, it is completely inconsistent with the two sentences that follow it.

One way to try to make sense of the sentence would be to rewrite it along these lines:

Some PDF documents have multiple layers.  Any PDF document with multiple layers needs to be flattened before certain internal processes take place within the USPTO, for example for review by the patent examiner or for use by the Publication Branch.  

When it is rewritten this way, it is simply explaining to the reader that there is such a thing as a multilayer PDF and that there is such a thing as flattening.  If you rewrite it this way, then when the reader encounters the next two sentences, the reader understands that the place where the flattening will happen is within the USPTO.

Another way to try to make sense of the sentence might be to rewrite it along these lines:

Some PDF documents have multiple layers.  Any PDF document with multiple layers needs to be flattened before certain internal processes take place within the USPTO, for example for review by the patent examiner or for use by the Publication Branch.  It costs the USPTO money in terms of electricity costs and processing time to do the flattening.  For this reason, the USPTO prefers that the filer do the flattening prior to submission.

Given that the USPTO actually does the flattening any time that the filer fails to do the flattening, it is hard to find any other way to give meaning to the part about “must be flattened prior to submission” other than a sort of gripe about the electricity cost of the USPTO having to do the flattening.  The real situation, however, is that for as many years as EFS-Web has been in operation, the USPTO has routinely carried out the flattening, and it has never actually been necessary for any filer to do such flattening.  I frankly cannot imagine that the first of the three sentences in that paragraph was ever actually talking about the electricity cost to the USPTO to do the flattening.

It is quite a stretch, but a third way to try to make sense of the first sentence might be something like this:

Some PDF documents have multiple layers.  Any PDF document with multiple layers needs to be flattened before certain internal processes take place within the USPTO, for example for review by the patent examiner or for use by the Publication Branch.  The filer can do the flattening, or the USPTO can do the flattening.  We at the USPTO don’t really understand flattening but we found a software tool that supposedly does flattening.  We don’t really know if there is some single unambiguous outcome of “flattening” or not.  If you leave it to us to do the flattening, please understand you are doing this at your own risk and if we end up with some flattened result that you decide you did not like, that is your problem.  We suggest you do your own flattening before submission, since then you will know exactly what the result of the flattening is.

I actually believe this is what the writer was trying to say.  In plain language, I think it is a sort of a warning like “don’t blame us if the flattening that you make us carry out leads to a result that you did not intend.”   If I am correct that this is what the writer was really trying to say, then two very sad things follow from this.   

The first sad thing that follows from this, if indeed this was what the writer was trying to say, is that the writer absolutely failed at his or her communications goal.  People who put words on the page do it because they are trying to communicate something, and if this is what the writer was trying to communicate, the writer simply failed.

The second sad thing that follows from this, if indeed this is what the writer was trying to say, is that the writer is needlessly creating a problem where no problem actually exists.  The real situation is that flattening is flattening.  It is not a sort of random thing that might have this outcome on a Monday and some other outcome on a Tuesday.  People who know what they are doing with PDF files know this.  Programmers who actually draw upon experience with PDF files know this.  What should have happened, back when these PDF Guidelines were being drafted, is that the people involved should have done whatever was needed to become familiar with how PDF files really work, and should have done whatever was needed to arrive at some level of confidence at what exactly “layers” mean, and what exactly it means for a layer to be “visible” or “invisible”, and what exactly it means to “flatten” a multilayer PDF file  The people involved should have educated themselves to the point of full familiarity with the various well-known software tools for PDF file manipulation, and in particular should have educated themselves to the point of appreciating that “flattening” always has a single unambiguous outcome.  

What should have happened then, after the USPTO people gained this familiarity with this aspect of how PDF files work, is that the USPTO should have made up their minds one way or the other about what they were going to do when a filer tried to upload a multilayer PDF file.  Here are examples of what might possibly have been acceptable ways to make up their minds.

Suck it up and quietly accept the multilayer PDFs from filers, and do the flattening, and don’t editorialize about it.  On this approach, the PDF Guidelines would not have contained this paragraph at all.  Or the PDF Guidelines could merely have mentioned in passing that if the filer e-files a multilayer PDF, the USPTO system will flatten it, discarding any invisible layers.  There would be no need to display any “warning” during the e-filing process.

Refuse to do the flattening.  Another approach would be for the USPTO to refuse to do the flattening.  On this approach, when the filer would try to upload a multilayer PDF file, the e-filing system would display an error (not a mere warning) and filing would not be permitted to continue.  

Why would USPTO select such an approach?  It is possible to imagine any of several reasons why USPTO might select such an approach.

Saving electricity.  Yes it costs some amount of electricity for USPTO’s computers to do the flattening.  So why not shift those costs to the filer?  Make the filer spend the money for the electricity to do the flattening.  I hate to even mention this idea since there is the danger the USPTO might be tempted to actually try this.

Avoiding blame or ambiguity.  If there were actually some uncertainty in what it means to “flatten” a multilayer PDF, then one could imagine a patent office taking the position that they are in a strange way actually doing the filer a favor by forcing the filer to do the flattening.  The idea is that by forcing the filer to do the flattening, then if the flattening were to yield some unexpected or unintended result in the PDF file, it would be the filer’s fault and not the USPTO’s fault.  The actual situation is that there is no uncertainty in the meaning of “flattening”, but if there were some uncertainty, then yes one could imagine this being a reason for a patent office to select this approach.

Avoiding having to learn pesky things that one ought to have to learn.  One could imagine a situation where the people involved within the patent office would prefer to avoid having to actually learn once and for all what “flattening” means exactly, and would prefer to avoid having to stick their necks out and take the position that “yeah I guess we can learn how to use the off-the-shelf flattening tools that everybody else knows how to use”.  If you offload the flattening burden to the filer, then you never actually have to figure out everything that would have to be figured out about how to do flattening.

What path did USPTO actually choose?

The path that the USPTO actually chose was not either of these paths.  The USPTO chose a very bad path.  The path the USPTO chose is that it does carry out the flattening.  But the USPTO also tells the filer in the PDF Guidelines that the filer “must” do the flattening, which is just not true.  Not only that, in the Patentcenter acknowledgment receipts, the USPTO displays an extremely poorly worded warning that parrots the three sentences from the PDF Guidelines.  The warning that appears in the Patentcenter ack receipts is:

Warning:  The PDF file contains multiple layers and must be flattened prior to submission.  Documents submitted with multiple layers will be flattened by the USPTO when converted to an image.  Layers marked as “invisible” will be lost during conversion.

There are several problems with this warning.  A first problem is that it says “must” which is simply false.  It is not the case that the PDF file “must” be flattened prior to submission.  The true situation is that the USPTO will do the flattening.

A second problem is that such a warning leads to wrong behavior by personnel in the Publication Branch.  In one of our cases, a USPTO person with the initials DP mailed out a Notice to File Corrected Application Papers filled with falsehoods, prompted by the Patentcenter ack receipt.  

• A first falsehood in DP’s Notice is a statement that we filed our Figure 1.1 “via EFS-Web”.  The truth is we filed it in Patentcenter.
• A second falsehood in DP’s Notice is a representation that the Legal Framework for Electronic Filing System-Web (EFS-Web) supposedly requires that “PDF documents with multiple layers must be flattened prior to submission.”  This is flatly false.  No such requirement appears anywhere in the Legal Framework document cited in DP’s Notice.
• A third falsehood in DP’s Notice is the claim that the filer must do the flattening at all, given that the USPTO actually does such flattening.

The USPTO needs to take several corrective actions.  

In the Publication Branch, corrective training needs to be given to DP and to all of the other people who do the kind of file review that leads to such Notices to File Corrected Application Papers.  DP and the other people need to be directed to stop mailing out such Notices if the only reason for the notice is the presence of such a warning in a Patentcenter ack receipt.

The wording of the warning in the Patentcenter ack receipt needs to be changed, or the warning needs to be scrapped.  Given that the USPTO actually does the flattening, and that flattening is a simple unambiguous process, probably the USPTO needs to simply scrap the warning.  If the USPTO nonetheless feels the need to preserve some sort of warning, the wording needs to be changed to make clear what the USPTO is really trying to say.  If for example the USPTO is really trying to say “we don’t really understand flattening and so you are at your own risk if you trust us to do it”, then that is what the USPTO should say.  

The responsible way to handle uncertainty.  There are some image conversions that can introduce some measure of uncertainty.  (PDF layer flattening does not, but maybe the USPTO fails to understand that point.)  An example of a type of image conversion that can introduce uncertainty is the handling of color or gray scale if the conversion is to a color depth of two (“bitonal” images).  When EFS-Web and Patentcenter do this, they usually ruin the image, often making it nearly illegible, and the filer finds out only after it is too late (only after the filer has clicked “submit”).  There are responsible ways to handle this.

A first example of a responsible way to handle this conversion uncertainty may be seen in ePCT.  The filer is warned that there is color or gray scale, after which the filer is invited to preview the conversion result.  This permits the filer to see what the result would be before clicking “submit”.  The filer can then make an informed decision and might consent to the converted image if it is still acceptably legible after conversion.

A second example of a responsible way to handle this conversion uncertainty may be seen in USPTO’s own EPAS and ETAS systems.  The filer uploads an assignment document.  What then appears on the screen are thumbnail images of the converted pages.  The filer can click on any of the thumbnails and can see the conversion result.  This permits the filer to see what the result would be before clicking “submit”.  The filer can then make an informed decision and might consent to the converted image if it is still acceptably legible after conversion.

Although users have repeatedly begged the USPTO to provide color and gray-scale conversion previews in EFS-Web, USPTO has never done so.  When the USPTO released Patentcenter for alpha testing, the alpha testers begged the USPTO to provide color and gray-scale conversion previews in Patentcenter.  USPTO has also failed to provide such previewing in Patentcenter.

Unlike the situation with the half-toning that converts gray scale or color to bitonal images, there is not any actual uncertainty or ambiguity in the outcome of flattening of multilayer PDF files.  Nonetheless if the USPTO were to perceive some uncertainty or ambiguity (even where none exists), the responsible way to address it would be to offer to the filer the opportunity to preview the results of the USPTO’s flattening process.  The filer could be shown what the outcome is from USPTO’s flattening and the filer could then review the result to see if the result is acceptable to the filer.

The real situation is that the result would always be acceptable to the filer, since the result of the flattening is predictable and it is just like looking at the PDF in any PDF viewer or printing the PDF on any printer.  So objectively it would be a waste of time to bother to write the software to provide such a preview.  But if such a preview process were the only way to get the USPTO people past any mistaken worry that there is some (imagined) randomness or uncertainty in the flattening process, it might be necessary anyway.

Back to the time-wasting consequences of this lack of clarity in the minds of the USPTO people.

What happens sometimes is that a USPTO person in the Publication Branch (such as good old DP) will be carrying out the Final Data Capture and file review after a Notice of Allowance.  And will encounter a Patentcenter ack receipt containing this needless warning about how some PDF file that got uploaded contained some dreaded “layers”.  And will then decide that there is no choice but to mail out a Notice to File Corrected Application Papers.  This wastes the time of the USPTO person.  But it also wastes my time as a patent practitioner.  I must then place a telephone call to the telephone number appearing in the Notice, which I am sorry to say is the same as the (misnamed) Application Assistance Unit.  The people who answer the telephone at the AAU are training to refuse to connect the caller to the person who actually mailed out the Notice.  It is impossible to actually reach the person who mailed out the Notice.