What USPTO still needs to do to remediate the system crashes

(I am delighted to observe (see blog) that USPTO has addressed the TEAS RF and TEAS Plus problem mentioned below.)

Today I review several aspects of USPTO’s massive system crash that began two days ago on December 22, 2015.  In other blog articles I discuss the status of various systems and the “deeming” remedy that USPTO has proposed and redundancy measures that USPTO needs to bring about urgently.  In this blog article I will talk about the administrative steps the USPTO needs to do (and has not yet done) to remediate the massive system crash.

When the USPTO’s patent e-filing servers (including the supposedly redundant server) crashed for some eighteen hours on May 14, 2014 (a crash that carried over to May 15), applicants were forced to file on paper.  Some hand-carried their patent applications to the USPTO.  Others (our firm included) used Priority Mail Express (née “Express Mail) at the post office to get a same-day filing date at the USPTO.

USPTO then slapped everyone who made use of either of these options with a penalty of $400 for failing to e-file.

Fairness would have suggested that USPTO should have waived the $400 penalty for failing to e-file, given that during the relevant time period USPTO’s e-filing systems (including the supposedly redundant system) were broken and it was impossible to e-file.

But fairness was not in the cards.  I personally asked a very high-up person at the USPTO to waive this fee.  USPTO’s official position in response was that the $400 penalty is statutory, not rule-based, and so it was not within the power of the Director to waive that penalty.

When I was first in practice as a patent attorney, nearly every big city had a 24-hour post office where you could go to send an Express Mail package to the USPTO at 11:59 PM and still obtain a same-day filing date.  (Many big cities, including my home towns of New York and Denver, had two 24-hour post offices, one downtown and another near the airport.)  But that was years ago, and nowadays the post office closes long before midnight.  In Denver, for example, the latest any post office is open is until 10 PM.  We managed to get our one drop-dead patent application on May 14, 2014 filed by reaching that post office by about 9:30 PM.

Applicants rely on USPTO’s EFS-Web server for filing patent applications.  And they rely on USPTO’s promise that even if the main EFS-Web server were to crash, USPTO’s “contingency” EFS-Web server would nonetheless be functioning.  But on May 14, 2014, both of these servers, including the “contingency” server, crashed.  It lasted for some eighteen hours, running well past midnight on May 14, 2014.

Many applicants simply lost the filing date.  Such an applicant, planning to e-file in the last few hours of May 14, 2014, and finding the main EFS-Web server to be broken, and then finding the “contingency” EFS-Web server to be broken, would likely find to his or her horror that there was no post office at all that was open at that hour of the evening.

Now comes this crash that began December 22, 2015 and has continued to today, December 24.

One thing that USPTO will need to do, as I detail here, is go to Capitol Hill to get a special bill passed deeming December 22-24 to have been a federal holiday in the District of Columbia for purposes of the USPTO.

In addition, there will doubtless be some customers who hand-carried or post-office-filed patent applications and US national-phase entries during the past couple of days, and who dutifully paid the $400 penalty for failing to e-file.  For each such customer, USPTO needs to refund the $400 penalty (after having gotten power from Congress to do so, as I discuss here).

After having gotten that power from Congress to refund the $400 penalty, USPTO should go back and refund it to everyone who was dinged with that fee during the massive EFS-Web outage of May 14-15, 2014.

For those customers who hand-carried or post-office-filed patent applications and US national-phase entries during the past couple of days, USPTO needs to not impose the $400 penalty for failing to e-file.

It gets worse.  The TEAS-Plus system imposes a $50 per trademark class penalty on any filer of a “Plus” or “RF” (reduced fee) application who files a response by means other than e-filing.  USPTO needs to not impose that penalty on those who responded to a Plus or RF office action by means other than e-filing during this system crash time.

The need for redundancy

Today I will post several blog posts relating to the USPTO’s system crash.  Inredundancy a first article I will review the status over the past few days of some of USPTO’s systems.  In a second article I will discuss USPTO’s proposed remedy of “deeming”several days to be federal holidays.   In a third article I will review the administrative remedies the USPTO needs to carry out. And in this article I will discuss the importance of redundancy. Continue reading “The need for redundancy”

Reviewing the status of USPTO’s crashed systems

Today I review several aspects of USPTO’s massive system crash that began two days ago on December 22, 2015.  (In other blog articles I discuss the “deeming” remedy that USPTO has proposed, and redundancy measures that USPTO needs to bring about urgently, as well as the administrative steps the USPTO needs to do (and has not yet done) to remediate the massive system crash.)

In this article I discuss the present and recent status of USPTO’s crashed systems.

The crash began on the afternoon of Tuesday, December 22, 2015.  We first noticed it when EFS-Web started to get flaky.  I was in the middle of trying to e-file a design patent application, and EFS-Web crashed shortly before I was going to click “submit”.  Others in our office noticed that PAIR was broken and that they were unable to e-file trademark applications and trademark responses in TEAS.

I posted a query on the EFS-Web listserv, wondering if it was “just me”.  No, it was not just me, I learned.  And soon it became clear that every external-facing e-commerce server at the USPTO was broken.  This included EFS-Web (normal and contingency), PAIR (public and private), TEAS (for e-filing trademark documents), EPAS and ETAS (for recording assignments), RAM (for paying maintenance fees), Financial Profile, AOTW (assignments on the web), ESTTA (e-filing of TTAB papers), OEMS (for ordering physical certified copies), PATFT and APPFT (patent full-text searching), TESS (trademark searching), the ID Manual, TSDR (trademark file inspection and status), and TTABVUE (TTAB proceeding file inspection), and the interference web portal.

A crash this massive, affecting every external-facing USPTO system except the main USPTO web site, has never been seen before.  It took hours and hours for USPTO to make any public comment, but eventually USPTO attributed this massive failure to a “power outage” that had “damaged” USPTO servers.

In a separate post I express skepticism as to how a well designed system could lead to “damage” to servers due to nothing more than a mere “power outage”.  In a well designed system, any server that has been switched to battery power rather than normal electrical power will shut itself down gracefully.  When power is restored, the server can be restarted as it normally would.  (In our office, after such a shutdown due to loss of electrical power, the servers actually restart automatically when power is restored.)

But it sort of does not matter what common sense might suggest about how long it ought to take to get things working after the power is turned back on.  What matters, on a practical level, is when the various servers actually returned to normal.

It is now 48 hours after the massive system crash began.  Which systems are working now?

This morning, alert list member Chico Gholz reported that the interference web portal is working again.  (He also reports that “Judge McKelvey put a lump of coal in someone’s stocking in 106023.”)

List member Mike Ervin reported that EPAS is working again.  Given that EPAS and ETAS are essentially the same thing, I’d guess this means that ETAS is also working again.

List member Ken Boone reports that TESS is working again, but as of earlier today it was not completely up to date.

List member Kevin Grierson reported that he was able to run Feathers today.  Meaning that at least some of the functionality of TSDR is back to normal.  But as Greg William pointed out, the “documents” function of TSDR was still broken as of earlier today.  Terry Carroll reported that the TSDR API was still broken.

Earlier today Private PAIR was still broken.  But just now (about 48 hours after the start of the outage) Private PAIR seems to be working again.  It has correspondence that was mailed early in the morning of Wednesday, December 23, but it does not contain any correspondence that was mailed early in the morning of Thursday, December 24.  (Maybe there was no correspondence mailed early in the morning of December 24, because maybe no work got done at the USPTO on December 23.)

EFS-Web is still broken — both the normal server and the contingency server.

TEAS is still broken.

RAM (for paying maintenance fees) is working again.

ESTTA (e-filing of TTAB papers) is still broken.

AOTW (assignments on the web) seems to be working normally.

PATFT and APPFT (patent full-text searching) are working normally, except that APPFT is a week behind, as it is missing the publications for December 24.

TTABVUE (TTAB proceeding file inspection) is working normally.

OEMS (for ordering physical certified copies) is working.

Financial Profile has a message saying that it is working intermittently.

The ID Manual seems to be working, albeit slowly.

USPTO gives free pass on filing deadlines until December 28

As you may know (see my previous blog posts here and here) the USPTO somehow ended up on December 22 with all of its e-commerce systems broken.  The system failures are said to be due to a “power failure”.  Oddly, it is apparently not enough to restore the power, because supposedly this power failure “damaged equipment” at the USPTO.  USPTO has posted this notice:

In light of this emergency situation, the USPTO will consider each day from Tuesday, December 22, 2015, through Thursday, December 24, 2015, to be a “Federal holiday within the District of Columbia” in accordance with the description and regulations in this official notice posted here: https://www.uspto.gov/blog/ebiz/.

It had already been determined some years ago that Friday, December 25 would be a federal holiday.  And December 26 and 27 are weekend days.  In practical terms this means that any filing that might have been due in the USPTO on December 22, 23, 24, 25, 26 or 27 will be considered by the USPTO to be timely filed if it is filed by Monday, December 28.  This applies to the filing of new patent applications, entry into the US national phase from a PCT application, or response to an Office Action.

USPTO unprepared for power outage

Every USPTO e-commerce system has been broken since about 5PM Eastern Time yesterday.   Many hours passed during which USPTO was silent about the cause of the outage, and it was left to customers to compare notes and to try to guess what was going on through discussions on the Trademark Practitioner’s Listserv and the Patent Practitioner’s Listserv.  (What?  You have not joined those listservs?  Click on the links to see how to join.)

About fifteen hours into the outage, USPTO finally revealed the cause of the outage — the electric power had gone out at the Madison Building in Alexandria.  USPTO does not, apparently, have backup generators to keep things going during such a power outage.  Nor does it have redundant electrical feeds from two different sources or feeds.

This power outage is like watching a bad movie again and again.  We all watched this bad movie on May 14, 2014 when all of the patent e-filing systems were broken for some eighteen hours.  The backup patent e-filing system was broken just like the main system.  The only way to file a patent application was to go to the Post Office or hand-carry it to the Patent Office.  Either of which would incur a $400 penalty for failing to e-file.

For some years now I have been urging USPTO (blog) to relocate its backup e-filing server to a geographically diverse location.  This has fallen on deaf ears at the USPTO.

Still no geographically diverse USPTO backup server for e-filing!

Well today every USPTO e-commerce system has crashed.  Users are unable to reach EFS-Web (patent filing) or PAIR or patent assignments on the web.  EPAS is broken.  The system for paying maintenance fees is broken.

Oh and EFS-Web Contingency, the “backup” patent e-filing system that is “guaranteed” to be working even if the main EFS-Web system is broken, is broken.

On the trademark side, TEAS and TESS are both broken.  TSDR is down.  TTABVue is down.

Oddly the main USPTO web site is not broken.  But then the main USPTO web site does not actually do anything for USPTO customers.  To actually get anything done one must migrate away from the main USPTO web site to one of the other e-commerce sites.

It is clear that even now there is some Single Point of Failure that is able to knock out both the main EFS-Web server and the backup EFS-Web server.  An alert twelve-year old, hearing about such a problem, would have no trouble figuring out that the correct protective step would be to move one of the two servers to some geographically diverse location.  No single errant backhoe would rip up both fiber optical cable feeds at the two locations.  No power line downed because of ice would shut down both locations.

Which brings me to the Big Problem.  The Big Problem, as I have blogged about again and again, is that USPTO continues to fail to place its backup patent e-filing system in a geographically diverse location such as one of the other Patent Offices.

Oh and of course USPTO’s answer would be “you can always use 37 CFR 1.10” meaning “you can go to the Post Office”.  Except then you would get a slap in the face, as our firm did, with a notice of having to pay a $400 penalty for failing to e-file.

Will the USPTO be closed this Thursday?

Every December there is a sort of game of chicken, with federal employees (and customers of the USPTO) wondering until the last minute whether the President will sign an executive order giving federal employees an extra day off from work around Christmas.  This affects customers of the USPTO because they need to know whether patent applications and responses can be postponed until the next working day.

Slightly over half of the time over the last twenty years or so, what has happened in December is that the President signs such an order.  There has never been more than two or three weeks’ advance notice, and sometimes the Presidential order has been signed just a few days before the would-be holiday.

In 2014 Christmas fell on a Thursday (and thus was a federal holiday), and on December 5 of that year the President signed an order giving everybody an extra day off on Friday the 26th.  This meant that any patent application or response that would normally be due on the 25th or 26th or 27th or 28th could be postponed until the 29th.

In 2013 Christmas fell on a Wednesday and the president did not give any extra time off.

So how will it be this Christmas season?  Will federal employees get the day off on Thursday the 24th?  Will customers of the USPTO be able to put off filings that would have been due on the 24th until Monday the 28th? Continue reading “Will the USPTO be closed this Thursday?”