Thursday, November 15, 2018

Visualizing Outcome Inconsistency at the USPTO

In an ideal world, your chance of getting a patent allowed is based on the merits of your patent application and independent of the largely random assignment of the patent examiner.  As any patent attorney knows, however, this is not the case.  Some examiners allow patents too easily and others seem predisposed against allowing any patents at all.

This ideal can be described as outcome consistency.  The outcome of a patent application should be largely the same regardless of the assigned patent examiner.  Outcome consistency is needed to ensure fairness.  It is unfair for an applicant to be denied a patent for a worthy invention because it was assigned a hard examiner, and it is unfair to the public for a patent to be granted for an unworthy invention because it was assigned to an easy examiner.  The lack of outcome consistency among patent examiners is a known issue that the USPTO is working on improving, and this article presents visualizations to help diagnose areas for improvement.

The patent application grant rate across the USPTO is 66% (computed as described here).  One would expect that a distribution of examiner grant rates would follow a bell-like curve with (i) the average examiner having a grant rate of 66% and (ii) a reasonably small standard deviation such that most examiners are close to the average.

Here is the actual distribution of examiner grant rates across the USPTO (this is a weighted histogram according to the number of cases handled by an examiner with SPEs excluded):


The distribution here is clearly far from ideal in that examiner grant rates run the full gamut from 0% to 100%.  The standard deviation is 22%, and the dashed line is the closest bell-like curve (a beta distribution).

Possible reasons for the wide spread of grant rates include the following:

  1. Technology areas may have inherently different grant rates based on the difficulty of discovering an invention in a technology area.
  2. Examiners may have different difficulty levels.

Because of the hierarchical structure of the USPTO, the first possible reason is quite easy to investigate.  The USPTO comprises three organizational levels: (i) 8 technical centers, (ii) 59 groups, and (iii) 568 art units.  As you go down the hierarchy, the technology addressed is more specific.  Accordingly, one would expect the examiner distributions for individual groups and art units to more closely resemble a bell-like curve with a smaller standard deviation.

Outcome Consistency Within Groups

I computed grant rate distributions for examiners of each of the 59 groups (I did not do so for art units because there is not enough data at only 10-20 examiners per art unit).  Most groups had better outcome consistency than the USPTO as a whole but some were worse.

For comparison, here are distributions for Group 2630 (Digital and Optical Communications) with a standard deviation of 10% and Group 2620 (Selective Visual Display Systems) with a standard deviation of 23%:
Group 2630:
Group 2620:

Because the distribution for Group 2630 is much closer to a bell curve and has a much smaller variance, we can conclude that Group 2630 is doing a much better job of achieving outcome consistency across examiners than Group 2620.

Why does Group 2620 have such poor outcome consistency?  Since all of the patents examined by Group 2620 relate to Selective Visual Display Systems, diversity of technology seems to be an unlikely culprit.  From my experience as a patent attorney and having worked with many different examiners, I would conclude that the poor outcome consistency is due to different difficulty levels of individual examiners.

Outcome Consistency Across Groups

Because different groups address different technologies, one would expect some variation of grant rates between different groups.  One would also expect that groups that address similar technologies (e.g., groups in the same tech center) would have similar grant rates.  The example above with groups 2620 and 2630, however, shows that this is not the case.

To view outcome consistency across groups, I computed the following distribution:

Each blue dot corresponds to an examiner, and the size of the dot corresponds to the number of cases handled by the examiner.  The horizontal axis is grant rate, and the vertical axis is the group.  Examiners in the same group are aligned horizontally so you can view the distribution of examiners in a group by scanning any horizontal line in the above distribution.  Divisions between tech centers are indicated by the yellow lines.  You can access an interactive version of the grant rate by group distribution, which allows you to click on a blue dot to see the details of an individual examiner.

As expected, the above distribution shows that some technology areas have, on average, higher grant rates than other technology areas.  What I find most striking about the above distribution, however, is the large variation of examiner grant rates across all technology areas.  While some groups (such as group 2630 discussed above) have lower variation of examiner grant rates, adjacent groups with similar technology (such as group 2620) do not share this lower variation, and accordingly there is a lack of outcome consistency across all technology areas of the USPTO.

If a patent applicant is fortunate enough to have his or her patent assigned to a group with low variation (such as group 2630) then, from that point forward, there is higher outcome consistency.  If the applicant had used slightly different words in the claims, however, then the application could have been assigned to a different group with a very different grant rate and/or a much higher variability of grant rates.

In my practice, I have applications with similar technology where some applications get assigned to one of the business method arts units with very low grant rates (in Tech Center 3600) and others get assigned to more technical art units with much higher grant rates (in different tech centers, such as Tech Centers 2100, 2400, 2600).  Accordingly, to get better results for my clients, I draft my claims in a manner to try to avoid getting classified into one of the business method art units.

Outcome consistency thus relates not only to the variability of examiners within a group, but also to the variability of examiners across the USPTO since applications with similar technology can be assigned to different groups and even to different tech centers.

Outcome Consistency and Patent Quality

The lack of outcome consistency is caused by at least two factors.  First, the inconsistency in application classification can cause applications with similar technology to be assigned to different groups with very different grant rates.  Second, in most groups, the variability of examiner grant rates is high.  As a result, your chance of getting a patent granted depends in large part on the mostly random assignment of a patent examiner to your application.

Patent quality is paramount to the mission of the USPTO, and patent quality presumably includes making sure (i) that unworthy patents are not granted and (ii) that worthy patents are granted.  When worthy patents are not granted, applicants are unfairly denied their patent rights.  When unworthy patents are granted, the public may be harmed by unfair competitive advantages.

The data presented here may help identify areas for improving the second cause of outcome consistency identified above.  The table at the end of this article shows the standard deviation of grant rates for each group in the USPTO.  The groups from this table with the largest standard deviations of examiner grant rates may be candidates for review to find ways to decrease variability.

In my view, improving outcome consistency across patent examiners is one of the most important things the USPTO can do to improve patent quality. I hope this article helps bring increased attention to finding a solution.

Group Grant Rate Std Dev Desc
1780 44% 26% Food, Miscellaneous Articles, Stock Material
1790 40% 23% Food, Analytical Chemistry, Sterilization, Biochemistry
37E 58% 23% Medical Instruments, Diagnostic Equipment, and Treatment Devices
2140-2170 46% 22% Graphical User Interface and Document Processing
2150-2160 64% 22% Data Bases & File Management
2620 63% 22% Selective Visual Display Systems
37A 57% 22% Amusement and Education Devices
37I 49% 22% Refrigeration, Vaporization, Ventilation, and Combustion
1720 62% 21% Fuel Cells, Battery, Flammable Gas, Electrophotography, Photolithography
1750 41% 21% Electrochemistry, Solar Cells, Thermoelectrics
2440-2450 66% 21% Computer Networks
1630 48% 20% Molecular Biology, Bioinformatics, Nucleic Acids, Recombinant DNA and RNA, Gene Regulation, Nucleic Acid Amplification, Animals and Plants, Combinatorial/ Computational Chemistry
1650 51% 20% Fermentation, Microbiology, Isolated and Recombinant Proteins/Enzymes
1710 53% 20% Coating, Etching, Cleaning, Single Crystal Growth
1770 63% 20% Chemical Apparatus, Separation and Purification, Liquid and Gas Contact Apparatus
37B 52% 20% Sheet Container Making, Package Making, Receptacles, Shoes, Apparel, and Tool Driving or Impacting
37D 59% 20% Medical and Surgical Instruments, Treatment Devices, Surgery and Surgical Supplies
37F 59% 20% Body Treatment, Kinestherapy, and Exercising
1730 66% 19% Metallurgy, Metal Working, Inorganic Chemistry, Catalyst
2120 71% 19% Miscellaneous Computer Applications
2130 71% 19% Memory Access and Control
2480 57% 19% Recording and Compression
2610 68% 19% Computer Graphic Processing, 3D Animation, Display Color Attribute, Object Processing, Hardware and Memory
2690 60% 19% Selective Visual Display Systems
3620 22% 19% Business Methods - Incentive Programs, Coupons; Operations Research; Electronic Shopping; Health Care; Point of Sale, Inventory, Accounting; Cost/Price, Reservations, Shipping and Transportation; Business Processing
3640 69% 19% Aeronautics, Agriculture, Fishing, Trapping, Vermin Destroying, Plant and Animal Husbandry, Weaponry, Nuclear Systems, and License and Review
3690 34% 19% Business Methods - Finance/Banking/ Insurance
1620 62% 18% Organic Chemistry
1670 62% 18% Process, Nucleac acid, Protein, Carbohydrate Chemistries and Diagnostics
1760 65% 18% Organic Chemistry, Polymers, Compositions
2420 62% 18% Cable and Television
2670 75% 18% Facsimile; Printer; Color; halftone; Scanner; Computer Graphic Processing; 3-D Animation; Display Color; Attributes; Object Processing; Hardware and Memory
2680 72% 18% Telemetry and Code Generation; Vehicles and System Alarms; Selective Communication; Dynamic Storage Systems; Mechanical parts of Disk Drives; Signal Processing and Control Processing in Disk Drives
3680 25% 18% Business Methods - Incentive Programs, Coupons; Electronic Shopping; Business Cryptography, Voting; Health Care; Point of Sale, Inventory, Accounting; Business Processing, Electronic Negotiation
1660 70% 17% Plants
2180 79% 17% Computer Architecture
1610 42% 16% Organic Compounds: Bio-affecting, Body Treating, Drug Delivery, Steroids, Herbicides, Pesticides, Cosmetics, and Drugs
1740 62% 16% Tires, Adhesive Bonding, Glass/Paper making, Plastics Shaping & Molding
2190 75% 16% Interprocess Communication and Software Development
2640 76% 16% Telecommunications: Analog Radio Telephone; Satellite and Power Control; Transceivers, Measuring and Testing; Bluetooth; Receivers and Transmitters; Equipment Details
2810-2820-2890 81% 16% Semiconductors/Memory
2850-2860 78% 16% Printing/Measuring and Testing
2870-2880 75% 16% Optics
3650 74% 16% Material and Article Handling
3670 67% 16% Wells, Earth Boring/Moving/Working, Excavating, Mining, Harvesters, Bridges, Roads, Petroleum, Closures, Connections, and Hardware
37C 67% 16% Manufacturing Devices and Processes, Machine Tools and Hand Tools
37G 69% 16% Thermal and Combustion Technology, Motive and Fluid Power Systems
2410-2460-2470 77% 15% Multiplex and VoIP
2650 74% 15% Videophones and Telephonic Communications; Audio Signals; Digital Audio Data Processing; Linguistics, Speech Processing and Audio Compression
3660 77% 15% Computerized Vehicle Controls and Navigation, Radio Wave, Optical and Acoustic Wave Communication, Robotics, and Nuclear Systems
1640 50% 14% Immunology, Receptor/Ligands, Cytokines Recombinant Hormones, and Molecular Biology
2110 83% 14% Computer Architecture
2430-2490 75% 14% Cryptography and Security
2830-2840 77% 14% Electrical Circuits and Systems
37H 64% 14% Fluid Handling and Dispensing
3630 63% 13% Static Structures, Supports and Furniture
2660 81% 11% Digital Cameras; Image Analysis; Applications; pattern Recognition; Color and Compression; Enhancement and Transformation
2630 84% 10% Digital and Optical Communications
3610 79% 10% Surface Transportation