Wednesday, May 30, 2018

Predicting Future Patent Outcomes

For patent examiners, past performance is indicative of future results.  Because of this, many patent practitioners use services that provide statistics about patent examiners to improve their prosecution strategy.  The most commonly used statistic is a grant rate or allowance rate that provides insight into the difficulty of an examiner.  This statistic looks backward to tell you what percentage of applications that have already been disposed, were granted (granted/(granted + abandoned)).  I’ll call this the backward grant rate.

For our examiner statistics, we compute a “three-year grant rate” that shows the probability of obtaining a granted patent within three years of the first office action.  This three-year grant rate tells you how difficult an examiner is and when you can expect to be granted a patent.  In this blog post, we compare the three-year grant rate to the conventional backward grant rate and demonstrate how it is a more accurate measure of examiner difficulty.

Here is an example of a grant rate timeline across the entire USPTO:



This timeline shows your chances of being granted a patent at different times after receiving your first office action.  For example, you have a 44% chance of being granted a patent by one year after the first office action, a 61% chance after two years, a 66% chance after three years, and so forth.  The timeline also shows the percentages of abandoned and pending applications at the same time intervals. 

Constructing this timeline is fairly straightforward.  You take all the patent applications of the USPTO and shift them in time so that the dates of their first office actions are the same.  Then, for each month afterwards, you compute the percentage of patent applications that were granted, still pending, or abandoned by that month.  As time goes on, the percentage of patent applications that are granted or abandoned always increases (see technical notes below).

Examiners in the USPTO can have very different grant rates.  For comparison, we present timelines for two examiners with very different grant rates.  The first, Examiner DT, has a very high grant rate with 90%, 97%, and 98% of applications granted at one, two, and three years after the first office action, respectively.

DT:

By contrast, Examiner SP has a very low grant rate with only 1% of applications granted at three years after the first office action.

SP:

In practice, it is easier to use a single number rather than an entire timeline to compare examiner difficulty.  For this reason, we use the three-year grant rate to compare examiners.  The precise time is somewhat arbitrary, but three years provides a balance between providing enough time for meaningful prosecution and obtaining a relatively near-term measure.  Here is a comparison of the above examiners and the USTPO average using the three-year grant rate:
  • USPTO — 3YGR of 66%
  • Examiner DT — 3YGR of 98%
  • Examiner SP — 3YGR of 1%
While the three-year grant rate can clearly be used to evaluate examiners, it begs the question of why one would use the three-year grant rate in place of the backward grant rate.

To compare the three-year grant rate with the backward grant rate, we created a scatter plot of examiners with the backward grant rate on the vertical axis and the 3-year grant rate on the horizontal axis (SPEs and examiners with a small number of cases have been excluded).

As you can see, the two grant rates are highly correlated with each other with the backward grant rate being, on average, a little higher than the three-year grant rate.  We have an interactive version of this graph here.


To compare the two grant rates, let’s look at two examiners with the same backward grant rate but with very different three-year grant rates.  We selected DY and VP on the plot above.   Examiner DY and VP each have backward grant rates of about 65%, but DY’s three-year grant rate is 17% and VP’s three-year grant rate is 70%.  Viewing their timelines provides an easy explanation.  Here is DY’s timeline:

DY:

And here is VP’s timeline:

VP:

While the two examiners have similar backward grant rates, their three-year grant rates are very different because it takes years longer to get an issued patent with DY than with VP.  It seems that examiner DY has many applications being prosecuted at even 5-6 years after the first office action.

DY is clearly a much more difficult examiner than VP and this is reflected in the three-year grant rate.  Here, the backward grant rate is not a good indicator because it assigns them the same difficulty level.

Now, let’s look at two examiners with the same three-year grant rate but with very different backward grant rates.  We again using DY and now comparing DY to BG from the scatter plot above.  For these two examiners, you have about an 18% chance of getting an issued patent at three years after the first office action.  Here is BG’s timeline.

BG:

The difference between DY and BG is the abandonment rate rather than the grant rate.  For some reason, 60% of BG’s cases are abandoned at three years and only 17% of DY’s cases are abandoned at three years.

My theory for the vastly different abandonment rates between DY and BG is that DY’s cases are much more valuable to the applicants (the group is “medical and surgical instruments”) than BG’s cases (the group is “amusement and education devices”).  As a result, DY’s applicants are willing to spend much more time and money on prosecution than BG’s applicants.

We believe it is more accurate to say that DY and BG have similar difficulty levels.  The fact that DY’s applicants are willing to spend more time and money on prosecution should not change the inherent difficulty level of the examiner.  Because the three-year grant rate assigns them the same difficulty level (18%) and the backward grant rate gives them very different difficulty levels (65% and 20%), the three-year grant rate is a more accurate indicator in this situation as well.

Takeaways

The greatest benefit of the three-year grant rate is that it incorporates information about both the difficulty of the examiner and the length of time to obtain a patent into a single, easy to understand number.  If your examiner has a three-year grant rate of 18%, it is easy to explain to your client that they have an 18% chance of getting a patent issued in three years.

For some examiners, such as in the examples above, the three-year grant rate also provides a more accurate depiction of the difficulty of an examiner than the backward grant rate.  The three-year grant rate is not influenced by the assiduousness of applicants while the backward grant rate is.

The full grant rate timeline also provides much more information in an easy-to-digest format.  For some examiners, you can see that they are difficult before the first RCE and much easier afterwards.  For other examiners, you can see that they make a decision early in prosecution and that it is hard to change their minds later.  You can leverage this additional information to plan your prosecution strategy, such as whether to file an RCE or a notice of appeal.

Caveats and Technical Notes

While the focus of this article is on raw statistics, we do want to note that there is an inherent unfairness in assigning a number to a person.  Many patent examiners at the USPTO work hard and do a fantastic job.  Some factors that influence an examiner’s grant rate have little to do with the actual difficulty of the examiner, such as the nature of the technology, the mostly random assignment of cases, or the influence of a primary examiner or SPE.  When using statistics, it is important to look beyond the numbers as well.

While computing a grant rate timeline is straightforward, there is a little time travel involved, which makes aspects of it a little surprising.  For a large enough amount of data, the grant and abandonment rates increase over time, as in the USPTO timeline above.  For individual examiners, there is much less data and the grant and abandonment rates can decrease over shorter time periods, such as for examiner VP’s abandonment rate above.  There are a few reasons for this, such as an examiner’s difficulty changing over time or variations in cases assigned to an examiner over time.

Also, the number of cases on the left of the timeline is much larger than the number of cases on the right side of the timeline.  For example, for DY there are 141 cases at year 0 (all cases in the data set) and 40 cases at year 4 (all cases in the data set with the first office action at least 4 years ago).  Accordingly, the lines on the right side will have greater variability than the lines on the left.