EPO Patent Quality: Where do we come from, where do we stand, and where are we going?
June 5, 2024
1. Preamble: The Iron Triangle
In project management there is a law known as the Iron Triangle, which states that in delivering any project, there are three points on a metaphorical triangle: time, cost and quality. In other words: how fast can the project be done; how cheaply; and what is the quality of the thing being delivered? Every project manager knows that there is always a trade-off: you can deliver two of these things, but never all three.
For example, imagine your project is to examine patent applications. The goal is to reward (only) those inventors with a (temporary) monopoly who have contributed to the state of the art by disclosing a true innovation. Granting monopolies to inventions that do not meet this threshold risks inhibiting the freedom of economic activity and jamming up the economy generally.
You can examine and grant patents cheaply and quickly, but they will be rubbish. You've sacrificed quality. In essence, they would be no better than unexamined utility models, as I argued earlier in my modest proposal.
Or you can examine patent applications quickly and with a high quality, but then you will need a large number of examiners working promptly on each case, which will be very expensive for the European Patent Organization and its users.
Or you can examine patent applications to a great quality and keep the costs down by limiting or even reducing the number of examiners, but that means you have sacrificed speed. And an unwanted side effect for applicants will be that they will have to wait very long for their “reward” (the patent), but they will have to pay the EPO’s considerable renewal fees every year nonetheless.
That being said, it is unrealistic to expect from the EPO, or indeed any project manager, to deliver the optimum in each dimension (e.g. a top quality patent within a few weeks and for free) to begin with. However, this should not be taken as an excuse or a reason for self-complacency. Striving for the best possible patent quality is still a goal worth pursuing, and serious applicants will understand that good quality will come with a price and time tag. Conversely, they will much less understand or approve a policy that puts speed (output) on top of everything, because this will only result in lower quality patents and a substantial fee rise, as happened in April of this year. For further explanations, see here. Unfortunately, this seems to be the EPO’s current trajectory, at least when managers talk turkey.
2. What are the current Indicators and Numbers for measuring Patent Quality?
Patent quality is notoriously difficult to measure, and comparing absolute numbers or scores of different tests with each other will not be very informative. I would also argue that no single test developed so far will provide a complete and accurate picture. Each methodology has its limitations and drawbacks. That being said, let us look into some of the current numbers and indicators in more depth.
On 08.03.2024, the EPO published a new Quality Dashboard and new “Key Performance Indicators” (KPI). One part of these KPI’s is measured by the Directorate Quality Audit (DQA) on the basis of a sample size of randomly selected 750 products, another reflects the rating of periodical user satisfaction surveys on the EPO’s search and examination products.
The EPO’s current internal figures show that, according to the DQA, 7.4% of grants were based on an incorrect assessment of novelty and inventive step and 4.6% on an incorrect assessment of added matter (it is possible that some products showed deficiencies in both dimensions). Yet 78% of the EPO’s users rated the examination products as good or very good. Different metrics, different “auditors” – so I would think that these figures are hard to compare directly. For example, the DQA also measured the quality of the EPO’s search products and found more relevant prior art in 3.7% of them. Whether the users factor in such a possibility in providing their marks is not clear to me.
Yet another possibility to indirectly measure patent quality is by looking at what happens to patents when they are properly challenged in opposition proceedings. In this regard, we owe a great deal of gratitude to Daniel X. Thomas, who has extensively researched this aspect, purely based the outcome on published decisions of the Boards of Appeal (BA) of the EPO in opposition appeal proceedings, which he has read and analyzed. Daniel published his first findings on the excellent IP Appify Blog in April, but kindly provided me with updated figures (status: 3.6.2024) and permission to publish them here.
Taking as a benchmark published decisions from 1.1.-3.6.2024, Daniel Thomas looked at 551 decisions, of which 502 decided on the validity of the opposed patent. His results were as follows:
- Maintenance in amended form 182, i.e. 36,3%
- Revocation in full 242, i.e. 48,2%
- Rejection of the opposition 78, i.e. 15,5%
Daniel further observed that in 64,4% of the published decisions, the decision of the OD was set aside by a BA. This alone is stunning, particularly in view of the fact that the Rules of Procedure before the Boards of Appeal render it extremely difficult to introduce new facts (e.g. new prior art or entirely new auxiliary requests) on appeal. Thus, at least in theory, the BA should “only” decide on questions of law, which the first instance examiners are supposed to know as well. Even though it would clearly be unrealistic to expect 100% convergence of first and second instance decisions, I find it amazing that the Opposition Divisions seem to get it, at least statistically, much more wrong than right.
In most cases, the BA are stricter than the Opposition Divisions. However, as Daniel correctly noted, it is striking for the first quarter of 2024 that in 17 of 78 cases, i.e. 21.8.%, the rejection by the BA was preceded by a revocation or maintenance in amended form by the Opposition Division.
The main divergences between the decision of the OD and the decision of the BA were the following (percentages are relative to the total number of OD decisions set aside by the BA):
- Maintenance in amended form to revocation of patent 31.3%
- Rejection of opposition to revocation of patent 17.5%
- Maintenance in amended form to maintenance in a different form 18.6%
- Rejection of the opposition to Maintenance in amended form 4.9%
- Revocation or Maintenance in amended form to rejection of oppo 15.5%
- Remittals for further prosecution 8.9%
A further and independent quality report was published end of 2023 by IP Quants, based on published EPO data and a similar analysis as conducted by Daniel Thomas, but including a number of further interesting aspects. The full report is downloadable as a pdf file here: Qthena_Insights_Report_3_2023_by_ipQuants AG and I highly recommend it to read it in full. For the purposes of this summary, though, it may suffice to reproduce IP Quants’ Key Findings:
Key Findings:
- Increased Pending Workload: There has been a 7.54% rise in pending patent applications at the EPO from 2018 to 2022, indicating an increased workload that traditionally would require a larger number of examiners.
- Reduced Examiner Workforce: The number of EPO patent examiners has decreased by 7% from 2018 to 2022, despite the rise in pending applications, suggesting a more significant workload per examiner and potential challenges in maintaining examination quality.
- Decreased Time to Decision: The time to grant for direct EP filings has been reduced by approximately 10% from 2019 to 2022, hinting at a more rapid, perhaps less thorough, examination process.
- Diminished Communication: A considerable proportion (48% in 2022) of granted EP patent applications received no Article 94(3) EPC Official Communications, reflecting an increase from 43% in 2013 and raising questions about the depth of examination.
- Opposition Outcomes as a Quality Indicator: The high rate of successful oppositions (72.9%) indicates that many EPO-granted patents do not withstand post-grant challenges, with only 27.1% of cases maintaining the patent as granted.
3. How to evaluate the EPO’s Patent Quality
Let us begin this chapter once more with IP Quants’ report, which concluded the following:
The data presented in this report substantiates the concerns about examination quality and workload at the EPO, echoing the issues raised by both industry stakeholders and the patent examiners themselves. The findings strongly suggest that the EPO must recognize and address these challenges. We propose a collaborative approach where EPO management, patent examiners, and industry stakeholders engage in open and constructive dialogue to recalibrate the examination process, ensuring that the quality of patent examination is not compromised.
I share these conclusions and would like to add a few observations of my own. Firstly, while patent quality is notoriously difficult to measure, this does not mean that one has to give up on this project altogether. At the very least, it is possible to compare timelines, i.e. the answers to the same questions and results of the same tests over the past five or ten years.
Unfortunately, the EPO’s own recent figures in the quality dashboard do not allow for such a direct comparison, because it is not clear at all that the methodology used in the current audits is the same as it was in the years from 2017-2021, for which we have official figures from the EPO. These published figures show a decrease in the audited quality of grants from about 85% to about 75%.
The statistical data from opposition proceedings (first instance only!) reported in the IPQuants report range from 2018-2022 and can be summarized in the following graph (data taken from page 13 of the IPQuants report and replotted with regression lines).
The trend seems to suggest that oppositions are getting more and more successful (about 0.75% per year), with “success” being defined as the sum of total revocation and maintenance of the patent in a limited form. This trend worries me (and should worry patent proprietors in general!), particularly when remembering that I started my career in a world where the statistical outcome in the first instance was about a 1/3 mix (1/3 of maintenance in full; 1/3 maintenance in part; 1/3 revocation). Today, only about 25% of opposed patents survive even the first instance (and, as Daniel Thomas has shown, see above) only about 10-15% survive two instances.
Similarly, the reversal rates of decisions of the opposition divisions by the BA used to be about 50% around the year 2000. Now it is about 65%, see above.
All of these figures point to a downward trend in the quality of patents granted by the EPO over the past decade and there is no end to this trend in sight. To be fair, it is also theoretically possible that opponents have gotten better in finding pertinent art and exploiting the Rules of Procedure of the Boards of Appeal, or that the Boards have tacitly “raised the bar” for patentability, or that “anything that can be invented has been invented” but I find it difficult to adduce any evidence for such any of these hypotheses, and note that the latter quote has been – probably wrongly - ascribed to Charles Holland Duell, who was Commissioner of the USPTO from 1898-1901. Therefore, I find it plausible that at least part of these changes is attributable to a less efficient filter function of the examination and opposition divisions, which also conforms with my own observations. At the same time, the average time to grant for a direct EP filing has been reduced from 57.8 months to 52.5 months – see page 9 of the IPQuants report. Put in one schematic drawing, the situation has developed like this:
The constructive critical voices from industry (IPQC) therefore have a point and are based on verifiable facts and data. They deserve to be taken seriously. Imagine how much more quality and how much more reduction of the time to grant would have been possible with the same ratio of examiners/applications as in the past!
4. What does the EPO do, and what should they be doing?
As is clear from the above, the EPO management seems to be firmly determined to squeeze ever more efficiency out of their machinery. Alas, the machinery that is supposed to deliver more and more products in shorter times is no magic mechanical or electrical engine but consists of ordinary human beings, so-called patent examiners. It stands to reason that there is a certain natural limit of how many “products” an average examiner is able to deliver with acceptable quality per working day.
Unfortunately, the EPO management does not seem to accept this simple truth or at least seems to believe that nothing bad will happen if they turn the wheel even faster in the future. The following slide that I received from an EPO source demonstrates this trend in an alarming manner:
Not only does the EPO management not recruit more examiners to cope with the increasing number of applications, it actually plans not to replace all examiners leaving the EPO in the next couple of years.
On top of that, there will be the problem of who will train the incoming new examiners. The EPO’s age pyramid has developed like this:
It is entirely predictable what will happen in a few years’ time, unless the EPO management finally makes a U-turn with its recruitment policy. There is still a short window of opportunity where the recruitment of new examiners could be reasonably handled by the existing experienced workforce. But if this window is not used, the EPO will suffer a further significant loss of experience and quality in the mid-term future.
5. The Happy End
Very fortunately, the Administrative Council (AC) of the EPO is fully aware of these developments and the trends though its subsidiary body, the Technical and Operational Support Committee (TOSC), which examines matters of quality in close detail and then presents them to Council. This committee is composed of a 1:1 mixture of external experts and members of the EPO and is instructed to advise the AC completely independently of the EPO management.
The AC is also perfectly aware that their role as the Office's supervisory body is to oversee the successful execution of the EPO’s mission to deliver high-quality patents and efficient services that foster innovation, competitiveness and economic growth. They appreciate that this is the key mission of the EPO, rather than to be “super-productive” in terms of the products per examiner, or “super-profitable” to the benefit of the European Patent Organization and its the contracting states.
The AC also knows that the EPO management are solely accountable to them and take this responsibility very seriously. For example, to ensure independence between the supervising body and the body that they are supervising, they have instituted a strict separation of powers and have now introduced conflict rules prohibiting its members from assuming senior management positions at the EPO after leaving the AC.
Moreover, on the instruction of the AC, the EPO management has meanwhile also recognized that the operation of its Standing Advisory Committee (SACEPO) working group on quality should be substantially changed so as to give external members a true forum to express and discuss their concerns with the responsible people at the EPO. As a result, the agendas of the meetings of this working group are now discussed and pre-agreed with its external members so as to make sure that at least 30% of the time is allocated to listening to users’ opinions and another 30% to an in depth-discussion on what could be done to further improve the quality of the EPO’s products.
In view of the above, the AC recently expressly welcomed being approached by a relevant group of applicants (the IPQC) bringing concerns about patent quality to their attention. As one would have expected, they took these as well as any other concerns about patent quality seriously and swiftly decided to investigate them thoroughly in view of the importance of patent quality for the EPO’s mission. Concerns about patent quality and constructive proposals for improving it are generally viewed as an opportunity rather than a nuisance. Therefore, such groups of users are usually also allowed – within the tight time constraints of an AC meeting – an opportunity to present their views directly to the AC.
And the EPO, the AC and its users lived happily ever after.
You don’t quite believe this ending? Well… consider Exhibit A: Letter to IPQC from AC.
Tags:
EPO,
Patent Quality
Become a contributor!
Interested in contributing? Submit your proposal for a blog post now and become a part of our legal community!
Contact Editorial Guidelines
Concerned observer
"For example, to ensure independence between the supervising body and the body that they are supervising, they have instituted a strict separation of powers and have now introduced conflict rules prohibiting its members from assuming senior management positions at the EPO after leaving the AC". If only. To individuals in the right circles, the EPO is the magic money tree that keeps on providing more than enough to satisfy even the largest of appetites. The trouble with satisfying large appetites is that regular over-consumption leads inevitably to greed. For greedy individuals, more is never enough. Hence the plan for fewer examiners to cope with an ever-increasing number of patent applications. The IPQC will, of course, be well aware of existing consultation mechanisms, such as SACEPO working groups. The mere existence of IPQC therefore represents the clearest possible indication that such consultation mechanisms have proven to be ineffective. The AC Chair's suggestion to use those mechanisms thus conveys to the IPQC the unambiguous message that they can take their concerns about patent quality and go forth and multiply.
DXThomas
Some further comments on the figures published by Thorsten. It is manifest that the result of oppositions after appeal cannot be taken as an absolute measure of quality of the granted patents, as the number of oppositions remains low and some technical areas barely see oppositions. That an opponent might make more efforts to find prior art, is also a factor allowing to relativise the figures. What is however striking is that opponents mainly come up with documents which were in the search files of the EPO. This is particularly noticeable when for instance, the opponent comes with prior art under Art 54(3). In some blogs I add in the "Comments" section a part "On the procedure" to this effect. The number of cases in which documents were truly not available in the search files is rather low, actually between 6 to 8%. Public prior uses, PhD dissertations, oral disclosures represent prior art which cannot be available at the EPO. To come to those figures there is no need to seek or search for confidential data, it is sufficient to look at publicly available data, i.e. published decisions of the boards after appeal on opposition. Any interested person can do this. My reasons for looking at published decisions is that I am still involved in training, be it with companies offering training in patents, representative’s firms or in-house patent departments. My analysis and the blog are actually a kind of side product of my involvement in training. No good to come to training with old case law. As case law evolves, it is best to keep abreast. Without giving too much importance on the published decisions of the boards, those represent the only directly available benchmark. A linear extrapolation to all granted patents would not be correct, but the gathered data are representative of a trend which cannot be denied. However, considering OD’s decisions which have not been appealed as being confirmed, is a view I have been confronted with when I published my figures for 2023. This view is also not correct. Technical areas in which oppositions are filed also vary with time. Just to give a few examples: for a long time there were lots of oppositions in absorbing devices. Presently, hair colouring compositions, means for improving document security against falsification, telecommunication or e-cigarettes seem to have the lead. In chemistry/pharmacy there seems to be a kind of constant level of oppositions, often filed by generics manufacturers. When looking at oppositions, new techniques are much more prone to oppositions as it is important to keep market parts in an expanding domain. I personally observed this with pacemakers. lithotripters or wind turbines. Once the technology has reached a cruising level, or subsidies by governments go down, oppositions decline and other domains take over. To sum it up, the figures based on published decisions of the boards after appeal on opposition show a trend, which alas is not positive, in spite of all possible denegation. This trend is actually confirmed by other sources as stated in Thorsten’s article.
Quality problems at the EPO? You are mistaken!
Further to the diminishing number of examiners in front of increasing filing figures, other reasons also explain the lowering of the quality delivered by first instance divisions of the EPO. A first aspect which contributes to difficulties in recruitment is the fact that examiners can only expect an unlimited work contract at the EPO after two 5 year periods! This fact is widely unknown outside the EPO. One could consider that the probationary time has been extended from one two 10 years. This has a direct effect on the recruitment. A potential candidate with some professional experience and a family will not file an application for a job at the EPO. It would mean severing all links with its national social security and pension systems when hiring at the EPO. With the possibility of getting fired at any moment if the performance is not to the liking of the EPO's management, who is proepared to take this risk for his family? Giving only two 5 years before an unlimited contract contracts is a perfect means for exerting pressure so that examiners do not become defiant and even join the union. At the beginning of BEST, training was set to three years. For examination before BEST it was three years for examination alone. The introduction of BEST already went on a par with a reduction of training time. With 5 years contracts for examiners, the training time has been reduced to two years. Keeping three years of training with a 5 year contract could not be sold to to the AC. It is doubtful that new candidates are more intelligent than their predecessors or that pedagogics in IP training has undergone a quantum leap. The production target is set at very high level from day 1 of the training. At the end of the training period the production/(productivity has to be that of an experienced examiner. Even with three years training, BEST examiners were well trained in drafting opinions. With two years the ensuing difficulties become even more noticeable. The problem starts when replies from applicants come in. Newcomers are then more or less left alone. It is then much easier or tempting to grant than to refuse. There are examiners who after quite a number of years at the EPO have never refused an application. For a while examiners got the double amount of points for a final action when it was a refusal. This has apparently gone. A file is a file. In order to find people, the "Young Professional program" and the "Pan-European Seal program" have been set up by the EPO. Young professionals are not recruited as full examiners, and are hence paid less. They actually can do an examiner job without being examiners. A similar program has been set up for examiners of national offices which can be seconded to the EPO under very specific conditions. They do not get the same pay as examiners. Under the heading "Bringing teams together” the upper management of the EPO has exactly done the contrary. Examiners do not any longer have dedicated offices and have to apply for a desk if they want to come into the office. They are confined to home-office! In the past, if a trainee wanted to speak to his trainer he just went along the corridor to speak with him. This possibility has gone. Coffee machine discussions or over lunch have also gone. They were a good source of knowledge exchange between examiners. In first instance, oral proceedings are only on possible in form of video conferences. Having a separate communication channel among the members of the division during OP does not replace the direct possibility of contact between their members. A lot of non-verbal communication between members of the division has gone, not to speak between parties and diviion. Even if oral proceedings have to be held in the form of a videoconference, it should at least be insured that all three members of the division sit together. This is not the case. In the past, the pay package and ancillary conditions at the EPO were certainly good, but this advantage has gone. A good engineer or scientist, with good knowledge of the three EPO official languages does not have any difficulty in finding a well paid job in his country of origin. Why should he come to the EPO under the present appalling conditions? There are thus many more factors than those brought forward in Mr Bausch’s article which explain the lowering quality of the work delivered by the EPO.
curious attorney
For me, the quality of oppositions has little to do with the quality of examination, it is a completely different job and I know for a fact that at the EPO few examiners do opposition and a former colleague of mine who works there assures me that the time spent on oppositions is fully recognised as extra work and is not subject to the new efficiency rules that govern examination. So the fact that opposition decisions are often overturned on appeal cannot have anything to do with the new working methods which apply only to examination, but perhaps it has to do with a new way in which applicants now interpret opposition or in which the BA now interprets certain aspects, which often puzzles me too. So to be honest, I don't see any connection between opposition appeals and first instance examinations: there are too many factors and differences between them. And whether more cases are successfully opposed depends only on the different strategies and policies of the applicants, who may now also have tools, resources and skills (AI?) that a big office like the EPO cannot always keep up with. Personally, I don't see any major changes in the last few years in the work delivered by examiners, but if there are, I don't see how the data presented here can somehow subrogate them.