Indicator Problems stressing out US Patent Office examiners October 22, 2007
Posted by Paul Duignan in : Outcomes theory, Indicators , trackback
In an earlier blog posting on the banking system, I commented on the problems associated with indicators which can be distorted by employees. You can either use such indicators to get an accurate measure of an outcome or use them for incentivizing employees, but not both. Of course, if they’re being distorted by employess to maximize their bonuses, then the wrong types of behavior are probably being incentivized. A different problem can arise in those situations where it much harder to distort indicators. These situations in which it’s relatively easy to independently verify indicators are described in the social sciences as being situations where such indicators are reliable. However, indicators need to not only be reliable, they also need to be valid - they need to actually measure what they claim to measure. The US Patent and Trademark Office is having indicator problems of this second type with their production quotas. According to a Washington Post article their production quotas have not been adjusted since 1976 and modern patents are more compex and therefore take more time to process. As a result 67% of staff see production quotas as among the top reasons they would consider leaving and the office has a turnover crisis according to the General Accounting Office.
From a technical outcomes theory point of view what is at issue here is simply whether a target set for an indicator accurately represents what can be expected of the average employee processing an average patent. Obviously, what needs to be done is a piece of workstudy to determine how long it takes to process a modern patent and subsequent adjustment of the production quotas. The fact that it seems from the article that this has not been done illustrates a complete lack of focus on the technical features of the Patent Office’s outcomes system (in this case it would be referred to as a performance management system, rather than an outcomes system). Interestingly, there seems to be a difference of opinion between management and staff about what is causing high turnover. In a survey, a majority of staff see the production quota system as of major importance, while management consider a number of other factors (e.g. the nature of the work, the high cost of living in Washington etc.) as of major importance.
Outcomes theory would argue that the problem here is a lack of attention to the technical features of the outcomes system the office is using. A basic question which can be asked at any time about the indicators within an outcomes or performance management system - is whether or not they are accurate. In this case, the question is whether the targets set on the particular indicator of the number of patents processes by each employee, accurately represents the average time it should take to process a modern patent application.
Presumably management and employees have been arguing about this for some time as part of various discussions around why turnover is so high. A clear outcomes theory approach would have focused very quickly on the technical characteristics of the outcomes (performance management) system and would have asked many years ago whether a production quota set in 1976 was still accurate.
Paul Duignan (outcomesblog.org)
Comments»
no comments yet - be the first?