Benefit-of-the-Doubt Approach to Workload Indicators: Simplifying the Use of Case Weights in Court Evaluations

In practice, the calculation of workloads in different entities and the allocation of resources between such entities is often done using simple task weighting models. Different tasks are assigned weights and the total workload is the weighted sum of the tasks performed. Traditional approaches for establishing the weights, including the using of Delphi methods or detailed time-studies, are however expensive, and the resulting weights are often challenged by the evaluated entities. In this paper, we discuss how to mitigate these problems.
We study the problem within the context of the judiciary system. To ensure an efficient judiciary, it is usually considered necessary to have a reliable case weighting system (CWS). Different types of court cases, e.g. criminal versus civil cases, have different resource needs, and to get a relevant aggregate measure of the tasks at hand, it is therefore necessary to weight the different court cases, i.e. to construct a case mix corrected workload measure.
We suggest a “benefit-of-the-doubt” (BoD) approach inspired by recent developments in Data Envelopment Analysis (DEA). We allow for uncertain weights based on only partial information about the “true” weights, and we evaluate individual courts with the weights that put them in their most favorable light. In addition to making the weight setting easier and less disputable in applications, our approach dispenses with several limitations of a traditional weighted caseload approach, including the implicit assumptions of constant returns to scale and a constant rate of substitution between caseloads. Moreover, most of the applications of detailed case weights are still available using our BoD approach. We can continue to evaluate the efficiency of individual courts and device sound and fair resource allocation procedures that are robust to the remaining uncertainty about weights.
We illustrate our approach by several real-world applications and discuss relevant extensions and applications in other areas, including regulation.