Weighing Sample Evidence

We model how sample evidence guides choice: An agent faces a number of alternative actions. For each action, she observes a sample of outcomes; she cannot see the distribution from where the sample was drawn. To make her choice, she evaluates the evidence for the hypothesis that an action is optimal. The strength of evidence in favor of the hypothesis is measured by the average decision utility of the outcomes in its sample; its weight gauges predictive validity and is approximated by the size of the sample. We identify necessary and sufficient conditions for her choice to be determined by the interaction of strength and weight, reflecting the determinants of confidence judgements documented in experiments (Griffin and Tversky, 1992). These conditions characterize a model consistent with non-trivial uncertainty attitudes and “frequentist" expected utility maximization.