We conduct an experiment to investigate how the gender composition of an audience interacts with the gender of a player thereby shaping her/his degree of responsibility in decision-making. Together with the measures of accountability based on decision theory, we employ two physiological measures, blood pressure and heart rate variability, which allow us to disentangle the separate effects of stress and accountability. Our results show that men are more sensitive to changes in the gender composition of the audience; specifically, men lower their accountability when paired with a female audience. By contrast, women display a level of accountability that does not change with gender pairing. Finally, we find that the variation in blood pressure has a significant but small effect only on men's behavior.