The 4 equity market factors from Fama and French (1993) and Carhart (1997) are pervasive in academia and practice. However, not much is known about their joint distribution and dynamics. We find striking evidence of asymmetric tail dependence across the factors. While the linear factor correlations are small and even negative, the extreme correlations are large and positive, so that the linear correlations drastically overstate the benefits of diversification across the factors. We model the nonlinear factor dependence dynamics and explore their economic importance in a portfolio allocation experiment showing that significant economic value is earned when acknowledging nonlinear dependence.