Multi-Agent Deep Reinforcement Learning for High-Frequency Multi-Market Making

High-frequency multi-market making is a liquidity-providing strategy that exercises cross-market latency arbitrage in order to simultaneously post multiple bids and asks in a fragmented market for a security or co-related securities, while maintaining a relatively low net position. By exploiting price discrepancies between markets, the strategy earns profit from the bid-ask spread for every trade against the risk of inventory, liquidity and adverse selection. We develop a multi-market simulation framework built over empirically verified heterogeneous agents, with a realistic market design and matching engine. We use it to design high-frequency market making agents based on deep attention recurrent Q-network architecture a with spatial and temporal attention module, to efficiently capture the non-linear features of the order book. We train heterogeneous market making agents, trading in the presence of other agents, with a simulation framework that employs independent Q-learning in a multi-agent deep reinforcement learning setting. We demonstrate the effectiveness of our agents in relation to traditional deep architecture and benchmark strategies using Deep Hawkes processes. We investigate the effect of latency and different market ecology on the market quality