Deep Reinforcement Learning for High-Frequency Market Making

Pankaj Kumar

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review


High-frequency market making is a algorithmic trading strategy in which an agent provides liquidity at the same time as quoting a bid price and an ask price on a security. The strategy reap profits in the form of the spread between the quoted price placed on the buy and sell prices. Due to complexity in inventory risk, counterparties to trades and information asymmetry, the understanding of high-frequency market making algorithms is relatively unexplored by academics across disciplines. In this paper, we develop realistic simulations of limit order markets and use them to design a high-frequency market making agent using Deep Recurrent Q-Networks. Our approach outperforms a prominent benchmark strategy from literature, which uses temporal-difference reinforcement learning to design market making agents. Using the simulation framework, we analyse how the maker-take fee, a feature of market design, affects market quality and the agent’s profitability. The agents successfully reproduce stylised facts in historical trade data from each simulation.
Original languageEnglish
Title of host publication14th Asian Conference on Machine Learning, ACML 2022
EditorsEmtiyaz Khan, Mehmet Gonen
Number of pages16
Place of PublicationCambridge, MA
PublisherML Research Press
Publication date2023
Publication statusPublished - 2023
Event14th Asian Conference on Machine Learning 2022 - Indian School of Business (ISB), Hyderabad, India
Duration: 12 Dec 202214 Dec 2022
Conference number: 14


Conference14th Asian Conference on Machine Learning 2022
LocationIndian School of Business (ISB)
Internet address
SeriesProceedings of Machine Learning Research


  • Agent-based models
  • Deep reinforcement learning
  • High-frequency trading

Cite this