Deep Reinforcement Learning for Market Making

Pankaj Kumar*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review


Market Making is high frequency trading strategy in which an agent provides liquidity simultaneously quoting a bid price and an ask price on an asset. Market Makers reaps profits in the form of the spread between the quoted price placed on the buy and sell prices. Due to complexity in inventory risk, counterparties to trades and information asymmetry, understanding of market making algorithms is relatively unexplored by academicians across disciplines. In this paper, we develop realistic simulations of limit order markets and use it to design a market making agent using Deep Recurrent Q-Networks. Our approach outperforms a prominent benchmark strategy from literature, which uses temporal-difference reinforcement learning to design market maker agents. The agents successfully reproduce stylized facts in historical trade data from each simulation.
Original languageEnglish
Title of host publicationAAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems
EditorsBo An, Neil Yorke-Smith, Amal El Fallah Seghrouchni, Gita Sukthankar
Number of pages3
Place of PublicationRichland, SC
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems
Publication date2020
ISBN (Print)9781450375184
Publication statusPublished - 2020
Event19th International Conference on Autonomous Agents and MultiAgent Systems. AAMAS 2020 - Auckland University of Technology, Auckland, New Zealand
Duration: 9 May 202013 May 2020
Conference number: 19


Conference19th International Conference on Autonomous Agents and MultiAgent Systems. AAMAS 2020
LocationAuckland University of Technology
CountryNew Zealand
Internet address


  • Deep reinforcement learning
  • Market making
  • Limit order books
  • High frequency trading strategies
  • Agent based models

Cite this