RL for Trading — Key Papers
| Paper | Year | Contribution | Relevance |
|---|---|---|---|
| Moody & Saffell, "Learning to trade via direct RL" | 2001 | First RL trading agent using policy gradient methods | Historical foundation. Showed RL could learn trading policies before deep learning existed. |
| Deng et al., "Deep direct RL for financial signal representation" | 2017 | Deep RL for financial feature learning with autoencoder pre-training | Feature engineering insight: pre-training on price structure helps RL convergence. |
| Jiang et al., "Deep RL for portfolio management" | 2017 | Continuous portfolio weights with CNN encoder for price images | Action space design: continuous weights are more expressive than discrete slot selection. |
| Liang et al., "Adversarial deep RL in portfolio management" | 2018 | Adversarial training for robustness to market shocks | Generalization: adversarial perturbations during training improve walk-forward performance. |
| Yang et al., "Deep RL for automated stock trading (FinRL)" | 2020 | Open-source RL trading framework with benchmark datasets | Closest comparable system. Our architecture differs in NSE-specific cost modeling and validation rigor. |
| Liu et al., "FinRL: DRL framework for quantitative finance" | 2021 | Comprehensive FinRL library with multiple algorithms and markets | Baseline comparison: FinRL's default cost models are US-centric and underestimate Indian fees. |
| Fang et al., "Universal trading for order execution" | 2021 | Execution optimization with RL using universal value function | Market impact: our slippage model is simpler, but this is the direction for future work. |
| Sun et al., "DeepScalper: RL-based high-frequency trading" | 2022 | Intraday RL strategies with minute-bar data | Intraday extension: higher SNR but higher infrastructure requirements. |
Meta-Learning & Regime Adaptation
| Paper | Year | Contribution | Relevance |
|---|---|---|---|
| Finn et al., "Model-Agnostic Meta-Learning (MAML)" | 2017 | Gradient-based meta-learning for rapid adaptation | Core algorithm for our Priority 1 research direction: regime-aware meta-learning. |
| Li et al., "Meta-learning for financial trading" | 2020 | Meta-RL for market regime adaptation | Directly relevant: shows MAML can adapt trading policies across regimes. |
| Xu et al., "Meta-learning for trading with switching regimes" | 2022 | HMM + meta-policy for regime detection and adaptation | Hybrid approach: explicit regime detection + meta-learning. Promising for NSE. |
Indian Market Specific
| Paper | Year | Contribution | Relevance |
|---|---|---|---|
| Jain & Jain, "ML for Indian stock market prediction" | 2019 | NSE-specific ML approaches and feature selection | Feature relevance: confirms technical indicators have predictive power on NSE, but lower than US markets. |
| Gupta et al., "DRL for Indian equity portfolio" | 2021 | SAC/PPO on NIFTY50 with simplified cost models | Closest Indian RL work. Our work extends this with realistic costs, walk-forward validation, and multi-granularity data. |
| Mondal et al., "Algorithmic trading in Indian markets" | 2023 | Market microstructure analysis of NSE | Microstructure: bid-ask spreads, liquidity patterns, and impact costs on NSE. Informs our slippage model. |
Risk Management in RL
| Paper | Year | Contribution | Relevance |
|---|---|---|---|
| Tamar et al., "Risk-sensitive RL" | 2015 | CVaR optimization in MDPs | Theoretical foundation for risk-aware reward design. We use tail-risk-aware penalties. |
| Chow et al., "Risk-constrained RL with CVaR" | 2017 | Constrained optimization for tail-risk control | Constrained RL: hard constraints on drawdown rather than soft penalties. Future direction. |
| Dabney et al., "Distributional RL with quantile regression" | 2018 | QR-DQN for modeling full return distribution | Tail risk: modeling the full distribution rather than just mean Q-values. Relevant for Q20. |
Our Commentary
The RL-for-trading literature has grown rapidly, but several gaps remain:
- Cost realism: Most papers use simplified cost models (flat 0.1% or less). Indian markets have complex, non-linear fee structures that significantly impact strategy design.
- Validation rigor: Many papers report results on a single train/test split. Walk-forward validation with multiple folds and statistical testing is rare but essential.
- Market specificity: Papers trained on US equities rarely generalize to Indian markets due to different microstructure, liquidity patterns, and regulatory environments.
- Reproducibility: Few papers release code or exact hyperparameters. We aim to be transparent about methodology even if source code remains proprietary.