<!-- omit in toc -->
# QuantRL-Lab
A Python testbed for Reinforcement Learning in finance, designed to enable researchers and developers to experiment with and evaluate RL algorithms in financial contexts. The project emphasizes modularity and configurability, allowing users to tailor the environment, data sources, and algorithmic settings to their specific needs
## Table of Contents
- [Table of Contents](#table-of-contents)
- [Motivation](#motivation)
- [Example usage:](#example-usage)
- [Roadmap 🔄](#roadmap-)
- [Setup Guide](#setup-guide)
- [Development \& Publishing](#development--publishing)
- [Literature Review](#literature-review)
### Motivation
**Addressing the Monolithic Environment Problem**
Most existing RL frameworks for finance suffer from tightly coupled, monolithic designs where action spaces, observation spaces, and reward functions are hardcoded into the environment initialization. This creates several critical limitations:
- **Limited Experimentation**: Users cannot easily test different reward formulations or action spaces without doing a lot of rewriting of the environments
- **Poor Scalability**: Adding new asset classes, trading strategies, or market conditions requires significant code restructuring
- **Reduced Reproducibility**: Inconsistent interfaces across different environment configurations make fair comparisons difficult
- **Development Overhead**: Simple modifications like testing different reward functions or adding new observation features require extensive refactoring
<u>The framework tries to demonstrate the following workflow:</u>
1. **Flexible Data Acquisition**: Aggregate market data from multiple heterogeneous sources with unified interfaces
2. **Feature Engineering**: Systematic selection and analysis of technical indicators (based on vectorized backtesting) for optimal signal generation
3. **Data Processing**: Enrich datasets with technical indicators and sentiment analysis from news sources
4. **Environment Configuration**: Define trading environments with customizable parameters (portfolio allocation, transaction costs, slippage, observation windows)
5. **Algorithm Training & Tuning**: Execute RL algorithm training with preset or configurable hyperparameters
6. **Performance Evaluation**: Assess model performance and action distribution
7. **Comparative Analysis**: Generate detailed performance reports
```mermaid
flowchart LR
A[Data Acquisition<br/>Multiple Sources] --> B[Data Processing<br/>Technical Indicators & Sentiment]
A -.-> C[Feature Engineering<br/>& Selection]
C -.-> B
B --> D[Environment Configuration<br/>Action/Reward/Observation Strategies]
D --> E[Algorithm Training<br/>RL Policy Learning]
E -.-> F[Hyperparameter Tuning<br/>Optuna Optimization]
F -.-> E
E --> G[Performance Evaluation<br/>Metrics & Action Analysis]
G --> H[Comparative Analysis<br/>Strategy Reports]
style A fill:#e1f5fe
style B fill:#f3e5f5
style C fill:#fff3e0
style D fill:#e8f5e8
style E fill:#fce4ec
style F fill:#fff8e1
style G fill:#e0f2f1
style H fill:#f1f8e9
classDef optional stroke-dasharray: 5 5
class C,F optional
```
```mermaid
graph LR
A[Start: Raw OHLCV DataFrame] --> B{Initialize DataProcessor};
B --> C[append_technical_indicators];
C --> D{News Data Provided?};
D -- Yes --> E[append_news_sentiment_data];
E --> F[Merge DataFrames];
D -- No --> F;
F --> G[convert_columns_to_numeric];
G --> H[dropna];
H --> I{Split Config Provided?};
I -- Yes --> J[_split_data];
J --> K[Drop Date Column from Splits];
K --> L[End: Dictionary of Split DataFrames & Metadata];
I -- No --> M[Drop Date Column];
M --> N[End: Single Processed DataFrame & Metadata];
```
---
### Example usage:
```python
# Easily swappable strategies for experimentation
# For in depth example, please refer to the backtesting_example.ipynb
sample_env_config = BacktestRunner.create_env_config_factory(
train_data=train_data_df,
test_data=test_data_df,
action_strategy=action_strategy,
reward_strategy=reward_strategies["conservative"],
observation_strategy=observation_strategy,
initial_balance=100000.0,
transaction_cost_pct=0.001,
window_size=20
)
runner = BacktestRunner(verbose=1)
# Single experiment
results = runner.run_single_experiment(
SAC, # Algorithm to use
sample_env_config,
# config=custom_sac_config, # an optional input arg
total_timesteps=50000, # Total timesteps for training
num_eval_episodes=3
)
BacktestRunner.inspect_single_experiment(results)
# More combinations
presets = ["default", "explorative", "conservative"]
algorithms = [PPO, A2C, SAC]
comprehensive_results = runner.run_comprehensive_backtest(
algorithms=algorithms,
env_configs=env_configs,
presets=presets,
# custom_configs=custom_configs, # either use presets or customize config by yourself
total_timesteps=50000,
n_envs=4,
num_eval_episodes=3
)
```
For more detailed use cases, please refer to the notebooks:
- Feature and window size selection: [`notebooks/feature_selection.ipynb`](notebooks/feature_selection.ipynb)
- Data processing example: [`notebooks/data_processing.ipynb`](notebooks/data_processing.ipynb)
- Backtesting: [`notebooks/backtesting_example.ipynb`](notebooks/backtesting_example.ipynb)
- Hyperparameter tuning for stablebaseline algo: [`notebooks/hyperparameter_tuning.ipynb`](notebooks/hyperparameter_tuning.ipynb)
- LLM hedge pair screener (for upcoming multi stock env): [`notebooks/llm_hedge_screener.ipynb`](notebooks/llm_hedge_screener.ipynb)
---
### Roadmap 🔄
- **Data Source Expansion**:
- Complete Integration for more (free) data sources
- Add Cryto data support
- Add OANDA forex data support
- **Technical Indicators**:
- Add more indicators (Ichimoku, Williams %R, CCI, etc.)
- **Trading Environments**:
- (In-progress) Multi-stock trading environment with hedging pair capabilities
- **Alternative Data for consideration in observable space**:
- Fundamental data (earnings, balance sheets, income statements, cash flow)
- Macroeconomic indicators (GDP, inflation, unemployment, interest rates)
- Economic calendar events
- Sector performance data
---
### Setup Guide
1. Clone the Repository
```bash
git clone https://github.com/whanyu1212/QuantRL-Lab.git
```
2. Install Poetry for dependency management
```bash
curl -sSL https://install.python-poetry.org | python3 -
```
3. Sync dependencies (It also installs the current project in dev mode)
```bash
poetry install
```
4. Activate virtual environment (Note that the `shell` command is deprecated in the latest poetry version)
```bash
poetry env activate
# a venv path will be printed in the terminal, just copy and run it
# e.g.,
source /home/codespace/.cache/pypoetry/virtualenvs/multi-agent-quant-cj6_z41n-py3.12/bin/activate
```
5. Install jupyter kernel
```bash
# You can change the name and display name according to your preference
python -m ipykernel install --user --name multi-agent-quant --display-name "Multi Agent Quant"
```
6. Set up environment variables
```bash
# Copy the example environment file
cp .env.example .env
# Open .env file and replace the placeholder values with your actual credentials
# You can use any text editor, here using VS Code
code .env
```
Make sure to replace all placeholder values in the `.env` file with your actual API keys and credentials. Never commit the `.env` file to version control.
<br>
7. Set up pre-commit hooks
```bash
# Install pre-commit
poetry add pre-commit
# Install the git hooks
pre-commit install
# Optional: run pre-commit on all files
pre-commit run --all-files
```
The pre-commit hooks will check for:
- Code formatting (black)
- Import sorting (isort)
- Code linting (flake8)
- Docstring formatting (docformatter)
- Basic file checks (trailing whitespace, YAML validation, etc.)
To skip pre-commit hooks temporarily:
```bash
git commit -m "your message" --no-verify
```
For more details, please refer to `.pre-commit-config.yaml` file.
---
### Development & Publishing
**CI/CD Pipeline**
This project uses GitHub Actions for continuous integration and deployment:
- **CI (Continuous Integration)**: Automatically runs tests, linting, and compatibility checks on every push/PR
- **CD (Continuous Deployment)**: Automatically publishes to PyPI when a new release is created
**Publishing to PyPI**
The package is published to PyPI for easy installation. For maintainers looking to publish updates:
1. **Quick release** (recommended):
```bash
# For bug fixes (0.1.0 → 0.1.1)
./release.sh patch
# For new features (0.1.0 → 0.2.0)
./release.sh minor
# For breaking changes (0.1.0 → 1.0.0)
./release.sh major
```
2. **Create GitHub Release**: After the script completes, create a release on GitHub to trigger automatic publishing to PyPI
For detailed setup instructions and troubleshooting, see:
- 📖 [Complete Publishing Guide](docs/PYPI_PUBLISHING.md)
- ✅ [Publishing Checklist](docs/PUBLISHING_CHECKLIST.md)
- 📝 [CD Setup Summary](docs/CD_SETUP_SUMMARY.md)
**Installation from PyPI** (after first release):
```bash
pip install quantrl-lab
```
---
### Literature Review
Raw data
{
"_id": null,
"home_page": null,
"name": "quantrl-lab",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "reinforcement-learning, trading, finance, quantitative, backtesting, gymnasium, stable-baselines3",
"author": "whanyu1212",
"author_email": "whanyu1212@hotmail.com",
"download_url": "https://files.pythonhosted.org/packages/3e/96/bdb547c735049e568b736da65e2869130be07e89b44d176bea3d53b2b0e7/quantrl_lab-0.1.0.tar.gz",
"platform": null,
"description": "<!-- omit in toc -->\n# QuantRL-Lab\nA Python testbed for Reinforcement Learning in finance, designed to enable researchers and developers to experiment with and evaluate RL algorithms in financial contexts. The project emphasizes modularity and configurability, allowing users to tailor the environment, data sources, and algorithmic settings to their specific needs\n\n## Table of Contents\n- [Table of Contents](#table-of-contents)\n - [Motivation](#motivation)\n - [Example usage:](#example-usage)\n - [Roadmap \ud83d\udd04](#roadmap-)\n - [Setup Guide](#setup-guide)\n - [Development \\& Publishing](#development--publishing)\n - [Literature Review](#literature-review)\n\n### Motivation\n\n**Addressing the Monolithic Environment Problem**\n\nMost existing RL frameworks for finance suffer from tightly coupled, monolithic designs where action spaces, observation spaces, and reward functions are hardcoded into the environment initialization. This creates several critical limitations:\n\n- **Limited Experimentation**: Users cannot easily test different reward formulations or action spaces without doing a lot of rewriting of the environments\n- **Poor Scalability**: Adding new asset classes, trading strategies, or market conditions requires significant code restructuring\n- **Reduced Reproducibility**: Inconsistent interfaces across different environment configurations make fair comparisons difficult\n- **Development Overhead**: Simple modifications like testing different reward functions or adding new observation features require extensive refactoring\n\n\n<u>The framework tries to demonstrate the following workflow:</u>\n1. **Flexible Data Acquisition**: Aggregate market data from multiple heterogeneous sources with unified interfaces\n2. **Feature Engineering**: Systematic selection and analysis of technical indicators (based on vectorized backtesting) for optimal signal generation\n3. **Data Processing**: Enrich datasets with technical indicators and sentiment analysis from news sources\n4. **Environment Configuration**: Define trading environments with customizable parameters (portfolio allocation, transaction costs, slippage, observation windows)\n5. **Algorithm Training & Tuning**: Execute RL algorithm training with preset or configurable hyperparameters\n6. **Performance Evaluation**: Assess model performance and action distribution\n7. **Comparative Analysis**: Generate detailed performance reports\n\n\n```mermaid\nflowchart LR\n A[Data Acquisition<br/>Multiple Sources] --> B[Data Processing<br/>Technical Indicators & Sentiment]\n A -.-> C[Feature Engineering<br/>& Selection]\n C -.-> B\n B --> D[Environment Configuration<br/>Action/Reward/Observation Strategies]\n D --> E[Algorithm Training<br/>RL Policy Learning]\n E -.-> F[Hyperparameter Tuning<br/>Optuna Optimization]\n F -.-> E\n E --> G[Performance Evaluation<br/>Metrics & Action Analysis]\n G --> H[Comparative Analysis<br/>Strategy Reports]\n\n style A fill:#e1f5fe\n style B fill:#f3e5f5\n style C fill:#fff3e0\n style D fill:#e8f5e8\n style E fill:#fce4ec\n style F fill:#fff8e1\n style G fill:#e0f2f1\n style H fill:#f1f8e9\n\n classDef optional stroke-dasharray: 5 5\n class C,F optional\n```\n```mermaid\ngraph LR\n A[Start: Raw OHLCV DataFrame] --> B{Initialize DataProcessor};\n B --> C[append_technical_indicators];\n C --> D{News Data Provided?};\n D -- Yes --> E[append_news_sentiment_data];\n E --> F[Merge DataFrames];\n D -- No --> F;\n F --> G[convert_columns_to_numeric];\n G --> H[dropna];\n H --> I{Split Config Provided?};\n I -- Yes --> J[_split_data];\n J --> K[Drop Date Column from Splits];\n K --> L[End: Dictionary of Split DataFrames & Metadata];\n I -- No --> M[Drop Date Column];\n M --> N[End: Single Processed DataFrame & Metadata];\n```\n---\n\n\n### Example usage:\n\n```python\n# Easily swappable strategies for experimentation\n# For in depth example, please refer to the backtesting_example.ipynb\n\nsample_env_config = BacktestRunner.create_env_config_factory(\n train_data=train_data_df,\n test_data=test_data_df,\n action_strategy=action_strategy,\n reward_strategy=reward_strategies[\"conservative\"],\n observation_strategy=observation_strategy,\n initial_balance=100000.0,\n transaction_cost_pct=0.001,\n window_size=20\n)\n\nrunner = BacktestRunner(verbose=1)\n\n# Single experiment\nresults = runner.run_single_experiment(\n SAC, # Algorithm to use\n sample_env_config,\n # config=custom_sac_config, # an optional input arg\n total_timesteps=50000, # Total timesteps for training\n num_eval_episodes=3\n)\n\nBacktestRunner.inspect_single_experiment(results)\n\n# More combinations\npresets = [\"default\", \"explorative\", \"conservative\"]\n\nalgorithms = [PPO, A2C, SAC]\n\ncomprehensive_results = runner.run_comprehensive_backtest(\n algorithms=algorithms,\n env_configs=env_configs,\n presets=presets,\n # custom_configs=custom_configs, # either use presets or customize config by yourself\n total_timesteps=50000,\n n_envs=4,\n num_eval_episodes=3\n)\n```\n\nFor more detailed use cases, please refer to the notebooks:\n- Feature and window size selection: [`notebooks/feature_selection.ipynb`](notebooks/feature_selection.ipynb)\n- Data processing example: [`notebooks/data_processing.ipynb`](notebooks/data_processing.ipynb)\n- Backtesting: [`notebooks/backtesting_example.ipynb`](notebooks/backtesting_example.ipynb)\n- Hyperparameter tuning for stablebaseline algo: [`notebooks/hyperparameter_tuning.ipynb`](notebooks/hyperparameter_tuning.ipynb)\n- LLM hedge pair screener (for upcoming multi stock env): [`notebooks/llm_hedge_screener.ipynb`](notebooks/llm_hedge_screener.ipynb)\n\n\n\n---\n\n### Roadmap \ud83d\udd04\n- **Data Source Expansion**:\n - Complete Integration for more (free) data sources\n - Add Cryto data support\n - Add OANDA forex data support\n- **Technical Indicators**:\n - Add more indicators (Ichimoku, Williams %R, CCI, etc.)\n- **Trading Environments**:\n - (In-progress) Multi-stock trading environment with hedging pair capabilities\n- **Alternative Data for consideration in observable space**:\n - Fundamental data (earnings, balance sheets, income statements, cash flow)\n - Macroeconomic indicators (GDP, inflation, unemployment, interest rates)\n - Economic calendar events\n - Sector performance data\n\n---\n\n### Setup Guide\n\n1. Clone the Repository\n```bash\ngit clone https://github.com/whanyu1212/QuantRL-Lab.git\n```\n\n2. Install Poetry for dependency management\n```bash\ncurl -sSL https://install.python-poetry.org | python3 -\n```\n\n3. Sync dependencies (It also installs the current project in dev mode)\n```bash\npoetry install\n```\n\n4. Activate virtual environment (Note that the `shell` command is deprecated in the latest poetry version)\n```bash\npoetry env activate\n# a venv path will be printed in the terminal, just copy and run it\n# e.g.,\nsource /home/codespace/.cache/pypoetry/virtualenvs/multi-agent-quant-cj6_z41n-py3.12/bin/activate\n```\n\n5. Install jupyter kernel\n```bash\n# You can change the name and display name according to your preference\npython -m ipykernel install --user --name multi-agent-quant --display-name \"Multi Agent Quant\"\n```\n\n6. Set up environment variables\n```bash\n# Copy the example environment file\ncp .env.example .env\n\n# Open .env file and replace the placeholder values with your actual credentials\n# You can use any text editor, here using VS Code\ncode .env\n```\n\nMake sure to replace all placeholder values in the `.env` file with your actual API keys and credentials. Never commit the `.env` file to version control.\n\n<br>\n\n7. Set up pre-commit hooks\n```bash\n# Install pre-commit\npoetry add pre-commit\n\n# Install the git hooks\npre-commit install\n\n# Optional: run pre-commit on all files\npre-commit run --all-files\n```\n\nThe pre-commit hooks will check for:\n- Code formatting (black)\n- Import sorting (isort)\n- Code linting (flake8)\n- Docstring formatting (docformatter)\n- Basic file checks (trailing whitespace, YAML validation, etc.)\n\nTo skip pre-commit hooks temporarily:\n```bash\ngit commit -m \"your message\" --no-verify\n```\n\nFor more details, please refer to `.pre-commit-config.yaml` file.\n\n---\n\n### Development & Publishing\n\n**CI/CD Pipeline**\n\nThis project uses GitHub Actions for continuous integration and deployment:\n- **CI (Continuous Integration)**: Automatically runs tests, linting, and compatibility checks on every push/PR\n- **CD (Continuous Deployment)**: Automatically publishes to PyPI when a new release is created\n\n**Publishing to PyPI**\n\nThe package is published to PyPI for easy installation. For maintainers looking to publish updates:\n\n1. **Quick release** (recommended):\n```bash\n# For bug fixes (0.1.0 \u2192 0.1.1)\n./release.sh patch\n\n# For new features (0.1.0 \u2192 0.2.0)\n./release.sh minor\n\n# For breaking changes (0.1.0 \u2192 1.0.0)\n./release.sh major\n```\n\n2. **Create GitHub Release**: After the script completes, create a release on GitHub to trigger automatic publishing to PyPI\n\nFor detailed setup instructions and troubleshooting, see:\n- \ud83d\udcd6 [Complete Publishing Guide](docs/PYPI_PUBLISHING.md)\n- \u2705 [Publishing Checklist](docs/PUBLISHING_CHECKLIST.md)\n- \ud83d\udcdd [CD Setup Summary](docs/CD_SETUP_SUMMARY.md)\n\n**Installation from PyPI** (after first release):\n```bash\npip install quantrl-lab\n```\n\n---\n\n### Literature Review\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A lightweight Python library for developing and backtesting RL Agents in financial markets",
"version": "0.1.0",
"project_urls": null,
"split_keywords": [
"reinforcement-learning",
" trading",
" finance",
" quantitative",
" backtesting",
" gymnasium",
" stable-baselines3"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d9f21550d1be5373af2dffd4b5f8ea0a05139617c887aae731d9f100f9b0d200",
"md5": "2c2380396e3b5d00051115f32632d70e",
"sha256": "eea2420b91a1cec47329ae9752ddfd88d0f74beae3d089998fb07e50f02a6e6a"
},
"downloads": -1,
"filename": "quantrl_lab-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2c2380396e3b5d00051115f32632d70e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 117205,
"upload_time": "2025-10-16T15:19:36",
"upload_time_iso_8601": "2025-10-16T15:19:36.539168Z",
"url": "https://files.pythonhosted.org/packages/d9/f2/1550d1be5373af2dffd4b5f8ea0a05139617c887aae731d9f100f9b0d200/quantrl_lab-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3e96bdb547c735049e568b736da65e2869130be07e89b44d176bea3d53b2b0e7",
"md5": "36bd4bf1da131bb59e40f813f553bbc6",
"sha256": "261866cac7eec6c2de55cc4350698ca92fe6d69bb5c335f739e4bb014ae1c4eb"
},
"downloads": -1,
"filename": "quantrl_lab-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "36bd4bf1da131bb59e40f813f553bbc6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 91249,
"upload_time": "2025-10-16T15:19:37",
"upload_time_iso_8601": "2025-10-16T15:19:37.693092Z",
"url": "https://files.pythonhosted.org/packages/3e/96/bdb547c735049e568b736da65e2869130be07e89b44d176bea3d53b2b0e7/quantrl_lab-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-16 15:19:37",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "quantrl-lab"
}
JSON