# Gradient Boosting Reinforcement Learning (GBRL)
GBRL is a Python-based Gradient Boosting Trees (GBT) library, similar to popular packages such as [XGBoost](https://xgboost.readthedocs.io/en/stable/), [CatBoost](https://catboost.ai/), but specifically designed and optimized for reinforcement learning (RL). GBRL is implemented in C++/CUDA aimed to seamlessly integrate within popular RL libraries.
[](https://github.com/NVlabs/gbrl/blob/master/LICENSE)
[](https://badge.fury.io/py/gbrl)
<!-- [](https://codecov.io/gh/Nvlabs/gbrl)
[](https://codecov.io/gh/Nvlabs/gbrl) -->
## Overview
GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets. The following diagram illustrates how GBRL uses gradient boosting trees in RL:
GBRL features a shared tree-based structure for policy and value functions, significantly reducing memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.
## Key Features:
- GBT Tailored for RL: GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets.
- Optimized Actor-Critic Architecture: GBRL features a shared tree-based structure for policy and value functions. This significantly reduces memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.
- Hardware Acceleration: GBRL leverages CUDA for hardware-accelerated computation, ensuring efficiency and speed.
- Seamless Integration: GBRL is designed for easy integration with popular RL libraries. We implemented GBT-based actor-critic algorithm implementations (A2C, PPO, and AWR) in stable_baselines3 [GBRL_SB3](https://github.com/NVlabs/gbrl_sb3).
## Performance
The following results, obtained using the `GBRL_SB3` repository, demonstrate the performance of PPO with GBRL compared to neural-networks across various scenarios and environments:
## Getting started
### Prerequisites
- Python 3.9 or higher
- LLVM and OpenMP (macOS).
### Installation
To install GBRL via pip, use the following command:
```
pip install gbrl
```
For further installation details and dependencies see the documentation.
### Usage Example
For a detailed usage example, see `tutorial.ipynb`
## Current Supported Features
### Tree Fitting
- Greedy (Depth-wise) tree building - (CPU/GPU)
- Oblivious (Symmetric) tree building - (CPU/GPU)
- L2 split score - (CPU/GPU)
- Cosine split score - (CPU/GPU)
- Uniform based candidate generation - (CPU/GPU)
- Quantile based candidate generation - (CPU/GPU)
- Supervised learning fitting / Multi-iteration fitting - (CPU/GPU)
- MultiRMSE loss (only)
- Categorical inputs
- Input feature weights - (CPU/GPU)
### GBT Inference
- SGD optimizer - (CPU/GPU)
- ADAM optimizer - (CPU only)
- Control Variates (gradient variance reduction technique) - (CPU only)
- Shared Tree for policy and value function - (CPU/GPU)
- Linear and constant learning rate scheduler - (CPU/GPU only constant)
- Support for up to two different optimizers (e.g, policy/value) - **(CPU/GPU if both are SGD)
- SHAP value calculation
# Documentation
For comprehensive documentation, visit the [GBRL documentation](https://nvlabs.github.io/gbrl/).
# Citation
```
@article{gbrl,
title={Gradient Boosting Reinforcement Learning},
author={Benjamin Fuhrer, Chen Tessler, Gal Dalal},
year={2024},
eprint={2407.08250},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2407.08250},
}
```
# Licenses
Copyright © 2024, NVIDIA Corporation. All rights reserved.
This work is made available under the NVIDIA Source Code License-NC. Click [here](https://nvlabs.github.io/gbrl/license.htm). to view a copy of this license.
Raw data
{
"_id": null,
"home_page": null,
"name": "gbrl",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Benjamin Fuhrer, Chen Tessler, Gal Dalal",
"author_email": "Benjamin Fuhrer <bfuhrer@nvidia.com>, Chen Tessler <ctessler@nvidia.com>, Gal Dalal <galal@nvidia.com>",
"download_url": "https://files.pythonhosted.org/packages/72/f5/9c1f44b7386b3a0afae51890aef925c3e9c79435d8db63173a0650911a0f/gbrl-1.0.7.tar.gz",
"platform": null,
"description": "# Gradient Boosting Reinforcement Learning (GBRL)\nGBRL is a Python-based Gradient Boosting Trees (GBT) library, similar to popular packages such as [XGBoost](https://xgboost.readthedocs.io/en/stable/), [CatBoost](https://catboost.ai/), but specifically designed and optimized for reinforcement learning (RL). GBRL is implemented in C++/CUDA aimed to seamlessly integrate within popular RL libraries. \n\n[](https://github.com/NVlabs/gbrl/blob/master/LICENSE)\n[](https://badge.fury.io/py/gbrl)\n<!-- [](https://codecov.io/gh/Nvlabs/gbrl)\n[](https://codecov.io/gh/Nvlabs/gbrl) -->\n\n## Overview\n\nGBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets. The following diagram illustrates how GBRL uses gradient boosting trees in RL:\n\n\n\nGBRL features a shared tree-based structure for policy and value functions, significantly reducing memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.\n\n## Key Features: \n- GBT Tailored for RL: GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets.\n- Optimized Actor-Critic Architecture: GBRL features a shared tree-based structure for policy and value functions. This significantly reduces memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.\n- Hardware Acceleration: GBRL leverages CUDA for hardware-accelerated computation, ensuring efficiency and speed.\n- Seamless Integration: GBRL is designed for easy integration with popular RL libraries. We implemented GBT-based actor-critic algorithm implementations (A2C, PPO, and AWR) in stable_baselines3 [GBRL_SB3](https://github.com/NVlabs/gbrl_sb3). \n\n## Performance\n\nThe following results, obtained using the `GBRL_SB3` repository, demonstrate the performance of PPO with GBRL compared to neural-networks across various scenarios and environments:\n\n\n\n## Getting started\n### Prerequisites\n- Python 3.9 or higher\n- LLVM and OpenMP (macOS).\n\n### Installation\nTo install GBRL via pip, use the following command:\n```\npip install gbrl\n```\n\nFor further installation details and dependencies see the documentation. \n\n### Usage Example\nFor a detailed usage example, see `tutorial.ipynb`\n\n## Current Supported Features\n### Tree Fitting\n- Greedy (Depth-wise) tree building - (CPU/GPU) \n- Oblivious (Symmetric) tree building - (CPU/GPU) \n- L2 split score - (CPU/GPU) \n- Cosine split score - (CPU/GPU) \n- Uniform based candidate generation - (CPU/GPU)\n- Quantile based candidate generation - (CPU/GPU)\n- Supervised learning fitting / Multi-iteration fitting - (CPU/GPU)\n - MultiRMSE loss (only)\n- Categorical inputs\n- Input feature weights - (CPU/GPU)\n### GBT Inference\n- SGD optimizer - (CPU/GPU)\n- ADAM optimizer - (CPU only)\n- Control Variates (gradient variance reduction technique) - (CPU only)\n- Shared Tree for policy and value function - (CPU/GPU)\n- Linear and constant learning rate scheduler - (CPU/GPU only constant)\n- Support for up to two different optimizers (e.g, policy/value) - **(CPU/GPU if both are SGD)\n- SHAP value calculation\n\n# Documentation \nFor comprehensive documentation, visit the [GBRL documentation](https://nvlabs.github.io/gbrl/).\n\n# Citation\n``` \n@article{gbrl,\n title={Gradient Boosting Reinforcement Learning},\n author={Benjamin Fuhrer, Chen Tessler, Gal Dalal},\n year={2024},\n eprint={2407.08250},\n archivePrefix={arXiv},\n primaryClass={cs.LG},\n url={https://arxiv.org/abs/2407.08250}, \n}\n```\n# Licenses\nCopyright \u00a9 2024, NVIDIA Corporation. All rights reserved.\n\nThis work is made available under the NVIDIA Source Code License-NC. Click [here](https://nvlabs.github.io/gbrl/license.htm). to view a copy of this license.\n\n",
"bugtrack_url": null,
"license": "NVIDIA Proprietary Software",
"summary": "Gradient Boosted Trees for RL",
"version": "1.0.7",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "72f59c1f44b7386b3a0afae51890aef925c3e9c79435d8db63173a0650911a0f",
"md5": "129b6bbcec813725e506238e8c969192",
"sha256": "627d952971273b3fa9203976bb3ca70a131347f1a5c32acf869f5afb7bee4597"
},
"downloads": -1,
"filename": "gbrl-1.0.7.tar.gz",
"has_sig": false,
"md5_digest": "129b6bbcec813725e506238e8c969192",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 107979,
"upload_time": "2025-01-23T19:53:57",
"upload_time_iso_8601": "2025-01-23T19:53:57.712625Z",
"url": "https://files.pythonhosted.org/packages/72/f5/9c1f44b7386b3a0afae51890aef925c3e9c79435d8db63173a0650911a0f/gbrl-1.0.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-23 19:53:57",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "gbrl"
}
JSON