# Gradient Boosting Reinforcement Learning (GBRL)
GBRL is a Python-based Gradient Boosting Trees (GBT) library, similar to popular packages such as [XGBoost](https://xgboost.readthedocs.io/en/stable/), [CatBoost](https://catboost.ai/), but specifically designed and optimized for reinforcement learning (RL). GBRL is implemented in C++/CUDA aimed to seamlessly integrate within popular RL libraries.
[](https://github.com/NVlabs/gbrl/blob/master/LICENSE)
[](https://badge.fury.io/py/gbrl)
## Overview
GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets. The following diagram illustrates how GBRL uses gradient boosting trees in RL:
GBRL features a shared tree-based structure for policy and value functions, significantly reducing memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.
## Key Features:
- GBT Tailored for RL: GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets.
- Optimized Actor-Critic Architecture: GBRL features a shared tree-based structure for policy and value functions. This significantly reduces memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.
- Hardware Acceleration: GBRL leverages CUDA for hardware-accelerated computation, ensuring efficiency and speed.
- Seamless Integration: GBRL is designed for easy integration with popular RL libraries. We implemented GBT-based actor-critic algorithm implementations (A2C, PPO, and AWR) in stable_baselines3 [GBRL_SB3](https://github.com/NVlabs/gbrl_sb3).
## Performance
The following results, obtained using the `GBRL_SB3` repository, demonstrate the performance of PPO with GBRL compared to neural-networks across various scenarios and environments:
## Getting started
### Dependencies
- Python 3.9 or higher
### Installation
GBRL provides pre-compiled binaries for easy installation. Choose **one** of the following options:
**CPU-only installation** (default):
```pip install gbrl```
**GPU-enabled installation** (requires CUDA 12 runtime libraries):
```pip install gbrl-gpu```
For further installation details and dependencies see the documentation.
### Usage Example
For a detailed usage example, see `tutorial.ipynb`
## Current Supported Features
### Tree Fitting
- Greedy (Depth-wise) tree building - (CPU/GPU)
- Oblivious (Symmetric) tree building - (CPU/GPU)
- L2 split score - (CPU/GPU)
- Cosine split score - (CPU/GPU)
- Uniform based candidate generation - (CPU/GPU)
- Quantile based candidate generation - (CPU/GPU)
- Supervised learning fitting / Multi-iteration fitting - (CPU/GPU)
- MultiRMSE loss (only)
- Categorical inputs
- Input feature weights - (CPU/GPU)
### GBT Inference
- SGD optimizer - (CPU/GPU)
- ADAM optimizer - (CPU only)
- Control Variates (gradient variance reduction technique) - (CPU only)
- Shared Tree for policy and value function - (CPU/GPU)
- Linear and constant learning rate scheduler - (CPU/GPU only constant)
- Support for up to two different optimizers (e.g, policy/value) - **(CPU/GPU if both are SGD)
- SHAP value calculation
# Documentation
For comprehensive documentation, visit the [GBRL documentation](https://nvlabs.github.io/gbrl/).
# Citation
```
@article{gbrl,
title={Gradient Boosting Reinforcement Learning},
author={Benjamin Fuhrer, Chen Tessler, Gal Dalal},
year={2024},
eprint={2407.08250},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2407.08250},
}
```
# Licenses
Copyright © 2024, NVIDIA Corporation. All rights reserved.
This work is made available under the NVIDIA Source Code License-NC. Click [here](https://github.com/NVlabs/gbrl/blob/master/LICENSE). to view a copy of this license.
Raw data
{
"_id": null,
"home_page": null,
"name": "gbrl",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Benjamin Fuhrer, Chen Tessler, Gal Dalal",
"author_email": "Benjamin Fuhrer <bfuhrer@nvidia.com>, Chen Tessler <ctessler@nvidia.com>, Gal Dalal <galal@nvidia.com>",
"download_url": "https://files.pythonhosted.org/packages/aa/97/faeb3288af97672e7335f068feeaa84a2c94548f2b26aa19565cc407cf2a/gbrl-1.1.1.tar.gz",
"platform": null,
"description": "# Gradient Boosting Reinforcement Learning (GBRL)\nGBRL is a Python-based Gradient Boosting Trees (GBT) library, similar to popular packages such as [XGBoost](https://xgboost.readthedocs.io/en/stable/), [CatBoost](https://catboost.ai/), but specifically designed and optimized for reinforcement learning (RL). GBRL is implemented in C++/CUDA aimed to seamlessly integrate within popular RL libraries. \n\n[](https://github.com/NVlabs/gbrl/blob/master/LICENSE)\n[](https://badge.fury.io/py/gbrl)\n\n## Overview\n\nGBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets. The following diagram illustrates how GBRL uses gradient boosting trees in RL:\n\n\n\nGBRL features a shared tree-based structure for policy and value functions, significantly reducing memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.\n\n## Key Features: \n- GBT Tailored for RL: GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets.\n- Optimized Actor-Critic Architecture: GBRL features a shared tree-based structure for policy and value functions. This significantly reduces memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.\n- Hardware Acceleration: GBRL leverages CUDA for hardware-accelerated computation, ensuring efficiency and speed.\n- Seamless Integration: GBRL is designed for easy integration with popular RL libraries. We implemented GBT-based actor-critic algorithm implementations (A2C, PPO, and AWR) in stable_baselines3 [GBRL_SB3](https://github.com/NVlabs/gbrl_sb3). \n\n## Performance\n\nThe following results, obtained using the `GBRL_SB3` repository, demonstrate the performance of PPO with GBRL compared to neural-networks across various scenarios and environments:\n\n\n\n## Getting started\n### Dependencies\n- Python 3.9 or higher\n\n### Installation\nGBRL provides pre-compiled binaries for easy installation. Choose **one** of the following options:\n\n**CPU-only installation** (default): \n```pip install gbrl```\n\n**GPU-enabled installation** (requires CUDA 12 runtime libraries): \n```pip install gbrl-gpu```\n\nFor further installation details and dependencies see the documentation. \n\n### Usage Example\nFor a detailed usage example, see `tutorial.ipynb`\n\n## Current Supported Features\n### Tree Fitting\n- Greedy (Depth-wise) tree building - (CPU/GPU) \n- Oblivious (Symmetric) tree building - (CPU/GPU) \n- L2 split score - (CPU/GPU) \n- Cosine split score - (CPU/GPU) \n- Uniform based candidate generation - (CPU/GPU)\n- Quantile based candidate generation - (CPU/GPU)\n- Supervised learning fitting / Multi-iteration fitting - (CPU/GPU)\n - MultiRMSE loss (only)\n- Categorical inputs\n- Input feature weights - (CPU/GPU)\n### GBT Inference\n- SGD optimizer - (CPU/GPU)\n- ADAM optimizer - (CPU only)\n- Control Variates (gradient variance reduction technique) - (CPU only)\n- Shared Tree for policy and value function - (CPU/GPU)\n- Linear and constant learning rate scheduler - (CPU/GPU only constant)\n- Support for up to two different optimizers (e.g, policy/value) - **(CPU/GPU if both are SGD)\n- SHAP value calculation\n\n# Documentation \nFor comprehensive documentation, visit the [GBRL documentation](https://nvlabs.github.io/gbrl/).\n\n# Citation\n``` \n@article{gbrl,\n title={Gradient Boosting Reinforcement Learning},\n author={Benjamin Fuhrer, Chen Tessler, Gal Dalal},\n year={2024},\n eprint={2407.08250},\n archivePrefix={arXiv},\n primaryClass={cs.LG},\n url={https://arxiv.org/abs/2407.08250}, \n}\n```\n# Licenses\nCopyright \u00a9 2024, NVIDIA Corporation. All rights reserved.\n\nThis work is made available under the NVIDIA Source Code License-NC. Click [here](https://github.com/NVlabs/gbrl/blob/master/LICENSE). to view a copy of this license.\n\n",
"bugtrack_url": null,
"license": "NVIDIA Proprietary Software",
"summary": "Gradient Boosted Trees for RL",
"version": "1.1.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "8de9655b4bd2333fd148e99d402928ee5f7c5365c830d3942b1a435348e49381",
"md5": "ea1c3c9a4520f6181a808fc485369af6",
"sha256": "83d9368f07d3b1d52c6cd6a96c401c1b2663570499dc68fbc07365858c3eadac"
},
"downloads": -1,
"filename": "gbrl-1.1.1-cp310-cp310-manylinux_2_31_x86_64.whl",
"has_sig": false,
"md5_digest": "ea1c3c9a4520f6181a808fc485369af6",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.9",
"size": 839968,
"upload_time": "2025-08-13T14:42:03",
"upload_time_iso_8601": "2025-08-13T14:42:03.674430Z",
"url": "https://files.pythonhosted.org/packages/8d/e9/655b4bd2333fd148e99d402928ee5f7c5365c830d3942b1a435348e49381/gbrl-1.1.1-cp310-cp310-manylinux_2_31_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "bcf1dd51cbd1692011da3046856697abec93df564e41fc58941f895a54acd6cb",
"md5": "4b7389062ac31dbad3784b1cc8d9ae36",
"sha256": "8d60a36989c50ce5c13b78c1c4362fa5ac9cf611a0166e9641ed8077738c2b3a"
},
"downloads": -1,
"filename": "gbrl-1.1.1-cp310-cp310-win_amd64.whl",
"has_sig": false,
"md5_digest": "4b7389062ac31dbad3784b1cc8d9ae36",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.9",
"size": 614329,
"upload_time": "2025-08-13T15:17:09",
"upload_time_iso_8601": "2025-08-13T15:17:09.846341Z",
"url": "https://files.pythonhosted.org/packages/bc/f1/dd51cbd1692011da3046856697abec93df564e41fc58941f895a54acd6cb/gbrl-1.1.1-cp310-cp310-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "e089a7a9d04e23d38212da848abbfd7a354f144ed030e39074240bfa5c4da49b",
"md5": "f417e5dc98f57b6f1958d0132ee00951",
"sha256": "77b6d00baa48f1726cb372d2c9edd20e49093446da425be6621e007173a4a016"
},
"downloads": -1,
"filename": "gbrl-1.1.1-cp311-cp311-manylinux_2_31_x86_64.whl",
"has_sig": false,
"md5_digest": "f417e5dc98f57b6f1958d0132ee00951",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.9",
"size": 841219,
"upload_time": "2025-08-13T14:42:06",
"upload_time_iso_8601": "2025-08-13T14:42:06.372902Z",
"url": "https://files.pythonhosted.org/packages/e0/89/a7a9d04e23d38212da848abbfd7a354f144ed030e39074240bfa5c4da49b/gbrl-1.1.1-cp311-cp311-manylinux_2_31_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "ad7aa33fb2cb2d200cd31c3449974e2a403557c59f1d064f782e95ced9edd254",
"md5": "0cf5ce7b46b57ae701f91d2970e0e495",
"sha256": "2d36c63430ff5e1066679ab559cea0c092eb2360adf5d68397cf150030167d09"
},
"downloads": -1,
"filename": "gbrl-1.1.1-cp311-cp311-win_amd64.whl",
"has_sig": false,
"md5_digest": "0cf5ce7b46b57ae701f91d2970e0e495",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.9",
"size": 615151,
"upload_time": "2025-08-13T15:17:11",
"upload_time_iso_8601": "2025-08-13T15:17:11.163056Z",
"url": "https://files.pythonhosted.org/packages/ad/7a/a33fb2cb2d200cd31c3449974e2a403557c59f1d064f782e95ced9edd254/gbrl-1.1.1-cp311-cp311-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b0ec8c8b10f177297578029ce1f1794243b3e7ac01db4017612f8d100d442b8e",
"md5": "9780fea4f43490fcfd7f75a97c736f28",
"sha256": "a972f936b0b898e688bc08bdba2ed6f6c21e43464909c13b6738572ce24e1e9a"
},
"downloads": -1,
"filename": "gbrl-1.1.1-cp312-cp312-manylinux_2_31_x86_64.whl",
"has_sig": false,
"md5_digest": "9780fea4f43490fcfd7f75a97c736f28",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.9",
"size": 840839,
"upload_time": "2025-08-13T14:42:09",
"upload_time_iso_8601": "2025-08-13T14:42:09.679012Z",
"url": "https://files.pythonhosted.org/packages/b0/ec/8c8b10f177297578029ce1f1794243b3e7ac01db4017612f8d100d442b8e/gbrl-1.1.1-cp312-cp312-manylinux_2_31_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "61c9bcab237492a8f1ce5fd1b205dbc07eea9feb37c6b4f9301315a96a768b21",
"md5": "418f1bf5b5f3b007bfc89eb64cf46baa",
"sha256": "b763045662e3e21e4b7d3451ee40ea5e70c9aa030743b9575ac05187335b83b5"
},
"downloads": -1,
"filename": "gbrl-1.1.1-cp312-cp312-win_amd64.whl",
"has_sig": false,
"md5_digest": "418f1bf5b5f3b007bfc89eb64cf46baa",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.9",
"size": 615521,
"upload_time": "2025-08-13T15:17:12",
"upload_time_iso_8601": "2025-08-13T15:17:12.212200Z",
"url": "https://files.pythonhosted.org/packages/61/c9/bcab237492a8f1ce5fd1b205dbc07eea9feb37c6b4f9301315a96a768b21/gbrl-1.1.1-cp312-cp312-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "dadc45dda8809bd0eb66c0bc30c40a334edc91e5ef9fc1fe1657553c01f5c415",
"md5": "e5ff19fb39c1623a3b0c03b40ad2ec54",
"sha256": "4edfec6f7c900e016d9adc002892cba29b045253b7d5e46f82fed45261480d99"
},
"downloads": -1,
"filename": "gbrl-1.1.1-cp39-cp39-manylinux_2_31_x86_64.whl",
"has_sig": false,
"md5_digest": "e5ff19fb39c1623a3b0c03b40ad2ec54",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.9",
"size": 839829,
"upload_time": "2025-08-13T14:42:13",
"upload_time_iso_8601": "2025-08-13T14:42:13.909787Z",
"url": "https://files.pythonhosted.org/packages/da/dc/45dda8809bd0eb66c0bc30c40a334edc91e5ef9fc1fe1657553c01f5c415/gbrl-1.1.1-cp39-cp39-manylinux_2_31_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "913a1acb3547cebea311f55ced8805f95346179eeb9496100f9fc8826b867518",
"md5": "f7f8651e850ad67735fbac43af70f3be",
"sha256": "54f2108fd6b240384c4abb2d0f293f6381c4858b0bacf805bae2287ef5788e82"
},
"downloads": -1,
"filename": "gbrl-1.1.1-cp39-cp39-win_amd64.whl",
"has_sig": false,
"md5_digest": "f7f8651e850ad67735fbac43af70f3be",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.9",
"size": 609950,
"upload_time": "2025-08-13T15:17:13",
"upload_time_iso_8601": "2025-08-13T15:17:13.277232Z",
"url": "https://files.pythonhosted.org/packages/91/3a/1acb3547cebea311f55ced8805f95346179eeb9496100f9fc8826b867518/gbrl-1.1.1-cp39-cp39-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "aa97faeb3288af97672e7335f068feeaa84a2c94548f2b26aa19565cc407cf2a",
"md5": "fde9fd95732df5ad52bd10ab8208ded7",
"sha256": "87f382da89d65ef6f8a3bfffa1a0a202a41f650eb81a1056b2beaf9712e89076"
},
"downloads": -1,
"filename": "gbrl-1.1.1.tar.gz",
"has_sig": false,
"md5_digest": "fde9fd95732df5ad52bd10ab8208ded7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 121552,
"upload_time": "2025-08-13T14:46:23",
"upload_time_iso_8601": "2025-08-13T14:46:23.688419Z",
"url": "https://files.pythonhosted.org/packages/aa/97/faeb3288af97672e7335f068feeaa84a2c94548f2b26aa19565cc407cf2a/gbrl-1.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-13 14:46:23",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "gbrl"
}
JSON