| Name | nxbench JSON |
| Version |
0.1.24
JSON |
| download |
| home_page | None |
| Summary | A centralized benchmarking suite to facilitate comparative profiling of tools across graph analytic libraries and datasets |
| upload_time | 2024-12-28 08:14:36 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | None |
| keywords |
networkx
graph
network
benchmarking
profiling
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
[](https://badge.fury.io/py/nxbench)
[](https://badge.fury.io/py/nxbench)
[](https://pre-commit.com/)
[](https://github.com/psf/black)
[](https://codecov.io/gh/dPys/nxbench)
# NxBench
<p align="center">
<img src="doc/_static/assets/nxbench_logo.png" alt="NxBench Logo" width="125"/>
</p>
**nxbench** is a comprehensive benchmarking suite designed to facilitate comparative profiling of graph analytic algorithms across NetworkX and compatible backends. Built using Prefect and Dask, nxbench places an emphasis on extensible and granular performance analysis, enabling developers and researchers to optimize their graph analysis workflows efficiently and reproducibly.
## Key Features
- **Cross-Backend Benchmarking**: Leverage NetworkX's backend system to profile algorithms across multiple implementations (NetworkX, nx-parallel, GraphBLAS, and CuGraph)
- **Configurable Suite**: YAML-based configuration for algorithms, datasets, and benchmarking parameters
- **Real-World Datasets**: Automated downloading and caching of networks and their metadata from NetworkRepository
- **Synthetic Graph Generation**: Support for generating benchmark graphs using any of NetworkX's built-in generators
- **Validation Framework**: Comprehensive result validation for correctness across implementations
- **Performance Monitoring**: Track execution time and memory usage with detailed metrics
- **Interactive Visualization**: Dynamic dashboard for exploring benchmark results using Plotly Dash
- **Flexible Storage**: SQL result storage with pandas integration for dowstream analysis
## Installation (Non-Docker Setup)
### Prerequisites
- **Python 3.10+**: Ensure you have a compatible Python environment.
- **PostgreSQL**: To run Prefect Orion with a persistent database, we recommend PostgreSQL for better concurrency and stability than an ephemeral in-memory database.
- **NetworkX Backend Installations**: To comparative benchmark graph algorithm performance across different [NetworkX backends](https://networkx.org/documentation/stable/reference/backends.html), you will need to install each backend.
#### Setting up PostgreSQL
In a terminal window:
1. **Install PostgreSQL**:
- On macOS (with Homebrew):
```bash
brew install postgresql
brew services start postgresql
```
- On Linux (Debian/Ubuntu):
```bash
sudo apt-get update && sudo apt-get install -y postgresql postgresql-contrib
sudo service postgresql start
```
- On Windows:
Download and run the [PostgreSQL installer](https://www.postgresql.org/download/windows/) and follow the prompts.
2. **Create a PostgreSQL User and Database**:
```bash
psql postgres
```
In the `psql` prompt, run:
```sql
CREATE USER prefect_user WITH PASSWORD 'pass';
CREATE DATABASE prefect_db OWNER prefect_user;
GRANT ALL PRIVILEGES ON DATABASE prefect_db TO prefect_user;
```
Exit the prompt with \q.
This sets up a prefect_user with password pass and a database named prefect_db.
#### Supported Backends
- NetworkX (default)
- nx-CuGraph (requires separate CuGraph installation and supported GPU hardware)
- GraphBLAS Algorithms (optional)
- nx-parallel (optional)
### Installing `nxbench`
In a new terminal window:
PyPi:
```bash
pip install nxbench
```
From source (local clone):
```bash
git clone https://github.com/dpys/nxbench.git
cd nxbench
make install
```
This should install nxbench and all required dependencies (including prefect, asyncpg, and related packages).
## Installation (Docker Setup)
Docker:
```bash
# CPU-only
docker-compose -f docker/docker-compose.cpu.yaml build
# With GPU
docker-compose -f docker/docker-compose.gpu.yaml build
```
## Quick Start
1. Configure your benchmarks in a yaml file (see `configs/example.yaml`):
```yaml
algorithms:
- name: "pagerank"
func: "networkx.pagerank"
params:
alpha: 0.85
groups: ["centrality"]
datasets:
- name: "karate"
source: "networkrepository"
```
2. Start an instance of an orion server in a separate terminal window:
```bash
export PREFECT_API_URL="http://127.0.0.1:4200/api"
export PREFECT_API_DATABASE_CONNECTION_URL="postgresql+asyncpg://prefect_user:pass@localhost:5432/prefect_db"
prefect server start
```
3. In the original terminal window, run benchmarks based on the configuration:
```bash
nxbench --config 'nxbench/configs/example.yaml' benchmark run
```
4. Export results:
```bash
nxbench --config 'nxbench/configs/example.yaml' benchmark export 'results/9e3e8baa4a3443c392dc8fee00373b11_20241220002902.json' --output-format csv --output-file 'results/results.csv' # convert benchmarked results from a run with hash `9e3e8baa4a3443c392dc8fee00373b11_20241220002902` into csv format.
```
5. View results:
```bash
nxbench viz serve # launch the interactive results visualization dashboard.
```
<p align="center">
<img src="doc/_static/assets/animation.gif" alt="Parallel Categories Animation" width="1000"/>
</p>
## Advanced Command Line Interface
The CLI provides comprehensive management of benchmarks, datasets, and visualization:
```bash
# Data Management
nxbench data download karate # download specific dataset
nxbench data list --category social # list available datasets
# Benchmarking
nxbench --config 'nxbench/configs/example.yaml' -vvv benchmark run # debug benchmark runs
nxbench --config 'nxbench/configs/example.yaml' benchmark export 'results/9e3e8baa4a3443c392dc8fee00373b11_20241220002902.json' --output-format sql --output-file 'results/benchmarks.sqlite' # export the results from a run with hash `9e3e8baa4a3443c392dc8fee00373b11_20241220002902` into a sql database
```
## Configuration
Benchmarks are configured through YAML files with the following structure:
```yaml
algorithms:
- name: "algorithm_name"
func: "fully.qualified.function.name"
params: {}
requires_directed: false
groups: ["category"]
validate_result: "validation.function"
datasets:
- name: "dataset_name"
source: "networkrepository"
params: {}
```
## Reproducible benchmarking through containerization
```bash
# Run benchmarks with GPU
NUM_GPU=1 docker-compose -f docker/docker-compose.gpu.yaml up nxbench
# Run benchmarks CPU-only
docker-compose -f docker/docker-compose.cpu.yaml up nxbench
# Start visualization dashboard
docker-compose -f docker/docker-compose.cpu.yaml up dashboard
# Run specific backend
docker-compose -f docker/docker-compose.cpu.yaml run --rm nxbench --config 'nxbench/configs/example.yaml' benchmark run --backend networkx
# Export results from a run with hash `9e3e8baa4a3443c392dc8fee00373b11_20241220002902`
docker-compose -f docker/docker-compose.cpu.yaml run --rm nxbench --config 'nxbench/configs/example.yaml' benchmark export 'nxbench_results/9e3e8baa4a3443c392dc8fee00373b11_20241220002902.json' --output-format csv --output-file 'nxbench_results/results.csv'
```
## Contributing
Contributions are welcome! Please read our [Contributing Guide](CONTRIBUTING.md) for details on:
- Code style guidelines
- Development setup
- Testing requirements
- Pull request process
## License
This project is licensed under the MIT License - see [LICENSE](LICENSE) for details.
## Acknowledgments
- NetworkX community for the core graph library and dispatching support
- NetworkRepository.com for harmonized dataset access
## Contact
For questions or suggestions:
- Open an issue on GitHub
- Email: <dpysalexander@gmail.com>
Raw data
{
"_id": null,
"home_page": null,
"name": "nxbench",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "networkx, graph, network, benchmarking, profiling",
"author": null,
"author_email": "Derek Alexander <dpysalexander@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/ed/ed/5da9fe9e1837b26a5f04c431b84cf2c07d92dd6f22eb01514c1befb7aaf7/nxbench-0.1.24.tar.gz",
"platform": null,
"description": "[](https://badge.fury.io/py/nxbench)\n[](https://badge.fury.io/py/nxbench)\n[](https://pre-commit.com/)\n[](https://github.com/psf/black)\n[](https://codecov.io/gh/dPys/nxbench)\n\n# NxBench\n\n<p align=\"center\">\n <img src=\"doc/_static/assets/nxbench_logo.png\" alt=\"NxBench Logo\" width=\"125\"/>\n</p>\n\n**nxbench** is a comprehensive benchmarking suite designed to facilitate comparative profiling of graph analytic algorithms across NetworkX and compatible backends. Built using Prefect and Dask, nxbench places an emphasis on extensible and granular performance analysis, enabling developers and researchers to optimize their graph analysis workflows efficiently and reproducibly.\n\n## Key Features\n\n- **Cross-Backend Benchmarking**: Leverage NetworkX's backend system to profile algorithms across multiple implementations (NetworkX, nx-parallel, GraphBLAS, and CuGraph)\n- **Configurable Suite**: YAML-based configuration for algorithms, datasets, and benchmarking parameters\n- **Real-World Datasets**: Automated downloading and caching of networks and their metadata from NetworkRepository\n- **Synthetic Graph Generation**: Support for generating benchmark graphs using any of NetworkX's built-in generators\n- **Validation Framework**: Comprehensive result validation for correctness across implementations\n- **Performance Monitoring**: Track execution time and memory usage with detailed metrics\n- **Interactive Visualization**: Dynamic dashboard for exploring benchmark results using Plotly Dash\n- **Flexible Storage**: SQL result storage with pandas integration for dowstream analysis\n\n## Installation (Non-Docker Setup)\n\n### Prerequisites\n\n- **Python 3.10+**: Ensure you have a compatible Python environment.\n- **PostgreSQL**: To run Prefect Orion with a persistent database, we recommend PostgreSQL for better concurrency and stability than an ephemeral in-memory database.\n- **NetworkX Backend Installations**: To comparative benchmark graph algorithm performance across different [NetworkX backends](https://networkx.org/documentation/stable/reference/backends.html), you will need to install each backend.\n\n#### Setting up PostgreSQL\n\nIn a terminal window:\n\n1. **Install PostgreSQL**:\n\n - On macOS (with Homebrew):\n\n ```bash\n brew install postgresql\n brew services start postgresql\n ```\n\n - On Linux (Debian/Ubuntu):\n\n ```bash\n sudo apt-get update && sudo apt-get install -y postgresql postgresql-contrib\n sudo service postgresql start\n ```\n\n - On Windows:\n Download and run the [PostgreSQL installer](https://www.postgresql.org/download/windows/) and follow the prompts.\n\n2. **Create a PostgreSQL User and Database**:\n\n ```bash\n psql postgres\n ```\n\n In the `psql` prompt, run:\n\n ```sql\n CREATE USER prefect_user WITH PASSWORD 'pass';\n\n CREATE DATABASE prefect_db OWNER prefect_user;\n GRANT ALL PRIVILEGES ON DATABASE prefect_db TO prefect_user;\n ```\n\n Exit the prompt with \\q.\n\n This sets up a prefect_user with password pass and a database named prefect_db.\n\n#### Supported Backends\n\n- NetworkX (default)\n- nx-CuGraph (requires separate CuGraph installation and supported GPU hardware)\n- GraphBLAS Algorithms (optional)\n- nx-parallel (optional)\n\n### Installing `nxbench`\n\nIn a new terminal window:\n\nPyPi:\n\n```bash\npip install nxbench\n```\n\nFrom source (local clone):\n\n```bash\ngit clone https://github.com/dpys/nxbench.git\ncd nxbench\nmake install\n```\n\nThis should install nxbench and all required dependencies (including prefect, asyncpg, and related packages).\n\n## Installation (Docker Setup)\n\nDocker:\n\n```bash\n# CPU-only\ndocker-compose -f docker/docker-compose.cpu.yaml build\n\n# With GPU\ndocker-compose -f docker/docker-compose.gpu.yaml build\n```\n\n## Quick Start\n\n1. Configure your benchmarks in a yaml file (see `configs/example.yaml`):\n\n```yaml\nalgorithms:\n - name: \"pagerank\"\n func: \"networkx.pagerank\"\n params:\n alpha: 0.85\n groups: [\"centrality\"]\n\ndatasets:\n - name: \"karate\"\n source: \"networkrepository\"\n```\n\n2. Start an instance of an orion server in a separate terminal window:\n\n```bash\nexport PREFECT_API_URL=\"http://127.0.0.1:4200/api\"\nexport PREFECT_API_DATABASE_CONNECTION_URL=\"postgresql+asyncpg://prefect_user:pass@localhost:5432/prefect_db\"\nprefect server start\n```\n\n3. In the original terminal window, run benchmarks based on the configuration:\n\n```bash\nnxbench --config 'nxbench/configs/example.yaml' benchmark run\n```\n\n4. Export results:\n\n```bash\nnxbench --config 'nxbench/configs/example.yaml' benchmark export 'results/9e3e8baa4a3443c392dc8fee00373b11_20241220002902.json' --output-format csv --output-file 'results/results.csv' # convert benchmarked results from a run with hash `9e3e8baa4a3443c392dc8fee00373b11_20241220002902` into csv format.\n```\n\n5. View results:\n\n```bash\nnxbench viz serve # launch the interactive results visualization dashboard.\n```\n\n<p align=\"center\">\n <img src=\"doc/_static/assets/animation.gif\" alt=\"Parallel Categories Animation\" width=\"1000\"/>\n</p>\n\n## Advanced Command Line Interface\n\nThe CLI provides comprehensive management of benchmarks, datasets, and visualization:\n\n```bash\n# Data Management\nnxbench data download karate # download specific dataset\nnxbench data list --category social # list available datasets\n\n# Benchmarking\nnxbench --config 'nxbench/configs/example.yaml' -vvv benchmark run # debug benchmark runs\nnxbench --config 'nxbench/configs/example.yaml' benchmark export 'results/9e3e8baa4a3443c392dc8fee00373b11_20241220002902.json' --output-format sql --output-file 'results/benchmarks.sqlite' # export the results from a run with hash `9e3e8baa4a3443c392dc8fee00373b11_20241220002902` into a sql database\n```\n\n## Configuration\n\nBenchmarks are configured through YAML files with the following structure:\n\n```yaml\nalgorithms:\n - name: \"algorithm_name\"\n func: \"fully.qualified.function.name\"\n params: {}\n requires_directed: false\n groups: [\"category\"]\n validate_result: \"validation.function\"\n\ndatasets:\n - name: \"dataset_name\"\n source: \"networkrepository\"\n params: {}\n```\n\n## Reproducible benchmarking through containerization\n\n```bash\n# Run benchmarks with GPU\nNUM_GPU=1 docker-compose -f docker/docker-compose.gpu.yaml up nxbench\n\n# Run benchmarks CPU-only\ndocker-compose -f docker/docker-compose.cpu.yaml up nxbench\n\n# Start visualization dashboard\ndocker-compose -f docker/docker-compose.cpu.yaml up dashboard\n\n# Run specific backend\ndocker-compose -f docker/docker-compose.cpu.yaml run --rm nxbench --config 'nxbench/configs/example.yaml' benchmark run --backend networkx\n\n# Export results from a run with hash `9e3e8baa4a3443c392dc8fee00373b11_20241220002902`\ndocker-compose -f docker/docker-compose.cpu.yaml run --rm nxbench --config 'nxbench/configs/example.yaml' benchmark export 'nxbench_results/9e3e8baa4a3443c392dc8fee00373b11_20241220002902.json' --output-format csv --output-file 'nxbench_results/results.csv'\n```\n\n## Contributing\n\nContributions are welcome! Please read our [Contributing Guide](CONTRIBUTING.md) for details on:\n\n- Code style guidelines\n- Development setup\n- Testing requirements\n- Pull request process\n\n## License\n\nThis project is licensed under the MIT License - see [LICENSE](LICENSE) for details.\n\n## Acknowledgments\n\n- NetworkX community for the core graph library and dispatching support\n- NetworkRepository.com for harmonized dataset access\n\n## Contact\n\nFor questions or suggestions:\n\n- Open an issue on GitHub\n- Email: <dpysalexander@gmail.com>\n",
"bugtrack_url": null,
"license": null,
"summary": "A centralized benchmarking suite to facilitate comparative profiling of tools across graph analytic libraries and datasets",
"version": "0.1.24",
"project_urls": null,
"split_keywords": [
"networkx",
" graph",
" network",
" benchmarking",
" profiling"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4696b5e17a50a2913484977787a0cbf0e0de55889040f473cd0ed9940da143a4",
"md5": "997edba49615ca0e88fd2a22a7171334",
"sha256": "143c662069200f50fe7ff8bf79eba4a8bfee073c4e2ad5198c07c05502b9559d"
},
"downloads": -1,
"filename": "nxbench-0.1.24-py3-none-any.whl",
"has_sig": false,
"md5_digest": "997edba49615ca0e88fd2a22a7171334",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 183088,
"upload_time": "2024-12-28T08:14:32",
"upload_time_iso_8601": "2024-12-28T08:14:32.328211Z",
"url": "https://files.pythonhosted.org/packages/46/96/b5e17a50a2913484977787a0cbf0e0de55889040f473cd0ed9940da143a4/nxbench-0.1.24-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "eded5da9fe9e1837b26a5f04c431b84cf2c07d92dd6f22eb01514c1befb7aaf7",
"md5": "ffd9df86ff649eb6843dac3a8067ba1e",
"sha256": "00123d12ff8fb24e2915a2671d65bb53eb5adf41bbc0e0fdd11a5ed8649378a3"
},
"downloads": -1,
"filename": "nxbench-0.1.24.tar.gz",
"has_sig": false,
"md5_digest": "ffd9df86ff649eb6843dac3a8067ba1e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 179750,
"upload_time": "2024-12-28T08:14:36",
"upload_time_iso_8601": "2024-12-28T08:14:36.276734Z",
"url": "https://files.pythonhosted.org/packages/ed/ed/5da9fe9e1837b26a5f04c431b84cf2c07d92dd6f22eb01514c1befb7aaf7/nxbench-0.1.24.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-28 08:14:36",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "nxbench"
}
