# 🖼️ Python Bindings for [`stable-diffusion.cpp`](https://github.com/leejet/stable-diffusion.cpp)
Simple Python bindings for **@leejet's** [`stable-diffusion.cpp`](https://github.com/leejet/stable-diffusion.cpp) library.
This package provides:
- Low-level access to C API via `ctypes` interface.
- High-level Python API for stable diffusion image generation.
## Installation
Requirements:
- Python 3.8+
- C compiler
- Linux: gcc or clang
- Windows: Visual Studio or MinGW
- MacOS: Xcode
To install the package, run:
```bash
pip install stable-diffusion-cpp-python
```
This will also build `stable-diffusion.cpp` from source and install it alongside this python package.
If this fails, add `--verbose` to the `pip install` see the full cmake build log.
### Installation Configuration
`stable-diffusion.cpp` supports a number of hardware acceleration backends to speed up inference as well as backend specific options. See the [stable-diffusion.cpp README](https://github.com/leejet/stable-diffusion.cpp#build) for a full list.
All `stable-diffusion.cpp` cmake build options can be set via the `CMAKE_ARGS` environment variable or via the `--config-settings / -C` cli flag during installation.
<details open>
<summary>Environment Variables</summary>
```bash
# Linux and Mac
CMAKE_ARGS="-DGGML_OPENBLAS=ON" \
pip install stable-diffusion-cpp-python
```
```powershell
# Windows
$env:CMAKE_ARGS = "-DGGML_OPENBLAS=ON"
pip install stable-diffusion-cpp-python
```
</details>
<details>
<summary>CLI / requirements.txt</summary>
They can also be set via `pip install -C / --config-settings` command and saved to a `requirements.txt` file:
```bash
pip install --upgrade pip # ensure pip is up to date
pip install stable-diffusion-cpp-python \
-C cmake.args="-DGGML_OPENBLAS=ON"
```
```txt
# requirements.txt
stable-diffusion-cpp-python -C cmake.args="-DGGML_OPENBLAS=ON"
```
</details>
### Supported Backends
Below are some common backends, their build commands and any additional environment variables required.
<details>
<summary>Using OpenBLAS (CPU)</summary>
```bash
CMAKE_ARGS="-DGGML_OPENBLAS=ON" pip install stable-diffusion-cpp-python
```
</details>
<details>
<summary>Using cuBLAS (CUDA)</summary>
This provides BLAS acceleration using the CUDA cores of your Nvidia GPU. Make sure to have the CUDA toolkit installed. You can download it from your Linux distro's package manager (e.g. `apt install nvidia-cuda-toolkit`) or from here: [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads). Recommended to have at least 4 GB of VRAM.
```bash
CMAKE_ARGS="-DSD_CUBLAS=on" pip install stable-diffusion-cpp-python
```
</details>
<details>
<summary>Using hipBLAS (ROCm)</summary>
This provides BLAS acceleration using the ROCm cores of your AMD GPU. Make sure to have the ROCm toolkit installed.
Windows Users Refer to [docs/hipBLAS_on_Windows.md](docs%2FhipBLAS_on_Windows.md) for a comprehensive guide.
```bash
CMAKE_ARGS="-G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DSD_HIPBLAS=ON -DCMAKE_BUILD_TYPE=Release -DAMDGPU_TARGETS=gfx1101" pip install stable-diffusion-cpp-python
```
</details>
<details>
<summary>Using Metal</summary>
Using Metal makes the computation run on the GPU. Currently, there are some issues with Metal when performing operations on very large matrices, making it highly inefficient at the moment. Performance improvements are expected in the near future.
```bash
CMAKE_ARGS="-DSD_METAL=ON" pip install stable-diffusion-cpp-python
```
</details>
<details>
<summary>Using Flash Attention</summary>
Enabling flash attention reduces memory usage by at least 400 MB. At the moment, it is not supported when CUBLAS is enabled because the kernel implementation is missing.
```bash
CMAKE_ARGS="-DSD_FLASH_ATTN=ON" pip install stable-diffusion-cpp-python
```
</details>
### Upgrading and Reinstalling
To upgrade and rebuild `stable-diffusion-cpp-python` add `--upgrade --force-reinstall --no-cache-dir` flags to the `pip install` command to ensure the package is rebuilt from source.
## High-level API
The high-level API provides a simple managed interface through the `StableDiffusion` class.
Below is a short example demonstrating how to use the high-level API to generate a simple image:
```python
>>> from stable_diffusion_cpp import StableDiffusion
>>> stable_diffusion = StableDiffusion(
model_path="../models/v1-5-pruned-emaonly.safetensors",
wtype="default", # Weight type (options: default, f32, f16, q4_0, q4_1, q5_0, q5_1, q8_0)
# seed=1337, # Uncomment to set a specific seed
)
>>> output = stable_diffusion.txt_to_img(
"a lovely cat", # Prompt
)
```
#### with LoRA
- You can specify the directory where the lora weights are stored via `lora_model_dir`. If not specified, the default is the current working directory.
- LoRA is specified via prompt, just like [stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#lora).
Here's a simple example:
```python
>>> from stable_diffusion_cpp import StableDiffusion
>>> stable_diffusion = StableDiffusion(
model_path="../models/v1-5-pruned-emaonly.safetensors",
lora_model_dir="../models/",
)
>>> output = stable_diffusion.txt_to_img(
"a lovely cat<lora:marblesh:1>", # Prompt
)
```
## Low-level API
The low-level API is a direct [`ctypes`](https://docs.python.org/3/library/ctypes.html) binding to the C API provided by `stable-diffusion.cpp`.
The entire low-level API can be found in [stable_diffusion_cpp/stable_diffusion_cpp.py](https://github.com/william-murray1204/stable-diffusion-cpp-python/blob/main/stable_diffusion_cpp/stable_diffusion_cpp.py) and directly mirrors the C API in [stable-diffusion.h](https://github.com/leejet/stable-diffusion.cpp/blob/master/stable-diffusion.h).
Below is a short example demonstrating how to use the low-level API:
```python
>>> import stable_diffusion_cpp as sd_cpp
>>> import ctypes
>>> from PIL import Image
>>> img = Image.open("path/to/image.png")
>>> img_bytes = img.tobytes()
>>> c_image = sd_cpp.sd_image_t(
width=img.width,
height=img.height,
channel=channel,
data=ctypes.cast(
(ctypes.c_byte * len(img_bytes))(*img_bytes),
ctypes.POINTER(ctypes.c_uint8),
),
) # Create a new C sd_image_t
>>> img = sd_cpp.upscale(
self.upscaler,
image_bytes,
upscale_factor,
) # Upscale the image
>>> sd_cpp.free_image(c_image)
```
## Development
To get started, clone the repository and install the package in editable / development mode.
```bash
git clone --recurse-submodules https://github.com/william-murray1204/stable-diffusion-cpp-python.git
cd stable-diffusion-cpp-python
# Upgrade pip (required for editable mode)
pip install --upgrade pip
# Install with pip
pip install -e .
```
Now you can make changes to the code within the `stable_diffusion_cpp` directory and test them in your python environment.
### Cleanup
To clear the cache.
```bash
make clean
```
## To Do
- [ ] Add options for these torch.FloatTensor, PIL.Image.Image, np.ndarray, List[torch.FloatTensor], List[PIL.Image.Image], or List[np.ndarray]
## References
- [stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp)
- [llama-cpp-python](https://github.com/abetlen/llama-cpp-python)
- [llama.cpp](https://github.com/ggerganov/llama.cpp)
- [whisper-cpp-python](https://github.com/carloscdias/whisper-cpp-python)
- [Golang stable-diffusion](https://github.com/seasonjs/stable-diffusion)
- [StableDiffusion.NET](https://github.com/DarthAffe/StableDiffusion.NET)
## License
This project is licensed under the terms of the MIT license. See [LICENSE](LICENSE) for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "stable-diffusion-cpp-python",
"maintainer": "William Murray",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "diffusion, stable-diffusion, python, cpp-bindings, artificial-intelligence, machine-learning",
"author": "William Murray",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/c8/c4/dee0b0f096c2699c268f6c81acdf525ee95846e4cb06c0f099c5af1635bc/stable_diffusion_cpp_python-0.1.6.tar.gz",
"platform": null,
"description": "# \ud83d\uddbc\ufe0f Python Bindings for [`stable-diffusion.cpp`](https://github.com/leejet/stable-diffusion.cpp)\n\nSimple Python bindings for **@leejet's** [`stable-diffusion.cpp`](https://github.com/leejet/stable-diffusion.cpp) library.\nThis package provides:\n\n- Low-level access to C API via `ctypes` interface.\n- High-level Python API for stable diffusion image generation.\n\n## Installation\n\nRequirements:\n\n- Python 3.8+\n- C compiler\n - Linux: gcc or clang\n - Windows: Visual Studio or MinGW\n - MacOS: Xcode\n\nTo install the package, run:\n\n```bash\npip install stable-diffusion-cpp-python\n```\n\nThis will also build `stable-diffusion.cpp` from source and install it alongside this python package.\n\nIf this fails, add `--verbose` to the `pip install` see the full cmake build log.\n\n### Installation Configuration\n\n`stable-diffusion.cpp` supports a number of hardware acceleration backends to speed up inference as well as backend specific options. See the [stable-diffusion.cpp README](https://github.com/leejet/stable-diffusion.cpp#build) for a full list.\n\nAll `stable-diffusion.cpp` cmake build options can be set via the `CMAKE_ARGS` environment variable or via the `--config-settings / -C` cli flag during installation.\n\n<details open>\n<summary>Environment Variables</summary>\n\n```bash\n# Linux and Mac\nCMAKE_ARGS=\"-DGGML_OPENBLAS=ON\" \\\n pip install stable-diffusion-cpp-python\n```\n\n```powershell\n# Windows\n$env:CMAKE_ARGS = \"-DGGML_OPENBLAS=ON\"\npip install stable-diffusion-cpp-python\n```\n\n</details>\n\n<details>\n<summary>CLI / requirements.txt</summary>\n\nThey can also be set via `pip install -C / --config-settings` command and saved to a `requirements.txt` file:\n\n```bash\npip install --upgrade pip # ensure pip is up to date\npip install stable-diffusion-cpp-python \\\n -C cmake.args=\"-DGGML_OPENBLAS=ON\"\n```\n\n```txt\n# requirements.txt\n\nstable-diffusion-cpp-python -C cmake.args=\"-DGGML_OPENBLAS=ON\"\n```\n\n</details>\n\n### Supported Backends\n\nBelow are some common backends, their build commands and any additional environment variables required.\n\n<details>\n<summary>Using OpenBLAS (CPU)</summary>\n\n```bash\nCMAKE_ARGS=\"-DGGML_OPENBLAS=ON\" pip install stable-diffusion-cpp-python\n```\n\n</details>\n\n<details>\n<summary>Using cuBLAS (CUDA)</summary>\n\nThis provides BLAS acceleration using the CUDA cores of your Nvidia GPU. Make sure to have the CUDA toolkit installed. You can download it from your Linux distro's package manager (e.g. `apt install nvidia-cuda-toolkit`) or from here: [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads). Recommended to have at least 4 GB of VRAM.\n\n```bash\nCMAKE_ARGS=\"-DSD_CUBLAS=on\" pip install stable-diffusion-cpp-python\n```\n\n</details>\n\n<details>\n<summary>Using hipBLAS (ROCm)</summary>\n\nThis provides BLAS acceleration using the ROCm cores of your AMD GPU. Make sure to have the ROCm toolkit installed.\nWindows Users Refer to [docs/hipBLAS_on_Windows.md](docs%2FhipBLAS_on_Windows.md) for a comprehensive guide.\n\n```bash\nCMAKE_ARGS=\"-G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DSD_HIPBLAS=ON -DCMAKE_BUILD_TYPE=Release -DAMDGPU_TARGETS=gfx1101\" pip install stable-diffusion-cpp-python\n```\n\n</details>\n\n<details>\n<summary>Using Metal</summary>\n\nUsing Metal makes the computation run on the GPU. Currently, there are some issues with Metal when performing operations on very large matrices, making it highly inefficient at the moment. Performance improvements are expected in the near future.\n\n```bash\nCMAKE_ARGS=\"-DSD_METAL=ON\" pip install stable-diffusion-cpp-python\n```\n\n</details>\n\n<details>\n<summary>Using Flash Attention</summary>\n\nEnabling flash attention reduces memory usage by at least 400 MB. At the moment, it is not supported when CUBLAS is enabled because the kernel implementation is missing.\n\n```bash\nCMAKE_ARGS=\"-DSD_FLASH_ATTN=ON\" pip install stable-diffusion-cpp-python\n```\n\n</details>\n\n### Upgrading and Reinstalling\n\nTo upgrade and rebuild `stable-diffusion-cpp-python` add `--upgrade --force-reinstall --no-cache-dir` flags to the `pip install` command to ensure the package is rebuilt from source.\n\n## High-level API\n\nThe high-level API provides a simple managed interface through the `StableDiffusion` class.\n\nBelow is a short example demonstrating how to use the high-level API to generate a simple image:\n\n```python\n>>> from stable_diffusion_cpp import StableDiffusion\n>>> stable_diffusion = StableDiffusion(\n model_path=\"../models/v1-5-pruned-emaonly.safetensors\",\n wtype=\"default\", # Weight type (options: default, f32, f16, q4_0, q4_1, q5_0, q5_1, q8_0)\n # seed=1337, # Uncomment to set a specific seed\n)\n>>> output = stable_diffusion.txt_to_img(\n \"a lovely cat\", # Prompt\n)\n```\n\n#### with LoRA\n\n- You can specify the directory where the lora weights are stored via `lora_model_dir`. If not specified, the default is the current working directory.\n\n- LoRA is specified via prompt, just like [stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#lora).\n\nHere's a simple example:\n\n```python\n>>> from stable_diffusion_cpp import StableDiffusion\n>>> stable_diffusion = StableDiffusion(\n model_path=\"../models/v1-5-pruned-emaonly.safetensors\",\n lora_model_dir=\"../models/\",\n)\n>>> output = stable_diffusion.txt_to_img(\n \"a lovely cat<lora:marblesh:1>\", # Prompt\n)\n```\n\n## Low-level API\n\nThe low-level API is a direct [`ctypes`](https://docs.python.org/3/library/ctypes.html) binding to the C API provided by `stable-diffusion.cpp`.\nThe entire low-level API can be found in [stable_diffusion_cpp/stable_diffusion_cpp.py](https://github.com/william-murray1204/stable-diffusion-cpp-python/blob/main/stable_diffusion_cpp/stable_diffusion_cpp.py) and directly mirrors the C API in [stable-diffusion.h](https://github.com/leejet/stable-diffusion.cpp/blob/master/stable-diffusion.h).\n\nBelow is a short example demonstrating how to use the low-level API:\n\n```python\n>>> import stable_diffusion_cpp as sd_cpp\n>>> import ctypes\n>>> from PIL import Image\n\n>>> img = Image.open(\"path/to/image.png\")\n>>> img_bytes = img.tobytes()\n\n>>> c_image = sd_cpp.sd_image_t(\n width=img.width,\n height=img.height,\n channel=channel,\n data=ctypes.cast(\n (ctypes.c_byte * len(img_bytes))(*img_bytes),\n ctypes.POINTER(ctypes.c_uint8),\n ),\n) # Create a new C sd_image_t\n\n>>> img = sd_cpp.upscale(\n self.upscaler,\n image_bytes,\n upscale_factor,\n) # Upscale the image\n\n>>> sd_cpp.free_image(c_image)\n```\n\n## Development\n\nTo get started, clone the repository and install the package in editable / development mode.\n\n```bash\ngit clone --recurse-submodules https://github.com/william-murray1204/stable-diffusion-cpp-python.git\ncd stable-diffusion-cpp-python\n\n# Upgrade pip (required for editable mode)\npip install --upgrade pip\n\n# Install with pip\npip install -e .\n```\n\nNow you can make changes to the code within the `stable_diffusion_cpp` directory and test them in your python environment.\n\n### Cleanup\n\nTo clear the cache.\n\n```bash\nmake clean\n```\n\n## To Do\n\n- [ ] Add options for these torch.FloatTensor, PIL.Image.Image, np.ndarray, List[torch.FloatTensor], List[PIL.Image.Image], or List[np.ndarray]\n\n## References\n\n- [stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp)\n- [llama-cpp-python](https://github.com/abetlen/llama-cpp-python)\n- [llama.cpp](https://github.com/ggerganov/llama.cpp)\n- [whisper-cpp-python](https://github.com/carloscdias/whisper-cpp-python)\n- [Golang stable-diffusion](https://github.com/seasonjs/stable-diffusion)\n- [StableDiffusion.NET](https://github.com/DarthAffe/StableDiffusion.NET)\n\n## License\n\nThis project is licensed under the terms of the MIT license. See [LICENSE](LICENSE) for details.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python bindings for the stable-diffusion.cpp library",
"version": "0.1.6",
"project_urls": {
"Documentation": "https://github.com/william-murray1204/stable-diffusion-cpp-python/blob/main/README.md",
"Homepage": "https://github.com/william-murray1204/stable-diffusion-cpp-python",
"Issues": "https://github.com/william-murray1204/stable-diffusion-cpp-python/issues"
},
"split_keywords": [
"diffusion",
" stable-diffusion",
" python",
" cpp-bindings",
" artificial-intelligence",
" machine-learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c8c4dee0b0f096c2699c268f6c81acdf525ee95846e4cb06c0f099c5af1635bc",
"md5": "9902e8c8c6f8bd31ec304d17327d58e4",
"sha256": "2535f2695cb774281c7d6fdff2a687db0f47b7f9ea1993a15f8ba10a126673e6"
},
"downloads": -1,
"filename": "stable_diffusion_cpp_python-0.1.6.tar.gz",
"has_sig": false,
"md5_digest": "9902e8c8c6f8bd31ec304d17327d58e4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 30490712,
"upload_time": "2024-05-19T14:49:41",
"upload_time_iso_8601": "2024-05-19T14:49:41.756870Z",
"url": "https://files.pythonhosted.org/packages/c8/c4/dee0b0f096c2699c268f6c81acdf525ee95846e4cb06c0f099c5af1635bc/stable_diffusion_cpp_python-0.1.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-19 14:49:41",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "william-murray1204",
"github_project": "stable-diffusion-cpp-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "stable-diffusion-cpp-python"
}
JSON
