| Name | sd-mecha JSON |
| Version |
0.0.26
JSON |
| download |
| home_page | None |
| Summary | State dict recipe merger |
| upload_time | 2024-09-06 16:11:22 |
| maintainer | None |
| docs_url | None |
| author | ljleb |
| requires_python | >=3.10 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# sd-mecha
[](https://badge.fury.io/py/sd-mecha)
[](https://discord.gg/invite/2EPaw6fxxm)
sd-mecha is a memory-efficient general-purpose model merger. It can merge any model architecture given appropriate configuration:
- diffusion models
- LLMs
- Depth models
- Scorers
- ...
## Features
- Memory efficient model merging -- merge a very large number of models at the same time
- Mecha recipes as a textual and interpretable format (.mecha)
- Extension API through python:
- add new architectures (i.e. Stable Cascade, Stable Diffusion 3, etc.)
- add new model types (i.e. OFT networks, LoKr, etc.)
- add new merge methods
- Recipe variables for general recipe templates
- Compose recipe templates to create mega recipes
- Builtin support for popular model architectures:
- SD1.5
- SDXL
- SD3
- Merge LoRAs together and to checkpoints
- Block-wise hyperparameters for precise control of blocks
- Class-wise hyperparameters for precise control of layer types
- Support arbitrary model architectures and types using the `sd_mecha.extensions` module
- Merge SDXL LoRAs to models and with other LoRAs
## Install
```commandline
pip install sd-mecha torch
```
sd-mecha depends additionally on:
- `torch>=2.0.1`
The pypi package does not ship with `torch` so that you can install the appropriate version for your system.
## Usage
### Merge models
To merge models, mecha needs a recipe as input. There are multiple ways to provide a recipe:
- using the python merging API
- using the CLI with .mecha recipes
#### Using the python merging API
Here's an example simple sum-twice merge setup:
```python
import sd_mecha
# create a simple weighted sum recipe
# all builtin merge methods are direct properties of the `sd_mecha` package for convenience
recipe = sd_mecha.weighted_sum(
sd_mecha.weighted_sum(
"ghostmix_v20Bakedvae",
"deliberate_v2",
alpha=0.5,
),
"dreamshaper_332BakedVaeClipFix",
alpha=0.33,
)
# merger contains default parameters
merger = sd_mecha.RecipeMerger(
models_dir=r"E:\sd\models\Stable-diffusion",
)
# perform the entire merge plan and save to output path
merger.merge_and_save(recipe, output="basic_merge")
```
See the [examples](/examples) directory for more examples.
#### Using the CLI with .mecha recipes
It is alternatively possible to merge recipes previously serialized to `.mecha`.
This is only possible if the recipe is concrete. (i.e. all potential parameters have been replaced with actual models)
```commandline
python -m sd_mecha merge path/to/recipe.mecha
```
For more information:
```commandline
python -m sd_mecha merge --help
```
### Get Model-Specific Information
The interface for block/class hyperparameters requires prior knowledge of the blocks and classes of the architecture being merged.
The command `info` was made to discover the names of the blocks and/or classes to use.
To show the registered model architectures:
```commandline
python -m sd_mecha info
```
Mecha has builtin support for the SD1.x and the SDXL architectures:
```
Available architectures:
- sd1
- sdxl
```
To view the available blocks and classes of an architecture, specify the architecture:
```commandline
python -m sd_mecha info sd1
```
```
Component "txt":
Blocks:
- in0
- in1
- in2
...
Classes:
- final_layer_norm
- layer_norm1
- layer_norm2
- mlp_fc1
...
Component "unet":
Blocks:
...
Classes:
...
```
Given this information, it is possible to set i.e. the value of block `in2` in the `txt` component specifically:
```python
import sd_mecha
recipe = sd_mecha.weighted_sum(
"ghostmix_v20Bakedvae",
"dreamshaper_332BakedVaeClipFix",
alpha=(
sd_mecha.default("sd1", "txt", 0.33) |
sd_mecha.blocks("sd1", "txt", in2=0.75)
),
)
```
See the [merging API section](#using-the-python-merging-api) above for more info.
If run as verbose, it also shows the keys that are associated with each block/class:
```commandline
python -m sd_mecha info sd1 -v
```
```
Component "txt":
Blocks:
in0:
- model.diffusion_model.input_blocks.0.0.bias
- model.diffusion_model.input_blocks.0.0.weight
in1:
- model.diffusion_model.input_blocks.1.0.emb_layers.1.bias
- model.diffusion_model.input_blocks.1.0.emb_layers.1.weight
- model.diffusion_model.input_blocks.1.0.in_layers.0.bias
- model.diffusion_model.input_blocks.1.0.in_layers.0.weight
...
...
...
```
### Compose recipes
It is possible to compose recipes together to create more complex recipes.
For this to work, the base recipe must be general: (i.e. the parameters to replace must exist in the base recipe)
```commandline
python -m sd_mecha compose path/to/recipe.mecha [options]
```
For example, here we compose the recipe [incompatible_fusion.mecha](examples/recipes/incompatible_fusion.mecha)
with another recipe for parameter "a" and
SD1.5 base for parameter "c":
```commandline
python -m sd_mecha compose examples/recipes/incompatible_fusion.mecha \
-p a examples/recipes/weighted_sum.mecha \
-p c v1-5-pruned.safetensors
```
For more information:
```shell
python -m sd_mecha compose --help
```
## Motivation
Keeping track of full merge recipes has always been annoying.
I needed something that allows to store merge recipes in a readable format while also being executable.
I also needed something that allows to fully merge an entire tree of models without having to save intermediate models to disk.
Typically, mergers load all models in memory before initiating the merge process.
This can be very inefficient when the merge focuses on each key individually:
sd-mecha doesn't have this problem as it saves keys as soon as it can:
This allows to merge a very large number of models simultaneously on low-end hardware.
Raw data
{
"_id": null,
"home_page": null,
"name": "sd-mecha",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": null,
"author": "ljleb",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/4f/e0/9d94b5a302838b76f6d3003e07ff62c653d651dadf750df3f369020da669/sd_mecha-0.0.26.tar.gz",
"platform": null,
"description": "# sd-mecha\r\n\r\n[](https://badge.fury.io/py/sd-mecha)\r\n[](https://discord.gg/invite/2EPaw6fxxm)\r\n\r\nsd-mecha is a memory-efficient general-purpose model merger. It can merge any model architecture given appropriate configuration:\r\n- diffusion models\r\n- LLMs\r\n- Depth models\r\n- Scorers\r\n- ...\r\n\r\n## Features\r\n\r\n- Memory efficient model merging -- merge a very large number of models at the same time\r\n- Mecha recipes as a textual and interpretable format (.mecha)\r\n- Extension API through python:\r\n - add new architectures (i.e. Stable Cascade, Stable Diffusion 3, etc.)\r\n - add new model types (i.e. OFT networks, LoKr, etc.)\r\n - add new merge methods\r\n- Recipe variables for general recipe templates\r\n- Compose recipe templates to create mega recipes\r\n- Builtin support for popular model architectures:\r\n - SD1.5\r\n - SDXL\r\n - SD3\r\n- Merge LoRAs together and to checkpoints\r\n- Block-wise hyperparameters for precise control of blocks\r\n- Class-wise hyperparameters for precise control of layer types\r\n- Support arbitrary model architectures and types using the `sd_mecha.extensions` module\r\n- Merge SDXL LoRAs to models and with other LoRAs\r\n\r\n## Install\r\n\r\n```commandline\r\npip install sd-mecha torch\r\n```\r\n\r\nsd-mecha depends additionally on:\r\n\r\n- `torch>=2.0.1`\r\n\r\nThe pypi package does not ship with `torch` so that you can install the appropriate version for your system.\r\n\r\n## Usage\r\n\r\n### Merge models\r\n\r\nTo merge models, mecha needs a recipe as input. There are multiple ways to provide a recipe:\r\n- using the python merging API\r\n- using the CLI with .mecha recipes\r\n\r\n#### Using the python merging API\r\n\r\nHere's an example simple sum-twice merge setup:\r\n\r\n```python\r\nimport sd_mecha\r\n\r\n# create a simple weighted sum recipe\r\n# all builtin merge methods are direct properties of the `sd_mecha` package for convenience\r\nrecipe = sd_mecha.weighted_sum(\r\n sd_mecha.weighted_sum(\r\n \"ghostmix_v20Bakedvae\",\r\n \"deliberate_v2\",\r\n alpha=0.5,\r\n ),\r\n \"dreamshaper_332BakedVaeClipFix\",\r\n alpha=0.33,\r\n)\r\n\r\n# merger contains default parameters\r\nmerger = sd_mecha.RecipeMerger(\r\n models_dir=r\"E:\\sd\\models\\Stable-diffusion\",\r\n)\r\n\r\n# perform the entire merge plan and save to output path\r\nmerger.merge_and_save(recipe, output=\"basic_merge\")\r\n```\r\n\r\nSee the [examples](/examples) directory for more examples.\r\n\r\n#### Using the CLI with .mecha recipes\r\n\r\nIt is alternatively possible to merge recipes previously serialized to `.mecha`.\r\nThis is only possible if the recipe is concrete. (i.e. all potential parameters have been replaced with actual models)\r\n\r\n```commandline\r\npython -m sd_mecha merge path/to/recipe.mecha\r\n```\r\n\r\nFor more information:\r\n\r\n```commandline\r\npython -m sd_mecha merge --help\r\n```\r\n\r\n### Get Model-Specific Information\r\n\r\nThe interface for block/class hyperparameters requires prior knowledge of the blocks and classes of the architecture being merged.\r\nThe command `info` was made to discover the names of the blocks and/or classes to use.\r\n\r\nTo show the registered model architectures:\r\n\r\n```commandline\r\npython -m sd_mecha info\r\n```\r\n\r\nMecha has builtin support for the SD1.x and the SDXL architectures:\r\n\r\n```\r\nAvailable architectures:\r\n- sd1\r\n- sdxl\r\n```\r\n\r\nTo view the available blocks and classes of an architecture, specify the architecture:\r\n\r\n```commandline\r\npython -m sd_mecha info sd1\r\n```\r\n```\r\nComponent \"txt\":\r\n Blocks:\r\n - in0\r\n - in1\r\n - in2\r\n ...\r\n Classes:\r\n - final_layer_norm\r\n - layer_norm1\r\n - layer_norm2\r\n - mlp_fc1\r\n ...\r\nComponent \"unet\":\r\n Blocks:\r\n ...\r\n Classes:\r\n ...\r\n```\r\n\r\nGiven this information, it is possible to set i.e. the value of block `in2` in the `txt` component specifically:\r\n\r\n```python\r\nimport sd_mecha\r\nrecipe = sd_mecha.weighted_sum(\r\n \"ghostmix_v20Bakedvae\",\r\n \"dreamshaper_332BakedVaeClipFix\",\r\n alpha=(\r\n sd_mecha.default(\"sd1\", \"txt\", 0.33) |\r\n sd_mecha.blocks(\"sd1\", \"txt\", in2=0.75)\r\n ),\r\n)\r\n```\r\n\r\nSee the [merging API section](#using-the-python-merging-api) above for more info.\r\n\r\nIf run as verbose, it also shows the keys that are associated with each block/class:\r\n\r\n```commandline\r\npython -m sd_mecha info sd1 -v\r\n```\r\n```\r\nComponent \"txt\":\r\n Blocks:\r\n in0:\r\n - model.diffusion_model.input_blocks.0.0.bias\r\n - model.diffusion_model.input_blocks.0.0.weight\r\n in1:\r\n - model.diffusion_model.input_blocks.1.0.emb_layers.1.bias\r\n - model.diffusion_model.input_blocks.1.0.emb_layers.1.weight\r\n - model.diffusion_model.input_blocks.1.0.in_layers.0.bias\r\n - model.diffusion_model.input_blocks.1.0.in_layers.0.weight\r\n ...\r\n ...\r\n...\r\n```\r\n\r\n### Compose recipes\r\n\r\nIt is possible to compose recipes together to create more complex recipes.\r\nFor this to work, the base recipe must be general: (i.e. the parameters to replace must exist in the base recipe)\r\n\r\n```commandline\r\npython -m sd_mecha compose path/to/recipe.mecha [options]\r\n```\r\n\r\nFor example, here we compose the recipe [incompatible_fusion.mecha](examples/recipes/incompatible_fusion.mecha)\r\nwith another recipe for parameter \"a\" and\r\nSD1.5 base for parameter \"c\":\r\n\r\n```commandline\r\npython -m sd_mecha compose examples/recipes/incompatible_fusion.mecha \\\r\n -p a examples/recipes/weighted_sum.mecha \\\r\n -p c v1-5-pruned.safetensors\r\n```\r\n\r\nFor more information:\r\n\r\n```shell\r\npython -m sd_mecha compose --help\r\n```\r\n\r\n## Motivation\r\n\r\nKeeping track of full merge recipes has always been annoying.\r\nI needed something that allows to store merge recipes in a readable format while also being executable.\r\nI also needed something that allows to fully merge an entire tree of models without having to save intermediate models to disk.\r\n\r\nTypically, mergers load all models in memory before initiating the merge process.\r\nThis can be very inefficient when the merge focuses on each key individually:\r\n\r\n\r\n\r\nsd-mecha doesn't have this problem as it saves keys as soon as it can:\r\n\r\n\r\n\r\nThis allows to merge a very large number of models simultaneously on low-end hardware.\r\n",
"bugtrack_url": null,
"license": null,
"summary": "State dict recipe merger",
"version": "0.0.26",
"project_urls": {
"Homepage": "https://github.com/ljleb/sd-mecha",
"Issues": "https://github.com/ljleb/sd-mecha/issues"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7a2f0bdb9fab465b9e1d10793416931cf12f3dc43e32d1508ff9a8a911e3bc3a",
"md5": "553e8e14fce8ce2931ff8206ed214169",
"sha256": "d343ca04824b414c85af2ae514f9cead52144f05d6fa80b9097c478918d66a64"
},
"downloads": -1,
"filename": "sd_mecha-0.0.26-py3-none-any.whl",
"has_sig": false,
"md5_digest": "553e8e14fce8ce2931ff8206ed214169",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 69667,
"upload_time": "2024-09-06T16:11:21",
"upload_time_iso_8601": "2024-09-06T16:11:21.065460Z",
"url": "https://files.pythonhosted.org/packages/7a/2f/0bdb9fab465b9e1d10793416931cf12f3dc43e32d1508ff9a8a911e3bc3a/sd_mecha-0.0.26-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4fe09d94b5a302838b76f6d3003e07ff62c653d651dadf750df3f369020da669",
"md5": "66f7db0b9672d578810a2b53ccc6044a",
"sha256": "14edb18d70c9bf724714beea7bbf121d9ca9e715bf36d93a1d5fcdd77f4a9dcd"
},
"downloads": -1,
"filename": "sd_mecha-0.0.26.tar.gz",
"has_sig": false,
"md5_digest": "66f7db0b9672d578810a2b53ccc6044a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 65813,
"upload_time": "2024-09-06T16:11:22",
"upload_time_iso_8601": "2024-09-06T16:11:22.358733Z",
"url": "https://files.pythonhosted.org/packages/4f/e0/9d94b5a302838b76f6d3003e07ff62c653d651dadf750df3f369020da669/sd_mecha-0.0.26.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-06 16:11:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ljleb",
"github_project": "sd-mecha",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "sd-mecha"
}
