# MindONE
This repository contains SoTA algorithms, models, and interesting projects in the area of multimodal understanding and content generation
ONE is short for "ONE for all"
## News
**Hello MindSpore** from **Stable Diffusion 3**!
<div>
<img src="https://github.com/townwish4git/mindone/assets/143256262/8c25ae9a-67b1-436f-abf6-eca36738cd17" alt="sd3" width="512" height="512">
</div>
- [mindone/diffusers](mindone/diffusers) now supports [Stable Diffusion 3](https://huggingface.co/stabilityai/stable-diffusion-3-medium). Give it a try yourself!
```py
import mindspore
from mindone.diffusers import StableDiffusion3Pipeline
pipe = StableDiffusion3Pipeline.from_pretrained(
"stabilityai/stable-diffusion-3-medium-diffusers",
mindspore_dtype=mindspore.float16,
)
prompt = "A cat holding a sign that says 'Hello MindSpore'"
image = pipe(prompt)[0][0]
image.save("sd3.png")
```
### supported models under mindone/examples
| model | features
| :--- | :-- |
| [hpcai open sora](https://github.com/mindspore-lab/mindone/blob/master/examples/opensora_hpcai) | support v1.0/1.1/1.2 large scale training with dp/sp/zero |
| [open sora plan](https://github.com/mindspore-lab/mindone/blob/master/examples/opensora_pku) | support v1.0/1.1/1.2 large scale training with dp/sp/zero |
| [stable diffusion](https://github.com/mindspore-lab/mindone/blob/master/examples/stable_diffusion_v2) | support sd 1.5/2.0/2.1, vanilla fine tune, lora, dreambooth, text inversion|
| [stable diffusion xl](https://github.com/mindspore-lab/mindone/blob/master/examples/stable_diffusion_xl) |support sai style(stability AI) vanilla fine tune, lora, dreambooth |
| [dit](https://github.com/mindspore-lab/mindone/blob/master/examples/dit) | support text to image fine tune |
| [hunyuan_dit](https://github.com/mindspore-lab/mindone/blob/master/examples/hunyuan_dit) | support text to image fine tune |
| [pixart_sigma](https://github.com/mindspore-lab/mindone/blob/master/examples/pixart_sigma) | suuport text to image fine tune at different aspect ratio |
| [latte](https://github.com/mindspore-lab/mindone/blob/master/examples/latte) | support uncondition text to image fine tune |
| [animate diff](https://github.com/mindspore-lab/mindone/blob/master/examples/animatediff) | support motion module and lora training |
| [dynamicrafter](https://github.com/mindspore-lab/mindone/blob/master/examples/dynamicrafter) | support image to video generation |
### run hf diffusers on mindspore
mindone diffusers is under active development.
| component | features
| :--- | :--
| [pipeline](https://github.com/mindspore-lab/mindone/tree/master/mindone/diffusers/pipelines) | support text2image,text2video,text2audio tasks 30+
| [models](https://github.com/mindspore-lab/mindone/tree/master/mindone/diffusers/models) | support audoencoder & transformers base models same as hf diffusers
| [schedulers](https://github.com/mindspore-lab/mindone/tree/master/mindone/diffusers/schedulers) | support ddpm & dpm solver 10+ schedulers same as hf diffusers
Raw data
{
"_id": null,
"home_page": null,
"name": "mindone",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "artificial intelligence, deep learning, diffusion, generative model, mindspore",
"author": null,
"author_email": "MindSpore Lab <mindspore-lab@example.com>",
"download_url": "https://files.pythonhosted.org/packages/1a/a4/4b90107504fccf1b83abf7a38f6f6a91876f79c80bbcb437a2f61651cb52/mindone-0.2.0.tar.gz",
"platform": null,
"description": "# MindONE\n\nThis repository contains SoTA algorithms, models, and interesting projects in the area of multimodal understanding and content generation\n\nONE is short for \"ONE for all\"\n\n## News\n\n**Hello MindSpore** from **Stable Diffusion 3**!\n\n<div>\n<img src=\"https://github.com/townwish4git/mindone/assets/143256262/8c25ae9a-67b1-436f-abf6-eca36738cd17\" alt=\"sd3\" width=\"512\" height=\"512\">\n</div>\n\n- [mindone/diffusers](mindone/diffusers) now supports [Stable Diffusion 3](https://huggingface.co/stabilityai/stable-diffusion-3-medium). Give it a try yourself!\n\n ```py\n import mindspore\n from mindone.diffusers import StableDiffusion3Pipeline\n\n pipe = StableDiffusion3Pipeline.from_pretrained(\n \"stabilityai/stable-diffusion-3-medium-diffusers\",\n mindspore_dtype=mindspore.float16,\n )\n prompt = \"A cat holding a sign that says 'Hello MindSpore'\"\n image = pipe(prompt)[0][0]\n image.save(\"sd3.png\")\n ```\n\n### supported models under mindone/examples\n| model | features \n| :--- | :-- |\n| [hpcai open sora](https://github.com/mindspore-lab/mindone/blob/master/examples/opensora_hpcai) | support v1.0/1.1/1.2 large scale training with dp/sp/zero |\n| [open sora plan](https://github.com/mindspore-lab/mindone/blob/master/examples/opensora_pku) | support v1.0/1.1/1.2 large scale training with dp/sp/zero |\n| [stable diffusion](https://github.com/mindspore-lab/mindone/blob/master/examples/stable_diffusion_v2) | support sd 1.5/2.0/2.1, vanilla fine tune, lora, dreambooth, text inversion|\n| [stable diffusion xl](https://github.com/mindspore-lab/mindone/blob/master/examples/stable_diffusion_xl) |support sai style(stability AI) vanilla fine tune, lora, dreambooth |\n| [dit](https://github.com/mindspore-lab/mindone/blob/master/examples/dit) | support text to image fine tune |\n| [hunyuan_dit](https://github.com/mindspore-lab/mindone/blob/master/examples/hunyuan_dit) | support text to image fine tune |\n| [pixart_sigma](https://github.com/mindspore-lab/mindone/blob/master/examples/pixart_sigma) | suuport text to image fine tune at different aspect ratio |\n| [latte](https://github.com/mindspore-lab/mindone/blob/master/examples/latte) | support uncondition text to image fine tune |\n| [animate diff](https://github.com/mindspore-lab/mindone/blob/master/examples/animatediff) | support motion module and lora training |\n| [dynamicrafter](https://github.com/mindspore-lab/mindone/blob/master/examples/dynamicrafter) | support image to video generation |\n\n### run hf diffusers on mindspore\nmindone diffusers is under active development.\n\n| component | features \n| :--- | :-- \n| [pipeline](https://github.com/mindspore-lab/mindone/tree/master/mindone/diffusers/pipelines) | support text2image,text2video,text2audio tasks 30+\n| [models](https://github.com/mindspore-lab/mindone/tree/master/mindone/diffusers/models) | support audoencoder & transformers base models same as hf diffusers\n| [schedulers](https://github.com/mindspore-lab/mindone/tree/master/mindone/diffusers/schedulers) | support ddpm & dpm solver 10+ schedulers same as hf diffusers\n",
"bugtrack_url": null,
"license": null,
"summary": "ONE for all, Optimal generator with No Exception.",
"version": "0.2.0",
"project_urls": {
"Documentation": "https://mindspore-lab.github.io/mindone/",
"Homepage": "https://pypi.org/project/mindone/",
"Issues": "https://github.com/mindspore-lab/mindone/issues",
"Source": "https://github.com/mindspore-lab/mindone"
},
"split_keywords": [
"artificial intelligence",
" deep learning",
" diffusion",
" generative model",
" mindspore"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "803cc0032d845933ba54cd380f33501b7aa5f44140729f4bd8c138ab57a4c9cd",
"md5": "ce39b018f36797862a7a637e12ff9325",
"sha256": "b855af3649949bd5b2f44cb2e574853a154e4c0d4329b2e49b4994d06b8e0d41"
},
"downloads": -1,
"filename": "mindone-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ce39b018f36797862a7a637e12ff9325",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 1766138,
"upload_time": "2024-11-06T08:11:12",
"upload_time_iso_8601": "2024-11-06T08:11:12.974931Z",
"url": "https://files.pythonhosted.org/packages/80/3c/c0032d845933ba54cd380f33501b7aa5f44140729f4bd8c138ab57a4c9cd/mindone-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1aa44b90107504fccf1b83abf7a38f6f6a91876f79c80bbcb437a2f61651cb52",
"md5": "8b5b7739d6d0a0b1ceccf464edef0d3a",
"sha256": "c52f0686513036069eca9f5f0d002de81e967decb68cbec8fa9f73b8af655203"
},
"downloads": -1,
"filename": "mindone-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "8b5b7739d6d0a0b1ceccf464edef0d3a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 1384405,
"upload_time": "2024-11-06T08:11:15",
"upload_time_iso_8601": "2024-11-06T08:11:15.172644Z",
"url": "https://files.pythonhosted.org/packages/1a/a4/4b90107504fccf1b83abf7a38f6f6a91876f79c80bbcb437a2f61651cb52/mindone-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-06 08:11:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mindspore-lab",
"github_project": "mindone",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "mindone"
}
JSON
