[](https://pepy.tech/project/cmind)
[](https://github.com/mlcommons/ck/tree/master/cm/cmind)
[](LICENSE.md)
[](https://pepy.tech/project/cmind)
[](https://arxiv.org/abs/2406.16791)
[](https://github.com/mlcommons/ck/actions/workflows/test-cm.yml)
[](https://github.com/mlcommons/ck/actions/workflows/test-cm-script-features.yml)
### About
Collective Mind (CM) is a small, modular, cross-platform and decentralized workflow automation framework
with a human-friendly interface to make it easier to build, run, benchmark and optimize applications
across diverse models, data sets, software and hardware.
CM is a part of [Collective Knowledge (CK)](https://github.com/mlcommons/ck) -
an educational community project to learn how to run emerging workloads
in the most efficient and cost-effective way across diverse
and continuously changing systems.
CM includes a collection of portable, extensible and technology-agnostic automation recipes
with a common API and CLI (aka CM scripts) to unify and automate different steps
required to compose, run, benchmark and optimize complex ML/AI applications
on any platform with any software and hardware.
CM scripts extend the concept of `cmake` with simple Python automations, native scripts
and JSON/YAML meta descriptions. They require Python 3.7+ with minimal dependencies and are
[continuously extended by the community and MLCommons members](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)
to run natively on Ubuntu, MacOS, Windows, RHEL, Debian, Amazon Linux
and any other operating system, in a cloud or inside automatically generated containers
while keeping backward compatibility.
CM scripts were originally developed based on the following requirements from the
[MLCommons members](https://mlcommons.org)
to help them automatically compose and optimize complex MLPerf benchmarks, applications and systems
across diverse and continuously changing models, data sets, software and hardware
from Nvidia, Intel, AMD, Google, Qualcomm, Amazon and other vendors:
* must work out of the box with the default options and without the need to edit some paths, environment variables and configuration files;
* must be non-intrusive, easy to debug and must reuse existing
user scripts and automation tools (such as cmake, make, ML workflows,
python poetry and containers) rather than substituting them;
* must have a very simple and human-friendly command line with a Python API and minimal dependencies;
* must require minimal or zero learning curve by using plain Python, native scripts, environment variables
and simple JSON/YAML descriptions instead of inventing new workflow languages;
* must have the same interface to run all automations natively, in a cloud or inside containers.
### Maintainers
* [Grigori Fursin](https://cKnowledge.org/gfursin)
### Resources
* CM v2.x (2022-cur) (stable): [installation on Linux, Windows, MacOS](https://access.cknowledge.org/playground/?action=install) ;
[docs](https://docs.mlcommons.org/ck) ; [popular commands](https://github.com/mlcommons/ck/tree/master/cm/docs/demos/some-cm-commands.md) ;
[getting started guide](https://github.com/mlcommons/ck/blob/master/docs/getting-started.md)
* CM v3.x aka CMX (2024-cur) (stable): [docs](https://github.com/orgs/mlcommons/projects/46)
* MLPerf inference benchmark automated via CM
* [Run MLPerf for submissions](https://docs.mlcommons.org/inference)
* [Run MLPerf at the Student Cluster Competition'24](https://docs.mlcommons.org/inference/benchmarks/text_to_image/reproducibility/scc24)
* Examples of modular containers and GitHub actions with CM commands:
* [GitHub action with CM commands to test MLPerf inference benchmark](https://github.com/mlcommons/inference/blob/master/.github/workflows/test-bert.yml)
* [Dockerfile to run MLPerf inference benchmark via CM](https://github.com/mlcommons/ck/blob/master/cm-mlops/script/app-mlperf-inference/dockerfiles/bert-99.9/ubuntu_22.04_python_onnxruntime_cpu.Dockerfile)
### License
[Apache 2.0](LICENSE.md)
### Citing CM project
If you found CM automations useful, please cite this article:
[ [ArXiv](https://arxiv.org/abs/2406.16791) ], [ [BibTex](https://github.com/mlcommons/ck/blob/master/citation.bib) ].
You can learn more about the motivation behind these projects from the following presentations:
* "Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments": [ [ArXiv](https://arxiv.org/abs/2406.16791) ]
* ACM REP'23 keynote about the MLCommons CM automation framework: [ [slides](https://doi.org/10.5281/zenodo.8105339) ]
* ACM TechTalk'21 about Collective Knowledge project: [ [YouTube](https://www.youtube.com/watch?v=7zpeIVwICa4) ] [ [slides](https://learning.acm.org/binaries/content/assets/leaning-center/webinar-slides/2021/grigorifursin_techtalk_slides.pdf) ]
### Acknowledgments
The Collective Mind (CM) automation framework was originally
developed by [Grigori Fursin](https://cKnowledge.org/gfursin),
as a part of the [Collective Knowledge educational initiative](https://cKnowledge.org),
sponsored by [cTuning.org](https://cTuning.org) and [cKnowledge.org](https://cKnowledge.org),
and contributed to MLCommons for the benefit of all.
This open-source technology, including CM4MLOps/CM4MLPerf, CM4ABTF, CM4Research, and more,
is a collaborative community-driven project made possible by our
[amazing volunteers, collaborators, and contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)!
Raw data
{
"_id": null,
"home_page": null,
"name": "cmind",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "Grigori Fursin <grigori.fursin@ctuning.org>",
"keywords": "cmind, cm, cmx, collective mind, automation, portability, reusability, mlops, devops, vmlops, api, cli",
"author": null,
"author_email": "Grigori Fursin <grigori.fursin@ctuning.org>",
"download_url": "https://files.pythonhosted.org/packages/7f/63/481c9dd4129e2f3dc034768192c9e064d0a6ca28dfe344f5fc6b259b88b1/cmind-3.4.2.tar.gz",
"platform": null,
"description": "[](https://pepy.tech/project/cmind)\n[](https://github.com/mlcommons/ck/tree/master/cm/cmind)\n[](LICENSE.md)\n[](https://pepy.tech/project/cmind)\n\n[](https://arxiv.org/abs/2406.16791)\n[](https://github.com/mlcommons/ck/actions/workflows/test-cm.yml)\n[](https://github.com/mlcommons/ck/actions/workflows/test-cm-script-features.yml)\n\n### About\n\nCollective Mind (CM) is a small, modular, cross-platform and decentralized workflow automation framework \nwith a human-friendly interface to make it easier to build, run, benchmark and optimize applications \nacross diverse models, data sets, software and hardware.\n\nCM is a part of [Collective Knowledge (CK)](https://github.com/mlcommons/ck) - \nan educational community project to learn how to run emerging workloads \nin the most efficient and cost-effective way across diverse \nand continuously changing systems.\n\nCM includes a collection of portable, extensible and technology-agnostic automation recipes\nwith a common API and CLI (aka CM scripts) to unify and automate different steps \nrequired to compose, run, benchmark and optimize complex ML/AI applications \non any platform with any software and hardware. \n\nCM scripts extend the concept of `cmake` with simple Python automations, native scripts\nand JSON/YAML meta descriptions. They require Python 3.7+ with minimal dependencies and are \n[continuously extended by the community and MLCommons members](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)\nto run natively on Ubuntu, MacOS, Windows, RHEL, Debian, Amazon Linux\nand any other operating system, in a cloud or inside automatically generated containers\nwhile keeping backward compatibility.\n\nCM scripts were originally developed based on the following requirements from the\n[MLCommons members](https://mlcommons.org) \nto help them automatically compose and optimize complex MLPerf benchmarks, applications and systems\nacross diverse and continuously changing models, data sets, software and hardware\nfrom Nvidia, Intel, AMD, Google, Qualcomm, Amazon and other vendors:\n* must work out of the box with the default options and without the need to edit some paths, environment variables and configuration files;\n* must be non-intrusive, easy to debug and must reuse existing \n user scripts and automation tools (such as cmake, make, ML workflows, \n python poetry and containers) rather than substituting them; \n* must have a very simple and human-friendly command line with a Python API and minimal dependencies;\n* must require minimal or zero learning curve by using plain Python, native scripts, environment variables \n and simple JSON/YAML descriptions instead of inventing new workflow languages;\n* must have the same interface to run all automations natively, in a cloud or inside containers.\n\n### Maintainers\n\n* [Grigori Fursin](https://cKnowledge.org/gfursin)\n\n### Resources\n\n* CM v2.x (2022-cur) (stable): [installation on Linux, Windows, MacOS](https://access.cknowledge.org/playground/?action=install) ; \n [docs](https://docs.mlcommons.org/ck) ; [popular commands](https://github.com/mlcommons/ck/tree/master/cm/docs/demos/some-cm-commands.md) ; \n [getting started guide](https://github.com/mlcommons/ck/blob/master/docs/getting-started.md)\n* CM v3.x aka CMX (2024-cur) (stable): [docs](https://github.com/orgs/mlcommons/projects/46)\n* MLPerf inference benchmark automated via CM\n * [Run MLPerf for submissions](https://docs.mlcommons.org/inference)\n * [Run MLPerf at the Student Cluster Competition'24](https://docs.mlcommons.org/inference/benchmarks/text_to_image/reproducibility/scc24)\n* Examples of modular containers and GitHub actions with CM commands:\n * [GitHub action with CM commands to test MLPerf inference benchmark](https://github.com/mlcommons/inference/blob/master/.github/workflows/test-bert.yml)\n * [Dockerfile to run MLPerf inference benchmark via CM](https://github.com/mlcommons/ck/blob/master/cm-mlops/script/app-mlperf-inference/dockerfiles/bert-99.9/ubuntu_22.04_python_onnxruntime_cpu.Dockerfile)\n\n### License\n\n[Apache 2.0](LICENSE.md)\n\n### Citing CM project\n\nIf you found CM automations useful, please cite this article: \n[ [ArXiv](https://arxiv.org/abs/2406.16791) ], [ [BibTex](https://github.com/mlcommons/ck/blob/master/citation.bib) ].\n\nYou can learn more about the motivation behind these projects from the following presentations:\n\n* \"Enabling more efficient and cost-effective AI/ML systems with Collective Mind, virtualized MLOps, MLPerf, Collective Knowledge Playground and reproducible optimization tournaments\": [ [ArXiv](https://arxiv.org/abs/2406.16791) ]\n* ACM REP'23 keynote about the MLCommons CM automation framework: [ [slides](https://doi.org/10.5281/zenodo.8105339) ] \n* ACM TechTalk'21 about Collective Knowledge project: [ [YouTube](https://www.youtube.com/watch?v=7zpeIVwICa4) ] [ [slides](https://learning.acm.org/binaries/content/assets/leaning-center/webinar-slides/2021/grigorifursin_techtalk_slides.pdf) ]\n\n### Acknowledgments\n\nThe Collective Mind (CM) automation framework was originally \ndeveloped by [Grigori Fursin](https://cKnowledge.org/gfursin), \nas a part of the [Collective Knowledge educational initiative](https://cKnowledge.org),\nsponsored by [cTuning.org](https://cTuning.org) and [cKnowledge.org](https://cKnowledge.org), \nand contributed to MLCommons for the benefit of all. \nThis open-source technology, including CM4MLOps/CM4MLPerf, CM4ABTF, CM4Research, and more, \nis a collaborative community-driven project made possible by our \n[amazing volunteers, collaborators, and contributors](https://github.com/mlcommons/ck/blob/master/CONTRIBUTING.md)! \n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Collective Mind automation framework (CM)",
"version": "3.4.2",
"project_urls": {
"Changelog": "https://github.com/mlcommons/ck/blob/master/cm/CHANGES.md",
"Homepage": "https://cKnowledge.org",
"Issues": "https://github.com/mlcommons/ck/issues",
"Repository": "https://github.com/mlcommons/ck/tree/master/cm"
},
"split_keywords": [
"cmind",
" cm",
" cmx",
" collective mind",
" automation",
" portability",
" reusability",
" mlops",
" devops",
" vmlops",
" api",
" cli"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "efe85cf9b1198bfffd68f4b8e0e1c1607fe7b7cd9fb7faeaec4f4de2c70cd017",
"md5": "d034df66d2e96b0462b5ab420d2209fc",
"sha256": "734dfdec35fb6e26ae2b599b7bcdddd4d038d80c2d437f0f80030ad220464b70"
},
"downloads": -1,
"filename": "cmind-3.4.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d034df66d2e96b0462b5ab420d2209fc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 78856,
"upload_time": "2024-11-22T13:58:40",
"upload_time_iso_8601": "2024-11-22T13:58:40.784703Z",
"url": "https://files.pythonhosted.org/packages/ef/e8/5cf9b1198bfffd68f4b8e0e1c1607fe7b7cd9fb7faeaec4f4de2c70cd017/cmind-3.4.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7f63481c9dd4129e2f3dc034768192c9e064d0a6ca28dfe344f5fc6b259b88b1",
"md5": "5e7e2ac11080bf9f7a4d16afc9c85ecb",
"sha256": "d75d646e89f21f99304157779ffc5c96a9fd96be40f3b83a70981e139f207e2e"
},
"downloads": -1,
"filename": "cmind-3.4.2.tar.gz",
"has_sig": false,
"md5_digest": "5e7e2ac11080bf9f7a4d16afc9c85ecb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 68914,
"upload_time": "2024-11-22T13:58:42",
"upload_time_iso_8601": "2024-11-22T13:58:42.057058Z",
"url": "https://files.pythonhosted.org/packages/7f/63/481c9dd4129e2f3dc034768192c9e064d0a6ca28dfe344f5fc6b259b88b1/cmind-3.4.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-22 13:58:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "mlcommons",
"github_project": "ck",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "cmind"
}
JSON
