| Name | seed-env JSON |
| Version |
0.1.0a1
JSON |
| download |
| home_page | None |
| Summary | A CLI tool for environment setup that combines a seeder project with your specific requirements. |
| upload_time | 2025-07-28 12:49:47 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.7 |
| license | None |
| keywords |
cli
environment
seeding
configuration
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Seed-Env CLI Tool
This seed-env command-line tool creates stable, reproducible Python environments for Machine Learning projects.
It's designed to solve complex dependency management challenges, especially for projects built on rapidly evolving
frameworks like JAX.
This seed-env command-line tool creates stable, reproducible Python environments for Machine Learning projects.
It's designed to solve complex dependency management challenges, especially for projects built on rapidly evolving
frameworks like JAX.
## What is seed-env tool?
`seed-env` is a Python command-line tool that automates the creation of consistent development and production environments.
The seed-env CLI tool's design centers around a methodology that bases (i.e. seeds) the Python environment on the thoroughly
tested dependency graph of [JAX](https://github.com/jax-ml/jax). The tool will take project-specific dependencies
(e.g., MaxText's [requirements.txt](https://github.com/AI-Hypercomputer/maxtext/blob/main/requirements.txt))
and intelligently layer them on top of the JAX seed, resolving conflicts to creat a stable final environment.
### Key concepts
- **Seed**: A Seed project (like [JAX](https://github.com/jax-ml/jax)) is the foundational dependency of a Host project.
A Seed project should include a `requirements_lock.txt` with only **pinned** and thoroughly tested dependencies, representing
a stable set for the Host project. For example, a Host project primarily dependent on JAX could use its
[requirements_lock_3_11.txt](https://github.com/jax-ml/jax/blob/main/build/requirements_lock_3_11.txt) as its seed for Python 3.11.
- **Host**: The Host project is your main project that you want to create a stable environment for.
The Host project depends on a foundational Seed project like JAX. The Host repository's `requirements.txt` should be **non-pinned**
(preferably lower-bound) dependencies to prevent conflicts with the Seed. Any dependencies from source links, e.g.,
`git+` prefix followed by the git repository URL, must be pinned to a specific
commit for reproducibility. For example, [MaxText](https://github.com/AI-Hypercomputer/maxtext/tree/main/MaxText) serves as a
Host repository, built upon the [JAX](https://github.com/jax-ml/jax) Seed environment.
### Key artifacts <a id="key-artifacts"></a>
The tool generates several key artifacts:
- **Lock Files**: It creates `uv.lock` and `requirements_lock.txt` files with every package version pinned, ensuring
the environment of the host repository can be recreated perfectly every time.
- **Project Definition File**: It creates a `pyproject.toml` file that defines the host repository's dependencies with
lower-bound version constraints.
- **(Optional) Installable Dependency PyPI Package**: If user specifies the `--build-pypi-package` flag when using the
`seed-env` tool, it builds a distributable Python package (`.whl` file and `tar.gz.` file) containing all of the
host repository's dependencies.
## How to Use It?
### Prerequisites
Before you begin, ensure you have the following:
- Ensure your Seed repo's `requirements_lock.txt` contains only **pinned dependencies**.
- Host Repo's `requirements.txt`: Make sure that your Host repo's `requirements.txt` contains **non-pinned dependencies**.
Additionally, all GitHub-importeddependencies must be pinned to a specific commit
(e.g., `google-jetstream @ https://github.com/AI-Hypercomputer/JetStream/archive/261f25007e4d12bb57cf8d5d61e291ba8f18430f.zip`).
### Install the `seed-env` tool
Clone the repository and install the tool:
```shell
git clone https://github.com/google-ml-infra/actions.git
cd actions/python_seed_env
# Build the seed-env CLI tool by running
pip install .
# Or run the following command if you want to edit and run pytest
# pip install -e . [dev]
```
### Example commands
Here are some common ways to use the seed-env tool:
```shell
# See all the arguments of the tool
seed-env --help
# Run the following command with minimal arguments needed to generate requirement lock files for maxtext based on the latest release jax as seed.
seed-env --host-repo=AI-Hypercomputer/maxtext --host-requirements=requirements.txt
# Run the following command to build lock files and a pypi package for maxtext at a specific commit and use jax at a specific commit/tag.
seed-env --host-repo=AI-Hypercomputer/maxtext --host-requirements=requirements.txt --host-commit=<a maxtext commit> --seed-config=jax_seed.yaml --seed-commit="jax-v0.6.2" --python-version="3.12" --hardware="tpu" --build-pypi-package
# Run the following command build lock files and a pypi package based on a local host requirement file and use the latest release jax as seed.
seed-env --local-requirements=<local path to a requirements.txt file> --build-pypi-package
```
### Utilize the generated artifacts to install host repo's environment
After running the `seed-env` tool, a few [artifacts](#key-artifacts) will be generated in the `generated_env` directory.
The typical structure looks like this:
```
generated_env
├── build
│ └── bdist
├── dist
│ └── <host-repo>-<version>.whl # generated dependency PyPI package
├── <host-repo>.egg-info
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ ├── dependency_links.txt
│ ├── requires.txt
│ └── top_level.txt
├── <host-repo>_requirements_lock_<python-version>.txt # generated requirements_lock.txt lock file
├── pyproject.toml # generated project definition file
└── uv.lock # generated uv.lock lock file
```
Before installing dependencies with these artifacts, it's **highly recommended** to start with a clean Python environment
(e.g., a new virtual environment) to avoid conflicts and ensure a predictable installation.
For example, you can create a virtual environment with the uv tool like so:
```shell
cd <path/to/your/project>
uv venv --python <python-version> --seed <venv-name>
source <venv-name>/bin/activate
python --version
```
With these generated artifacts, you can set up the development environment for your host repository using either of the following approaches:
#### Option A: Using Lock Files
- For `uv` users: Move the generated `uv.lock` file to your project root and execute `uv sync`.
- For `pip` users: Run `python -m pip install -r <path_to_the_project_requirements_lock_file>`.
#### Option B: Using the Dependency PyPI Packages
```shell
uv pip install <path_to_generated_dependency_package.whl> --resolution=lowest
```
## How to Add a New Seed Project?
To add a new seed project, refer to the [jax_seed.yaml](https://github.com/google-ml-infra/actions/blob/main/python_seed_env/src/seed_env/seeder_configs/jax_seed.yaml) file located in src/seed_env/seeder_configs. This folder stores seeder project configuration YAMLs for runtime data access (currently, only JAX is supported).
Create a similar YAML file, updating the configuration values to match your seeder project. Then, invoke the seed-env CLI tool using the `--seed-config` flag, providing either a relative or absolute path to your new YAML file. The tool will first check its package data, then look for the file locally if not found.
> [!WARNING]
> This tool is still under construction at this time.
Raw data
{
"_id": null,
"home_page": null,
"name": "seed-env",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "cli, environment, seeding, configuration",
"author": null,
"author_email": "ml-velocity authors <ml-velocity-team+seed-env@google.com>",
"download_url": "https://files.pythonhosted.org/packages/76/cf/599c6482b8a74f52a9724545c2d4fa0f3fbe072d64705591264b95140b2c/seed_env-0.1.0a1.tar.gz",
"platform": null,
"description": "# Seed-Env CLI Tool\n\nThis seed-env command-line tool creates stable, reproducible Python environments for Machine Learning projects.\nIt's designed to solve complex dependency management challenges, especially for projects built on rapidly evolving\nframeworks like JAX.\nThis seed-env command-line tool creates stable, reproducible Python environments for Machine Learning projects.\nIt's designed to solve complex dependency management challenges, especially for projects built on rapidly evolving\nframeworks like JAX.\n\n## What is seed-env tool?\n\n`seed-env` is a Python command-line tool that automates the creation of consistent development and production environments.\nThe seed-env CLI tool's design centers around a methodology that bases (i.e. seeds) the Python environment on the thoroughly\ntested dependency graph of [JAX](https://github.com/jax-ml/jax). The tool will take project-specific dependencies\n(e.g., MaxText's [requirements.txt](https://github.com/AI-Hypercomputer/maxtext/blob/main/requirements.txt))\nand intelligently layer them on top of the JAX seed, resolving conflicts to creat a stable final environment.\n\n### Key concepts\n\n- **Seed**: A Seed project (like [JAX](https://github.com/jax-ml/jax)) is the foundational dependency of a Host project.\nA Seed project should include a `requirements_lock.txt` with only **pinned** and thoroughly tested dependencies, representing\na stable set for the Host project. For example, a Host project primarily dependent on JAX could use its\n[requirements_lock_3_11.txt](https://github.com/jax-ml/jax/blob/main/build/requirements_lock_3_11.txt) as its seed for Python 3.11.\n- **Host**: The Host project is your main project that you want to create a stable environment for.\nThe Host project depends on a foundational Seed project like JAX. The Host repository's `requirements.txt` should be **non-pinned**\n(preferably lower-bound) dependencies to prevent conflicts with the Seed. Any dependencies from source links, e.g.,\n`git+` prefix followed by the git repository URL, must be pinned to a specific\ncommit for reproducibility. For example, [MaxText](https://github.com/AI-Hypercomputer/maxtext/tree/main/MaxText) serves as a\nHost repository, built upon the [JAX](https://github.com/jax-ml/jax) Seed environment.\n\n### Key artifacts <a id=\"key-artifacts\"></a>\n\nThe tool generates several key artifacts:\n\n- **Lock Files**: It creates `uv.lock` and `requirements_lock.txt` files with every package version pinned, ensuring\nthe environment of the host repository can be recreated perfectly every time.\n- **Project Definition File**: It creates a `pyproject.toml` file that defines the host repository's dependencies with\nlower-bound version constraints.\n- **(Optional) Installable Dependency PyPI Package**: If user specifies the `--build-pypi-package` flag when using the\n`seed-env` tool, it builds a distributable Python package (`.whl` file and `tar.gz.` file) containing all of the\nhost repository's dependencies.\n\n## How to Use It?\n\n### Prerequisites\n\nBefore you begin, ensure you have the following:\n\n- Ensure your Seed repo's `requirements_lock.txt` contains only **pinned dependencies**.\n- Host Repo's `requirements.txt`: Make sure that your Host repo's `requirements.txt` contains **non-pinned dependencies**.\nAdditionally, all GitHub-importeddependencies must be pinned to a specific commit\n(e.g., `google-jetstream @ https://github.com/AI-Hypercomputer/JetStream/archive/261f25007e4d12bb57cf8d5d61e291ba8f18430f.zip`).\n\n### Install the `seed-env` tool\n\nClone the repository and install the tool:\n\n```shell\ngit clone https://github.com/google-ml-infra/actions.git\ncd actions/python_seed_env\n\n# Build the seed-env CLI tool by running\npip install .\n\n\n# Or run the following command if you want to edit and run pytest\n# pip install -e . [dev]\n```\n\n### Example commands\n\nHere are some common ways to use the seed-env tool:\n\n```shell\n# See all the arguments of the tool\nseed-env --help\n\n\n# Run the following command with minimal arguments needed to generate requirement lock files for maxtext based on the latest release jax as seed.\nseed-env --host-repo=AI-Hypercomputer/maxtext --host-requirements=requirements.txt\n\n\n# Run the following command to build lock files and a pypi package for maxtext at a specific commit and use jax at a specific commit/tag.\nseed-env --host-repo=AI-Hypercomputer/maxtext --host-requirements=requirements.txt --host-commit=<a maxtext commit> --seed-config=jax_seed.yaml --seed-commit=\"jax-v0.6.2\" --python-version=\"3.12\" --hardware=\"tpu\" --build-pypi-package\n\n\n# Run the following command build lock files and a pypi package based on a local host requirement file and use the latest release jax as seed.\nseed-env --local-requirements=<local path to a requirements.txt file> --build-pypi-package\n```\n\n### Utilize the generated artifacts to install host repo's environment\n\nAfter running the `seed-env` tool, a few [artifacts](#key-artifacts) will be generated in the `generated_env` directory.\nThe typical structure looks like this:\n\n```\ngenerated_env\n\u251c\u2500\u2500 build\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 bdist\n\u251c\u2500\u2500 dist\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 <host-repo>-<version>.whl # generated dependency PyPI package\n\u251c\u2500\u2500 <host-repo>.egg-info\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 PKG-INFO\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 SOURCES.txt\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 dependency_links.txt\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 requires.txt\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 top_level.txt\n\u251c\u2500\u2500 <host-repo>_requirements_lock_<python-version>.txt # generated requirements_lock.txt lock file\n\u251c\u2500\u2500 pyproject.toml # generated project definition file\n\u2514\u2500\u2500 uv.lock # generated uv.lock lock file\n```\n\nBefore installing dependencies with these artifacts, it's **highly recommended** to start with a clean Python environment\n(e.g., a new virtual environment) to avoid conflicts and ensure a predictable installation.\n\nFor example, you can create a virtual environment with the uv tool like so:\n\n```shell\ncd <path/to/your/project>\nuv venv --python <python-version> --seed <venv-name>\nsource <venv-name>/bin/activate\npython --version\n```\n\nWith these generated artifacts, you can set up the development environment for your host repository using either of the following approaches:\n\n#### Option A: Using Lock Files\n\n- For `uv` users: Move the generated `uv.lock` file to your project root and execute `uv sync`.\n- For `pip` users: Run `python -m pip install -r <path_to_the_project_requirements_lock_file>`.\n\n#### Option B: Using the Dependency PyPI Packages\n\n```shell\nuv pip install <path_to_generated_dependency_package.whl> --resolution=lowest\n```\n\n## How to Add a New Seed Project?\n\nTo add a new seed project, refer to the [jax_seed.yaml](https://github.com/google-ml-infra/actions/blob/main/python_seed_env/src/seed_env/seeder_configs/jax_seed.yaml) file located in src/seed_env/seeder_configs. This folder stores seeder project configuration YAMLs for runtime data access (currently, only JAX is supported).\n\nCreate a similar YAML file, updating the configuration values to match your seeder project. Then, invoke the seed-env CLI tool using the `--seed-config` flag, providing either a relative or absolute path to your new YAML file. The tool will first check its package data, then look for the file locally if not found.\n\n> [!WARNING]\n> This tool is still under construction at this time.\n",
"bugtrack_url": null,
"license": null,
"summary": "A CLI tool for environment setup that combines a seeder project with your specific requirements.",
"version": "0.1.0a1",
"project_urls": {
"Homepage": "https://github.com/google-ml-infra/actions/python_seed_env",
"Source Code": "https://github.com/google-ml-infra/actions/python_seed_env"
},
"split_keywords": [
"cli",
" environment",
" seeding",
" configuration"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "79442d70a74c377a5c7b7389903ee297eac2122125e138a63bb70f336367a159",
"md5": "d69f3e85f9a56b060dde21fe35ae5b5e",
"sha256": "df9fc53f24b83454f4444af043ac57ed637a25c819b7e6ab3c1d8aeb52032d98"
},
"downloads": -1,
"filename": "seed_env-0.1.0a1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d69f3e85f9a56b060dde21fe35ae5b5e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 24427,
"upload_time": "2025-07-28T12:49:46",
"upload_time_iso_8601": "2025-07-28T12:49:46.038372Z",
"url": "https://files.pythonhosted.org/packages/79/44/2d70a74c377a5c7b7389903ee297eac2122125e138a63bb70f336367a159/seed_env-0.1.0a1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "76cf599c6482b8a74f52a9724545c2d4fa0f3fbe072d64705591264b95140b2c",
"md5": "74aa60b09f556eaf74bcc98e3f8462ec",
"sha256": "ec66c813b6d26f1902b85989566d6330a95947f84d709439f293a17179b4e911"
},
"downloads": -1,
"filename": "seed_env-0.1.0a1.tar.gz",
"has_sig": false,
"md5_digest": "74aa60b09f556eaf74bcc98e3f8462ec",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 26956,
"upload_time": "2025-07-28T12:49:47",
"upload_time_iso_8601": "2025-07-28T12:49:47.465691Z",
"url": "https://files.pythonhosted.org/packages/76/cf/599c6482b8a74f52a9724545c2d4fa0f3fbe072d64705591264b95140b2c/seed_env-0.1.0a1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-28 12:49:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "google-ml-infra",
"github_project": "actions",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "seed-env"
}
