# mgnify-pipelines-toolkit
This Python package contains a collection of scripts and tools for including in MGnify pipelines. Scripts stored here are mainly for:
- One-off production scripts that perform specific tasks in pipelines
- Scripts that have few dependencies
- Scripts that don't have existing containers built to run them
- Scripts for which building an entire container would be too bulky of a solution to deploy in pipelines
This package is built and uploaded to PyPi and bioconda. The package bundles scripts and makes them executable from the command-line when this package is installed.
## How to install
This package is available both on [PyPi](https://pypi.org/project/mgnify-pipelines-toolkit/) and bioconda.
To install from PyPi with pip:
`pip install mgnify-pipelines-toolkit`
To install from bioconda with conda/mamba:
`conda install -c bioconda mgnify-pipelines-toolkit`
You should then be able to run the packages from the command-line. For example to run the `get_subunits.py` script:
`get_subunits -i ${easel_coords} -n ${meta.id}`
## Adding a new script to the package
### Local development requirements
Before starting any development, you should do these few steps:
- Clone the repo if you haven't already and create a feature branch from the `dev` branch (NOT `main`).
- Create a virtual environment with the tool of your choice (i.e. `conda create --name my_new_env`)
- Activate you new environment (i.e. `conda activate my_new_env`)
- Install dev dependencies `pip install -e '.[dev]'`
- Install pre-commit hooks `pre-commit install`
When doing these steps above, you ensure that the code you add will be linted and formatted properly.
### New script requirements
There are a few requirements for your script:
- It needs to have a named main function of some kind. See `mgnify_pipelines_toolkit/analysis/shared/get_subunits.py` and the `main()` function for an example
- Because this package is meant to be run from the command-line, make sure your script can easily pass arguments using tools like `argparse` or `click`
- A small amount of dependencies. This requirement is subjective, but for example if your script only requires a handful of basic packages like `Biopython`, `numpy`, `pandas`, etc., then it's fine. However if the script has a more extensive list of dependencies, a container is probably a better fit.
### How to add a new script
To add a new Python script, first copy it over to the `mgnify_pipelines_toolkit` directory in this repository, specifically to the subdirectory that makes the most sense. If none of the subdirectories make sense for your script, create a new one. If your script doesn't have a `main()` type function yet, write one.
Then, open `pyproject.toml` as you will need to add some bits. First, add any missing dependencies (include the version) to the `dependencies` field.
Then, if you created a new subdirectory to add your script in, go to the `packages` line under `[tool.setuptools]` and add the new subdirectory following the same syntax.
Then, scroll down to the `[project.scripts]` line. Here, you will create an alias command for running your script from the command-line. In the example line:
`get_subunits = "mgnify_pipelines_toolkit.analysis.shared.get_subunits:main"`
- `get_subunits` is the alias
- `mgnify_pipelines_toolkit.analysis.shared.get_subunits` will link the alias to the script with the path `mgnify_pipelines_toolkit/analysis/shared/get_subunits.py`
- `:main` will specifically call the function named `main()` when the alias is run.
When you have setup this command, executing `get_subunits` on the command-line will be the equivalent of doing:
`from mgnify_pipelines_toolkit.analysis.shared.get_subunits import main; main()`
You should then write at least one unit test for your addition. This package uses `pytest` at the moment for this purpose. A GitHub Action workflow will run all of the unit tests whenever a commit is pushed to any branch.
Finally, you will need to bump up the version in the `version` line.
At the moment, these should be the only steps required to setup your script in this package (which is subject to change).
### Building and uploading to PyPi
The building and pushing of the package is automated by GitHub Actions, which will activate only on a new release. Bioconda should then automatically pick up the new PyPi release and push it to their recipes, though it's worth keeping an eye on their automated pull requests just in case [here](https://github.com/bioconda/bioconda-recipes/pulls).
Raw data
{
"_id": null,
"home_page": null,
"name": "mgnify-pipelines-toolkit",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "bioinformatics, pipelines, metagenomics",
"author": null,
"author_email": "MGnify team <metagenomics-help@ebi.ac.uk>",
"download_url": "https://files.pythonhosted.org/packages/88/0a/006fe1047af4e2c0e69aa20f69051e6995e8b774b05f6f6b4e40b3219f8f/mgnify_pipelines_toolkit-0.1.8.tar.gz",
"platform": null,
"description": "# mgnify-pipelines-toolkit\n\nThis Python package contains a collection of scripts and tools for including in MGnify pipelines. Scripts stored here are mainly for:\n\n- One-off production scripts that perform specific tasks in pipelines\n- Scripts that have few dependencies\n- Scripts that don't have existing containers built to run them\n- Scripts for which building an entire container would be too bulky of a solution to deploy in pipelines\n\nThis package is built and uploaded to PyPi and bioconda. The package bundles scripts and makes them executable from the command-line when this package is installed.\n\n## How to install\n\nThis package is available both on [PyPi](https://pypi.org/project/mgnify-pipelines-toolkit/) and bioconda.\n\nTo install from PyPi with pip:\n\n`pip install mgnify-pipelines-toolkit`\n\nTo install from bioconda with conda/mamba:\n\n`conda install -c bioconda mgnify-pipelines-toolkit`\n\nYou should then be able to run the packages from the command-line. For example to run the `get_subunits.py` script:\n\n`get_subunits -i ${easel_coords} -n ${meta.id}`\n\n## Adding a new script to the package\n\n### Local development requirements\nBefore starting any development, you should do these few steps:\n- Clone the repo if you haven't already and create a feature branch from the `dev` branch (NOT `main`).\n- Create a virtual environment with the tool of your choice (i.e. `conda create --name my_new_env`)\n- Activate you new environment (i.e. `conda activate my_new_env`)\n- Install dev dependencies `pip install -e '.[dev]'`\n- Install pre-commit hooks `pre-commit install`\n\nWhen doing these steps above, you ensure that the code you add will be linted and formatted properly.\n\n### New script requirements\n\nThere are a few requirements for your script:\n\n- It needs to have a named main function of some kind. See `mgnify_pipelines_toolkit/analysis/shared/get_subunits.py` and the `main()` function for an example\n- Because this package is meant to be run from the command-line, make sure your script can easily pass arguments using tools like `argparse` or `click`\n- A small amount of dependencies. This requirement is subjective, but for example if your script only requires a handful of basic packages like `Biopython`, `numpy`, `pandas`, etc., then it's fine. However if the script has a more extensive list of dependencies, a container is probably a better fit.\n\n### How to add a new script\n\nTo add a new Python script, first copy it over to the `mgnify_pipelines_toolkit` directory in this repository, specifically to the subdirectory that makes the most sense. If none of the subdirectories make sense for your script, create a new one. If your script doesn't have a `main()` type function yet, write one.\n\nThen, open `pyproject.toml` as you will need to add some bits. First, add any missing dependencies (include the version) to the `dependencies` field.\n\nThen, if you created a new subdirectory to add your script in, go to the `packages` line under `[tool.setuptools]` and add the new subdirectory following the same syntax.\n\nThen, scroll down to the `[project.scripts]` line. Here, you will create an alias command for running your script from the command-line. In the example line:\n\n`get_subunits = \"mgnify_pipelines_toolkit.analysis.shared.get_subunits:main\"`\n\n- `get_subunits` is the alias\n- `mgnify_pipelines_toolkit.analysis.shared.get_subunits` will link the alias to the script with the path `mgnify_pipelines_toolkit/analysis/shared/get_subunits.py`\n- `:main` will specifically call the function named `main()` when the alias is run.\n\nWhen you have setup this command, executing `get_subunits` on the command-line will be the equivalent of doing:\n\n`from mgnify_pipelines_toolkit.analysis.shared.get_subunits import main; main()`\n\nYou should then write at least one unit test for your addition. This package uses `pytest` at the moment for this purpose. A GitHub Action workflow will run all of the unit tests whenever a commit is pushed to any branch.\n\nFinally, you will need to bump up the version in the `version` line.\n\nAt the moment, these should be the only steps required to setup your script in this package (which is subject to change).\n\n### Building and uploading to PyPi\n\nThe building and pushing of the package is automated by GitHub Actions, which will activate only on a new release. Bioconda should then automatically pick up the new PyPi release and push it to their recipes, though it's worth keeping an eye on their automated pull requests just in case [here](https://github.com/bioconda/bioconda-recipes/pulls).\n",
"bugtrack_url": null,
"license": "Apache Software License 2.0",
"summary": "Collection of scripts and tools for MGnify pipelines",
"version": "0.1.8",
"project_urls": null,
"split_keywords": [
"bioinformatics",
" pipelines",
" metagenomics"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "30fa9828d400db08f09119dcf81e13fe038a29bebc8aee35aafe9e75e47f41a0",
"md5": "ecc56de0e4f1fa64aba43da3d80b7253",
"sha256": "793aa0dfd7e2a3472bf6db3c0d773fe582daf15c65f204fad8852ddb51ad5bec"
},
"downloads": -1,
"filename": "mgnify_pipelines_toolkit-0.1.8-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ecc56de0e4f1fa64aba43da3d80b7253",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 53130,
"upload_time": "2024-10-23T15:23:48",
"upload_time_iso_8601": "2024-10-23T15:23:48.732819Z",
"url": "https://files.pythonhosted.org/packages/30/fa/9828d400db08f09119dcf81e13fe038a29bebc8aee35aafe9e75e47f41a0/mgnify_pipelines_toolkit-0.1.8-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "880a006fe1047af4e2c0e69aa20f69051e6995e8b774b05f6f6b4e40b3219f8f",
"md5": "4bda226e76b2d456b0c7048c8adad528",
"sha256": "57c299db9d80ab66c343d53a12382871eb166cf5e55470197ff4447fb7f93e78"
},
"downloads": -1,
"filename": "mgnify_pipelines_toolkit-0.1.8.tar.gz",
"has_sig": false,
"md5_digest": "4bda226e76b2d456b0c7048c8adad528",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 34481,
"upload_time": "2024-10-23T15:23:49",
"upload_time_iso_8601": "2024-10-23T15:23:49.920466Z",
"url": "https://files.pythonhosted.org/packages/88/0a/006fe1047af4e2c0e69aa20f69051e6995e8b774b05f6f6b4e40b3219f8f/mgnify_pipelines_toolkit-0.1.8.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-23 15:23:49",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "mgnify-pipelines-toolkit"
}
JSON