![CI](https://github.com/wandb/nb_helpers/workflows/CI/badge.svg)
[![PyPI](https://img.shields.io/pypi/v/nb-helpers?color=blue&label=pypi%20version)](https://pypi.org/project/nb_helpers/#description)
# nb_helpers
A simple tool to clean, test and fix notebooks for your repo
## Install
You can install from pypi:
```bash
pip install nb_helpers
```
or get latest:
```bash
pip install -e .
```
## Usage
This little library gives you command line tools to clean, test and check your jupyter notebooks.
- Clean: When you call `clean_nbs` it will strip notebooks from the metadata, this helps prevent git conflicts. You can also pass the flag `--clear_outs` and also remove cell outputs.
```bash
$ nb_helpers.clean_nbs --help tcapelle at MBP14.local (-)(main)
usage: nb_helpers.clean_nbs [-h] [--path PATH] [--clear_outs] [--verbose]
Clean notebooks on `path` from useless metadata
options:
-h, --help show this help message and exit
--path PATH The path to notebooks (default: .)
--clear_outs Remove cell outputs (default: False)
--verbose Rnun on verbose mdoe (default: False)
```
You can run this comman on this repo:
```bash
$ nb_helpers.clean_nbs
>
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ Notebook Path ┃ Status ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ tests/data/dummy_folder/fail_nb.ipynb │ Ok✔ │
│ tests/data/dummy_folder/test_nb2.ipynb │ Ok✔ │
│ tests/data/dummy_folder/test_nb_all_slow.ipynb │ Ok✔ │
│ tests/data/dummy_folder/test_nb_some_slow.ipynb │ Ok✔ │
│ tests/data/features_nb.ipynb │ Ok✔ │
│ tests/data/test_nb.ipynb │ Ok✔ │
└─────────────────────────────────────────────────┴────────┘
```
- Run: One can run the notebooks in `path` and get info about the execution.
```bash
$ nb_helpers.run_nbs --help tcapelle at MBP14 (--)(main)
usage: nb_helpers.run_nbs [-h] [--verbose] [--lib_name LIB_NAME] [--no_run] [--pip_install] [--github_issue] [--repo REPO] [--owner OWNER] [path]
positional arguments:
path A path to nb files (default: /Users/tcapelle/wandb/nb_helpers)
options:
-h, --help show this help message and exit
--verbose Print errors along the way (default: False)
--lib_name LIB_NAME Python lib names to filter, eg: tensorflow
--no_run Do not run any notebook (default: False)
--pip_install Run cells with !pip install (default: False)
--github_issue Create a github issue if notebook fails (default: False)
--repo REPO Github repo to create issue in (default: nb_helpers)
--owner OWNER Github owner to create issue in (default: wandb)
```
> You can now post github issues when running fails, the cool thing is that it can be posted to another repo than the one from the notebooks. Just pass the `--repo` name and the `--owner` (for example `wandb/other_cool_repo`)
You get the following output inside this repo:
```bash
$ nb_helpers.run_nbs
CONSOLE.is_terminal(): True
Writing output to run.csv
```
| Notebook Path |Status |Run Time |colab |
|--- | --- | --- | --- |
| dev_nbs/search.ipynb | Fail | 1 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/dev_nbs/search.ipynb) |
| tests/data/dummy_folder/fail_nb.ipynb | Fail | 1 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/dummy_folder/fail_nb.ipynb) |
| tests/data/dummy_folder/test_nb2.ipynb | Ok | 0 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/dummy_folder/test_nb2.ipynb) |
| tests/data/dummy_folder/test_nb_all_slow.ipynb | Skipped | 0 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/dummy_folder/test_nb_all_slow.ipynb) |
| tests/data/dummy_folder/test_nb_some_slow.ipynb | Ok | 0 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/dummy_folder/test_nb_some_slow.ipynb) |
| tests/data/features_nb.ipynb | Ok | 0 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/features_nb.ipynb) |
| tests/data/test_nb.ipynb | Ok | 0 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/test_nb.ipynb) |
- Summary:
You can get a summary of the notebooks in your project with the `nb_helpers.summary_nbs` function.
```bash
$ nb_helpers.summary_nbs
CONSOLE.is_terminal(): True
Writing output to /Users/tcapelle/wandb/nb_helpers/logs/summary.csv
Reading 6 notebooks
┌───┬─────────────────────────────────────────────────┬────────────┬────────────────┬────────────────────────────────────────────────┬────────────┬───────┐
│ # │ nb name │ tracker │ wandb features │ python libs │ colab_cell │ colab │
├───┼─────────────────────────────────────────────────┼────────────┼────────────────┼────────────────────────────────────────────────┼────────────┼───────┤
│ 1 │ tests/data/dummy_folder/fail_nb.ipynb │ │ │ │ │ open │
│ 2 │ tests/data/dummy_folder/test_nb2.ipynb │ │ │ │ │ open │
│ 3 │ tests/data/dummy_folder/test_nb_all_slow.ipynb │ │ │ time │ │ open │
│ 4 │ tests/data/dummy_folder/test_nb_some_slow.ipynb │ │ │ time │ │ open │
│ 5 │ tests/data/features_nb.ipynb │ │ │ typing, itertools │ │ open │
│ 6 │ tests/data/test_nb.ipynb │ 0: tracker │ │ os, sys, logging, pathlib, fastcore, itertools │ 1 │ open │
└───┴─────────────────────────────────────────────────┴────────────┴────────────────┴────────────────────────────────────────────────┴────────────┴───────┘
```
------------
## Python Lib
All this functions can also be used inside python:
```python
from pathlib import Path
from nb_helpers.run import run_nbs
examples_path = Path("examples/colabs")
errors = run_nbs(path=examples_path, verbose=True, timeout=600)
```
Also the library has many little functions to make your life easier inside the repo you are orchestrating:
```python
from pathlib import Path
from nb_helpers.utils import *
from nb_helpers.colab import *
examples_path = Path("examples/colabs")
# get all nbs in the folder recursevely, filters hidden, non nb stuff
nb_files = find_nbs(example_path)
one_nb_path = nb_files[0]
notebook = read_nb(one_nb_path)
# get all libs imported
libs = detect_imported_libs(notebook)
# get remote github repo
github_repo = git_origin_repo(one_nb_path)
# detect if master is called main or master
master_name = git_main_name(one_nb_path)
# get colab link
colab_url = get_colab_url(one_nb_path, branch)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/wandb/nb_helpers/tree/main/",
"name": "nb-helpers",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "jupyter notebook",
"author": "Thomas Capelle",
"author_email": "tcapelle@wandb.com",
"download_url": "https://files.pythonhosted.org/packages/3e/a0/22b53253444c052a8b17ca750d4f1e370d98b9ffe13bae4ce4d75fcedaf9/nb_helpers-0.6.2.tar.gz",
"platform": null,
"description": "![CI](https://github.com/wandb/nb_helpers/workflows/CI/badge.svg)\n[![PyPI](https://img.shields.io/pypi/v/nb-helpers?color=blue&label=pypi%20version)](https://pypi.org/project/nb_helpers/#description)\n# nb_helpers\n\nA simple tool to clean, test and fix notebooks for your repo\n\n## Install\nYou can install from pypi:\n```bash\npip install nb_helpers\n```\nor get latest:\n```bash\npip install -e .\n```\n\n## Usage\n\nThis little library gives you command line tools to clean, test and check your jupyter notebooks.\n\n- Clean: When you call `clean_nbs` it will strip notebooks from the metadata, this helps prevent git conflicts. You can also pass the flag `--clear_outs` and also remove cell outputs.\n\n```bash\n$ nb_helpers.clean_nbs --help tcapelle at MBP14.local (-)(main)\nusage: nb_helpers.clean_nbs [-h] [--path PATH] [--clear_outs] [--verbose]\n\nClean notebooks on `path` from useless metadata\n\noptions:\n -h, --help show this help message and exit\n --path PATH The path to notebooks (default: .)\n --clear_outs Remove cell outputs (default: False)\n --verbose Rnun on verbose mdoe (default: False)\n```\n\nYou can run this comman on this repo:\n\n```bash\n$ nb_helpers.clean_nbs\n> \n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513 \n\u2503 Notebook Path \u2503 Status \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 tests/data/dummy_folder/fail_nb.ipynb \u2502 Ok\u2714 \u2502\n\u2502 tests/data/dummy_folder/test_nb2.ipynb \u2502 Ok\u2714 \u2502\n\u2502 tests/data/dummy_folder/test_nb_all_slow.ipynb \u2502 Ok\u2714 \u2502\n\u2502 tests/data/dummy_folder/test_nb_some_slow.ipynb \u2502 Ok\u2714 \u2502\n\u2502 tests/data/features_nb.ipynb \u2502 Ok\u2714 \u2502\n\u2502 tests/data/test_nb.ipynb \u2502 Ok\u2714 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n- Run: One can run the notebooks in `path` and get info about the execution.\n\n```bash\n$ nb_helpers.run_nbs --help tcapelle at MBP14 (--)(main)\nusage: nb_helpers.run_nbs [-h] [--verbose] [--lib_name LIB_NAME] [--no_run] [--pip_install] [--github_issue] [--repo REPO] [--owner OWNER] [path]\n\npositional arguments:\n path A path to nb files (default: /Users/tcapelle/wandb/nb_helpers)\n\noptions:\n -h, --help show this help message and exit\n --verbose Print errors along the way (default: False)\n --lib_name LIB_NAME Python lib names to filter, eg: tensorflow\n --no_run Do not run any notebook (default: False)\n --pip_install Run cells with !pip install (default: False)\n --github_issue Create a github issue if notebook fails (default: False)\n --repo REPO Github repo to create issue in (default: nb_helpers)\n --owner OWNER Github owner to create issue in (default: wandb)\n```\n\n> You can now post github issues when running fails, the cool thing is that it can be posted to another repo than the one from the notebooks. Just pass the `--repo` name and the `--owner` (for example `wandb/other_cool_repo`)\n\nYou get the following output inside this repo:\n\n```bash\n$ nb_helpers.run_nbs\nCONSOLE.is_terminal(): True\nWriting output to run.csv\n```\n\n | Notebook Path |Status |Run Time |colab |\n |--- | --- | --- | --- |\n | dev_nbs/search.ipynb | Fail | 1 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/dev_nbs/search.ipynb) |\n | tests/data/dummy_folder/fail_nb.ipynb | Fail | 1 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/dummy_folder/fail_nb.ipynb) |\n | tests/data/dummy_folder/test_nb2.ipynb | Ok | 0 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/dummy_folder/test_nb2.ipynb) |\n | tests/data/dummy_folder/test_nb_all_slow.ipynb | Skipped | 0 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/dummy_folder/test_nb_all_slow.ipynb) |\n | tests/data/dummy_folder/test_nb_some_slow.ipynb | Ok | 0 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/dummy_folder/test_nb_some_slow.ipynb) |\n | tests/data/features_nb.ipynb | Ok | 0 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/features_nb.ipynb) |\n | tests/data/test_nb.ipynb | Ok | 0 s | [open](https://colab.research.google.com/github/wandb/nb_helpers/blob/main/tests/data/test_nb.ipynb) |\n\n- Summary:\nYou can get a summary of the notebooks in your project with the `nb_helpers.summary_nbs` function.\n\n```bash\n$ nb_helpers.summary_nbs\nCONSOLE.is_terminal(): True\nWriting output to /Users/tcapelle/wandb/nb_helpers/logs/summary.csv\nReading 6 notebooks\n\u250c\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 # \u2502 nb name \u2502 tracker \u2502 wandb features \u2502 python libs \u2502 colab_cell \u2502 colab \u2502\n\u251c\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 1 \u2502 tests/data/dummy_folder/fail_nb.ipynb \u2502 \u2502 \u2502 \u2502 \u2502 open \u2502\n\u2502 2 \u2502 tests/data/dummy_folder/test_nb2.ipynb \u2502 \u2502 \u2502 \u2502 \u2502 open \u2502\n\u2502 3 \u2502 tests/data/dummy_folder/test_nb_all_slow.ipynb \u2502 \u2502 \u2502 time \u2502 \u2502 open \u2502\n\u2502 4 \u2502 tests/data/dummy_folder/test_nb_some_slow.ipynb \u2502 \u2502 \u2502 time \u2502 \u2502 open \u2502\n\u2502 5 \u2502 tests/data/features_nb.ipynb \u2502 \u2502 \u2502 typing, itertools \u2502 \u2502 open \u2502\n\u2502 6 \u2502 tests/data/test_nb.ipynb \u2502 0: tracker \u2502 \u2502 os, sys, logging, pathlib, fastcore, itertools \u2502 1 \u2502 open \u2502\n\u2514\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n------------\n\n## Python Lib\n\nAll this functions can also be used inside python:\n\n```python\nfrom pathlib import Path\nfrom nb_helpers.run import run_nbs\n\nexamples_path = Path(\"examples/colabs\")\n\nerrors = run_nbs(path=examples_path, verbose=True, timeout=600)\n```\n\nAlso the library has many little functions to make your life easier inside the repo you are orchestrating:\n\n```python\nfrom pathlib import Path\nfrom nb_helpers.utils import *\nfrom nb_helpers.colab import *\n\nexamples_path = Path(\"examples/colabs\")\n\n# get all nbs in the folder recursevely, filters hidden, non nb stuff\nnb_files = find_nbs(example_path)\n\none_nb_path = nb_files[0]\nnotebook = read_nb(one_nb_path)\n\n# get all libs imported\nlibs = detect_imported_libs(notebook)\n\n# get remote github repo\ngithub_repo = git_origin_repo(one_nb_path)\n\n# detect if master is called main or master\nmaster_name = git_main_name(one_nb_path)\n\n# get colab link\ncolab_url = get_colab_url(one_nb_path, branch) \n```\n\n\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "A set of tools for nb handling",
"version": "0.6.2",
"project_urls": {
"Homepage": "https://github.com/wandb/nb_helpers/tree/main/"
},
"split_keywords": [
"jupyter",
"notebook"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5f224b8b07c7922bd52eebc17b9727a590d6eb4fdc77128fa0eab97f72d34ef0",
"md5": "14a90f8c05da2783f80240d3f54d4b20",
"sha256": "0b76640ef66a5d92b5ca4966c0620d3ec63d1087497b8e37804a09262ea75f44"
},
"downloads": -1,
"filename": "nb_helpers-0.6.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "14a90f8c05da2783f80240d3f54d4b20",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 21888,
"upload_time": "2023-06-09T22:17:55",
"upload_time_iso_8601": "2023-06-09T22:17:55.557211Z",
"url": "https://files.pythonhosted.org/packages/5f/22/4b8b07c7922bd52eebc17b9727a590d6eb4fdc77128fa0eab97f72d34ef0/nb_helpers-0.6.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3ea022b53253444c052a8b17ca750d4f1e370d98b9ffe13bae4ce4d75fcedaf9",
"md5": "582ae0439fa53ca956fe78110b843f66",
"sha256": "1a99383e68f31cf12720386a76069b7afd273dc2f66c274a4eb4e81b94a11fe4"
},
"downloads": -1,
"filename": "nb_helpers-0.6.2.tar.gz",
"has_sig": false,
"md5_digest": "582ae0439fa53ca956fe78110b843f66",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 20491,
"upload_time": "2023-06-09T22:17:57",
"upload_time_iso_8601": "2023-06-09T22:17:57.363788Z",
"url": "https://files.pythonhosted.org/packages/3e/a0/22b53253444c052a8b17ca750d4f1e370d98b9ffe13bae4ce4d75fcedaf9/nb_helpers-0.6.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-09 22:17:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "wandb",
"github_project": "nb_helpers",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "nb-helpers"
}