| Name | kit4dl JSON |
| Version |
2023.11b1
JSON |
| download |
| home_page | |
| Summary | Kit4DL - A quick way to start with machine and deep learning |
| upload_time | 2023-11-24 16:41:02 |
| maintainer | |
| docs_url | None |
| author | Jakub Walczak, Marco Mancini, Mirko Stojiljkovic, Shahbaz Alvi |
| requires_python | >=3.10 |
| license | MIT License Copyright (c) 2023 opengeokube Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
deep learning
training pipeline
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<div align="center">
<img src="https://raw.githubusercontent.com/opengeokube/ml-kit/56dc56c1be7f6332c0f75cdfb3160d29cebc3c58/static/logo.svg" width="20%" height="20%">
# A quick way to start with machine and deep learning
[](https://www.python.org/downloads)
[](https://black.readthedocs.io/en/stable/)
[](https://pycqa.github.io/isort/)
[](https://github.com/pylint-dev/pylint)
[](http://mypy-lang.org/)
[](http://www.pydocstyle.org/en/stable/)
[](https://github.com/opengeokube/kit4dl/blob/main/LICENSE)
[](https://github.com/opengeokube/kit4dl/actions/workflows/test.yml)
[](https://doi.org/10.5281/zenodo.8328176)
[](https://anaconda.org/conda-forge/kit4dl)
[](https://anaconda.org/conda-forge/kit4dl)
[](https://badge.fury.io/py/kit4dl)
</div>
## ποΈ Authors
OpenGeokube Developers:
1. Jakub Walczak <a href="https://orcid.org/0000-0002-5632-9484"><img alt="ORCID logo" src="https://info.orcid.org/wp-content/uploads/2019/11/orcid_16x16.png" width="16" height="16" /></a>
1. Marco Macini <a href="https://orcid.org/0000-0002-9150-943X"><img alt="ORCID logo" src="https://info.orcid.org/wp-content/uploads/2019/11/orcid_16x16.png" width="16" height="16" /></a>
1. Mirko Stojiljkovic <a href="https://orcid.org/0000-0003-2256-1645"><img alt="ORCID logo" src="https://info.orcid.org/wp-content/uploads/2019/11/orcid_16x16.png" width="16" height="16" /></a>
1. Shahbaz Alvi <a href="https://orcid.org/0000-0001-5779-8568"><img alt="ORCID logo" src="https://info.orcid.org/wp-content/uploads/2019/11/orcid_16x16.png" width="16" height="16" /></a>
## π Cite Us
```bibtex
@SOFTWARE{kit4dl,
author = {Walczak, Jakub and
Mancini, Marco and
StojiljkoviΔ, Mirko and
Alvi, Shahbaz},
title = {Kit4DL},
month = sep,
year = 2023,
note = {{Available in GitHub: https://github.com/opengeokube/kit4dl}},
publisher = {Zenodo},
version = {2023.9b1},
doi = {10.5281/zenodo.8328176},
url = {https://doi.org/10.5281/zenodo.8328176}
}
```
## π§ Roadmap
> **Warning**: Kit4DL is currently in its alpha stage. All recommendations are welcomed.
- [ ] add handling sklearn-like models
- [ ] add functionality to serve the model
- [x] enable custom metrics
- [x] enable using callbacks (also custom ones)
- [ ] write more unit tests
## π¬ Quickstart
### Getting started
#### Installation
```bash
pip install kit4dl
```
or
```bash
conda install -c conda-forge kit4dl
```
For contributing:
Download and install the `make` tool unless it is already available in your system.
```text
git clone https://github.com/opengeokube/kit4dl
cd kit4dl
conda env create -f dev-env.yaml
pip install -e .
```
#### Preparing simple project
To start the new project in the current working directory, just run the following command:
```bash
kit4dl init --name=my-new-project
```
It will create a directory with the name `my-new-project` where you'll find sample files.
Implement necessery methods for datamodule (`dataset.py`) and network (`model.py`).
Then, adjust `conf.toml` according to your needs.
That's all π
#### Running the training
To run the training just type the following command:
```bash
kit4dl train
```
> **Note**: If you want to run also test for best saved weight, use flag `--test`
If the `conf.toml` file is present in your current working directory, the training will start.
If you need to specify the path to the configuration file, use `--conf` argument:
```bash
kit4dl train --conf=/path/to/your/conf.toml
```
#### Serving the model
The packuge does not yet support model serving.
## πͺ Playground
At first, install `kit4dl` package as indicated in the Section [Installation](#installation).
#### Handwritten digit recognition
Just navigate to the directory `/examples/cnn_mnist_classification` and run
```bash
kit4dl train
```
#### Point cloud instance segmentation
Just navigate to the directory `/examples/cnn_s3dis_segmentation` and run
```bash
kit4dl train
```
## π‘ Instruction
1. [Configuring base setup](#configuring-base-setup)
1. [Configuring logging](#configuring-logging)
1. [Defining model](#defining-model)
1. [Defining datamodule](#defining-datamodule)
1. [Configuring training](#configuring-training)
1. [Configuring optimizer](#configuring-optimizer)
1. [Configuring criterion](#configuring-criterion)
1. [Configuring metrics](#configuring-metrics)
1. [Configuring checkpoint](#configuring-checkpoint)
1. [Defining `target`](#defining-target)
1. [Substitutable symbols](#substitutable-symbols)
1. [Context constants](#context-constants)
#### Configuring base setup
Most of the training/validation procedure is managed by a configuration file in the TOML format (recommended name is `conf.toml`).
Each aspect is covered by separate sections. The general one is called `[base]`.
It has the following properties:
| **Property** | **Type** | **Details** |
|--------------- |----------------- | ------------------------------------------------------------------------------------------------------- |
| `seed` | `int` | seed of the random numbers generators for `NumPy` and `PyTorch` |
| `cuda_id` | `int` or `None` | ID of the cuda device (if available) or `None` for `CPU` |
|`experiment_name`* | `str` | name of the experiment |
> **Note**: Arguments marked with `*` are obligatory!
> **Warning**: Remember to install the version of `pytorch-cuda` package compliant to your CUDA Toolkit version.
##### βοΈ Example
```toml
[base]
seed = 0
cuda_id = 1
experiment_name = "point_clout_segmentation"
```
#### Configuring logging
Logging section is optional but it provides you with some extra flexibility regarding the logging.
All configuration related to logging is included in the `[logging]` section of the configuration file.
You can define following properties:
| **Property** | **Type** | **Details** |
|--------------- |----------------- | ------------------------------------------------------------------------------------------------------- |
| `type` | `str` | type of metric logger (one of the value: `"comet"`, `"csv"`, `"mlflow"`, `"neptune"`, `"tensorboard"`, `"wandb"` - metric loggers supported by PyTorch Lightning [https://lightning.ai/docs/pytorch/stable/api_references.html#loggers](https://lightning.ai/docs/pytorch/stable/api_references.html#loggers). **DEFAULT:** `csv`) |
| `level` | `str` | Python-supported logging levels (i.e. `"DEBUG"`, `"INFO"`, `"WARN"`, `"ERROR"`, `"CRITICAL"`) **DEFAULT:** `INFO` |
|`format` | `str` | logging message format as defined for the Python `logging` package (see [https://docs.python.org/3/library/logging.html#logging.LogRecord](https://docs.python.org/3/library/logging.html#logging.LogRecord)) |
> **Warning**: Logger `level` and `format` are related to the Python `logging` Loggers you can use in your model and datamodule classes with approperiate methods `self.debure`, `self.info`, etc. In `type`, in turn, you just specify the metric logger as used in PyTorch Lightning package!
> **Note**: All required arguments for metric logger can be specified as extra arguments in the `[logging]`section.
##### βοΈ Example
```toml
[logging]
# we gonna use CSVLogger
type = "csv"
# for CSVLogger, we need to define 'save_dir' argument and/or
# other extra ones (https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.loggers.csv_logs.html#module-lightning.pytorch.loggers.csv_logs)
save_dir = "{{ PROJECT_DIR }}/my_metrics.csv"
# then we define level and format for logging messages
level = "info"
format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
```
> **Note**: If you don't pass a `name` or `experiment_name` argument explicitly for the metric logger, the `experiment_name` value defined in the `[base]` section will be applied as, respectively: `name` argument for `csv`, `neptune`, `tensorboard`, `wandb`, and as `experiment_name` for `comet` and `mlflow`.
#### Defining model
The machine learning/deep learning model definition is realized in two aspects.
1. The definition of the model (e.g. PyTorch model) in the `.py` file.
1. The configuration in the `[model]` section of the configuration file.
The file with the model definition should contain a subclass of `Kit4DLAbstractModule` abstract class of the `kit4dl` package.
The subclass should implement, at least, abstract methods `configure` and `run_step`.
In the `configure` method, the architecture of the network should be defined.
In `run_step` method, it turn, the logic for single forward pass should be implemented.
##### βοΈ Example
```python
import torch
from torch import nn
from kit4dl.nn.base import Kit4DLAbstractModule
class SimpleCNN(Kit4DLAbstractModule):
def configure(self, input_dims, output_dims) -> None:
self.l1 = nn.Sequential(
nn.Conv2d(
input_dims, 16, kernel_size=3, padding="same", bias=True
),
nn.ReLU(),
)
def run_step(self, batch, batch_idx) -> tuple[torch.Tensor, ...]:
x, label = batch
logits = self.l1(x)
preds = logits.argmax(dim=-1)
return label, logits, preds
```
> **Note**: `run_step` method should return a tuple of 2 (ground-truth, scores) or 3 (ground-truth, scores, loss) tensors.
> **Note**: `batch` argument can be unpacked depending on how you define your dataset for datamodule (see [Defining datamodule](#defining-datamodule))
In the configuration file, in the dedicated `[model]` section, at least `target` property should be set. The extra arguments are treated as the arguments for the `configure` method.
> **Note**: Arguments' values of the `configure` method (i.e. `input_dims` and `output_dims`) are taken from the configuration files. Those names can be arbitrary.
##### βοΈ Example
```toml
[model]
target = "./model.py::SimpleCNN"
input_dims = 1
output_dims = 10
```
> **Note**: `target` is a required parameter that **must** be set. It contains a path to the class (a subclass of `Kit4DLAbstractModule`). To learn how `target` could be defined, see Section [Defining `target`](#defining-target).
If a forward pass for your model differs for the training, validation, test, or prediction stages, you can define separate methods for them:
##### βοΈ Example
```python
import torch
from torch import nn
from kit4dl.nn.base import Kit4DLAbstractModule
class SimpleCNN(Kit4DLAbstractModule):
...
def run_val_step(self, batch, batch_idx) -> tuple[torch.Tensor, torch.Tensor]:
pass
def run_test_step(self, batch, batch_idx) -> tuple[torch.Tensor, torch.Tensor]:
pass
def run_predict_step(self, batch, batch_idx) -> torch.Tensor:
pass
```
> **Note**: If you need more customization of the process, you can always override the existing methods according to your needs.
#### Defining datamodule
Similarily to the model, datamodule instance is fully defined by the Python class and its configuration.
The datamodule need to be a subclass of the `Kit4DLAbstractDataModule` abstract class from the `kit4dl` package.
The class has to implement, at least, `prepare_trainvaldataset` (if preparing is the same for the train and validation splits) or `prepare_traindataset` and `prepare_valdataset` (if preparing data differs). Besides those, you can define `prepare_testdataset` and `prepare_predictdataset`, for test and prediction, respectively.
##### βοΈ Example
```python
from torch.utils.data import Dataset, random_split
from torchvision import transforms
from torchvision.datasets import MNIST
from kit4dl.dataset import Kit4DLAbstractDataModule
class MNISTCustomDatamodule(Kit4DLAbstractDataModule):
def prepare_trainvaldataset(
self, root_dir: str
) -> tuple[Dataset, Dataset]:
dset = MNIST(
root=root_dir,
train=True,
download=True,
transform=transforms.ToTensor(),
)
train_dset, val_dset = random_split(dset, [0.8, 0.2])
return (train_dset, val_dset)
def prepare_testdataset(self, root_dir: str) -> Dataset:
return MNIST(
root=root_dir,
train=False,
download=True,
transform=transforms.ToTensor(),
)
```
If you need to acquire data or do some other processing, implement `prepare_data` method. In that method you can use extra attributes you defined in the `[dataset]` section of the configuration file.
##### βοΈ Example
```toml
[dataset]
target = "./datamodule.py::MNISTCustomDatamodule"
my_variable = 10
```
```python
...
class MNISTCustomDatamodule(Kit4DLAbstractDataModule):
my_variable: int # NOTE: To make attribute visible, we can declare it here
def prepare_data(self):
result = self.my_variable * 2
```
> **Warning**: **DO NOT** set state inside `prepare_data` method (~~`self.x = ...`~~).
If you need more customization, feel free to override the other methods of `Kit4DLAbstractDataModule` superclass.
To force custom batch collation, override selected methods out of the following ones. They should return the proper callable object!
```python
def some_collate_func(samples: list): ...
class MNISTCustomDatamodule(Kit4DLAbstractDataModule):
...
def get_train_collate_fn(self):
return some_collate_func
def get_val_collate_fn(self):
return some_collate_func
def get_test_collate_fn(self):
return some_collate_func
def get_predict_collate_fn(self):
return some_collate_func
```
> **Warning**: **DO NOT** use nested function as a callation callable. It will fail due to pickling nested function error.
If you need a custom batch collation but the same for each stage (train/val/test/predict), implement the method `get_collate_fn()`:
```python
def get_collate_fn(self):
return some_collate_func
```
In the configuration file, there are dedicated `[dataset]`-related sections.
##### βοΈ Example
```toml
[dataset]
target = "./datamodule.py::MNISTCustomDatamodule"
[dataset.trainval]
root_dir = "./mnist"
[dataset.train.loader]
batch_size = 150
shuffle = true
num_workers = 4
[dataset.validation.loader]
batch_size = 150
shuffle = false
num_workers = 4
```
In the root `[dataset]` you should define `target` property being a path to the subclass of the `Kit4DLAbstractDataModule` module (see [Defining `target`](#defining-target)).
Then, you need to define either `[dataset.trainval]` section or two separate sections: `[dataset.train]`, `[dataset.validation]`. There are also optional sections: `[dataset.test]` and `[dataset.predict]`.
In `[dataset.trainval]` you pass values for parameters of the `prepare_trainvaldataset` method.
Respectively, in the `[dataset.train]` you pass values for the parameters of the `prepare_traindataset` method, in `[dataset.validation]` β `prepare_valdataset`, `[dataset.test]` β `prepare_testdataset`, `[dataset.predict]` β `prepare_predictdataset`.
Besides dataset configuration, you need to specify data loader arguments as indicated in the PyTorch docs [torch.utils.data.DataLoader](https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader).
> **Warning**: You **cannot** specify loader arguments for in the `[dataset.trainval.loader]`. Loaders should be defined for each split separately.
#### Configuring training
Training-related arguments should be defined in the `[training]` section of the configuration file.
You can define the following arguments.
| **Property** | **Type** | **Details** |
|--------------- |----------------- | -------------------------------- |
| `epochs`* | `int > 0` | number of epochs |
| `callbacks` | `list` | list of callbacks |
|`epoch_schedulers` | `list of dict` | list of schedulers definitions |
> **Note**: Arguments marked with `*` are obligatory!
You can define a list of custom callbacks applied in the training process. Your callbacks need to be subclasses of `lightning.pytorch.callbacks.Callback` or `kit4dl.Kit4DLCallback` (for convenience) class and define one/some of the methods indicated in the [PyTorch-Lightning callback API](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html#callback-api). You can always use one of the [predefined callbacks](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html#built-in-callbacks).
```toml
[training]
callbacks = [
{target = "./callbacks.py::SaveConfusionMatrixCallback", task="multiclass", num_classes=10, save_dir="{{ PROJECT_DIR }}/cm},
{target = "lightning.pytorch.callbacks::DeviceStatsMonitor"}
]
```
Where the 1st callback is user-defined and the other - PyTorch-Loghtning built-in. For the custom callback we need to provide a class (here: located in the `callbacks.py` file in the project directory, the class is named `SaveConfusionMatrixCallback`).
```python
import os
from typing import Any
import lightning.pytorch as pl
import torchmetrics as tm
from kit4dl import Kit4DLCallback
class SaveConfusionMatrixCallback(Kit4DLCallback):
_cm: tm.ConfusionMatrix
_num_classes: int
_task: str
_save_dir: str
def __init__(self, task: str, num_classes: int, save_dir: str) -> None:
super().__init__()
self._num_classes = num_classes
self._save_dir = save_dir
self._task = task
os.makedirs(self._save_dir, exist_ok=True)
def on_validation_epoch_start(
self, trainer: pl.Trainer, pl_module: pl.LightningModule
) -> None:
self._cm = tm.ConfusionMatrix(
task=self._task, num_classes=self._num_classes
)
def on_validation_batch_end(
self,
trainer: pl.Trainer,
pl_module: pl.LightningModule,
outputs: dict,
batch: Any,
batch_idx: int,
dataloader_idx: int = 0,
) -> None:
self._cm.update(outputs['pred'], outputs['true'])
def on_validation_epoch_end(
self, trainer: pl.Trainer, pl_module: pl.LightningModule
) -> None:
"""Called when the val epoch ends."""
fig, _ = self._cm.plot()
target_file = os.path.join(
self._save_dir,
f"confusion_matrix_for_epoch_{pl_module.current_epoch}",
)
fig.savefig(target_file)
```
Besides those listed in the table above, you can specify PyTorch Lightning-related `Trainer` arguments, like:
1. `accumulate_grad_batches`
1. `gradient_clip_val`
1. `gradient_clip_algorithm`
1. ...
##### βοΈ Example
```toml
[training]
epochs = 10
epoch_schedulers = [
{target = "torch.optim.lr_scheduler::CosineAnnealingLR", T_max = 100}
]
accumulate_grad_batches = 2
```
#### Configuring optimizer
Optimizer configuration is located in the subsection `[training.optimizer]`.
There, you should define `target` (see [Defining `target`](#defining-target)) and extra keyword arguments passed to the optimizer initializer.
##### βοΈ Example
```toml
[training.optimizer]
target = "torch.optim::Adam"
lr = 0.001
weight_decay = 0.01
```
> **Note**: The section `[training.optimizer]` is **mandatory**.
> **Note**: You can always define the custom optimizer. Then, you just need to set the proper `target` value.
#### Configuring criterion
Similarily to the optimizer configuration, there is a subsection dedicated for the critarion.
You need to specify, at least, the `target` (see [Defining `target`](#defining-target)) and other mandatory or optional
properties of the selected critarion (loss function).
##### βοΈ Example
```toml
[training.criterion]
target = "torch.nn::CrossEntropyLoss"
weight = [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]
```
> **Note**: The section `[training.criterion]` is **mandatory**.
> **Note**: You can always define the custom optimizer. Then, you just need to set the proper `target` value.
#### Configuring metrics
Metrics are configured in the section `[metrics]` of the configuration file. You can define several metrics (including the custom ones).
The only thing you need to do is to define all desired metrics. For each metric dictionary, you need to set `target` (see Section [Defining `target`](#defining-target)) value and, eventually, extra arguments. **REMEMBER** to have metric names (here `MyPrecision` and `FBetaScore`) unique!
##### βοΈ Example
```toml
[metrics]
MyPrecision = {target = "torchmetrics::Precision", task = "multiclass", num_classes=10}
FBetaScore = {target = "torchmetrics::FBetaScore", task = "multiclass", num_classes=10, beta = 0.1}
```
> **Note**: You can define custom metrics. Just properly set `target` value. **REMEMBER!** The custom metric need to be a subclass of `torchmetrics.Metric` class!
```python
import torch
import torchmetrics as tm
class MyMetric(tm.Metric):
def __init__(self):
...
def update(self, preds: torch.Tensor, target: torch.Tensor):
...
def compute(self):
...
```
#### Configuring checkpoint
If you need to save your intermediate weights (do checkpoints) you can configure the optional subsection `[training.checkpoint]`.
In the section, you can define the following proeprties:
| **Property** | **Type** | **Details** |
|--------------- |----------------- | -------------------------------- |
| `path`* | `str` | path to a directory where checkpoints should be stored |
|`monitor`* | `dict` | a dictionary with two keys: `metric` and `stage`. `metrics` is a metric name as defined in the `[metrics]` section ([Configuring metrics](#configuring-metrics)), `stage` is one of the following: [`train`, `val`] |
| `filename`* | `str` | filename pattern of the checkpoint (see (PyTorch Lightning `ModelCheckpoint`)[https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.callbacks.ModelCheckpoint.html]) you can use value of the defined metric for the stage. if you want `MyPrecision` score for the validation stage, use `{val_myprecision}` in the filename |
| `mode` | `min` | `max` | to save checkpoint for `min`imum or `max`imum value of the metric being tracked (`monitor`). **default: `max`** |
| `save_top_k` | `int` | save checkepoints for the top `k` values of the metric. **default: `1`** |
| `save_weights_only` | `bool` | if only weights should be saved (`True`) or other states (optimizer, scheduler) also (`False`). **default: `True`** |
| `every_n_epochs` | `int` | The number of training epochs between saving sucessive checkpoints. **default: `1`** |
| `save_on_train_epoch_end` | `bool` | if `False` checkpointing is run at the end of the validation, otherwise - training **default: `False`** |
> **Note**: Arguments marked with `*` are obligatory!
##### βοΈ Example
```toml
[training.checkpoint]
path = "{{ PROJECT_DIR }}/chckpt"
monitor = {"metric" = "Precision", "stage" = "val"}
filename = "{epoch}_{val_precision:.2f}_cnn"
mode = "max"
save_top_k = 1
```
> **Note**: You can see we used substitutable symbol `{{ PROJECT_DIR }}`. More about them in the Section [Substitutable symbols](#substitutable-symbols).
#### Defining `target`
Target property in the Kit4DL package is kind of extended fully qualified name pointing to the classes supposed to use in the
given context, like for:
1. neural network class (`target = "./model.py::SimpleCNN"`)
1. datamodule (`target = "./datamodule.py::MNISTCustomDatamodule"`)
1. optimizer (`target = "torch.optim::Adam"`)
1. criterion (`target = "torch.nn::CrossEntropyLoss"`)
1. schedulers (`target = "torch.optim.lr_scheduler::CosineAnnealingLR"`)
> **Note**: As a package/module - class separator the double colon is used `::`!
It might be set in several different ways:
1. **By using a built-and installed package**. Then, you just need to specify the package/module name and the class name, like `target = "torch.nn::CrossEntropyLoss"` (we use module `torch.nn` and class `CrossEntropyLoss` defined within).
1. **By using a custom module in the project directory**. The project directory, i.e. the directory where the confguration TOML file is located, is added to the `PYTHONPATH`, so you can freely use `.py` files defined there as modules. Having the module `model.py` with the `SimpleCNN` class definition, we can write `target` as `target = "model::SimpleCNN"`.
1. **By using a custom `.py` file.** In this case, you specify `target` as an absolute or relative (w.r.t. the configuration file) to a `.py` file, like `target = "./model.py::SimpleCNN"` or `target = "/usr/neural_nets/my_net/model.py::SimpleCNN"`.
> **Note**: For `target` definition you can use substitutable symbols defined below.
#### Substitutable symbols
In the configuration file you can use symbols that will be substituted during the runtime.
The symbols should be surrended by single spaces and in double curly brackets (e.g. `{{ PROJECT_DIR }}`.)
| **Symbol** | **Meaning of the symbol** | **Example** |
|------------- |----------------------------------------------------------------------- | ----------------------------------- |
| `PROJECT_DIR` | the home directory of the TOML configuration file (project directory) | `target = {{ PROJECT_DIR }}/model.py` |
> **Note**: You can also use environmental variables. Just use `env` dict, e.g.: `{{ env['your_var_name'] }}`.
##### βοΈ Example
First, let's define some environmental variable: using Python or system tool.
```python
import os
os.environ["MY_LOG_LEVEL"] = "INFO"
```
or
```bash
export MY_LOG_LEVEL="MY_LOG_LEVEL"
```
Now, we can use the environmental variable `MY_LOG_LEVEL` in our config file:
```toml
[logging]
level = "{{ env['MY_LOG_LEVEL'] }}"
format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
```
> **Warning**: If you use **double quote** for text values in TOML configuration file, then use **single quote** to access `env` values.
#### Context constants
When you run training using `kit4dl train` command, all custom modules have access to context constant values (defined for the current Python interpreter session).
You can access them via `context` module:
##### βοΈ Example
```python
from kit4dl import context
print(context.PROJECT_DIR)
```
The constants currently available in `kit4dl` are the following:
| **Symbol** | **Meaning of the symbol** | **Example** |
|------------- |----------------------------------------------------------------------- | ----------------------------------- |
| `PROJECT_DIR` | the home directory of the TOML configuration file (project directory) | `context.PROJECT_DIR` |
| `LOG_LEVEL` | logging level as defined in the configuration TOML file | `context.LOG_LEVEL` |
| `LOG_FORMAT` | logging message format as defined in the configuration TOML file | `context.LOG_FORMAT` |
| `VERSION` | the current version of the package | `context.VERSION` |
Raw data
{
"_id": null,
"home_page": "",
"name": "kit4dl",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "",
"keywords": "deep learning,training pipeline",
"author": "Jakub Walczak, Marco Mancini, Mirko Stojiljkovic, Shahbaz Alvi",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/33/a8/aa82c6bd0c235b7845c2dbf5faf49fe5f73a9c269c5edf1f10c85aeebb3c/kit4dl-2023.11b1.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/opengeokube/ml-kit/56dc56c1be7f6332c0f75cdfb3160d29cebc3c58/static/logo.svg\" width=\"20%\" height=\"20%\">\n\n# A quick way to start with machine and deep learning\n[](https://www.python.org/downloads)\n\n[](https://black.readthedocs.io/en/stable/)\n[](https://pycqa.github.io/isort/) \n[](https://github.com/pylint-dev/pylint)\n[](http://mypy-lang.org/)\n[](http://www.pydocstyle.org/en/stable/)\n\n[](https://github.com/opengeokube/kit4dl/blob/main/LICENSE)\n\n[](https://github.com/opengeokube/kit4dl/actions/workflows/test.yml)\n\n[](https://doi.org/10.5281/zenodo.8328176)\n\n[](https://anaconda.org/conda-forge/kit4dl) \n[](https://anaconda.org/conda-forge/kit4dl)\n\n[](https://badge.fury.io/py/kit4dl)\n</div>\n\n\n\n\n## \ud83d\udd8b\ufe0f Authors\nOpenGeokube Developers:\n1. Jakub Walczak <a href=\"https://orcid.org/0000-0002-5632-9484\"><img alt=\"ORCID logo\" src=\"https://info.orcid.org/wp-content/uploads/2019/11/orcid_16x16.png\" width=\"16\" height=\"16\" /></a>\n1. Marco Macini <a href=\"https://orcid.org/0000-0002-9150-943X\"><img alt=\"ORCID logo\" src=\"https://info.orcid.org/wp-content/uploads/2019/11/orcid_16x16.png\" width=\"16\" height=\"16\" /></a>\n1. Mirko Stojiljkovic <a href=\"https://orcid.org/0000-0003-2256-1645\"><img alt=\"ORCID logo\" src=\"https://info.orcid.org/wp-content/uploads/2019/11/orcid_16x16.png\" width=\"16\" height=\"16\" /></a>\n1. Shahbaz Alvi <a href=\"https://orcid.org/0000-0001-5779-8568\"><img alt=\"ORCID logo\" src=\"https://info.orcid.org/wp-content/uploads/2019/11/orcid_16x16.png\" width=\"16\" height=\"16\" /></a>\n\n## \ud83d\udcdc Cite Us\n```bibtex\n@SOFTWARE{kit4dl,\n author = {Walczak, Jakub and\n Mancini, Marco and\n Stojiljkovi\u0107, Mirko and\n Alvi, Shahbaz},\n title = {Kit4DL},\n month = sep,\n year = 2023,\n note = {{Available in GitHub: https://github.com/opengeokube/kit4dl}},\n publisher = {Zenodo},\n version = {2023.9b1},\n doi = {10.5281/zenodo.8328176},\n url = {https://doi.org/10.5281/zenodo.8328176}\n}\n```\n## \ud83d\udea7 Roadmap\n\n> **Warning**: Kit4DL is currently in its alpha stage. All recommendations are welcomed.\n\n- [ ] add handling sklearn-like models\n- [ ] add functionality to serve the model\n- [x] enable custom metrics\n- [x] enable using callbacks (also custom ones)\n- [ ] write more unit tests\n\n\n## \ud83c\udfac Quickstart\n\n### Getting started\n\n#### Installation\n```bash\npip install kit4dl\n```\n\nor\n\n```bash\nconda install -c conda-forge kit4dl \n```\n\nFor contributing:\n\nDownload and install the `make` tool unless it is already available in your system. \n\n```text\ngit clone https://github.com/opengeokube/kit4dl\ncd kit4dl\nconda env create -f dev-env.yaml\npip install -e .\n```\n\n#### Preparing simple project\nTo start the new project in the current working directory, just run the following command:\n\n```bash\nkit4dl init --name=my-new-project\n```\n\nIt will create a directory with the name `my-new-project` where you'll find sample files.\nImplement necessery methods for datamodule (`dataset.py`) and network (`model.py`).\nThen, adjust `conf.toml` according to your needs. \nThat's all \ud83c\udf89\n\n#### Running the training\nTo run the training just type the following command:\n\n```bash\nkit4dl train\n```\n> **Note**: If you want to run also test for best saved weight, use flag `--test`\n\n\nIf the `conf.toml` file is present in your current working directory, the training will start.\n\nIf you need to specify the path to the configuration file, use `--conf` argument:\n```bash\nkit4dl train --conf=/path/to/your/conf.toml\n```\n\n#### Serving the model\nThe packuge does not yet support model serving.\n\n## \ud83e\ude81 Playground\nAt first, install `kit4dl` package as indicated in the Section [Installation](#installation).\n\n#### Handwritten digit recognition\nJust navigate to the directory `/examples/cnn_mnist_classification` and run\n```bash\nkit4dl train\n```\n\n#### Point cloud instance segmentation\nJust navigate to the directory `/examples/cnn_s3dis_segmentation` and run\n```bash\nkit4dl train\n```\n\n\n## \ud83d\udca1 Instruction\n1. [Configuring base setup](#configuring-base-setup)\n1. [Configuring logging](#configuring-logging)\n1. [Defining model](#defining-model)\n1. [Defining datamodule](#defining-datamodule)\n1. [Configuring training](#configuring-training)\n1. [Configuring optimizer](#configuring-optimizer)\n1. [Configuring criterion](#configuring-criterion)\n1. [Configuring metrics](#configuring-metrics)\n1. [Configuring checkpoint](#configuring-checkpoint)\n1. [Defining `target`](#defining-target)\n1. [Substitutable symbols](#substitutable-symbols)\n1. [Context constants](#context-constants)\n\n#### Configuring base setup\nMost of the training/validation procedure is managed by a configuration file in the TOML format (recommended name is `conf.toml`).\nEach aspect is covered by separate sections. The general one is called `[base]`.\nIt has the following properties:\n\n| **Property** \t| **Type** | **Details** \t|\n|---------------\t|----------------- | -------------------------------------------------------------------------------------------------------\t| \n| `seed`\t | `int` | seed of the random numbers generators for `NumPy` and `PyTorch` \t| \n| `cuda_id` | `int` or `None` | ID of the cuda device (if available) or `None` for `CPU` |\n|`experiment_name`* | `str` | name of the experiment |\n\n> **Note**: Arguments marked with `*` are obligatory!\n\n> **Warning**: Remember to install the version of `pytorch-cuda` package compliant to your CUDA Toolkit version.\n\n\n##### \u270d\ufe0f Example\n```toml\n[base]\nseed = 0\ncuda_id = 1\nexperiment_name = \"point_clout_segmentation\"\n```\n\n#### Configuring logging\nLogging section is optional but it provides you with some extra flexibility regarding the logging.\nAll configuration related to logging is included in the `[logging]` section of the configuration file. \nYou can define following properties:\n\n| **Property** \t| **Type** | **Details** \t|\n|---------------\t|----------------- | -------------------------------------------------------------------------------------------------------\t| \n| `type`\t | `str` | type of metric logger (one of the value: `\"comet\"`, `\"csv\"`, `\"mlflow\"`, `\"neptune\"`, `\"tensorboard\"`, `\"wandb\"` - metric loggers supported by PyTorch Lightning [https://lightning.ai/docs/pytorch/stable/api_references.html#loggers](https://lightning.ai/docs/pytorch/stable/api_references.html#loggers). **DEFAULT:** `csv`) \t| \n| `level` | `str` | Python-supported logging levels (i.e. `\"DEBUG\"`, `\"INFO\"`, `\"WARN\"`, `\"ERROR\"`, `\"CRITICAL\"`) **DEFAULT:** `INFO` |\n|`format` | `str` | logging message format as defined for the Python `logging` package (see [https://docs.python.org/3/library/logging.html#logging.LogRecord](https://docs.python.org/3/library/logging.html#logging.LogRecord)) |\n\n> **Warning**: Logger `level` and `format` are related to the Python `logging` Loggers you can use in your model and datamodule classes with approperiate methods `self.debure`, `self.info`, etc. In `type`, in turn, you just specify the metric logger as used in PyTorch Lightning package!\n\n> **Note**: All required arguments for metric logger can be specified as extra arguments in the `[logging]`section.\n\n##### \u270d\ufe0f Example\n```toml\n[logging]\n# we gonna use CSVLogger\ntype = \"csv\"\n# for CSVLogger, we need to define 'save_dir' argument and/or\n# other extra ones (https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.loggers.csv_logs.html#module-lightning.pytorch.loggers.csv_logs)\nsave_dir = \"{{ PROJECT_DIR }}/my_metrics.csv\"\n\n# then we define level and format for logging messages\nlevel = \"info\"\nformat = \"%(asctime)s - %(name)s - %(levelname)s - %(message)s\"\n```\n\n> **Note**: If you don't pass a `name` or `experiment_name` argument explicitly for the metric logger, the `experiment_name` value defined in the `[base]` section will be applied as, respectively: `name` argument for `csv`, `neptune`, `tensorboard`, `wandb`, and as `experiment_name` for `comet` and `mlflow`.\n\n\n#### Defining model\nThe machine learning/deep learning model definition is realized in two aspects. \n1. The definition of the model (e.g. PyTorch model) in the `.py` file.\n1. The configuration in the `[model]` section of the configuration file.\n\nThe file with the model definition should contain a subclass of `Kit4DLAbstractModule` abstract class of the `kit4dl` package.\nThe subclass should implement, at least, abstract methods `configure` and `run_step`.\nIn the `configure` method, the architecture of the network should be defined. \nIn `run_step` method, it turn, the logic for single forward pass should be implemented.\n\n##### \u270d\ufe0f Example\n```python\nimport torch\nfrom torch import nn\nfrom kit4dl.nn.base import Kit4DLAbstractModule\n\nclass SimpleCNN(Kit4DLAbstractModule):\n def configure(self, input_dims, output_dims) -> None:\n self.l1 = nn.Sequential(\n nn.Conv2d(\n input_dims, 16, kernel_size=3, padding=\"same\", bias=True\n ),\n nn.ReLU(),\n )\n\n def run_step(self, batch, batch_idx) -> tuple[torch.Tensor, ...]:\n x, label = batch\n logits = self.l1(x)\n preds = logits.argmax(dim=-1)\n return label, logits, preds\n```\n> **Note**: `run_step` method should return a tuple of 2 (ground-truth, scores) or 3 (ground-truth, scores, loss) tensors.\n\n> **Note**: `batch` argument can be unpacked depending on how you define your dataset for datamodule (see [Defining datamodule](#defining-datamodule))\n\nIn the configuration file, in the dedicated `[model]` section, at least `target` property should be set. The extra arguments are treated as the arguments for the `configure` method.\n\n> **Note**: Arguments' values of the `configure` method (i.e. `input_dims` and `output_dims`) are taken from the configuration files. Those names can be arbitrary.\n\n##### \u270d\ufe0f Example\n```toml\n[model]\ntarget = \"./model.py::SimpleCNN\" \ninput_dims = 1\noutput_dims = 10\n```\n> **Note**: `target` is a required parameter that **must** be set. It contains a path to the class (a subclass of `Kit4DLAbstractModule`). To learn how `target` could be defined, see Section [Defining `target`](#defining-target).\n\nIf a forward pass for your model differs for the training, validation, test, or prediction stages, you can define separate methods for them:\n\n##### \u270d\ufe0f Example\n```python\nimport torch\nfrom torch import nn\nfrom kit4dl.nn.base import Kit4DLAbstractModule\n\nclass SimpleCNN(Kit4DLAbstractModule):\n ...\n def run_val_step(self, batch, batch_idx) -> tuple[torch.Tensor, torch.Tensor]:\n pass\n\n def run_test_step(self, batch, batch_idx) -> tuple[torch.Tensor, torch.Tensor]:\n pass\n\n def run_predict_step(self, batch, batch_idx) -> torch.Tensor:\n pass \n```\n\n> **Note**: If you need more customization of the process, you can always override the existing methods according to your needs.\n\n#### Defining datamodule\nSimilarily to the model, datamodule instance is fully defined by the Python class and its configuration.\nThe datamodule need to be a subclass of the `Kit4DLAbstractDataModule` abstract class from the `kit4dl` package.\nThe class has to implement, at least, `prepare_trainvaldataset` (if preparing is the same for the train and validation splits) or `prepare_traindataset` and `prepare_valdataset` (if preparing data differs). Besides those, you can define `prepare_testdataset` and `prepare_predictdataset`, for test and prediction, respectively.\n\n##### \u270d\ufe0f Example\n```python\nfrom torch.utils.data import Dataset, random_split\nfrom torchvision import transforms\nfrom torchvision.datasets import MNIST\n\nfrom kit4dl.dataset import Kit4DLAbstractDataModule\n\n\nclass MNISTCustomDatamodule(Kit4DLAbstractDataModule):\n def prepare_trainvaldataset(\n self, root_dir: str\n ) -> tuple[Dataset, Dataset]:\n dset = MNIST(\n root=root_dir,\n train=True,\n download=True,\n transform=transforms.ToTensor(),\n )\n train_dset, val_dset = random_split(dset, [0.8, 0.2])\n return (train_dset, val_dset)\n\n def prepare_testdataset(self, root_dir: str) -> Dataset:\n return MNIST(\n root=root_dir,\n train=False,\n download=True,\n transform=transforms.ToTensor(),\n )\n```\n\nIf you need to acquire data or do some other processing, implement `prepare_data` method. In that method you can use extra attributes you defined in the `[dataset]` section of the configuration file.\n\n##### \u270d\ufe0f Example\n```toml\n[dataset]\ntarget = \"./datamodule.py::MNISTCustomDatamodule\"\nmy_variable = 10\n```\n\n```python\n...\nclass MNISTCustomDatamodule(Kit4DLAbstractDataModule):\n my_variable: int # NOTE: To make attribute visible, we can declare it here\n\n def prepare_data(self):\n result = self.my_variable * 2\n```\n\n> **Warning**: **DO NOT** set state inside `prepare_data` method (~~`self.x = ...`~~).\n\nIf you need more customization, feel free to override the other methods of `Kit4DLAbstractDataModule` superclass.\nTo force custom batch collation, override selected methods out of the following ones. They should return the proper callable object!\n\n```python\ndef some_collate_func(samples: list): ...\n\nclass MNISTCustomDatamodule(Kit4DLAbstractDataModule):\n ...\n def get_train_collate_fn(self):\n return some_collate_func\n\n def get_val_collate_fn(self):\n return some_collate_func\n\n def get_test_collate_fn(self):\n return some_collate_func\n\n def get_predict_collate_fn(self):\n return some_collate_func\n```\n\n> **Warning**: **DO NOT** use nested function as a callation callable. It will fail due to pickling nested function error.\n\nIf you need a custom batch collation but the same for each stage (train/val/test/predict), implement the method `get_collate_fn()`:\n```python\ndef get_collate_fn(self):\n return some_collate_func\n```\n\nIn the configuration file, there are dedicated `[dataset]`-related sections.\n\n##### \u270d\ufe0f Example\n```toml\n[dataset]\ntarget = \"./datamodule.py::MNISTCustomDatamodule\"\n\n[dataset.trainval]\nroot_dir = \"./mnist\"\n\n[dataset.train.loader]\nbatch_size = 150\nshuffle = true\nnum_workers = 4\n\n[dataset.validation.loader]\nbatch_size = 150\nshuffle = false\nnum_workers = 4\n```\n\nIn the root `[dataset]` you should define `target` property being a path to the subclass of the `Kit4DLAbstractDataModule` module (see [Defining `target`](#defining-target)).\nThen, you need to define either `[dataset.trainval]` section or two separate sections: `[dataset.train]`, `[dataset.validation]`. There are also optional sections: `[dataset.test]` and `[dataset.predict]`.\nIn `[dataset.trainval]` you pass values for parameters of the `prepare_trainvaldataset` method.\nRespectively, in the `[dataset.train]` you pass values for the parameters of the `prepare_traindataset` method, in `[dataset.validation]` \u2014 `prepare_valdataset`, `[dataset.test]` \u2014 `prepare_testdataset`, `[dataset.predict]` \u2014 `prepare_predictdataset`.\n\nBesides dataset configuration, you need to specify data loader arguments as indicated in the PyTorch docs [torch.utils.data.DataLoader](https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader).\n\n> **Warning**: You **cannot** specify loader arguments for in the `[dataset.trainval.loader]`. Loaders should be defined for each split separately.\n\n\n#### Configuring training\nTraining-related arguments should be defined in the `[training]` section of the configuration file.\nYou can define the following arguments.\n\n| **Property** \t| **Type** | **Details** |\n|---------------\t|----------------- | -------------------------------- | \n| `epochs`* | `int > 0` | number of epochs\t | \n| `callbacks` | `list` | list of callbacks\t | \n|`epoch_schedulers` | `list of dict` | list of schedulers definitions |\n\n> **Note**: Arguments marked with `*` are obligatory!\n\nYou can define a list of custom callbacks applied in the training process. Your callbacks need to be subclasses of `lightning.pytorch.callbacks.Callback` or `kit4dl.Kit4DLCallback` (for convenience) class and define one/some of the methods indicated in the [PyTorch-Lightning callback API](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html#callback-api). You can always use one of the [predefined callbacks](https://lightning.ai/docs/pytorch/stable/extensions/callbacks.html#built-in-callbacks).\n\n\n```toml\n[training]\ncallbacks = [\n {target = \"./callbacks.py::SaveConfusionMatrixCallback\", task=\"multiclass\", num_classes=10, save_dir=\"{{ PROJECT_DIR }}/cm},\n {target = \"lightning.pytorch.callbacks::DeviceStatsMonitor\"}\n]\n```\n\nWhere the 1st callback is user-defined and the other - PyTorch-Loghtning built-in. For the custom callback we need to provide a class (here: located in the `callbacks.py` file in the project directory, the class is named `SaveConfusionMatrixCallback`).\n\n```python\nimport os\nfrom typing import Any\n\nimport lightning.pytorch as pl\nimport torchmetrics as tm\n\nfrom kit4dl import Kit4DLCallback\n\n\nclass SaveConfusionMatrixCallback(Kit4DLCallback):\n _cm: tm.ConfusionMatrix\n _num_classes: int\n _task: str\n _save_dir: str\n\n def __init__(self, task: str, num_classes: int, save_dir: str) -> None:\n super().__init__()\n self._num_classes = num_classes\n self._save_dir = save_dir\n self._task = task\n os.makedirs(self._save_dir, exist_ok=True)\n\n def on_validation_epoch_start(\n self, trainer: pl.Trainer, pl_module: pl.LightningModule\n ) -> None:\n self._cm = tm.ConfusionMatrix(\n task=self._task, num_classes=self._num_classes\n )\n\n def on_validation_batch_end(\n self,\n trainer: pl.Trainer,\n pl_module: pl.LightningModule,\n outputs: dict,\n batch: Any,\n batch_idx: int,\n dataloader_idx: int = 0,\n ) -> None:\n self._cm.update(outputs['pred'], outputs['true'])\n\n def on_validation_epoch_end(\n self, trainer: pl.Trainer, pl_module: pl.LightningModule\n ) -> None:\n \"\"\"Called when the val epoch ends.\"\"\"\n fig, _ = self._cm.plot()\n target_file = os.path.join(\n self._save_dir,\n f\"confusion_matrix_for_epoch_{pl_module.current_epoch}\",\n )\n fig.savefig(target_file)\n\n```\n\n\n\n\n\n\nBesides those listed in the table above, you can specify PyTorch Lightning-related `Trainer` arguments, like:\n1. `accumulate_grad_batches`\n1. `gradient_clip_val`\n1. `gradient_clip_algorithm`\n1. ...\n\n##### \u270d\ufe0f Example\n\n```toml\n[training]\nepochs = 10\nepoch_schedulers = [\n {target = \"torch.optim.lr_scheduler::CosineAnnealingLR\", T_max = 100}\n]\naccumulate_grad_batches = 2\n```\n\n#### Configuring optimizer\nOptimizer configuration is located in the subsection `[training.optimizer]`.\nThere, you should define `target` (see [Defining `target`](#defining-target)) and extra keyword arguments passed to the optimizer initializer.\n\n##### \u270d\ufe0f Example\n```toml\n[training.optimizer]\ntarget = \"torch.optim::Adam\"\nlr = 0.001\nweight_decay = 0.01\n```\n> **Note**: The section `[training.optimizer]` is **mandatory**.\n> **Note**: You can always define the custom optimizer. Then, you just need to set the proper `target` value.\n\n\n#### Configuring criterion\nSimilarily to the optimizer configuration, there is a subsection dedicated for the critarion. \nYou need to specify, at least, the `target` (see [Defining `target`](#defining-target)) and other mandatory or optional\nproperties of the selected critarion (loss function).\n\n##### \u270d\ufe0f Example\n```toml\n[training.criterion]\ntarget = \"torch.nn::CrossEntropyLoss\"\nweight = [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]\n```\n\n> **Note**: The section `[training.criterion]` is **mandatory**.\n> **Note**: You can always define the custom optimizer. Then, you just need to set the proper `target` value.\n\n#### Configuring metrics\nMetrics are configured in the section `[metrics]` of the configuration file. You can define several metrics (including the custom ones). \nThe only thing you need to do is to define all desired metrics. For each metric dictionary, you need to set `target` (see Section [Defining `target`](#defining-target)) value and, eventually, extra arguments. **REMEMBER** to have metric names (here `MyPrecision` and `FBetaScore`) unique!\n\n##### \u270d\ufe0f Example\n```toml\n[metrics]\nMyPrecision = {target = \"torchmetrics::Precision\", task = \"multiclass\", num_classes=10}\nFBetaScore = {target = \"torchmetrics::FBetaScore\", task = \"multiclass\", num_classes=10, beta = 0.1}\n```\n> **Note**: You can define custom metrics. Just properly set `target` value. **REMEMBER!** The custom metric need to be a subclass of `torchmetrics.Metric` class!\n\n```python\nimport torch\nimport torchmetrics as tm\n\nclass MyMetric(tm.Metric):\n def __init__(self):\n ...\n def update(self, preds: torch.Tensor, target: torch.Tensor):\n ...\n def compute(self):\n ...\n```\n\n#### Configuring checkpoint\nIf you need to save your intermediate weights (do checkpoints) you can configure the optional subsection `[training.checkpoint]`.\nIn the section, you can define the following proeprties:\n\n| **Property** \t| **Type** | **Details** |\n|---------------\t|----------------- | -------------------------------- | \n| `path`* | `str` | path to a directory where checkpoints should be stored\t | \n|`monitor`* | `dict` | a dictionary with two keys: `metric` and `stage`. `metrics` is a metric name as defined in the `[metrics]` section ([Configuring metrics](#configuring-metrics)), `stage` is one of the following: [`train`, `val`] |\n| `filename`* | `str` | filename pattern of the checkpoint (see (PyTorch Lightning `ModelCheckpoint`)[https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.callbacks.ModelCheckpoint.html])\tyou can use value of the defined metric for the stage. if you want `MyPrecision` score for the validation stage, use `{val_myprecision}` in the filename | \n| `mode` | `min` | `max` | to save checkpoint for `min`imum or `max`imum value of the metric being tracked (`monitor`). **default: `max`**\t | \n| `save_top_k` | `int` | save checkepoints for the top `k` values of the metric. **default: `1`**\t |\n| `save_weights_only` | `bool` | if only weights should be saved (`True`) or other states (optimizer, scheduler) also (`False`). **default: `True`**\t |\n| `every_n_epochs` | `int` | The number of training epochs between saving sucessive checkpoints. **default: `1`**\t | \n| `save_on_train_epoch_end` | `bool` | if `False` checkpointing is run at the end of the validation, otherwise - training **default: `False`**\t | \n\n> **Note**: Arguments marked with `*` are obligatory!\n\n##### \u270d\ufe0f Example\n```toml\n[training.checkpoint]\npath = \"{{ PROJECT_DIR }}/chckpt\"\nmonitor = {\"metric\" = \"Precision\", \"stage\" = \"val\"}\nfilename = \"{epoch}_{val_precision:.2f}_cnn\"\nmode = \"max\"\nsave_top_k = 1\n```\n\n> **Note**: You can see we used substitutable symbol `{{ PROJECT_DIR }}`. More about them in the Section [Substitutable symbols](#substitutable-symbols).\n\n\n\n\n#### Defining `target`\nTarget property in the Kit4DL package is kind of extended fully qualified name pointing to the classes supposed to use in the\ngiven context, like for:\n1. neural network class (`target = \"./model.py::SimpleCNN\"`)\n1. datamodule (`target = \"./datamodule.py::MNISTCustomDatamodule\"`)\n1. optimizer (`target = \"torch.optim::Adam\"`)\n1. criterion (`target = \"torch.nn::CrossEntropyLoss\"`)\n1. schedulers (`target = \"torch.optim.lr_scheduler::CosineAnnealingLR\"`)\n\n> **Note**: As a package/module - class separator the double colon is used `::`!\n\nIt might be set in several different ways:\n1. **By using a built-and installed package**. Then, you just need to specify the package/module name and the class name, like `target = \"torch.nn::CrossEntropyLoss\"` (we use module `torch.nn` and class `CrossEntropyLoss` defined within).\n1. **By using a custom module in the project directory**. The project directory, i.e. the directory where the confguration TOML file is located, is added to the `PYTHONPATH`, so you can freely use `.py` files defined there as modules. Having the module `model.py` with the `SimpleCNN` class definition, we can write `target` as `target = \"model::SimpleCNN\"`.\n1. **By using a custom `.py` file.** In this case, you specify `target` as an absolute or relative (w.r.t. the configuration file) to a `.py` file, like `target = \"./model.py::SimpleCNN\"` or `target = \"/usr/neural_nets/my_net/model.py::SimpleCNN\"`.\n\n> **Note**: For `target` definition you can use substitutable symbols defined below.\n\n#### Substitutable symbols\nIn the configuration file you can use symbols that will be substituted during the runtime.\nThe symbols should be surrended by single spaces and in double curly brackets (e.g. `{{ PROJECT_DIR }}`.)\n\n| **Symbol** \t| **Meaning of the symbol** \t| **Example** |\n|-------------\t|-----------------------------------------------------------------------\t| -----------------------------------\t|\n| `PROJECT_DIR`\t| the home directory of the TOML configuration file (project directory) \t| `target = {{ PROJECT_DIR }}/model.py` |\n\n> **Note**: You can also use environmental variables. Just use `env` dict, e.g.: `{{ env['your_var_name'] }}`.\n\n##### \u270d\ufe0f Example\nFirst, let's define some environmental variable: using Python or system tool.\n```python\nimport os\n\nos.environ[\"MY_LOG_LEVEL\"] = \"INFO\"\n```\nor\n```bash\nexport MY_LOG_LEVEL=\"MY_LOG_LEVEL\"\n```\nNow, we can use the environmental variable `MY_LOG_LEVEL` in our config file:\n\n```toml\n[logging]\nlevel = \"{{ env['MY_LOG_LEVEL'] }}\"\nformat = \"%(asctime)s - %(name)s - %(levelname)s - %(message)s\"\n```\n> **Warning**: If you use **double quote** for text values in TOML configuration file, then use **single quote** to access `env` values. \n\n#### Context constants\nWhen you run training using `kit4dl train` command, all custom modules have access to context constant values (defined for the current Python interpreter session).\nYou can access them via `context` module:\n\n##### \u270d\ufe0f Example\n```python\nfrom kit4dl import context\n\nprint(context.PROJECT_DIR)\n```\n\nThe constants currently available in `kit4dl` are the following:\n| **Symbol** \t| **Meaning of the symbol** \t| **Example** |\n|-------------\t|-----------------------------------------------------------------------\t| -----------------------------------\t|\n| `PROJECT_DIR`\t| the home directory of the TOML configuration file (project directory) \t| `context.PROJECT_DIR` |\n| `LOG_LEVEL`\t| logging level as defined in the configuration TOML file \t| `context.LOG_LEVEL` |\n| `LOG_FORMAT`\t| logging message format as defined in the configuration TOML file \t| `context.LOG_FORMAT` |\n| `VERSION`\t| the current version of the package \t| `context.VERSION` |\n\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2023 opengeokube Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "Kit4DL - A quick way to start with machine and deep learning",
"version": "2023.11b1",
"project_urls": null,
"split_keywords": [
"deep learning",
"training pipeline"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "13b8450ddf00420cf211a58afcd55d4729d1ccdf943efe45d22d2c0fda9dcc29",
"md5": "894290d833f5d5cc5eeb3e139ff651ae",
"sha256": "94d51f7d4003b6af8f5f491b042a1a19920a662d89621b69fcb49944e58b1d66"
},
"downloads": -1,
"filename": "kit4dl-2023.11b1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "894290d833f5d5cc5eeb3e139ff651ae",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 53291,
"upload_time": "2023-11-24T16:41:01",
"upload_time_iso_8601": "2023-11-24T16:41:01.375392Z",
"url": "https://files.pythonhosted.org/packages/13/b8/450ddf00420cf211a58afcd55d4729d1ccdf943efe45d22d2c0fda9dcc29/kit4dl-2023.11b1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "33a8aa82c6bd0c235b7845c2dbf5faf49fe5f73a9c269c5edf1f10c85aeebb3c",
"md5": "8346b96f61ea1676b62dc6c41c92991b",
"sha256": "75d680a402b30cb3d001c83dddf20ee67a6bd7b066a1ab90f28e2dbbcfdb120c"
},
"downloads": -1,
"filename": "kit4dl-2023.11b1.tar.gz",
"has_sig": false,
"md5_digest": "8346b96f61ea1676b62dc6c41c92991b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 53659,
"upload_time": "2023-11-24T16:41:02",
"upload_time_iso_8601": "2023-11-24T16:41:02.841512Z",
"url": "https://files.pythonhosted.org/packages/33/a8/aa82c6bd0c235b7845c2dbf5faf49fe5f73a9c269c5edf1f10c85aeebb3c/kit4dl-2023.11b1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-24 16:41:02",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "kit4dl"
}
