# CVTool
**This is a tool box for computer vision and deep learning, while providing [Flask Web API](#web-ui-application) for advanced applications.**
Note : The annotations must follow the [labelme](https://github.com/wkentaro/labelme) format.
The available model tools are as follows：
| MODEL | TASK | REFERENCE |
| ---------------------------------------------------- | --------------------- | ------------------------------------------------------------ |
| [Torchvision](#torchvision-for-image-classification) | Image Classification | [LINK](https://pytorch.org/vision/stable/models.html#classification) |
| [YOLOv7](#yolov7-for-object-detection) | Object Detection | [LINK](https://github.com/WongKinYiu/yolov7) |
| [UNet](#unet-for-semantic-segmentation) | Semantic Segmentation | [LINK](https://github.com/xiaoyufenfei/Efficient-Segmentation-Networks) |
| [HRNet](#hrnet-for-semantic-segmentation) | Semantic Segmentation | [LINK](https://github.com/CSAILVision/semantic-segmentation-pytorch) |
| [PointRend](#pointrend-for-instance-segmentation) | Instance Segmentation | [LINK](https://github.com/ayoolaolafenwa/PixelLib) |
## Requirements
- CUDA >= 11.8
- Python >= 3.7
- Windows Installation
- Tested environment : CUDA==12.0, Python==3.8.10
- Requirements :
```bash
pip install -r requirements/windows.txt
```
```
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
```
```shell
pip install pycocotools-windows
```
```shell
pip install flask
```
- Linux Installation
- Tested environment : CUDA==12.0, Python==3.8.10
- Requirements :
```bash
pip3 install -r requirements/linux.txt
```
## Dataset Preparation
Directory of the labelme dataset must contain all images and corresponding label json files, as shown below.
```markdown
├── dataset
│ ├── image0001.jpg
│ ├── image0001.json
│ ├── image0002.jpg
│ ├── image0002.json
│ ├── image0003.jpg
│ ├── image0003.json
│ ├── .
│ ├── .
│ ├── .
│ ├── image9999.jpg
└── └── image9999.json
```
### YOLO format to Labelme format
If your dataset is in the format of [Yolo_mark](https://github.com/AlexeyAB/Yolo_mark), it can be converted to the [labelme](https://github.com/wkentaro/labelme) format by the script yolo2labelme.py. The method of use is as follows:
```shell
python cvtool/utils/yolo2labelme.py <label_dir> <output_dir> <classes>
```
- **label_dir** : path to a directory of yolo marks
- **output_dir** : path to a directory to save [labelme](https://github.com/wkentaro/labelme) annotations
- **classes** : path to a file with list of class names or a class string separated with comma, e.g., person,car,dog,cat
- note：This tool can still be used even if there are subfolders in your path, it will automatically search for all images and corresponding annotations json in the directory
You can also convert the format through script as follows,
```python
from cvtool.utils import yolo2labelme
yolo2labelme.convert(label_dir, output_dir, classes)
```
### MS COCO format to Labelme format
In addition, if you want to convert COCO's json format into Labelme, you can implement it as follows:
```shell
python cvtool/utils/coco2labelme.py <label_path> <image_dir> <output_dir>
```
- **label_path** : path to a JSON label file with [COCO](https://cocodataset.org/) format, i.e., instances_train2017.json
- **image_dir** : path to a root directory containing all images
- **output_dir** : path to a directory to save [labelme](https://github.com/wkentaro/labelme) annotations
You can also convert the format through script as follows,
```python
from cvtool.utils import coco2labelme
coco2labelme.convert(label_path, image_dir, output_dir)
```
### PASCAL VOC format to Labelme format
The following method can convert PASCAL VOC data into Labelme format:
```shell
python cvtool/utils/pascal2labelme.py <label_dir> <output_dir>
```
- **label_dir** : path to a directory of images and pascal poc labels <.xml>
- **output_dir** : path to a directory to save [labelme](https://github.com/wkentaro/labelme) annotations
- note：This tool can still be used even if there are subfolders in your path, it will automatically search for all images and corresponding annotations xml in the directory
You can also convert the format through script as follows,
```python
from cvtool.utils import pascal2labelme
pascal2labelme.convert(label_dir, output_dir)
```
### Mask image annotations to Labelme format
The following method can convert mask image annotations data into Labelme format:
```shell
python masks2labelme.py <mask_dir> <output_dir> <mask_mark>
```
- **mask_dir** : path to a directory of original images and mask images <.jpg or .png>
- **output_dir** : path to a directory to save [labelme](https://github.com/wkentaro/labelme) annotations
- **mask_mark**： feature words of annotations data that can distinguish between the mask image and the original image on the path. (ex："_GT" or "annotations")
- note：This tool can still be used even if there are subfolders in your path, it will automatically search for all images and corresponding annotations mask in the directory
You can also convert the format through script as follows,
```python
from cvtool.utils import masks2labelme
masks2labelme.convert(mask_dir, output_dir, mask_mark)
```
### Examples for building datasets
1. Create dataset by the directory
```python
from cvtool.data import build_dataset
dataset = build_dataset("path/to/labelme/dataset/")
```
2. Split dataset into several subsets based on the given split ratios
```python
train_set, valid_set = dataset.random_split([0.8, 0.2], seed=21)
train_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)
```
3. Merge datasets
```python
dataset1 = build_dataset("path/to/the/first/labelme/dataset")
dataset2 = build_dataset("path/to/the/second/labelme/dataset")
# Append all data in dataset2 to dataset1
dataset1.merge(dataset2)
```
4. Flag dataset as the training set
```python
dataset.is_train = True
```
5. Dump dataset to a file that lists all image paths
```python
dataset.dump_list("train.txt")
```
6. View all object and class names
```python
print(dataset.object_names) # search all labels in `shapes`
print(dataset.class_names) # search all flags in `flags`
```
7. Register dataset to a global variable `DATASETS`
```python
dataset.register("a-unique-dataset-name")
from cvtool.data import DATASETS # DATASETS is a global variable
dataset = DATASETS["a-unique-dataset-name"]
```
8. Iterate all images and labels
```python
for img, lbl in dataset:
print(img, lbl)
```
9. Map class names, i.e., modify all stingray objects to be treated as `fish`
```python
dataset.map({
"stingray": "fish",
"shark": "fish",
"penguin": "bird",
"puffin": "bird",
"starfish": "cnidaria",
"jellyfish": "cnidaria",
})
```
10. Dump and Load cache file
```python
from cvtool.data import build_dataset_from_cache
dataset.dump_cache("dataset.cache")
dataset = build_dataset_from_cache("dataset.cache", is_train=False)
```
## Torchvision for Image Classification
**Note:**
Depending on the version of torchvision installed, this classifier can support dozens of backbone networks.
You can build classifiers by calling the global variable `AVAILABLE_MODELS` to see the available backbone names.
```python
from cvtool.models.classification import AVAILABLE_MODELS
print(AVAILABLE_MODELS)
```
### Training Example
Dataset Preparation
You can split images into several folders as their categories as follows.
```
├── dataset
│ ├── class_name1
│ │ ├── image0001.jpg
│ │ ├── image0002.jpg
│ │ ├── image0003.jpg
│ ├── .
│ ├── .
│ ├── class_name2
│ │ ├── image0999.jpg
│ │ ├── image1000.jpg
│ │ ├── image1001.jpg
│ ├── .
│ ├── .
└── └── .
```
The following method can convert image folders into into Labelme format as classification dataset
```shell
python cvtool/utils/dirs2labelme.py <base_dir> <output_dir>
```
- **base_dir** : path to the base directory of the images
- **output_dir** : path to a directory to save [labelme](https://github.com/wkentaro/labelme) annotations
```python
from cvtool.models.classification import efficientnet_b0
from cvtool.data import build_dataset
dataset = build_dataset("path/to/labelme/dataset/")
train_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)
trainer = efficientnet_b0.build_trainer()
trainer.cfg.learning_rate = 0.005 # learning rate (optional)
trainer.cfg.batch_size = 16 # training batch size (optional)
trainer.cfg.height = 512 # input height (optional)
trainer.cfg.width = 512 # input width (optional)
trainer.cfg.project = "output/" # path to save training results (optional)
# If you want to classify object boxes as ROIs, set trainer.cfg.roi = True
# You can use the pretrained weights initial model by setting trainer.cfg.weights = "weights.pth"
# You can resume training based on pretrained weights by setting trainer.cfg.resume = True
# start training
trainer.train(train_set, valid_set)
```
### Evaluation Example
```python
from cvtool.models.classification import efficientnet_b0
from cvtool.data import build_dataset
dataset = build_dataset("path/to/labelme/dataset/")
train_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)
evaluator = efficientnet_b0.build_evaluator()
evaluator.cfg.weights = "path/to/the/model_best.pth" # (required)
evaluator.cfg.cfg = "path/to/the/config.yaml" # (required)
# start evaluation
evaluator.evaluate(test_set)
```
### Test Example
```python
from cvtool.models.classification import efficientnet_b0
predictor = efficientnet_b0.build_predictor()
predictor.cfg.weights = "path/to/the/model_best.pth" # (required)
predictor.cfg.cfg = "path/to/the/config.yaml" # (required)
predictor.reset()
# start prediction
# The returned viz is the original image without any marks
(pred_name, prob), viz = predictor.predict("path/to/an/image.jpg", return_viz=True)
# If you want to predict ROI, slice the image before predicting, i.e.,
# x1, y1, x2, y2 = bbox
# (pred_name, prob), viz = predictor.predict(img[y1:y2, x1:x2])
```
## YOLOv7 for Object Detection
### Note:
When using YOLOv7 in the Windows OS, in order to avoid repeated execution of threads, the code should be written as follows.
```python
if __name__ == "__main__":
### Code written here
```
or
```python
def main():
### Code written here
if __name__ == "__main__":
main()
```
### Training Example (Resume, Transform)
```python
from cvtool.models.detection import yolov7
from cvtool.data import build_dataset
import os, cv2
dataset = build_dataset("path/to/labelme/dataset/")
train_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)
trainer = yolov7.build_trainer()
# def grayscale(img, lbl):
# gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# return np.stack((gray,) * 3, -1), lbl
# 1. Resume training from pretrained weights
# trainer.cfg.weights = "path/to/last.pt"
# trainer.cfg.resume = True
# 2. transform images before forward pass
# trainer.cfg.transform_callback = grayscale
trainer.train(train_set, valid_set)
```
### Evaluation Example
```python
evaluator = yolov7.build_evaluator()
# the data was stored in the trainer.cfg.output_dir
evaluator.cfg.data = "path/to/data.yaml"
# the best weights were stored in the trainer.cfg.output_dir/weights/best.pt
evaluator.cfg.weights = "path/to/model/weights.pt"
# start evaluation
evaluator.evaluate(test_set)
```
### Test Example
```python
predictor = yolov7.build_predictor()
# the data was stored in the trainer.cfg.output_dir
predictor.cfg.data = "path/to/data.yaml"
# the best weights were stored in the trainer.cfg.output_dir/weights/best.pt
predictor.cfg.weights = "path/to/model/weights.pt"
# pretrained yolov7 weights for object detection (downloaded from https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7.pt)
predictor.cfg.weights = "path/to/cvtool/models/yolov7/weights/yolov7.pt"
predictor.reset()
# start prediction
detections, viz = predictor.predict("path/to/an/image.jpg", return_viz=True)
cv2.imwrite("detections.png", viz)
```
### YOLOv7 for Pose Estimation
```python
predictor = yolov7.build_predictor()
# assign the pretrained yolov7 weights for post estimation (downloaded from https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7-w6-pose.pt)
predictor.cfg.weights = "path/to/cvtool/models/yolov7/weights/yolov7-w6-pose.pt"
predictor.reset()
image_path = "path/to/an/image.jpg"
detections, viz = predictor.predict(image_path, return_viz=True)
cv2.imwrite("keypoints.png", viz)
```
## UNet for Semantic Segmentation
### Training Example
```python
from cvtool.models.segmentation import unet
from cvtool.data import build_dataset
dataset = build_dataset("path/to/labelme/dataset/")
train_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)
trainer = unet.build_trainer()
trainer.cfg.project = "path/to/save/model"
# total epochs for training
trainer.cfg.epochs = 250
trainer.train(train_set, valid_set)
```
### Evaluation Example
```python
evaluator = unet.build_evaluator()
# path to the pretrained weights
evaluator.cfg.weights = "path/to/the/pretrained/model_best.pth"
# path to the config file stored in the [project] folder
evaluator.cfg.cfg = "path/to/the/config.yaml"
# start evaluation
metrics = evaluator.evaluate(test_set)
```
### Test Example
```python
predictor = unet.build_predictor()
# path to the pretrained weights
predictor.cfg.weights = "path/to/the/pretrained/model_best.pth"
# path to the config file stored in the [project] folder
predictor.cfg.cfg = "path/to/the/config.yaml"
predictor.reset()
# start prediction
outputs, viz = predictor.predict("path/to/an/image", return_viz=True)
cv2.imwrite("segmentation.png", viz)
```
## HRNet for Semantic Segmentation
### Note:
When using hrnet segmentation you need to set annotations mode, "ori_mask" or "fill_poly".
- "fill_poly"：Use the polygon list in the json file to fill to generate a mask image.
**Note** : In "fill_poly" mode, if your labelme polygon list is generate by Contours detect (like masks2labelme.py). It might have some loss in detect process, so in this mode the final annotations may not be equal to the original mask annotations. If not necessary, use "ori_mask" mode.
- "ori_mask" : When you original annotations are mask images, and you don't need to change original annotations, please use this mode.
If you set "ori_mask" mode, please put directory of the original dataset and labelme dataset in the same dir, and use "_Labelme" to distinguish between the original dataset and the dataset converted to labelme(labelme format as described above), detailed structure as shown below.
```
├── all_dataset_dir
│ ├── dataset
│ │ ├── annotations
│ | | ├──image0001.jpg
│ | | ├──image0002.jpg
│ | | | .
│ | | | .
│ │ ├── images (Non-essential)
│ | | ├──image0001.jpg
│ | | ├──image0002.jpg
│ │ | | .
│ │ | | .
│ ├── dataset_Labelme
│ │ ├── image0001.jpg
│ │ ├── image0001.json
│ │ ├── image0002.jpg
│ │ ├── image0002.json
│ │ | .
│ │ | .
└── └── .
```
### Training Example
```python
from cvtool.models.segmentation import hrnet
from cvtool.data import build_dataset
dataset = build_dataset("path/to/labelme/dataset/")
train_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)
trainer = hrnet.build_trainer()
# Parameter configuration files, you can find more model mode configuration files in the config folder
trainer.cfg.cfg ="./cvtool/models/hrnet/utils/config/ade20k-hrnetv2.yaml"
# If you want to resume the weight you train before, just set start_epoch to last time epoch end number
# set 'ori_mask' or 'fill_poly' for annotations mode
trainer.cfg.mode = 'ori_mask'
trainer.train(train_set, valid_set)
```
### Evaluation Example
```python
evaluator = hrnet.build_evaluator()
# path/to/your/weights/folder/XXX.pth, XXX.pth can select best.pth or last.pth. Please ignore encoder_and decoder_ word system will automatically capture them.(required)
predictor.cfg.weights = "path/to/your/weights/folder/best.pth"
# path to the config file stored in the [project] folder
evaluator.cfg.cfg = "path/to/the/config.yaml"
# start evaluation
evaluator.evaluate(test_set)
```
### Test Example
```python
predictor = hrnet.build_predictor()
# path/to/your/weights/folder/XXX.pth, XXX.pth can select best.pth or last.pth. Please ignore encoder_and decoder_ word system will automatically capture them.(required)
predictor.cfg.weights = "path/to/your/weights/folder/best.pth"
# path to the config file stored in the [project] folder
predictor.cfg.cfg = "path/to/the/config.yaml"
# predictor.cfg.class_names_csv = './cvtool/models/hrnet/utils/data/object150_info.csv'
# start prediction
# result will save in TEST.result path which set by config.yaml
predictor.reset()
segm, viz = predictor.predict("path/to/an/image", return_viz=True)
```
- **class_names_csv**： if in cfg your class_name is named by id_num, you can use a csv to replace it. sample csv is in './cvtool/models/hrnet/utils/data/object150_info.csv'
## PointRend for Instance Segmentation
### Training Example
```python
from cvtool.models.detection import pointrend
from cvtool.data import build_dataset
import os, cv2
dataset = build_dataset("path/to/labelme/dataset/")
train_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)
trainer = pointrend.build_trainer()
# path to save results
trainer.cfg.project = "path/to/save/model"
# total epochs for training
trainer.cfg.epochs = 250
# resume from the last checkpoint
trainer.cfg.resume = False
# start training
trainer.train(train_set, valid_set)
```
### Evaluation Example
```python
evaluator = pointrend.build_evaluator()
# path to the pretrained weights
evaluator.cfg.weights = "path/to/the/pretrained/model_final.pth"
# path to the config file stored in the [project] folder
evaluator.cfg.cfg = "path/to/the/config.yaml"
# start evaluation
metrics = evaluator.evaluate(test_set)
```
### Test Example
```python
predictor = pointrend.build_predictor()
# path to the pretrained weights
evaluator.cfg.weights = "path/to/the/pretrained/model_final.pth"
# path to the config file stored in the [project] folder
evaluator.cfg.cfg = "path/to/the/config.yaml"
predictor.reset()
# start prediction
outputs, viz = predictor.predict("path/to/an/image")
cv2.imwrite("detections.png", viz)
```
## Deployment
*This section mainly describes the functions or objects that may be used when deploying*
### Training in a thread
```python
# This method does not affect the execution of the main program when training
from cvtool.utils.thread import StoppableThread
training_thread = StoppableThread(target=trainer.train, args=(train_set, valid_set), daemon=True)
training_thread.start()
# Do something in the main program
# progress = trainer.get_progress() # Get the current training progress (0.0 ~ 1.0)
# training_thread.stop() # Stop training
```
### Setup
Run ```python setup.py`` to compile cvtool.
Tested environment :
- Linux : Python==3.8.10, Cython==0.29.34
- Windows : Python==3.7.9, Cython==0.29.33 on Visual Studio 2019 Community
## Change Logs
2023.04.11：Upload YOLOv7 engines
2023.04.14：Modify some YOLOv7 default configurations
2023.04.23：Update YOLOv7 for Pose Estimation
2023.04.27：Add annotation converter from YOLO mark to labelme format
2023.05.09：Indicate data and weights location in README
2023.05.16：Update the dataset cache dumping and loading method
2023.05.17 : Upload PointRend engines
2023.05.26 : Update the method of model training in a thread
2023.06.01 : Upload the engine and instructions of the classifiers
2023.06.20 : Upload the engine and instructions of the UNet. Add descriptions while printing the engine configs.
2023.07.10 : Upload CVTool Web UI. Update requirements for web UI.
2023.07.20 : Change blueprint import method in the app.
2023.07.31 : Update blueprint method for UI design. Add descriptions for custom pipeline.
2023.08.21 : Move the instructions related to Web UI to cvtool/app/README.md
2023.09.18 : Upload HRNet engines for semantic segmentation
2023.09.28 : Upload setup.py for compiling
Raw data
{
"_id": null,
"home_page": "",
"name": "pycvtool",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "cvtool,computer vision,deep learning,object detection,instance segmentation,pose estimation,image classification",
"author": "Bruce Chuang",
"author_email": "brucechuang1109@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/28/47/362793f73be1d4fd603478076cafd9d42d081604a054f37315ba64eff6a3/pycvtool-0.0.2.tar.gz",
"platform": null,
"description": "# CVTool\n\n**This is a tool box for computer vision and deep learning, while providing [Flask Web API](#web-ui-application) for advanced applications.**\n\nNote : The annotations must follow the [labelme](https://github.com/wkentaro/labelme) format.\n\nThe available model tools are as follows\uff1a\n\n| MODEL | TASK | REFERENCE |\n| ---------------------------------------------------- | --------------------- | ------------------------------------------------------------ |\n| [Torchvision](#torchvision-for-image-classification) | Image Classification | [LINK](https://pytorch.org/vision/stable/models.html#classification) |\n| [YOLOv7](#yolov7-for-object-detection) | Object Detection | [LINK](https://github.com/WongKinYiu/yolov7) |\n| [UNet](#unet-for-semantic-segmentation) | Semantic Segmentation | [LINK](https://github.com/xiaoyufenfei/Efficient-Segmentation-Networks) |\n| [HRNet](#hrnet-for-semantic-segmentation) | Semantic Segmentation | [LINK](https://github.com/CSAILVision/semantic-segmentation-pytorch) |\n| [PointRend](#pointrend-for-instance-segmentation) | Instance Segmentation | [LINK](https://github.com/ayoolaolafenwa/PixelLib) |\n\n## Requirements\n\n- CUDA >= 11.8\n\n- Python >= 3.7\n\n- Windows Installation\n\n - Tested environment : CUDA==12.0, Python==3.8.10\n\n - Requirements : \n\n ```bash\n pip install -r requirements/windows.txt\n ```\n\n ```\n pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118\n ```\n\n ```shell\n pip install pycocotools-windows\n ```\n \n ```shell\n pip install flask\n ```\n\n- Linux Installation\n\n - Tested environment : CUDA==12.0, Python==3.8.10\n\n - Requirements : \n\n ```bash\n pip3 install -r requirements/linux.txt\n ```\n\n## Dataset Preparation\n\nDirectory of the labelme dataset must contain all images and corresponding label json files, as shown below.\n\n```markdown\n\u251c\u2500\u2500 dataset\n\u2502 \u251c\u2500\u2500 image0001.jpg\n\u2502 \u251c\u2500\u2500 image0001.json\n\u2502 \u251c\u2500\u2500 image0002.jpg\n\u2502 \u251c\u2500\u2500 image0002.json\n\u2502 \u251c\u2500\u2500 image0003.jpg\n\u2502 \u251c\u2500\u2500 image0003.json\n\u2502 \u251c\u2500\u2500 \u00a0\u00a0\u00a0\u00a0.\n\u2502 \u251c\u2500\u2500 \u00a0\u00a0\u00a0\u00a0.\n\u2502 \u251c\u2500\u2500 \u00a0\u00a0\u00a0\u00a0.\n\u2502 \u251c\u2500\u2500 image9999.jpg\n\u2514\u2500\u2500 \u2514\u2500\u2500 image9999.json\n```\n\n### YOLO format to Labelme format\n\nIf your dataset is in the format of [Yolo_mark](https://github.com/AlexeyAB/Yolo_mark), it can be converted to the [labelme](https://github.com/wkentaro/labelme) format by the script yolo2labelme.py. The method of use is as follows:\n\n```shell\npython cvtool/utils/yolo2labelme.py <label_dir> <output_dir> <classes>\n```\n\n- **label_dir** : path to a directory of yolo marks\n- **output_dir** : path to a directory to save [labelme](https://github.com/wkentaro/labelme) annotations\n- **classes** : path to a file with list of class names or a class string separated with comma, e.g., person,car,dog,cat\n- note\uff1aThis tool can still be used even if there are subfolders in your path, it will automatically search for all images and corresponding annotations json in the directory\n\nYou can also convert the format through script as follows,\n\n```python\nfrom cvtool.utils import yolo2labelme\n\nyolo2labelme.convert(label_dir, output_dir, classes)\n```\n\n### MS COCO format to Labelme format\n\nIn addition, if you want to convert COCO's json format into Labelme, you can implement it as follows:\n\n```shell\npython cvtool/utils/coco2labelme.py <label_path> <image_dir> <output_dir>\n```\n\n- **label_path** : path to a JSON label file with [COCO](https://cocodataset.org/) format, i.e., instances_train2017.json\n- **image_dir** : path to a root directory containing all images\n- **output_dir** : path to a directory to save [labelme](https://github.com/wkentaro/labelme) annotations\n\nYou can also convert the format through script as follows,\n\n```python\nfrom cvtool.utils import coco2labelme\n\ncoco2labelme.convert(label_path, image_dir, output_dir)\n```\n\n### PASCAL VOC format to Labelme format\n\nThe following method can convert PASCAL VOC data into Labelme format:\n\n```shell\npython cvtool/utils/pascal2labelme.py <label_dir> <output_dir>\n```\n\n- **label_dir** : path to a directory of images and pascal poc labels <.xml>\n- **output_dir** : path to a directory to save [labelme](https://github.com/wkentaro/labelme) annotations\n- note\uff1aThis tool can still be used even if there are subfolders in your path, it will automatically search for all images and corresponding annotations xml in the directory\n\nYou can also convert the format through script as follows,\n\n```python\nfrom cvtool.utils import pascal2labelme\n\npascal2labelme.convert(label_dir, output_dir)\n```\n### Mask image annotations to Labelme format\nThe following method can convert mask image annotations data into Labelme format:\n\n```shell\npython masks2labelme.py <mask_dir> <output_dir> <mask_mark> \n```\n\n- **mask_dir** : path to a directory of original images and mask images <.jpg or .png>\n- **output_dir** : path to a directory to save [labelme](https://github.com/wkentaro/labelme) annotations\n- **mask_mark**\uff1a feature words of annotations data that can distinguish between the mask image and the original image on the path. (ex\uff1a\"_GT\" or \"annotations\")\n- note\uff1aThis tool can still be used even if there are subfolders in your path, it will automatically search for all images and corresponding annotations mask in the directory\n\nYou can also convert the format through script as follows,\n\n```python\nfrom cvtool.utils import masks2labelme\n\nmasks2labelme.convert(mask_dir, output_dir, mask_mark)\n```\n\n### Examples for building datasets\n\n1. Create dataset by the directory\n\n```python\nfrom cvtool.data import build_dataset\ndataset = build_dataset(\"path/to/labelme/dataset/\")\n```\n\n2. Split dataset into several subsets based on the given split ratios\n\n```python\ntrain_set, valid_set = dataset.random_split([0.8, 0.2], seed=21)\ntrain_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)\n```\n\n3. Merge datasets\n\n```python\ndataset1 = build_dataset(\"path/to/the/first/labelme/dataset\")\ndataset2 = build_dataset(\"path/to/the/second/labelme/dataset\")\n# Append all data in dataset2 to dataset1\ndataset1.merge(dataset2)\n```\n\n4. Flag dataset as the training set\n\n```python\ndataset.is_train = True\n```\n\n5. Dump dataset to a file that lists all image paths\n\n```python\ndataset.dump_list(\"train.txt\")\n```\n\n6. View all object and class names\n\n```python\nprint(dataset.object_names) # search all labels in `shapes`\nprint(dataset.class_names) # search all flags in `flags`\n```\n\n7. Register dataset to a global variable `DATASETS`\n\n```python\ndataset.register(\"a-unique-dataset-name\")\nfrom cvtool.data import DATASETS # DATASETS is a global variable\ndataset = DATASETS[\"a-unique-dataset-name\"]\n```\n\n8. Iterate all images and labels\n\n```python\nfor img, lbl in dataset:\n print(img, lbl)\n```\n\n9. Map class names, i.e., modify all stingray objects to be treated as `fish`\n\n```python\ndataset.map({\n \"stingray\": \"fish\", \n \"shark\": \"fish\",\n \"penguin\": \"bird\",\n \"puffin\": \"bird\",\n \"starfish\": \"cnidaria\",\n \"jellyfish\": \"cnidaria\",\n})\n```\n\n10. Dump and Load cache file\n\n```python\nfrom cvtool.data import build_dataset_from_cache\n\ndataset.dump_cache(\"dataset.cache\")\ndataset = build_dataset_from_cache(\"dataset.cache\", is_train=False)\n```\n\n## Torchvision for Image Classification\n\n**Note:**\n\nDepending on the version of torchvision installed, this classifier can support dozens of backbone networks.\n\nYou can build classifiers by calling the global variable `AVAILABLE_MODELS` to see the available backbone names.\n\n```python\nfrom cvtool.models.classification import AVAILABLE_MODELS\nprint(AVAILABLE_MODELS)\n```\n\n### Training Example\n\nDataset Preparation\n\nYou can split images into several folders as their categories as follows.\n\n```\n\u251c\u2500\u2500 dataset\n\u2502 \u251c\u2500\u2500 class_name1\n\u2502 \u2502 \u251c\u2500\u2500 image0001.jpg\n\u2502 \u2502 \u251c\u2500\u2500 image0002.jpg\n\u2502 \u2502 \u251c\u2500\u2500 image0003.jpg\n\u2502 \u251c\u2500\u2500 \u00a0\u00a0\u00a0\u00a0.\n\u2502 \u251c\u2500\u2500 \u00a0\u00a0\u00a0\u00a0.\n\u2502 \u251c\u2500\u2500 class_name2\n\u2502 \u2502 \u251c\u2500\u2500 image0999.jpg\n\u2502 \u2502 \u251c\u2500\u2500 image1000.jpg\n\u2502 \u2502 \u251c\u2500\u2500 image1001.jpg\n\u2502 \u251c\u2500\u2500 \u00a0\u00a0\u00a0\u00a0.\n\u2502 \u251c\u2500\u2500 \u00a0\u00a0\u00a0\u00a0.\n\u2514\u2500\u2500 \u2514\u2500\u2500 .\n```\n\nThe following method can convert image folders into into Labelme format as classification dataset\n\n```shell\npython cvtool/utils/dirs2labelme.py <base_dir> <output_dir>\n```\n\n- **base_dir** : path to the base directory of the images\n- **output_dir** : path to a directory to save [labelme](https://github.com/wkentaro/labelme) annotations\n\n```python\nfrom cvtool.models.classification import efficientnet_b0\nfrom cvtool.data import build_dataset\n\ndataset = build_dataset(\"path/to/labelme/dataset/\")\ntrain_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)\n\ntrainer = efficientnet_b0.build_trainer()\ntrainer.cfg.learning_rate = 0.005 # learning rate (optional)\ntrainer.cfg.batch_size = 16 # training batch size (optional)\ntrainer.cfg.height = 512 # input height (optional)\ntrainer.cfg.width = 512 # input width (optional)\ntrainer.cfg.project = \"output/\" # path to save training results (optional)\n\n# If you want to classify object boxes as ROIs, set trainer.cfg.roi = True\n# You can use the pretrained weights initial model by setting trainer.cfg.weights = \"weights.pth\"\n# You can resume training based on pretrained weights by setting trainer.cfg.resume = True\n\n# start training\ntrainer.train(train_set, valid_set)\n```\n\n### Evaluation Example\n\n```python\nfrom cvtool.models.classification import efficientnet_b0\nfrom cvtool.data import build_dataset\n\ndataset = build_dataset(\"path/to/labelme/dataset/\")\ntrain_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)\n\nevaluator = efficientnet_b0.build_evaluator()\nevaluator.cfg.weights = \"path/to/the/model_best.pth\" # (required)\nevaluator.cfg.cfg = \"path/to/the/config.yaml\" # (required)\n\n# start evaluation\nevaluator.evaluate(test_set)\n```\n\n### Test Example\n\n```python\nfrom cvtool.models.classification import efficientnet_b0\n\npredictor = efficientnet_b0.build_predictor()\npredictor.cfg.weights = \"path/to/the/model_best.pth\" # (required)\npredictor.cfg.cfg = \"path/to/the/config.yaml\" # (required)\npredictor.reset()\n\n# start prediction\n# The returned viz is the original image without any marks\n(pred_name, prob), viz = predictor.predict(\"path/to/an/image.jpg\", return_viz=True)\n\n# If you want to predict ROI, slice the image before predicting, i.e.,\n# x1, y1, x2, y2 = bbox\n# (pred_name, prob), viz = predictor.predict(img[y1:y2, x1:x2])\n```\n\n## YOLOv7 for Object Detection\n\n### Note: \n\nWhen using YOLOv7 in the Windows OS, in order to avoid repeated execution of threads, the code should be written as follows.\n\n```python\nif __name__ == \"__main__\":\n ### Code written here\n```\n\nor\n\n```python\ndef main():\n ### Code written here\n\nif __name__ == \"__main__\":\n main()\n```\n\n### Training Example (Resume, Transform)\n\n```python\nfrom cvtool.models.detection import yolov7\nfrom cvtool.data import build_dataset\nimport os, cv2\n\ndataset = build_dataset(\"path/to/labelme/dataset/\")\ntrain_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)\n\ntrainer = yolov7.build_trainer()\n# def grayscale(img, lbl):\n# gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n# return np.stack((gray,) * 3, -1), lbl\n\n# 1. Resume training from pretrained weights\n# trainer.cfg.weights = \"path/to/last.pt\"\n# trainer.cfg.resume = True\n\n# 2. transform images before forward pass\n# trainer.cfg.transform_callback = grayscale\n\ntrainer.train(train_set, valid_set)\n```\n\n### Evaluation Example\n\n```python\nevaluator = yolov7.build_evaluator()\n\n# the data was stored in the trainer.cfg.output_dir\nevaluator.cfg.data = \"path/to/data.yaml\"\n# the best weights were stored in the trainer.cfg.output_dir/weights/best.pt\nevaluator.cfg.weights = \"path/to/model/weights.pt\"\n\n# start evaluation\nevaluator.evaluate(test_set)\n```\n\n### Test Example\n\n```python\npredictor = yolov7.build_predictor()\n\n# the data was stored in the trainer.cfg.output_dir\npredictor.cfg.data = \"path/to/data.yaml\"\n# the best weights were stored in the trainer.cfg.output_dir/weights/best.pt\npredictor.cfg.weights = \"path/to/model/weights.pt\"\n# pretrained yolov7 weights for object detection (downloaded from https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7.pt)\npredictor.cfg.weights = \"path/to/cvtool/models/yolov7/weights/yolov7.pt\"\npredictor.reset()\n\n# start prediction\ndetections, viz = predictor.predict(\"path/to/an/image.jpg\", return_viz=True)\ncv2.imwrite(\"detections.png\", viz)\n```\n\n### YOLOv7 for Pose Estimation\n\n```python\npredictor = yolov7.build_predictor()\n# assign the pretrained yolov7 weights for post estimation (downloaded from https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7-w6-pose.pt)\npredictor.cfg.weights = \"path/to/cvtool/models/yolov7/weights/yolov7-w6-pose.pt\"\npredictor.reset()\n\nimage_path = \"path/to/an/image.jpg\"\n\ndetections, viz = predictor.predict(image_path, return_viz=True)\ncv2.imwrite(\"keypoints.png\", viz)\n```\n\n## UNet for Semantic Segmentation\n\n### Training Example\n\n```python\nfrom cvtool.models.segmentation import unet\nfrom cvtool.data import build_dataset\n\ndataset = build_dataset(\"path/to/labelme/dataset/\")\ntrain_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)\n\ntrainer = unet.build_trainer()\n\ntrainer.cfg.project = \"path/to/save/model\"\n# total epochs for training\ntrainer.cfg.epochs = 250\n\ntrainer.train(train_set, valid_set)\n```\n\n### Evaluation Example\n\n```python\nevaluator = unet.build_evaluator()\n\n# path to the pretrained weights\nevaluator.cfg.weights = \"path/to/the/pretrained/model_best.pth\"\n# path to the config file stored in the [project] folder\nevaluator.cfg.cfg = \"path/to/the/config.yaml\"\n\n# start evaluation\nmetrics = evaluator.evaluate(test_set)\n```\n\n### Test Example\n\n```python\npredictor = unet.build_predictor()\n\n# path to the pretrained weights\npredictor.cfg.weights = \"path/to/the/pretrained/model_best.pth\"\n# path to the config file stored in the [project] folder\npredictor.cfg.cfg = \"path/to/the/config.yaml\"\npredictor.reset()\n\n# start prediction\noutputs, viz = predictor.predict(\"path/to/an/image\", return_viz=True)\ncv2.imwrite(\"segmentation.png\", viz)\n```\n## HRNet for Semantic Segmentation\n\n### Note: \nWhen using hrnet segmentation you need to set annotations mode, \"ori_mask\" or \"fill_poly\".\n- \"fill_poly\"\uff1aUse the polygon list in the json file to fill to generate a mask image.\n\n **Note** : In \"fill_poly\" mode, if your labelme polygon list is generate by Contours detect (like masks2labelme.py). It might have some loss in detect process, so in this mode the final annotations may not be equal to the original mask annotations. If not necessary, use \"ori_mask\" mode.\n\n- \"ori_mask\" : When you original annotations are mask images, and you don't need to change original annotations, please use this mode.\nIf you set \"ori_mask\" mode, please put directory of the original dataset and labelme dataset in the same dir, and use \"_Labelme\" to distinguish between the original dataset and the dataset converted to labelme(labelme format as described above), detailed structure as shown below.\n```\n\u251c\u2500\u2500 all_dataset_dir\n\u2502 \u251c\u2500\u2500 dataset\n\u2502 \u2502 \u251c\u2500\u2500 annotations\n\u2502 | | \u251c\u2500\u2500image0001.jpg\n\u2502 | | \u251c\u2500\u2500image0002.jpg\n\u2502 | | \u00a0\u00a0|\u00a0.\n\u2502 | \u00a0\u00a0| |\u00a0.\n\u2502 \u2502 \u251c\u2500\u2500 images (Non-essential)\n\u2502 | | \u251c\u2500\u2500image0001.jpg\n\u2502 | | \u251c\u2500\u2500image0002.jpg\n\u2502 \u2502 \u00a0\u00a0|\u00a0 | .\n\u2502 \u2502 \u00a0\u00a0| |\u00a0.\n\u2502 \u251c\u2500\u2500 dataset_Labelme\n\u2502 \u2502 \u251c\u2500\u2500 image0001.jpg\n\u2502 \u2502 \u251c\u2500\u2500 image0001.json\n\u2502 \u2502 \u251c\u2500\u2500 image0002.jpg\n\u2502 \u2502 \u251c\u2500\u2500 image0002.json\n\u2502 \u2502 \u00a0\u00a0| \u00a0\u00a0.\n\u2502 \u2502 \u00a0\u00a0| \u00a0\u00a0.\n\u2514\u2500\u2500 \u2514\u2500\u2500 .\n```\n\n### Training Example\n\n```python\nfrom cvtool.models.segmentation import hrnet\nfrom cvtool.data import build_dataset\n\ndataset = build_dataset(\"path/to/labelme/dataset/\")\ntrain_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)\n\ntrainer = hrnet.build_trainer()\n# Parameter configuration files, you can find more model mode configuration files in the config folder\ntrainer.cfg.cfg =\"./cvtool/models/hrnet/utils/config/ade20k-hrnetv2.yaml\"\n# If you want to resume the weight you train before, just set start_epoch to last time epoch end number\n\n# set 'ori_mask' or 'fill_poly' for annotations mode\ntrainer.cfg.mode = 'ori_mask'\n \ntrainer.train(train_set, valid_set)\n```\n### Evaluation Example\n\n```python\nevaluator = hrnet.build_evaluator()\n\n# path/to/your/weights/folder/XXX.pth, XXX.pth can select best.pth or last.pth. Please ignore encoder_and decoder_ word system will automatically capture them.(required)\npredictor.cfg.weights = \"path/to/your/weights/folder/best.pth\"\n\n# path to the config file stored in the [project] folder\nevaluator.cfg.cfg = \"path/to/the/config.yaml\"\n\n# start evaluation\nevaluator.evaluate(test_set)\n```\n\n### Test Example\n\n```python\npredictor = hrnet.build_predictor()\n\n# path/to/your/weights/folder/XXX.pth, XXX.pth can select best.pth or last.pth. Please ignore encoder_and decoder_ word system will automatically capture them.(required)\npredictor.cfg.weights = \"path/to/your/weights/folder/best.pth\"\n# path to the config file stored in the [project] folder\npredictor.cfg.cfg = \"path/to/the/config.yaml\"\n\n# predictor.cfg.class_names_csv = './cvtool/models/hrnet/utils/data/object150_info.csv'\n\n# start prediction\n# result will save in TEST.result path which set by config.yaml\npredictor.reset()\nsegm, viz = predictor.predict(\"path/to/an/image\", return_viz=True)\n```\n- **class_names_csv**\uff1a if in cfg your class_name is named by id_num, you can use a csv to replace it. sample csv is in './cvtool/models/hrnet/utils/data/object150_info.csv'\n\n## PointRend for Instance Segmentation\n\n### Training Example\n\n```python\nfrom cvtool.models.detection import pointrend\nfrom cvtool.data import build_dataset\nimport os, cv2\n\ndataset = build_dataset(\"path/to/labelme/dataset/\")\ntrain_set, valid_set, test_set = dataset.random_split([0.7, 0.2, 0.1], seed=42)\n\ntrainer = pointrend.build_trainer()\n\n# path to save results\ntrainer.cfg.project = \"path/to/save/model\"\n# total epochs for training\ntrainer.cfg.epochs = 250\n# resume from the last checkpoint\ntrainer.cfg.resume = False\n\n# start training\ntrainer.train(train_set, valid_set)\n```\n\n### Evaluation Example\n\n```python\nevaluator = pointrend.build_evaluator()\n\n# path to the pretrained weights\nevaluator.cfg.weights = \"path/to/the/pretrained/model_final.pth\"\n# path to the config file stored in the [project] folder\nevaluator.cfg.cfg = \"path/to/the/config.yaml\"\n\n# start evaluation\nmetrics = evaluator.evaluate(test_set)\n```\n\n### Test Example\n\n```python\npredictor = pointrend.build_predictor()\n\n# path to the pretrained weights\nevaluator.cfg.weights = \"path/to/the/pretrained/model_final.pth\"\n# path to the config file stored in the [project] folder\nevaluator.cfg.cfg = \"path/to/the/config.yaml\"\npredictor.reset()\n\n# start prediction\noutputs, viz = predictor.predict(\"path/to/an/image\")\ncv2.imwrite(\"detections.png\", viz)\n```\n\n## Deployment\n\n*This section mainly describes the functions or objects that may be used when deploying*\n\n### Training in a thread\n\n```python\n# This method does not affect the execution of the main program when training\nfrom cvtool.utils.thread import StoppableThread\n\ntraining_thread = StoppableThread(target=trainer.train, args=(train_set, valid_set), daemon=True)\ntraining_thread.start()\n\n# Do something in the main program\n# progress = trainer.get_progress() # Get the current training progress (0.0 ~ 1.0)\n# training_thread.stop()\t\t\t# Stop training\n```\n\n### Setup\n\nRun ```python setup.py`` to compile cvtool.\n\nTested environment :\n\n- Linux : Python==3.8.10, Cython==0.29.34\n- Windows : Python==3.7.9, Cython==0.29.33 on Visual Studio 2019 Community\n\n## Change Logs\n\n2023.04.11\uff1aUpload YOLOv7 engines\n\n2023.04.14\uff1aModify some YOLOv7 default configurations\n\n2023.04.23\uff1aUpdate YOLOv7 for Pose Estimation\n\n2023.04.27\uff1aAdd annotation converter from YOLO mark to labelme format\n\n2023.05.09\uff1aIndicate data and weights location in README\n\n2023.05.16\uff1aUpdate the dataset cache dumping and loading method\n\n2023.05.17 : Upload PointRend engines\n\n2023.05.26 : Update the method of model training in a thread\n\n2023.06.01 : Upload the engine and instructions of the classifiers\n\n2023.06.20 : Upload the engine and instructions of the UNet. Add descriptions while printing the engine configs.\n\n2023.07.10 : Upload CVTool Web UI. Update requirements for web UI.\n\n2023.07.20 : Change blueprint import method in the app.\n\n2023.07.31 : Update blueprint method for UI design. Add descriptions for custom pipeline.\n\n2023.08.21 : Move the instructions related to Web UI to cvtool/app/README.md\n\n2023.09.18 : Upload HRNet engines for semantic segmentation\n\n2023.09.28 : Upload setup.py for compiling\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Deep learning tool for computer vision",
"version": "0.0.2",
"project_urls": null,
"split_keywords": [
"cvtool",
"computer vision",
"deep learning",
"object detection",
"instance segmentation",
"pose estimation",
"image classification"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0a4146ee3e4a8e02216de039ea4028bee7810ce76765e326511e6a7a02ab09ed",
"md5": "0e88d8cecfd284702cd2a976c6b47f3b",
"sha256": "67ccc28038c19c29c5a8a822b92d0f6c29d23b46da748a4e946938bd4906de5d"
},
"downloads": -1,
"filename": "pycvtool-0.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0e88d8cecfd284702cd2a976c6b47f3b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 7087,
"upload_time": "2023-10-03T06:32:31",
"upload_time_iso_8601": "2023-10-03T06:32:31.852076Z",
"url": "https://files.pythonhosted.org/packages/0a/41/46ee3e4a8e02216de039ea4028bee7810ce76765e326511e6a7a02ab09ed/pycvtool-0.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2847362793f73be1d4fd603478076cafd9d42d081604a054f37315ba64eff6a3",
"md5": "c9f6b4002f7a7559c0728ffc64c60e12",
"sha256": "4dd215f2a684095d63d8b80bfcc5e457c3254d966e282a4ad3ef6ba797f64b77"
},
"downloads": -1,
"filename": "pycvtool-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "c9f6b4002f7a7559c0728ffc64c60e12",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 7603,
"upload_time": "2023-10-03T06:32:34",
"upload_time_iso_8601": "2023-10-03T06:32:34.060826Z",
"url": "https://files.pythonhosted.org/packages/28/47/362793f73be1d4fd603478076cafd9d42d081604a054f37315ba64eff6a3/pycvtool-0.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-03 06:32:34",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "pycvtool"
}
JSON