# BaseNet: A simpler way to build AI models.
<p align="center">
<img src="https://raw.githubusercontent.com/iTzAlver/basenet_api/main/doc/multimedia/basenet_logo.png" width="400px">
</p>
<p align="center">
<a href="https://github.com/iTzAlver/basenet_api/blob/main/LICENSE">
<img src="https://img.shields.io/github/license/iTzAlver/basenet_api?color=purple&style=plastic" /></a>
<a href="https://github.com/iTzAlver/basenet_api/tree/main/test">
<img src="https://img.shields.io/badge/tests-passed-green?color=green&style=plastic" /></a>
<a href="https://github.com/iTzAlver/basenet_api/blob/main/requirements.txt">
<img src="https://img.shields.io/badge/requirements-pypi-red?color=red&style=plastic" /></a>
<a href="https://htmlpreview.github.io/?https://github.com/iTzAlver/basenet_api/blob/main/doc/basenet.html">
<img src="https://img.shields.io/badge/doc-available-green?color=yellow&style=plastic" /></a>
<a href="https://github.com/iTzAlver/BaseNet-API/releases/tag/1.5.0-release">
<img src="https://img.shields.io/badge/release-1.5.0-white?color=white&style=plastic" /></a>
</p>
<p align="center">
<a href="https://www.tensorflow.org/">
<img src="https://img.shields.io/badge/dependencies-tensorflow-red?color=orange&style=for-the-badge" /></a>
<a href="https://keras.io/">
<img src="https://img.shields.io/badge/dependencies-keras-red?color=red&style=for-the-badge" /></a>
<a href="https://www.ray.io/">
<img src="https://img.shields.io/badge/dependencies-ray-red?color=blue&style=for-the-badge" /></a>
</p>
# Basenet API Package - 1.9.3: (2.0.0 Is coming!!)
This package implements an API over Keras and Tensorflow to build Deep Learning models easily without losing the
framework flexibility. BaseNet API tries to implement almost everything from a few lines of code.
> **Disclaimer**: This API is under development. This means that there isn't a stable release yet.
> However, some features are stable and can be used. Check the tutorials for more info.
> Please, check the roadmap below to see the future updates.
> Any feature request is wellcome and appreciated; and has a high probability to be implemented.
> Any issue is wellcome and appreciated; and it will often be inspected.
## About ##
Author: A.Palomo-Alonso (a.palomo@uah.es)
Universidad de Alcalá.
Escuela Politécnica Superior.
Departamento de Teoría De la Señal y Comunicaciones (TDSC).
ISDEFE Chair of Research.
## Features
#### **MACHINE LEARNING**
* **Feature 01:** Database train, validation and test automatic and random segmentation for DeepLearning.
* **Feature 02:** Solving optimization problems for MetaHeuristic models.
#### **EFFICIENCY**
* **Feature 03:** Real multiprocessing training process (CPU usage optimization).
* **Feature 04:** Automatic and custom GPU usage and assignment.
#### **SIMPLICITY**
* **Feature 05:** Easy-to-use API.
* **Feature 06:** API documentation.
* **Feature 07:** JuPyter Notebooks tutorials included.
* **Feature 08:** Python Packaging and PyPi indexing.
#### **MONITORIZATION**
* **Feature 09:** Real-time logging.
* **Feature 10:** Dashboards included.
* **Feature 11:** Model printing and easy debugging.
#### **CONNECTIVITY**
* **Feature 12:** Model merging and multiple model inputs.
* **Feature 13:** Computational cluster linking.
* **Feature 14:** The different parts of the API are designed to interact.
* **Feature 15:** It allows to create dynamic databases for data ingestion and math problems that require synthetic data.
#### **RELIABILITY**
* **Feature 16:** Depends on huge and reliable frameworks: KERAS, TENSORFLOW, RAY.
* **Feature 17:** Code updating and active support.
### Cons:
#### **FLEXIBILITY**
* An API must look for a balance between simplicity and flexibility. In this case, we bet on simplicity; but the API
is still highly flexible.
* The API is designed for high level research and design. But it is not optimal for low-level research.
#### **DEPENDENCE**
* This API is built over highly reliable frameworks, however, the API depends on those frameworks to run the models.
#### **DEPLOYMENT**
* The API can not deploy models by itself yet in this current version. But I am planning!
### Roadmap:
#### Discipline coverage:
- [x] Database building.
- [x] Supervised learning.
- [x] MetaHeuristic optimization.
- [ ] Unsupervised learning. (Planning in 3.0 release)
- [ ] Reinforcement learning. (Planning in 4.0 release)
- [ ] Computer Vision. (Planning in 5.0 release)
- [ ] Natural Language Processing. (Will be in development from 3.0 to 6.0)
#### Feature roadmap:
- [x] Feeder databases.
- [x] Computational clustering.
- [x] Monitoring.
- [ ] BaseNetDeployment:
- Deploy your model in your infrastructure with high connectivity and scalability.
- With preprocess, model, postprocess and front-end sector.
- Automatic workload balance (probably with ``Ray`` or ``Spark`` clusters).
- Dynamic training after deployment. The model will still be learning from input data if desired.
- [ ] Reinforcement learning:
- This will be hard to add in deployment.
- Custom environments.
- Connectivity with BaseNetCompiler: It will work with RL policy and DL!
- State-of-the-art RL.
- [ ] Computer Vision:
- It is already possible to be implemented in DeepLearning API; but can be improved.
- Database visualization.
- Database size optimization.
- Computer vision utils.
- [ ] Natural Language Processing:
- Hard to tell here, this discipline is very ad-hoc, and it is hard to pack it in an API.
- Transformers, LSTM and Word Vectorization will be included.
- Preprocessing will be a must.
- Probably will use ``HuggingFace`` package.
- Will be in constant development.
## What's new?
### < 0.1.0
1. BaseNetModel included.
2. BaseNetDatabase included.
3. BaseNetCompiler included.
4. Inheritance from CorNetAPI project.
5. Multi-processing fitting.
6. Tensorboard launching.
### 0.2.0
1. BaseNetResults included (working).
2. Now the model is callable.
3. Switched print to logging.
4. Project documentation.
### 1.0.0 - 1.0.3
1. Python packaging
3. 1.0.x: Upload bug solving.
### 1.1.0
1. Functional package.
2. PyPi indexing.
### 1.2.0:
1. Loss results included in the BaseNetResults while multiprocessing.
2. GPU auto set up to avoid TensorFlow memory errors.
3. Method ``BaseNetCompiler.set_up_devices()`` configures the GPUs according to the free RAM to be used in the API.
### 1.3.0
1. Included WindowDiff to the project scope.
### 1.4.0
1. Solved python packaging problems.
2. Included force stop callback in the ```BaseNetModel.fit_stop()``` method.
### 1.5.0
1. BaseNetDatabase now has the attributes ``BaseNetDatabase.size`` and ``BaseNetDatabase.distribution``.
2. Solved forced stopping bugs with multiprocessing in the method ``BaseNetDatabase.fit_stop()``.
3. ```BaseNetModel._threshold()``` private method now takes a set of outputs instead only one.
This was only for optimization.
4. Solved wrong ```BaseNetModel.recover()```.
5. **Auto recover implemented**, now ```BaseNetModel.recover()``` is a private method: ```BaseNetModel._recover()```.
Now the used does not need to recover it. *The model recovers by itself. -- Hans Niemann 2022.*
**NOTE**: RECOVER IS NECESARY WHEN THE MODEL IS EARLY STOPPED; CONSIDER RECOVERING ALWAYS THE MODEL.
### 1.5.1 - 1.5.3
1. Solved a bug where ``BaseNetDatabase`` modified the incoming list of instances in the database; avoiding checkpoints
for large database generators.
2. Exception handler for ``nvml`` library if NVIDIA Drivers are not installed`in the machine.
### 1.5.4
1. Added some ``BaseNetDatabase`` utils: merge and split databases.
2. Added ``BaseNetDatabase`` equality check.
3. Added a ``BaseNetDatabase._reversion()``, ``BaseNetCompiler._reversion()`` and ``BaseNetModel.__version__``. Which
rebuilds the Classes to the current version of the API.
### 1.6.0
1. Start to develop the second branch of the API: ``BaseNetHeuristic``
2. Start to create JuPyter Notebook tutorials of the API.
3. Included ``BaseNetDatabase`` binarization and normalization of databases.
4. Included in ``BaseNetDatabase`` to read ``TensorFlow`` and ``Pandas`` databases.
5. Some rework was done for bug-fixing and providing more logging information.
### 1.7.0 - 1.9.1
1. Reworked ``BaseNetDatabase`` for minor bug fixing. (1.7.0)
2. Added the ``BaseNetFeeder`` Class to generate dynamic ``BaseNetDatabase``s. (1.7.0)
3. Jupyter tutorials. (1.8.0)
4. Added the ``BaseNetDeployment`` Class to deploy scalable Machine Learning models (in alpha, do not use until 3.0).
(1.9.0)
5. Added ``BaseNetHeuristic`` algorithms that implements MetaHeuristic methods such as PSO. (1.8.0)
### 1.9.2
1. Added ``BaseNetLMSE``, solving linear problems with matrix multiplication.
### 1.9.3
1. Added ``CassandraDatabase``.
## Basic and fast usage
There are Jupyter Notebooks with usage tutorials! Refer to them
[HERE](https://github.com/iTzAlver/BaseNet-API/tree/main/jupyter). You should run your notebook creating a
virtual environment in this README path.
### Cite as
Please, cite this library as:
@misc{basenetapi,
title={BaseNet: A simpler way to build AI models.},
author={A. Palomo-Alonso},
booktitle={PhD in TIC: Machine Learning and NLP.},
year={2022}
}
Raw data
{
"_id": null,
"home_page": "https://github.com/iTzAlver/basenet_api.git",
"name": "basenet-api",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "deeplearning,ml,api",
"author": "Palomo-Alonso, Alberto",
"author_email": "a.palomo@edu.uah",
"download_url": "https://files.pythonhosted.org/packages/14/6f/0793bf1b9b373d670d83b08f61ae24a90e73f9b5fcb3be4ac7575655ba1e/basenet_api-1.9.3.tar.gz",
"platform": null,
"description": "# BaseNet: A simpler way to build AI models.\r\n\r\n<p align=\"center\">\r\n <img src=\"https://raw.githubusercontent.com/iTzAlver/basenet_api/main/doc/multimedia/basenet_logo.png\" width=\"400px\">\r\n</p>\r\n\r\n<p align=\"center\">\r\n <a href=\"https://github.com/iTzAlver/basenet_api/blob/main/LICENSE\">\r\n <img src=\"https://img.shields.io/github/license/iTzAlver/basenet_api?color=purple&style=plastic\" /></a>\r\n <a href=\"https://github.com/iTzAlver/basenet_api/tree/main/test\">\r\n <img src=\"https://img.shields.io/badge/tests-passed-green?color=green&style=plastic\" /></a>\r\n <a href=\"https://github.com/iTzAlver/basenet_api/blob/main/requirements.txt\">\r\n <img src=\"https://img.shields.io/badge/requirements-pypi-red?color=red&style=plastic\" /></a>\r\n <a href=\"https://htmlpreview.github.io/?https://github.com/iTzAlver/basenet_api/blob/main/doc/basenet.html\">\r\n <img src=\"https://img.shields.io/badge/doc-available-green?color=yellow&style=plastic\" /></a>\r\n <a href=\"https://github.com/iTzAlver/BaseNet-API/releases/tag/1.5.0-release\">\r\n <img src=\"https://img.shields.io/badge/release-1.5.0-white?color=white&style=plastic\" /></a>\r\n</p>\r\n\r\n<p align=\"center\">\r\n <a href=\"https://www.tensorflow.org/\">\r\n <img src=\"https://img.shields.io/badge/dependencies-tensorflow-red?color=orange&style=for-the-badge\" /></a>\r\n <a href=\"https://keras.io/\">\r\n <img src=\"https://img.shields.io/badge/dependencies-keras-red?color=red&style=for-the-badge\" /></a>\r\n <a href=\"https://www.ray.io/\">\r\n <img src=\"https://img.shields.io/badge/dependencies-ray-red?color=blue&style=for-the-badge\" /></a>\r\n</p>\r\n\r\n# Basenet API Package - 1.9.3: (2.0.0 Is coming!!)\r\n\r\nThis package implements an API over Keras and Tensorflow to build Deep Learning models easily without losing the\r\nframework flexibility. BaseNet API tries to implement almost everything from a few lines of code.\r\n\r\n> **Disclaimer**: This API is under development. This means that there isn't a stable release yet.\r\n> However, some features are stable and can be used. Check the tutorials for more info. \r\n> Please, check the roadmap below to see the future updates.\r\n\r\n> Any feature request is wellcome and appreciated; and has a high probability to be implemented.\r\n\r\n> Any issue is wellcome and appreciated; and it will often be inspected.\r\n\r\n## About ##\r\n\r\n Author: A.Palomo-Alonso (a.palomo@uah.es)\r\n Universidad de Alcal\u00e1.\r\n Escuela Polit\u00e9cnica Superior.\r\n Departamento de Teor\u00eda De la Se\u00f1al y Comunicaciones (TDSC).\r\n ISDEFE Chair of Research.\r\n\r\n## Features\r\n\r\n#### **MACHINE LEARNING**\r\n* **Feature 01:** Database train, validation and test automatic and random segmentation for DeepLearning.\r\n* **Feature 02:** Solving optimization problems for MetaHeuristic models.\r\n\r\n#### **EFFICIENCY**\r\n* **Feature 03:** Real multiprocessing training process (CPU usage optimization).\r\n* **Feature 04:** Automatic and custom GPU usage and assignment.\r\n\r\n#### **SIMPLICITY**\r\n* **Feature 05:** Easy-to-use API.\r\n* **Feature 06:** API documentation.\r\n* **Feature 07:** JuPyter Notebooks tutorials included.\r\n* **Feature 08:** Python Packaging and PyPi indexing.\r\n\r\n#### **MONITORIZATION**\r\n* **Feature 09:** Real-time logging.\r\n* **Feature 10:** Dashboards included.\r\n* **Feature 11:** Model printing and easy debugging.\r\n\r\n#### **CONNECTIVITY**\r\n* **Feature 12:** Model merging and multiple model inputs.\r\n* **Feature 13:** Computational cluster linking.\r\n* **Feature 14:** The different parts of the API are designed to interact.\r\n* **Feature 15:** It allows to create dynamic databases for data ingestion and math problems that require synthetic data.\r\n\r\n#### **RELIABILITY**\r\n* **Feature 16:** Depends on huge and reliable frameworks: KERAS, TENSORFLOW, RAY.\r\n* **Feature 17:** Code updating and active support.\r\n\r\n\r\n### Cons:\r\n#### **FLEXIBILITY**\r\n* An API must look for a balance between simplicity and flexibility. In this case, we bet on simplicity; but the API\r\nis still highly flexible.\r\n* The API is designed for high level research and design. But it is not optimal for low-level research.\r\n\r\n#### **DEPENDENCE**\r\n* This API is built over highly reliable frameworks, however, the API depends on those frameworks to run the models.\r\n\r\n#### **DEPLOYMENT**\r\n* The API can not deploy models by itself yet in this current version. But I am planning!\r\n\r\n\r\n### Roadmap:\r\n#### Discipline coverage:\r\n- [x] Database building.\r\n- [x] Supervised learning.\r\n- [x] MetaHeuristic optimization.\r\n- [ ] Unsupervised learning. (Planning in 3.0 release)\r\n- [ ] Reinforcement learning. (Planning in 4.0 release)\r\n- [ ] Computer Vision. (Planning in 5.0 release)\r\n- [ ] Natural Language Processing. (Will be in development from 3.0 to 6.0)\r\n\r\n#### Feature roadmap:\r\n\r\n- [x] Feeder databases.\r\n- [x] Computational clustering.\r\n- [x] Monitoring.\r\n\r\n\r\n- [ ] BaseNetDeployment: \r\n - Deploy your model in your infrastructure with high connectivity and scalability. \r\n - With preprocess, model, postprocess and front-end sector.\r\n - Automatic workload balance (probably with ``Ray`` or ``Spark`` clusters).\r\n - Dynamic training after deployment. The model will still be learning from input data if desired.\r\n\r\n\r\n- [ ] Reinforcement learning:\r\n - This will be hard to add in deployment.\r\n - Custom environments.\r\n - Connectivity with BaseNetCompiler: It will work with RL policy and DL!\r\n - State-of-the-art RL.\r\n\r\n\r\n- [ ] Computer Vision:\r\n - It is already possible to be implemented in DeepLearning API; but can be improved.\r\n - Database visualization.\r\n - Database size optimization.\r\n - Computer vision utils.\r\n\r\n\r\n- [ ] Natural Language Processing:\r\n - Hard to tell here, this discipline is very ad-hoc, and it is hard to pack it in an API.\r\n - Transformers, LSTM and Word Vectorization will be included.\r\n - Preprocessing will be a must.\r\n - Probably will use ``HuggingFace`` package.\r\n - Will be in constant development.\r\n\r\n\r\n## What's new?\r\n\r\n### < 0.1.0\r\n1. BaseNetModel included.\r\n2. BaseNetDatabase included.\r\n3. BaseNetCompiler included.\r\n4. Inheritance from CorNetAPI project.\r\n5. Multi-processing fitting.\r\n6. Tensorboard launching.\r\n\r\n### 0.2.0\r\n1. BaseNetResults included (working).\r\n2. Now the model is callable.\r\n3. Switched print to logging.\r\n4. Project documentation.\r\n\r\n\r\n### 1.0.0 - 1.0.3\r\n1. Python packaging\r\n3. 1.0.x: Upload bug solving.\r\n\r\n### 1.1.0\r\n1. Functional package.\r\n2. PyPi indexing.\r\n\r\n### 1.2.0:\r\n1. Loss results included in the BaseNetResults while multiprocessing.\r\n2. GPU auto set up to avoid TensorFlow memory errors.\r\n3. Method ``BaseNetCompiler.set_up_devices()`` configures the GPUs according to the free RAM to be used in the API.\r\n\r\n### 1.3.0\r\n1. Included WindowDiff to the project scope.\r\n\r\n### 1.4.0\r\n1. Solved python packaging problems.\r\n2. Included force stop callback in the ```BaseNetModel.fit_stop()``` method.\r\n\r\n### 1.5.0\r\n1. BaseNetDatabase now has the attributes ``BaseNetDatabase.size`` and ``BaseNetDatabase.distribution``.\r\n2. Solved forced stopping bugs with multiprocessing in the method ``BaseNetDatabase.fit_stop()``.\r\n3. ```BaseNetModel._threshold()``` private method now takes a set of outputs instead only one. \r\nThis was only for optimization.\r\n4. Solved wrong ```BaseNetModel.recover()```.\r\n5. **Auto recover implemented**, now ```BaseNetModel.recover()``` is a private method: ```BaseNetModel._recover()```.\r\nNow the used does not need to recover it. *The model recovers by itself. -- Hans Niemann 2022.* \r\n**NOTE**: RECOVER IS NECESARY WHEN THE MODEL IS EARLY STOPPED; CONSIDER RECOVERING ALWAYS THE MODEL.\r\n\r\n### 1.5.1 - 1.5.3\r\n1. Solved a bug where ``BaseNetDatabase`` modified the incoming list of instances in the database; avoiding checkpoints\r\nfor large database generators.\r\n2. Exception handler for ``nvml`` library if NVIDIA Drivers are not installed`in the machine.\r\n\r\n### 1.5.4\r\n1. Added some ``BaseNetDatabase`` utils: merge and split databases.\r\n2. Added ``BaseNetDatabase`` equality check.\r\n3. Added a ``BaseNetDatabase._reversion()``, ``BaseNetCompiler._reversion()`` and ``BaseNetModel.__version__``. Which\r\nrebuilds the Classes to the current version of the API.\r\n\r\n### 1.6.0\r\n1. Start to develop the second branch of the API: ``BaseNetHeuristic``\r\n2. Start to create JuPyter Notebook tutorials of the API.\r\n3. Included ``BaseNetDatabase`` binarization and normalization of databases.\r\n4. Included in ``BaseNetDatabase`` to read ``TensorFlow`` and ``Pandas`` databases.\r\n5. Some rework was done for bug-fixing and providing more logging information.\r\n\r\n### 1.7.0 - 1.9.1\r\n1. Reworked ``BaseNetDatabase`` for minor bug fixing. (1.7.0)\r\n2. Added the ``BaseNetFeeder`` Class to generate dynamic ``BaseNetDatabase``s. (1.7.0)\r\n3. Jupyter tutorials. (1.8.0)\r\n4. Added the ``BaseNetDeployment`` Class to deploy scalable Machine Learning models (in alpha, do not use until 3.0).\r\n(1.9.0)\r\n5. Added ``BaseNetHeuristic`` algorithms that implements MetaHeuristic methods such as PSO. (1.8.0)\r\n\r\n### 1.9.2\r\n1. Added ``BaseNetLMSE``, solving linear problems with matrix multiplication.\r\n\r\n### 1.9.3\r\n1. Added ``CassandraDatabase``.\r\n\r\n\r\n## Basic and fast usage\r\n\r\nThere are Jupyter Notebooks with usage tutorials! Refer to them \r\n[HERE](https://github.com/iTzAlver/BaseNet-API/tree/main/jupyter). You should run your notebook creating a \r\nvirtual environment in this README path.\r\n\r\n### Cite as\r\n\r\nPlease, cite this library as:\r\n\r\n\r\n @misc{basenetapi,\r\n title={BaseNet: A simpler way to build AI models.},\r\n author={A. Palomo-Alonso},\r\n booktitle={PhD in TIC: Machine Learning and NLP.},\r\n year={2022}\r\n }\r\n",
"bugtrack_url": null,
"license": "",
"summary": "Basenet API: A simpler way to build ML models.",
"version": "1.9.3",
"split_keywords": [
"deeplearning",
"ml",
"api"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f81c6712978f67bc7a41bb458d4cd4cd474fe0e7eeb625c8967ff04fa19472c8",
"md5": "e5087021bd704fa9ddbb65b5bcac3932",
"sha256": "6b5e77ce425f73ec6ab9bea0a054d792903f7c39dc849bb92fd57ca7c320207c"
},
"downloads": -1,
"filename": "basenet_api-1.9.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e5087021bd704fa9ddbb65b5bcac3932",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 822016,
"upload_time": "2023-01-24T10:47:07",
"upload_time_iso_8601": "2023-01-24T10:47:07.144971Z",
"url": "https://files.pythonhosted.org/packages/f8/1c/6712978f67bc7a41bb458d4cd4cd474fe0e7eeb625c8967ff04fa19472c8/basenet_api-1.9.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "146f0793bf1b9b373d670d83b08f61ae24a90e73f9b5fcb3be4ac7575655ba1e",
"md5": "a6b820916e5a27be4aae01027f3e564f",
"sha256": "170c0b0a226198ad65fb2c181c4d552abe92d5dd405f67ade44a0049a3c6fbc5"
},
"downloads": -1,
"filename": "basenet_api-1.9.3.tar.gz",
"has_sig": false,
"md5_digest": "a6b820916e5a27be4aae01027f3e564f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 799011,
"upload_time": "2023-01-24T10:47:10",
"upload_time_iso_8601": "2023-01-24T10:47:10.327608Z",
"url": "https://files.pythonhosted.org/packages/14/6f/0793bf1b9b373d670d83b08f61ae24a90e73f9b5fcb3be4ac7575655ba1e/basenet_api-1.9.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-24 10:47:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "iTzAlver",
"github_project": "basenet_api.git",
"lcname": "basenet-api"
}
JSON
