# ToyNet
[](https://www.python.org/downloads/)
[](LICENSE)
[](http://mypy-lang.org/)
[](https://github.com/astral-sh/ruff)
An educational but robust deep learning framework implemented from scratch using NumPy.
Data loading is handled with Pandas.
ToyNet provides a clean, extensible architecture for building and training neural networks without external ML dependencies. Perfect for understanding deep learning mathematical fundamentals or experimenting with custom architectures on small to medium datasets.
## Table of Contents
- [Features](#features)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Architecture Overview](#architecture-overview)
- [Comprehensive Example](#comprehensive-example)
- [Mathematical Foundations](#mathematical-foundations)
- [Development](#development)
- [License](#license)
## Features
### **Core Capabilities**
- **Pure NumPy implementation** - No external ML libraries, transparent mathematical operations
- **Educational focus** - Clear, readable code designed for learning and understanding
- **Modular architecture** - Easily extensible with custom components
- **Type safety** - Full mypy compatibility with comprehensive type hints
- **Comprehensive tests** - Unit, integration, and end-to-end tests with high coverage
### **Data Handling**
- **Data loaders** - Support for CSV files and in-memory arrays
- **Preprocessing** - Programmable feature scaling, train/validation splits in data loaders
- **Batch processing** - Efficient mini-batch training with configurable sizes
### **Neural Network Components**
- **Layers**: Customizable with activation functions, weight initialization methods
- **Activation and loss functions**: Classic activations and losses for various tasks with the protocols to create custom ones
- **Optimizers**: Classic GD variants or Adam with adaptive learning rates
### **Training Features**
- **Training policies** - Early stopping, learning rate scheduling, model checkpointing, protocol for custom policies
- **Model persistence** - Save/load trained models in NumPy format
- **Logging** - Comprehensive training progress tracking
## Installation
### From PyPI (Recommended)
```bash
pip install toynet-ml
```
## Quick Start
### Simple XOR Problem
```python
import numpy as np
from toynet import MultiLayerPerceptron, Dense, BasicDataLoader
from toynet.functions import ReLU, Sigmoid, BinaryCrossEntropy
from toynet.optimizers import Adam
# Create XOR dataset
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]).reshape(4, 1, 2)
y = np.array([[0], [1], [1], [0]]).reshape(4, 1, 1)
# Build network
network = MultiLayerPerceptron(
layers=[
Dense(2, 8, ReLU),
Dense(8, 1, Sigmoid)
],
loss_function=BinaryCrossEntropy,
optimizer=Adam(learning_rate=0.01)
)
# Train
data_loader = BasicDataLoader(X, y, batch_size=2)
network.train(data_loader, epochs=1000)
# Make batch predictions
predictions = network(X)
predictions_rounded = np.round(predictions, 2)
print(f"Input: \n{X.reshape(4, 2)}")
print(f"Predictions: {predictions_rounded.flatten()}")
```
### Simple Regression Problem: Predict y = 2x + 1
```python
import numpy as np
from toynet import MultiLayerPerceptron, Dense, BasicDataLoader
from toynet.functions import ReLU, Identity, MeanSquaredError
from toynet.optimizers import Adam
# Create regression dataset
X = np.array([[-4], [-3], [-2], [-1], [0], [1], [2], [3], [4], [5], [6]]).reshape(11, 1, 1)
y = np.array([[-7], [-5], [-3], [-1], [1], [3], [5], [7], [9], [11], [13]]).reshape(11, 1, 1)
# Build network
network = MultiLayerPerceptron(
layers=[
Dense(1, 8, ReLU),
Dense(8, 8, ReLU),
Dense(8, 1, Identity)
],
loss_function=MeanSquaredError,
optimizer=Adam(learning_rate=0.01)
)
# Train
data_loader = BasicDataLoader(X, y, batch_size=4)
network.train(data_loader, epochs=1000)
# Make unseen data prediction
prediction = network(np.array([9]).reshape(1, 1)) # 2x + 1 = 19
print(f"Predictions: {prediction.flatten()}")
```
## Architecture Overview
ToyNet follows a modular, object-oriented design:
```
MultiLayerPerceptron
├── Layers (Dense)
│ ├── Input/Output dimensions
│ ├── Activation Functions (ReLU, Sigmoid, etc.)
│ └── Weight Initializers (He (default), Xavier)
├── Loss Function (CrossEntropy, MSE)
└── Optimizer (GD variants, Adam)
Training
├── Data Loader (Basic, CSV)
├── Fixed epochs
└── Training Policies (EarlyStopping, LR Scheduling etc.)
```
## Comprehensive Example
### End to end training on Kaggle Digit Recognizer CSV dataset
```python
import numpy as np
from toynet import Adam, CSVDataloader, Dense, MultiLayerPerceptron
from toynet.functions import (
CategoricalCrossEntropy,
ReLU,
Softmax,
)
from toynet.policies import ReduceLROnPlateau, SaveBestModel, ValidationLossEarlyStop
if __name__ == "__main__":
data_loader = CSVDataloader(
"train.csv",
batch_size=128,
label_cols=["label"],
validation_split=0.2,
transform=lambda X, y: (X / 255.0, np.eye(10)[y.astype(int)]),
)
nnet = MultiLayerPerceptron(
[
Dense(784, 256, ReLU),
Dense(256, 128, ReLU),
Dense(128, 64, ReLU),
Dense(64, 10, Softmax),
],
loss_function=CategoricalCrossEntropy,
optimizer=Adam(
learning_rate=0.01,
),
)
nnet.train(
data_loader,
epochs=250,
policies=[
ValidationLossEarlyStop("mnist.npz", patience=6),
ReduceLROnPlateau(factor=0.1, patience=4, min_lr=1e-6),
SaveBestModel("mnist_best_checkpoint.npz", save_grace_period=20),
],
)
```
### Benchmarks:
- Training time: ~40 minutes on modern CPU
- Best Kaggle test accuracy achieved: 96.6%
For production workloads, use PyTorch or TensorFlow which offer GPU acceleration and distributed training.
## Mathematical Foundations
ToyNet implements core neural network mathematics from scratch:
### Forward Propagation
```
h = σ(Wx + b)
```
Where:
- `W`: Weight matrix
- `x`: Input vector
- `b`: Bias vector
- `σ`: Activation function
### Backpropagation
Automatic gradient computation using the chain rule:
```
∂L/∂W = ∂L/∂h × ∂h/∂z × ∂z/∂W
```
Gradients are computed and stored in layer objects during backpropagation.
### Weight Updates
- **GD variants**: `W ← W - η∇W`
- **Adam**: Adaptive learning with momentum and RMSprop
## Development
### Running tests and code checks
After cloning the repository and setting up a virtual environment:
```bash
# Install development dependencies
pip install -e ".[dev]"
# Run all tests with coverage
pytest --cov=toynet --cov-report=html
# Type checking
mypy --install-types --config-file pyproject.toml ./src
# Code formatting
ruff format
ruff check --fix
```
### Project Structure
```
toynet/
├── src/toynet/ # Main package
│ ├── data_loaders/ # Data loading utilities
│ ├── functions/ # Activation and loss functions
│ ├── initializers/ # Weight initialization
│ ├── layers/ # Layer implementations
│ ├── networks/ # Neural network architectures
│ ├── optimizers/ # Gradient descent algorithms
│ ├── policies/ # Training policies
│ └── config.py # Configuration settings
│
├── tests/ # Test suite
│ ├── uts/ # Unit tests
│ ├── integration/ # Integration tests
└───└── e2e/ # End-to-end tests
```
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "toynet-ml",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.12",
"maintainer_email": null,
"keywords": "deep learning, machine learning, neural networks, numpy, artificial intelligence, framework, education",
"author": "Maciej Kmie\u0107",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/a7/1e/17ceafe1ee5844ccbc7da3fa07aacb295877b6861c731fee5df7df5bd61b/toynet_ml-1.0.0.tar.gz",
"platform": null,
"description": "# ToyNet\n[](https://www.python.org/downloads/)\n[](LICENSE)\n[](http://mypy-lang.org/)\n[](https://github.com/astral-sh/ruff)\n\nAn educational but robust deep learning framework implemented from scratch using NumPy. \nData loading is handled with Pandas.\n\nToyNet provides a clean, extensible architecture for building and training neural networks without external ML dependencies. Perfect for understanding deep learning mathematical fundamentals or experimenting with custom architectures on small to medium datasets.\n\n## Table of Contents\n- [Features](#features)\n- [Installation](#installation)\n- [Quick Start](#quick-start)\n- [Architecture Overview](#architecture-overview)\n- [Comprehensive Example](#comprehensive-example)\n- [Mathematical Foundations](#mathematical-foundations)\n- [Development](#development)\n- [License](#license)\n\n## Features\n### **Core Capabilities**\n- **Pure NumPy implementation** - No external ML libraries, transparent mathematical operations\n- **Educational focus** - Clear, readable code designed for learning and understanding\n- **Modular architecture** - Easily extensible with custom components\n- **Type safety** - Full mypy compatibility with comprehensive type hints\n- **Comprehensive tests** - Unit, integration, and end-to-end tests with high coverage\n\n### **Data Handling**\n- **Data loaders** - Support for CSV files and in-memory arrays\n- **Preprocessing** - Programmable feature scaling, train/validation splits in data loaders\n- **Batch processing** - Efficient mini-batch training with configurable sizes\n\n### **Neural Network Components**\n- **Layers**: Customizable with activation functions, weight initialization methods\n- **Activation and loss functions**: Classic activations and losses for various tasks with the protocols to create custom ones\n- **Optimizers**: Classic GD variants or Adam with adaptive learning rates\n\n### **Training Features**\n- **Training policies** - Early stopping, learning rate scheduling, model checkpointing, protocol for custom policies\n- **Model persistence** - Save/load trained models in NumPy format\n- **Logging** - Comprehensive training progress tracking\n\n## Installation\n### From PyPI (Recommended)\n```bash\npip install toynet-ml\n```\n\n## Quick Start\n### Simple XOR Problem\n```python\nimport numpy as np\nfrom toynet import MultiLayerPerceptron, Dense, BasicDataLoader\nfrom toynet.functions import ReLU, Sigmoid, BinaryCrossEntropy\nfrom toynet.optimizers import Adam\n\n# Create XOR dataset\nX = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]).reshape(4, 1, 2)\ny = np.array([[0], [1], [1], [0]]).reshape(4, 1, 1)\n\n# Build network\nnetwork = MultiLayerPerceptron(\n layers=[\n Dense(2, 8, ReLU),\n Dense(8, 1, Sigmoid)\n ],\n loss_function=BinaryCrossEntropy,\n optimizer=Adam(learning_rate=0.01)\n)\n\n# Train\ndata_loader = BasicDataLoader(X, y, batch_size=2)\nnetwork.train(data_loader, epochs=1000)\n\n# Make batch predictions\npredictions = network(X)\npredictions_rounded = np.round(predictions, 2)\nprint(f\"Input: \\n{X.reshape(4, 2)}\")\nprint(f\"Predictions: {predictions_rounded.flatten()}\")\n```\n\n### Simple Regression Problem: Predict y = 2x + 1\n```python\nimport numpy as np\nfrom toynet import MultiLayerPerceptron, Dense, BasicDataLoader\nfrom toynet.functions import ReLU, Identity, MeanSquaredError\nfrom toynet.optimizers import Adam\n\n# Create regression dataset\nX = np.array([[-4], [-3], [-2], [-1], [0], [1], [2], [3], [4], [5], [6]]).reshape(11, 1, 1)\ny = np.array([[-7], [-5], [-3], [-1], [1], [3], [5], [7], [9], [11], [13]]).reshape(11, 1, 1)\n\n# Build network\nnetwork = MultiLayerPerceptron(\n layers=[\n Dense(1, 8, ReLU),\n Dense(8, 8, ReLU),\n Dense(8, 1, Identity)\n ],\n loss_function=MeanSquaredError,\n optimizer=Adam(learning_rate=0.01)\n)\n\n# Train\ndata_loader = BasicDataLoader(X, y, batch_size=4)\nnetwork.train(data_loader, epochs=1000)\n\n# Make unseen data prediction\nprediction = network(np.array([9]).reshape(1, 1)) # 2x + 1 = 19\nprint(f\"Predictions: {prediction.flatten()}\") \n```\n\n## Architecture Overview\nToyNet follows a modular, object-oriented design:\n\n```\nMultiLayerPerceptron\n\u251c\u2500\u2500 Layers (Dense)\n\u2502 \u251c\u2500\u2500 Input/Output dimensions\n\u2502 \u251c\u2500\u2500 Activation Functions (ReLU, Sigmoid, etc.)\n\u2502 \u2514\u2500\u2500 Weight Initializers (He (default), Xavier)\n\u251c\u2500\u2500 Loss Function (CrossEntropy, MSE)\n\u2514\u2500\u2500 Optimizer (GD variants, Adam)\n\nTraining\n\u251c\u2500\u2500 Data Loader (Basic, CSV)\n\u251c\u2500\u2500 Fixed epochs\n\u2514\u2500\u2500 Training Policies (EarlyStopping, LR Scheduling etc.)\n```\n\n## Comprehensive Example\n### End to end training on Kaggle Digit Recognizer CSV dataset\n```python\nimport numpy as np\n\nfrom toynet import Adam, CSVDataloader, Dense, MultiLayerPerceptron\nfrom toynet.functions import (\n CategoricalCrossEntropy,\n ReLU,\n Softmax,\n)\nfrom toynet.policies import ReduceLROnPlateau, SaveBestModel, ValidationLossEarlyStop\n\nif __name__ == \"__main__\":\n data_loader = CSVDataloader(\n \"train.csv\",\n batch_size=128,\n label_cols=[\"label\"],\n validation_split=0.2,\n transform=lambda X, y: (X / 255.0, np.eye(10)[y.astype(int)]),\n )\n\n nnet = MultiLayerPerceptron(\n [\n Dense(784, 256, ReLU),\n Dense(256, 128, ReLU),\n Dense(128, 64, ReLU),\n Dense(64, 10, Softmax),\n ],\n loss_function=CategoricalCrossEntropy,\n optimizer=Adam(\n learning_rate=0.01,\n ),\n )\n\n nnet.train(\n data_loader,\n epochs=250,\n policies=[\n ValidationLossEarlyStop(\"mnist.npz\", patience=6),\n ReduceLROnPlateau(factor=0.1, patience=4, min_lr=1e-6),\n SaveBestModel(\"mnist_best_checkpoint.npz\", save_grace_period=20),\n ],\n )\n```\n### Benchmarks:\n- Training time: ~40 minutes on modern CPU\n- Best Kaggle test accuracy achieved: 96.6%\n\nFor production workloads, use PyTorch or TensorFlow which offer GPU acceleration and distributed training.\n\n\n## Mathematical Foundations\nToyNet implements core neural network mathematics from scratch:\n\n### Forward Propagation\n```\nh = \u03c3(Wx + b)\n```\nWhere:\n- `W`: Weight matrix\n- `x`: Input vector \n- `b`: Bias vector\n- `\u03c3`: Activation function\n\n### Backpropagation\nAutomatic gradient computation using the chain rule:\n```\n\u2202L/\u2202W = \u2202L/\u2202h \u00d7 \u2202h/\u2202z \u00d7 \u2202z/\u2202W\n```\nGradients are computed and stored in layer objects during backpropagation.\n\n### Weight Updates\n- **GD variants**: `W \u2190 W - \u03b7\u2207W`\n- **Adam**: Adaptive learning with momentum and RMSprop\n\n## Development\n### Running tests and code checks\nAfter cloning the repository and setting up a virtual environment:\n```bash\n# Install development dependencies\npip install -e \".[dev]\"\n\n# Run all tests with coverage\npytest --cov=toynet --cov-report=html\n\n# Type checking\nmypy --install-types --config-file pyproject.toml ./src\n\n# Code formatting\nruff format\nruff check --fix\n```\n\n### Project Structure\n```\ntoynet/\n\u251c\u2500\u2500 src/toynet/ # Main package\n\u2502 \u251c\u2500\u2500 data_loaders/ # Data loading utilities\n\u2502 \u251c\u2500\u2500 functions/ # Activation and loss functions\n\u2502 \u251c\u2500\u2500 initializers/ # Weight initialization\n\u2502 \u251c\u2500\u2500 layers/ # Layer implementations\n\u2502 \u251c\u2500\u2500 networks/ # Neural network architectures\n\u2502 \u251c\u2500\u2500 optimizers/ # Gradient descent algorithms\n\u2502 \u251c\u2500\u2500 policies/ # Training policies\n\u2502 \u2514\u2500\u2500 config.py # Configuration settings\n\u2502\n\u251c\u2500\u2500 tests/ # Test suite\n\u2502 \u251c\u2500\u2500 uts/ # Unit tests\n\u2502 \u251c\u2500\u2500 integration/ # Integration tests\n\u2514\u2500\u2500\u2500\u2514\u2500\u2500 e2e/ # End-to-end tests\n```\n\n## License\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n",
"bugtrack_url": null,
"license": null,
"summary": "An educational but robust deep learning framework implemented from scratch using NumPy.",
"version": "1.0.0",
"project_urls": null,
"split_keywords": [
"deep learning",
" machine learning",
" neural networks",
" numpy",
" artificial intelligence",
" framework",
" education"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "4fa260a810711386407d84dcbd7111e720d4efc082c214b845f26788b3119c06",
"md5": "7771a91a38cbb93f28408c3220989a51",
"sha256": "03e20e4e5bb5c881e2b6c35073265f858ae67f32ab1f24e236629da18caf4162"
},
"downloads": -1,
"filename": "toynet_ml-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7771a91a38cbb93f28408c3220989a51",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.12",
"size": 20206,
"upload_time": "2025-08-24T13:56:20",
"upload_time_iso_8601": "2025-08-24T13:56:20.152505Z",
"url": "https://files.pythonhosted.org/packages/4f/a2/60a810711386407d84dcbd7111e720d4efc082c214b845f26788b3119c06/toynet_ml-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "a71e17ceafe1ee5844ccbc7da3fa07aacb295877b6861c731fee5df7df5bd61b",
"md5": "1e7a41b517ecdd27cdc7e1c3c9242ab1",
"sha256": "6bc58ddbdf46b60fd3903495c03a293a9f68c86117281a93ce8388abd5854b85"
},
"downloads": -1,
"filename": "toynet_ml-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "1e7a41b517ecdd27cdc7e1c3c9242ab1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12",
"size": 16053,
"upload_time": "2025-08-24T13:56:22",
"upload_time_iso_8601": "2025-08-24T13:56:22.869116Z",
"url": "https://files.pythonhosted.org/packages/a7/1e/17ceafe1ee5844ccbc7da3fa07aacb295877b6861c731fee5df7df5bd61b/toynet_ml-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-24 13:56:22",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "toynet-ml"
}
JSON