# SelfProjection Module for PyTorch
- [SelfProjection Module for PyTorch](#selfprojection-module-for-pytorch)
- [Overview](#overview)
- [Approach](#approach)
- [Installation](#installation)
- [Usage](#usage)
- [Evaluation](#evaluation)
- [Future Directions](#future-directions)
- [Contribution](#contribution)
## Overview
The `SelfProjection` module is a PyTorch-based neural network layer designed to transform and project high-dimensional data. It is particularly useful in contexts requiring sophisticated analysis and representation of feature relationships, such as outputs from Transformer models.
## Approach
The `SelfProjection` module employs a dual projection mechanism to process input tensors, capturing different perspectives of the data. Key aspects include:
- **Original and Permuted Projections**: The module processes the input tensor in its original form and a permuted form, creating two distinct representations.
- **Relational Interference**: By computing relational matrices, the module captures the interplay between different projections, emphasizing the relationships between data dimensions.
- **Normalization**: Custom normalization steps, which involve mean subtraction and variance scaling similar to Layer Normalization, are applied to the projections, ensuring stable feature scaling and potentially improved model performance.
- **Trainable Parameters**: The module includes several trainable parameters, allowing it to learn optimal feature transformations during training.
## Installation
__Using pip:__
To install the `SelfProjection` module using pip, simply run the following command:
```bash
pip install self-projection
```
__From source:__
To install the `SelfProjection` module, clone this repository and import the module into your PyTorch project.
```bash
git clone https://github.com/Sombressoul/self-projection ./self_projection
python -m pip install -e ./self_projection
```
## Usage
Here's a simple example of how to use the `SelfProjection` with PyTorch:
```python
import torch
from self_projection import SelfProjection
# Define the input tensor dimensions and projection size
input_tensor = torch.randn((batch_size, sequence_length, embedding_dim))
size_projection = 128
# Initialize the SelfProjection module
self_projection = SelfProjection(size_input=input_tensor.size()[1::], size_projection=size_projection)
# Apply the module to the input tensor
projected, relations = self_projection(input_tensor)
print(projected.shape)
# >>> torch.Size([<batch_size>, 128, 128])
print(relations.shape)
# >>> torch.Size([<batch_size>, 128, 128])
```
## Evaluation
(deleted as outdated since 20.01.2024)
Pending.
## Future Directions
The ongoing development of the `SelfProjection` module is a component of a personal endeavor in the field of machine learning. Future plans include conducting experiments with more complex datasets to further assess and refine the module's capabilities. These steps are aimed at exploring a broader range of applications and enhancing the module's performance in diverse settings.
This module is a reflection of my interest in contributing to the machine learning community through individual efforts and is one aspect of a larger personal project dedicated to exploring innovative approaches in the field.
## Contribution
Contributions to the `SelfProjection` module are welcome. Please submit a pull request or open an issue if you have suggestions or improvements.
Raw data
{
"_id": null,
"home_page": "https://github.com/Sombressoul/self-projection",
"name": "self-projection",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10.0",
"maintainer_email": "",
"keywords": "machine learning,deep learning,pytorch layers,pytorch modules",
"author": "Konstantin Bashinskiy",
"author_email": "sombressoul@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ac/26/35b38a3e963d022a7bc4018f5ac6c2066f8aaf780f2ad8eab919038cb176/self-projection-0.2.2.tar.gz",
"platform": null,
"description": "# SelfProjection Module for PyTorch\n- [SelfProjection Module for PyTorch](#selfprojection-module-for-pytorch)\n - [Overview](#overview)\n - [Approach](#approach)\n - [Installation](#installation)\n - [Usage](#usage)\n - [Evaluation](#evaluation)\n - [Future Directions](#future-directions)\n - [Contribution](#contribution)\n\n## Overview\nThe `SelfProjection` module is a PyTorch-based neural network layer designed to transform and project high-dimensional data. It is particularly useful in contexts requiring sophisticated analysis and representation of feature relationships, such as outputs from Transformer models.\n\n## Approach\nThe `SelfProjection` module employs a dual projection mechanism to process input tensors, capturing different perspectives of the data. Key aspects include:\n\n- **Original and Permuted Projections**: The module processes the input tensor in its original form and a permuted form, creating two distinct representations.\n- **Relational Interference**: By computing relational matrices, the module captures the interplay between different projections, emphasizing the relationships between data dimensions.\n- **Normalization**: Custom normalization steps, which involve mean subtraction and variance scaling similar to Layer Normalization, are applied to the projections, ensuring stable feature scaling and potentially improved model performance.\n- **Trainable Parameters**: The module includes several trainable parameters, allowing it to learn optimal feature transformations during training.\n\n## Installation\n\n__Using pip:__\n\nTo install the `SelfProjection` module using pip, simply run the following command:\n\n```bash\npip install self-projection\n```\n\n__From source:__\n\nTo install the `SelfProjection` module, clone this repository and import the module into your PyTorch project.\n\n```bash\ngit clone https://github.com/Sombressoul/self-projection ./self_projection\npython -m pip install -e ./self_projection\n```\n\n## Usage\n\nHere's a simple example of how to use the `SelfProjection` with PyTorch:\n\n```python\nimport torch\nfrom self_projection import SelfProjection\n\n# Define the input tensor dimensions and projection size\ninput_tensor = torch.randn((batch_size, sequence_length, embedding_dim))\nsize_projection = 128\n\n# Initialize the SelfProjection module\nself_projection = SelfProjection(size_input=input_tensor.size()[1::], size_projection=size_projection)\n\n# Apply the module to the input tensor\nprojected, relations = self_projection(input_tensor)\n\nprint(projected.shape)\n# >>> torch.Size([<batch_size>, 128, 128])\nprint(relations.shape)\n# >>> torch.Size([<batch_size>, 128, 128])\n```\n\n## Evaluation\n\n(deleted as outdated since 20.01.2024)\n\nPending.\n\n## Future Directions\n\nThe ongoing development of the `SelfProjection` module is a component of a personal endeavor in the field of machine learning. Future plans include conducting experiments with more complex datasets to further assess and refine the module's capabilities. These steps are aimed at exploring a broader range of applications and enhancing the module's performance in diverse settings.\n\nThis module is a reflection of my interest in contributing to the machine learning community through individual efforts and is one aspect of a larger personal project dedicated to exploring innovative approaches in the field.\n\n## Contribution\n\nContributions to the `SelfProjection` module are welcome. Please submit a pull request or open an issue if you have suggestions or improvements.\n",
"bugtrack_url": null,
"license": "Apache",
"summary": "SelfProjection is a PyTorch-based neural network layer designed to transform and project high-dimensional data.",
"version": "0.2.2",
"project_urls": {
"Homepage": "https://github.com/Sombressoul/self-projection"
},
"split_keywords": [
"machine learning",
"deep learning",
"pytorch layers",
"pytorch modules"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "862159253b2c170f2ee9442957dbf8525f1e9f3716a686f442864fcd66341630",
"md5": "0ced6b94f9991aaa32eab5097ce8fdea",
"sha256": "3e535f89b0af42b09a619318190ab68af9c2670684db6a260ba701e470ef5bd4"
},
"downloads": -1,
"filename": "self_projection-0.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0ced6b94f9991aaa32eab5097ce8fdea",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10.0",
"size": 20556,
"upload_time": "2024-01-29T17:35:39",
"upload_time_iso_8601": "2024-01-29T17:35:39.182934Z",
"url": "https://files.pythonhosted.org/packages/86/21/59253b2c170f2ee9442957dbf8525f1e9f3716a686f442864fcd66341630/self_projection-0.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ac2635b38a3e963d022a7bc4018f5ac6c2066f8aaf780f2ad8eab919038cb176",
"md5": "ff53ddf9df0567aec673ff88063f0c23",
"sha256": "b85ad323523b5552316bf459c2c16967f658acf18c0631302de650f20fbfd165"
},
"downloads": -1,
"filename": "self-projection-0.2.2.tar.gz",
"has_sig": false,
"md5_digest": "ff53ddf9df0567aec673ff88063f0c23",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10.0",
"size": 17397,
"upload_time": "2024-01-29T17:35:40",
"upload_time_iso_8601": "2024-01-29T17:35:40.899225Z",
"url": "https://files.pythonhosted.org/packages/ac/26/35b38a3e963d022a7bc4018f5ac6c2066f8aaf780f2ad8eab919038cb176/self-projection-0.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-29 17:35:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Sombressoul",
"github_project": "self-projection",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "self-projection"
}
JSON
