gpt4-torch


Namegpt4-torch JSON
Version 0.0.3 PyPI version JSON
download
home_pagehttps://github.com/kyegomez/gpt3
SummaryGPT4 - Pytorch
upload_time2023-08-26 03:19:29
maintainer
docs_urlNone
authorKye Gomez
requires_python>=3.6,<4.0
licenseMIT
keywords artificial intelligence attention mechanism transformers
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            [![Multi-Modality](agorabanner.png)](https://discord.gg/qUtxnK2NMf)



# GPT4
The open source implementation of the base model behind GPT-4 from OPENAI [Language + Multi-Modal], click here for the [Research Paper](https://arxiv.org/pdf/2303.08774.pdf)


# Installation
`pip install gpt4-torch`


# Usage

Here's an illustrative code snippet that showcases GPT-3 in action:


```python
import torch
from gpt4 import GPT4

# Generate a random input sequence
x = torch.randint(0, 256, (1, 1024)).cuda()

# Initialize GPT-3 model
model = GPT4()

# Pass the input sequence through the model
output = model(x)
```

## MultiModal Iteration
* Pass in text and and image tensors into GPT4
```python
import torch
from gpt4.gpt4 import GPT4MultiModal

#usage
img = torch.randn(1, 3, 256, 256)
text = torch.randint(0, 20000, (1, 1024))


model = GPT4MultiModal()
output = model(text, img)

```


# 📚 Training

```python
from gpt4 import train

train()

```

For further instructions, refer to the [Training SOP](DOCs/TRAINING.md).


1. Set the environment variables:
   - `ENTITY_NAME`: Your wandb project name
   - `OUTPUT_DIR`: Directory to save the weights (e.g., `./weights`)
   - `MASTER_ADDR`: For distributed training
   - `MASTER_PORT` For master port distributed training
   - `RANK`- Number of nodes services
   - `WORLD_SIZE` Number of gpus

2. Configure the training:
   - Accelerate Config
   - Enable Deepspeed 3
   - Accelerate launch train_distributed_accelerate.py

For more information, refer to the [Training SOP](DOCs/TRAINING.md).

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/kyegomez/gpt3",
    "name": "gpt4-torch",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.6,<4.0",
    "maintainer_email": "",
    "keywords": "artificial intelligence,attention mechanism,transformers",
    "author": "Kye Gomez",
    "author_email": "kye@apac.ai",
    "download_url": "https://files.pythonhosted.org/packages/ac/00/bd320b816761bdd79d7170151b4e04a1224e893550c632761c0b3c743bad/gpt4_torch-0.0.3.tar.gz",
    "platform": null,
    "description": "[![Multi-Modality](agorabanner.png)](https://discord.gg/qUtxnK2NMf)\n\n\n\n# GPT4\nThe open source implementation of the base model behind GPT-4 from OPENAI [Language + Multi-Modal], click here for the [Research Paper](https://arxiv.org/pdf/2303.08774.pdf)\n\n\n# Installation\n`pip install gpt4-torch`\n\n\n# Usage\n\nHere's an illustrative code snippet that showcases GPT-3 in action:\n\n\n```python\nimport torch\nfrom gpt4 import GPT4\n\n# Generate a random input sequence\nx = torch.randint(0, 256, (1, 1024)).cuda()\n\n# Initialize GPT-3 model\nmodel = GPT4()\n\n# Pass the input sequence through the model\noutput = model(x)\n```\n\n## MultiModal Iteration\n* Pass in text and and image tensors into GPT4\n```python\nimport torch\nfrom gpt4.gpt4 import GPT4MultiModal\n\n#usage\nimg = torch.randn(1, 3, 256, 256)\ntext = torch.randint(0, 20000, (1, 1024))\n\n\nmodel = GPT4MultiModal()\noutput = model(text, img)\n\n```\n\n\n# \ud83d\udcda Training\n\n```python\nfrom gpt4 import train\n\ntrain()\n\n```\n\nFor further instructions, refer to the [Training SOP](DOCs/TRAINING.md).\n\n\n1. Set the environment variables:\n   - `ENTITY_NAME`: Your wandb project name\n   - `OUTPUT_DIR`: Directory to save the weights (e.g., `./weights`)\n   - `MASTER_ADDR`: For distributed training\n   - `MASTER_PORT` For master port distributed training\n   - `RANK`- Number of nodes services\n   - `WORLD_SIZE` Number of gpus\n\n2. Configure the training:\n   - Accelerate Config\n   - Enable Deepspeed 3\n   - Accelerate launch train_distributed_accelerate.py\n\nFor more information, refer to the [Training SOP](DOCs/TRAINING.md).\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "GPT4 - Pytorch",
    "version": "0.0.3",
    "project_urls": {
        "Homepage": "https://github.com/kyegomez/gpt3"
    },
    "split_keywords": [
        "artificial intelligence",
        "attention mechanism",
        "transformers"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "6eb3b670acb4b361f07ff2756939d3676db22d2d42ce2883a45ce0cd3dbd2586",
                "md5": "c215fe941a9acdd2486d4f8523fd0315",
                "sha256": "ce7dec810c55b4022634ca276e7f5a82939b845e57287e508b23bdb6fb533d39"
            },
            "downloads": -1,
            "filename": "gpt4_torch-0.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "c215fe941a9acdd2486d4f8523fd0315",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.6,<4.0",
            "size": 28098,
            "upload_time": "2023-08-26T03:19:27",
            "upload_time_iso_8601": "2023-08-26T03:19:27.786997Z",
            "url": "https://files.pythonhosted.org/packages/6e/b3/b670acb4b361f07ff2756939d3676db22d2d42ce2883a45ce0cd3dbd2586/gpt4_torch-0.0.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "ac00bd320b816761bdd79d7170151b4e04a1224e893550c632761c0b3c743bad",
                "md5": "3466f84d98ffe8327c03339e3f2aae69",
                "sha256": "04801544de0ad4c9a2a1b8b91d07b9f363774077921a2df48dfaad9fecc8ea5a"
            },
            "downloads": -1,
            "filename": "gpt4_torch-0.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "3466f84d98ffe8327c03339e3f2aae69",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.6,<4.0",
            "size": 28797,
            "upload_time": "2023-08-26T03:19:29",
            "upload_time_iso_8601": "2023-08-26T03:19:29.093214Z",
            "url": "https://files.pythonhosted.org/packages/ac/00/bd320b816761bdd79d7170151b4e04a1224e893550c632761c0b3c743bad/gpt4_torch-0.0.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-08-26 03:19:29",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "kyegomez",
    "github_project": "gpt3",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "gpt4-torch"
}
        
Elapsed time: 0.27830s