A simple Python package that wraps existing model fine-tuning and generation scripts for OpenAI GPT-2 text generation model (specifically the "small", 124M hyperparameter version). Additionally, this package allows easier generation of text, generating to a file for easy curation, allowing for prefixes to force the text to start with a given phrase.
## Usage
An example for downloading the model to the local system, fineturning it on a dataset. and generating some text.
Warning: the pretrained model, and thus any finetuned model, is 500 MB!
```python
import gpt_2_simple as gpt2
gpt2.download_gpt2() # model is saved into current directory under /models/124M/
sess = gpt2.start_tf_sess()
gpt2.finetune(sess, 'shakespeare.txt', steps=1000) # steps is max number of training steps
gpt2.generate(sess)
```
The generated model checkpoints are by default in `/checkpoint/run1`. If you want to load a model from that folder and generate text from it:
```python
import gpt_2_simple as gpt2
sess = gpt2.start_tf_sess()
gpt2.load_gpt2(sess)
gpt2.generate(sess)
```
As with textgenrnn, you can generate and save text for later use (e.g. an API or a bot) by using the `return_as_list` parameter.
```python
single_text = gpt2.generate(sess, return_as_list=True)[0]
print(single_text)
```
You can pass a `run_name` parameter to `finetune` and `load_gpt2` if you want to store/load multiple models in a `checkpoint` folder.
NB: *Restart the Python session first* if you want to finetune on another dataset or load another model.
Raw data
{
"_id": null,
"home_page": "https://github.com/minimaxir/gpt-2-simple",
"name": "gpt-2-simple",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "deep learning,tensorflow,text generation",
"author": "Max Woolf",
"author_email": "max@minimaxir.com",
"download_url": "https://files.pythonhosted.org/packages/ec/d5/d1e9ab56bd82bc206fbd26a284115bee8101c2057a03d6ce8bcd069a1525/gpt_2_simple-0.8.1.tar.gz",
"platform": "",
"description": "A simple Python package that wraps existing model fine-tuning and generation scripts for OpenAI GPT-2 text generation model (specifically the \"small\", 124M hyperparameter version). Additionally, this package allows easier generation of text, generating to a file for easy curation, allowing for prefixes to force the text to start with a given phrase.\n\n## Usage\n\nAn example for downloading the model to the local system, fineturning it on a dataset. and generating some text.\n\nWarning: the pretrained model, and thus any finetuned model, is 500 MB!\n\n```python\nimport gpt_2_simple as gpt2\n\ngpt2.download_gpt2() # model is saved into current directory under /models/124M/\n\nsess = gpt2.start_tf_sess()\ngpt2.finetune(sess, 'shakespeare.txt', steps=1000) # steps is max number of training steps\n\ngpt2.generate(sess)\n```\n\nThe generated model checkpoints are by default in `/checkpoint/run1`. If you want to load a model from that folder and generate text from it:\n\n```python\nimport gpt_2_simple as gpt2\n\nsess = gpt2.start_tf_sess()\ngpt2.load_gpt2(sess)\n\ngpt2.generate(sess)\n```\n\nAs with textgenrnn, you can generate and save text for later use (e.g. an API or a bot) by using the `return_as_list` parameter.\n\n```python\nsingle_text = gpt2.generate(sess, return_as_list=True)[0]\nprint(single_text)\n```\n\nYou can pass a `run_name` parameter to `finetune` and `load_gpt2` if you want to store/load multiple models in a `checkpoint` folder.\n\nNB: *Restart the Python session first* if you want to finetune on another dataset or load another model.",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python package to easily retrain OpenAI's GPT-2 text-generating model on new texts.",
"version": "0.8.1",
"project_urls": {
"Homepage": "https://github.com/minimaxir/gpt-2-simple"
},
"split_keywords": [
"deep learning",
"tensorflow",
"text generation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ecd5d1e9ab56bd82bc206fbd26a284115bee8101c2057a03d6ce8bcd069a1525",
"md5": "d6065545a8dae4e8c6c263e1e60b2450",
"sha256": "0d620a17b4c4592190b637548afa0aaf26231c71a107165ee91e90929e398c11"
},
"downloads": -1,
"filename": "gpt_2_simple-0.8.1.tar.gz",
"has_sig": false,
"md5_digest": "d6065545a8dae4e8c6c263e1e60b2450",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 26465,
"upload_time": "2021-10-18T01:47:20",
"upload_time_iso_8601": "2021-10-18T01:47:20.224652Z",
"url": "https://files.pythonhosted.org/packages/ec/d5/d1e9ab56bd82bc206fbd26a284115bee8101c2057a03d6ce8bcd069a1525/gpt_2_simple-0.8.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2021-10-18 01:47:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "minimaxir",
"github_project": "gpt-2-simple",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "tensorflow",
"specs": [
[
">=",
"2.5.1"
]
]
},
{
"name": "regex",
"specs": []
},
{
"name": "requests",
"specs": []
},
{
"name": "tqdm",
"specs": []
},
{
"name": "numpy",
"specs": []
},
{
"name": "toposort",
"specs": []
}
],
"lcname": "gpt-2-simple"
}
JSON
