| Name | gpt-trim JSON |
| Version |
0.1
JSON |
| download |
| home_page | |
| Summary | Trims the messages array for ChatGPT API |
| upload_time | 2023-09-17 13:58:47 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3 |
| license | MIT License Copyright (c) 2023 JC Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| keywords |
gpt
trim
gpt-trim
chatgpt
token
tiktoken
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# gpt-trim
This is a (slightly) faster version of [KillianLucas/tokentrim](https://pypi.org/project/tokentrim) for longer message arrays.
In average, gpt-trim is \~80% faster than tokentrim, and that tokentrim is around 5x\~7x slower.
Although gpt-trim is fast, I still need to finish my LeetCode problems that I left years ago, just so that I can make it 20x faster than 95% of people.
## Usage
The usage is quite similiar to `tokentrim`.
```python
import gpt_trim
trimmed = gpt_trim.trim(
messages,
model="gpt-3.5-turbo"
)
print(trimmed)
```
Alternatively, you can assign the token limit manually:
```python
gpt_trim.trim(
messages,
max_tokens=100
)
```
You can also add system messages with ease:
```python
import gpt_trim
messages = [
..., # long, long content
{
"role": "user",
"content": "It's about drive, it's about power"
}
]
trimmed = gpt_trim.advanced_trim(
messages,
system_messages=[
{
"role": "system",
"content": "You'll act like the celebrity: The Rock."
}
],
model="gpt-3.5-turbo",
)
print(trimmed)
```
The catch? It's slower. With great power comes great... patience.
## Comparison
You can compare this project to [KillianLucas/tokentrim](https://pypi.org/project/tokentrim) like so:
```python
import time
import gpt_trim
import tiktoken
import tokentrim
pattern = "d!3h.l7$fj" # 10 tokens
messages = [
{
"role": "user",
"content": pattern * 5000 # 50000 tokens
}
]
# cache first
enc = tiktoken.get_encoding("cl100k_base")
gpt_trim.num_tokens_from_messages(
messages,
enc
)
def test(provider):
print("Testing", provider.__name__)
s = time.time()
result = provider.trim(
messages,
model="gpt-3.5-turbo",
)
print(f"took {(time.time() - s):.4f}s\n")
# Swap the following for every test and see tokentrim
# struggles when dealing with longer context.
test(gpt_trim)
test(tokentrim)
```
***
Right. I was bored.
Raw data
{
"_id": null,
"home_page": "",
"name": "gpt-trim",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3",
"maintainer_email": "",
"keywords": "gpt,trim,gpt-trim,chatgpt,token,tiktoken",
"author": "",
"author_email": "AWeirdDev <aweirdscratcher@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/b5/76/8689552465e3a4678982170f2a2509edccbc1dae4656518b97d0d52a4801/gpt-trim-0.1.tar.gz",
"platform": null,
"description": "# gpt-trim\n\nThis is a (slightly) faster version of [KillianLucas/tokentrim](https://pypi.org/project/tokentrim) for longer message arrays.\n\nIn average, gpt-trim is \\~80% faster than tokentrim, and that tokentrim is around 5x\\~7x slower.\n\nAlthough gpt-trim is fast, I still need to finish my LeetCode problems that I left years ago, just so that I can make it 20x faster than 95% of people.\n\n## Usage\n\nThe usage is quite similiar to `tokentrim`.\n\n```python\nimport gpt_trim\n\ntrimmed = gpt_trim.trim(\n messages, \n model=\"gpt-3.5-turbo\"\n)\nprint(trimmed)\n```\n\nAlternatively, you can assign the token limit manually:\n\n```python\ngpt_trim.trim(\n messages,\n max_tokens=100\n)\n```\n\nYou can also add system messages with ease:\n\n```python\nimport gpt_trim\n\nmessages = [\n ..., # long, long content\n {\n \"role\": \"user\",\n \"content\": \"It's about drive, it's about power\"\n }\n]\ntrimmed = gpt_trim.advanced_trim(\n messages,\n system_messages=[\n {\n \"role\": \"system\",\n \"content\": \"You'll act like the celebrity: The Rock.\"\n }\n ],\n model=\"gpt-3.5-turbo\",\n)\nprint(trimmed)\n```\n\nThe catch? It's slower. With great power comes great... patience.\n\n## Comparison\n\nYou can compare this project to [KillianLucas/tokentrim](https://pypi.org/project/tokentrim) like so:\n\n```python\nimport time\n\nimport gpt_trim\nimport tiktoken\nimport tokentrim\n\npattern = \"d!3h.l7$fj\" # 10 tokens\nmessages = [\n {\n \"role\": \"user\",\n \"content\": pattern * 5000 # 50000 tokens\n }\n]\n\n# cache first\nenc = tiktoken.get_encoding(\"cl100k_base\")\ngpt_trim.num_tokens_from_messages(\n messages,\n enc\n)\n\ndef test(provider):\n print(\"Testing\", provider.__name__)\n\n s = time.time()\n result = provider.trim(\n messages,\n model=\"gpt-3.5-turbo\",\n )\n\n print(f\"took {(time.time() - s):.4f}s\\n\")\n\n# Swap the following for every test and see tokentrim \n# struggles when dealing with longer context.\ntest(gpt_trim)\ntest(tokentrim)\n```\n\n***\n\nRight. I was bored.\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2023 JC Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ",
"summary": "Trims the messages array for ChatGPT API",
"version": "0.1",
"project_urls": {
"Bug Tracker": "https://github.com/AWeirdScratcher/gpt-trim/issues",
"Homepage": "https://github.com/AWeirdScratcher/gpt-trim"
},
"split_keywords": [
"gpt",
"trim",
"gpt-trim",
"chatgpt",
"token",
"tiktoken"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b5768689552465e3a4678982170f2a2509edccbc1dae4656518b97d0d52a4801",
"md5": "52dd44b8f350a21987f200cf716aaeba",
"sha256": "edda599ed622283d7b002ef992976a89f216c10655927f84fac68aae2e4a5526"
},
"downloads": -1,
"filename": "gpt-trim-0.1.tar.gz",
"has_sig": false,
"md5_digest": "52dd44b8f350a21987f200cf716aaeba",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3",
"size": 4751,
"upload_time": "2023-09-17T13:58:47",
"upload_time_iso_8601": "2023-09-17T13:58:47.862379Z",
"url": "https://files.pythonhosted.org/packages/b5/76/8689552465e3a4678982170f2a2509edccbc1dae4656518b97d0d52a4801/gpt-trim-0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-17 13:58:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "AWeirdScratcher",
"github_project": "gpt-trim",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "gpt-trim"
}
