# semantic-text-splitter
[](https://semantic-text-splitter.readthedocs.io/en/latest/?badge=latest) [](https://github.com/benbrandt/text-splitter/blob/main/LICENSE.txt)
Large language models (LLMs) can be used for many tasks, but often have a limited context size that can be smaller than documents you might want to use. To use documents of larger length, you often have to split your text into chunks to fit within this context size.
This crate provides methods for splitting longer pieces of text into smaller chunks, aiming to maximize a desired chunk size, but still splitting at semantically sensible boundaries whenever possible.
## Get Started
### By Number of Characters
```python
from semantic_text_splitter import TextSplitter
# Maximum number of characters in a chunk
max_characters = 1000
# Optionally can also have the splitter not trim whitespace for you
splitter = TextSplitter(max_characters)
# splitter = TextSplitter(max_characters, trim=False)
chunks = splitter.chunks("your document text")
```
### Using a Range for Chunk Capacity
You also have the option of specifying your chunk capacity as a range.
Once a chunk has reached a length that falls within the range it will be returned.
It is always possible that a chunk may be returned that is less than the `start` value, as adding the next piece of text may have made it larger than the `end` capacity.
```python
from semantic_text_splitter import TextSplitter
# Maximum number of characters in a chunk. Will fill up the
# chunk until it is somewhere in this range.
splitter = TextSplitter((200,1000))
chunks = splitter.chunks("your document text")
```
### Using a Hugging Face Tokenizer
```python
from semantic_text_splitter import TextSplitter
from tokenizers import Tokenizer
# Maximum number of tokens in a chunk
max_tokens = 1000
tokenizer = Tokenizer.from_pretrained("bert-base-uncased")
splitter = TextSplitter.from_huggingface_tokenizer(tokenizer, max_tokens)
chunks = splitter.chunks("your document text")
```
### Using a Tiktoken Tokenizer
```python
from semantic_text_splitter import TextSplitter
# Maximum number of tokens in a chunk
max_tokens = 1000
splitter = TextSplitter.from_tiktoken_model("gpt-3.5-turbo", max_tokens)
chunks = splitter.chunks("your document text")
```
### Using a Custom Callback
```python
from semantic_text_splitter import TextSplitter
splitter = TextSplitter.from_callback(lambda text: len(text), 1000)
chunks = splitter.chunks("your document text")
```
### Markdown
All of the above examples also can also work with Markdown text. You can use the `MarkdownSplitter` in the same ways as the `TextSplitter`.
```python
from semantic_text_splitter import MarkdownSplitter
# Maximum number of characters in a chunk
max_characters = 1000
# Optionally can also have the splitter not trim whitespace for you
splitter = MarkdownSplitter(max_characters)
# splitter = MarkdownSplitter(max_characters, trim=False)
chunks = splitter.chunks("# Header\n\nyour document text")
```
## Method
To preserve as much semantic meaning within a chunk as possible, each chunk is composed of the largest semantic units that can fit in the next given chunk. For each splitter type, there is a defined set of semantic levels. Here is an example of the steps used:
1. Split the text by a increasing semantic levels.
2. Check the first item for each level and select the highest level whose first item still fits within the chunk size.
3. Merge as many of these neighboring sections of this level or above into a chunk to maximize chunk length. Boundaries of higher semantic levels are always included when merging, so that the chunk doesn't inadvertantly cross semantic boundaries.
The boundaries used to split the text if using the `chunks` method, in ascending order:
### `TextSplitter` Semantic Levels
1. Characters
2. [Unicode Grapheme Cluster Boundaries](https://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries)
3. [Unicode Word Boundaries](https://www.unicode.org/reports/tr29/#Word_Boundaries)
4. [Unicode Sentence Boundaries](https://www.unicode.org/reports/tr29/#Sentence_Boundaries)
5. Ascending sequence length of newlines. (Newline is `\r\n`, `\n`, or `\r`) Each unique length of consecutive newline sequences is treated as its own semantic level. So a sequence of 2 newlines is a higher level than a sequence of 1 newline, and so on.
Splitting doesn't occur below the character level, otherwise you could get partial bytes of a char, which may not be a valid unicode str.
### `MarkdownSplitter` Semantic Levels
Markdown is parsed according to the CommonMark spec, along with some optional features such as GitHub Flavored Markdown.
1. Characters
2. [Unicode Grapheme Cluster Boundaries](https://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries)
3. [Unicode Word Boundaries](https://www.unicode.org/reports/tr29/#Word_Boundaries)
4. [Unicode Sentence Boundaries](https://www.unicode.org/reports/tr29/#Sentence_Boundaries)
5. Soft line breaks (single newline) which isn't necessarily a new element in Markdown.
6. Inline elements such as: text nodes, emphasis, strong, strikethrough, link, image, table cells, inline code, footnote references, task list markers, and inline html.
7. Block elements suce as: paragraphs, code blocks, footnote definitions, metadata. Also, a block quote or row/item within a table or list that can contain other "block" type elements, and a list or table that contains items.
8. Thematic breaks or horizontal rules.
9. Headings by level
Splitting doesn't occur below the character level, otherwise you could get partial bytes of a char, which may not be a valid unicode str.
### Note on sentences
There are lots of methods of determining sentence breaks, all to varying degrees of accuracy, and many requiring ML models to do so. Rather than trying to find the perfect sentence breaks, we rely on unicode method of sentence boundaries, which in most cases is good enough for finding a decent semantic breaking point if a paragraph is too large, and avoids the performance penalties of many other methods.
## Inspiration
This crate was inspired by [LangChain's TextSplitter](https://api.python.langchain.com/en/latest/character/langchain_text_splitters.character.RecursiveCharacterTextSplitter.html#langchain_text_splitters.character.RecursiveCharacterTextSplitter). But, looking into the implementation, there was potential for better performance as well as better semantic chunking.
A big thank you to the unicode-rs team for their [unicode-segmentation](https://crates.io/crates/unicode-segmentation) crate that manages a lot of the complexity of matching the Unicode rules for words and sentences.
Raw data
{
"_id": null,
"home_page": null,
"name": "semantic-text-splitter",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "text, split, tokenizer, nlp, ai",
"author": "Ben Brandt <benjamin.j.brandt@gmail.com>",
"author_email": "Ben Brandt <benjamin.j.brandt@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/26/d2/f6daf9f29fc196e27cb2ea8f7d0c176ff9bc358889dfd6bb27b02d18f2ec/semantic_text_splitter-0.12.3.tar.gz",
"platform": null,
"description": "# semantic-text-splitter\n\n[](https://semantic-text-splitter.readthedocs.io/en/latest/?badge=latest) [](https://github.com/benbrandt/text-splitter/blob/main/LICENSE.txt)\n\nLarge language models (LLMs) can be used for many tasks, but often have a limited context size that can be smaller than documents you might want to use. To use documents of larger length, you often have to split your text into chunks to fit within this context size.\n\nThis crate provides methods for splitting longer pieces of text into smaller chunks, aiming to maximize a desired chunk size, but still splitting at semantically sensible boundaries whenever possible.\n\n## Get Started\n\n### By Number of Characters\n\n```python\nfrom semantic_text_splitter import TextSplitter\n\n# Maximum number of characters in a chunk\nmax_characters = 1000\n# Optionally can also have the splitter not trim whitespace for you\nsplitter = TextSplitter(max_characters)\n# splitter = TextSplitter(max_characters, trim=False)\n\nchunks = splitter.chunks(\"your document text\")\n```\n\n### Using a Range for Chunk Capacity\n\nYou also have the option of specifying your chunk capacity as a range.\n\nOnce a chunk has reached a length that falls within the range it will be returned.\n\nIt is always possible that a chunk may be returned that is less than the `start` value, as adding the next piece of text may have made it larger than the `end` capacity.\n\n```python\nfrom semantic_text_splitter import TextSplitter\n\n\n# Maximum number of characters in a chunk. Will fill up the\n# chunk until it is somewhere in this range.\nsplitter = TextSplitter((200,1000))\n\nchunks = splitter.chunks(\"your document text\")\n```\n\n### Using a Hugging Face Tokenizer\n\n```python\nfrom semantic_text_splitter import TextSplitter\nfrom tokenizers import Tokenizer\n\n# Maximum number of tokens in a chunk\nmax_tokens = 1000\ntokenizer = Tokenizer.from_pretrained(\"bert-base-uncased\")\nsplitter = TextSplitter.from_huggingface_tokenizer(tokenizer, max_tokens)\n\nchunks = splitter.chunks(\"your document text\")\n```\n\n### Using a Tiktoken Tokenizer\n\n```python\nfrom semantic_text_splitter import TextSplitter\n\n# Maximum number of tokens in a chunk\nmax_tokens = 1000\nsplitter = TextSplitter.from_tiktoken_model(\"gpt-3.5-turbo\", max_tokens)\n\nchunks = splitter.chunks(\"your document text\")\n```\n\n### Using a Custom Callback\n\n```python\nfrom semantic_text_splitter import TextSplitter\n\nsplitter = TextSplitter.from_callback(lambda text: len(text), 1000)\n\nchunks = splitter.chunks(\"your document text\")\n```\n\n### Markdown\n\nAll of the above examples also can also work with Markdown text. You can use the `MarkdownSplitter` in the same ways as the `TextSplitter`.\n\n```python\nfrom semantic_text_splitter import MarkdownSplitter\n\n# Maximum number of characters in a chunk\nmax_characters = 1000\n# Optionally can also have the splitter not trim whitespace for you\nsplitter = MarkdownSplitter(max_characters)\n# splitter = MarkdownSplitter(max_characters, trim=False)\n\nchunks = splitter.chunks(\"# Header\\n\\nyour document text\")\n```\n\n## Method\n\nTo preserve as much semantic meaning within a chunk as possible, each chunk is composed of the largest semantic units that can fit in the next given chunk. For each splitter type, there is a defined set of semantic levels. Here is an example of the steps used:\n\n1. Split the text by a increasing semantic levels.\n2. Check the first item for each level and select the highest level whose first item still fits within the chunk size.\n3. Merge as many of these neighboring sections of this level or above into a chunk to maximize chunk length. Boundaries of higher semantic levels are always included when merging, so that the chunk doesn't inadvertantly cross semantic boundaries.\n\nThe boundaries used to split the text if using the `chunks` method, in ascending order:\n\n### `TextSplitter` Semantic Levels\n\n1. Characters\n2. [Unicode Grapheme Cluster Boundaries](https://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries)\n3. [Unicode Word Boundaries](https://www.unicode.org/reports/tr29/#Word_Boundaries)\n4. [Unicode Sentence Boundaries](https://www.unicode.org/reports/tr29/#Sentence_Boundaries)\n5. Ascending sequence length of newlines. (Newline is `\\r\\n`, `\\n`, or `\\r`) Each unique length of consecutive newline sequences is treated as its own semantic level. So a sequence of 2 newlines is a higher level than a sequence of 1 newline, and so on.\n\nSplitting doesn't occur below the character level, otherwise you could get partial bytes of a char, which may not be a valid unicode str.\n\n### `MarkdownSplitter` Semantic Levels\n\nMarkdown is parsed according to the CommonMark spec, along with some optional features such as GitHub Flavored Markdown.\n\n1. Characters\n2. [Unicode Grapheme Cluster Boundaries](https://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries)\n3. [Unicode Word Boundaries](https://www.unicode.org/reports/tr29/#Word_Boundaries)\n4. [Unicode Sentence Boundaries](https://www.unicode.org/reports/tr29/#Sentence_Boundaries)\n5. Soft line breaks (single newline) which isn't necessarily a new element in Markdown.\n6. Inline elements such as: text nodes, emphasis, strong, strikethrough, link, image, table cells, inline code, footnote references, task list markers, and inline html.\n7. Block elements suce as: paragraphs, code blocks, footnote definitions, metadata. Also, a block quote or row/item within a table or list that can contain other \"block\" type elements, and a list or table that contains items.\n8. Thematic breaks or horizontal rules.\n9. Headings by level\n\nSplitting doesn't occur below the character level, otherwise you could get partial bytes of a char, which may not be a valid unicode str.\n\n### Note on sentences\n\nThere are lots of methods of determining sentence breaks, all to varying degrees of accuracy, and many requiring ML models to do so. Rather than trying to find the perfect sentence breaks, we rely on unicode method of sentence boundaries, which in most cases is good enough for finding a decent semantic breaking point if a paragraph is too large, and avoids the performance penalties of many other methods.\n\n## Inspiration\n\nThis crate was inspired by [LangChain's TextSplitter](https://api.python.langchain.com/en/latest/character/langchain_text_splitters.character.RecursiveCharacterTextSplitter.html#langchain_text_splitters.character.RecursiveCharacterTextSplitter). But, looking into the implementation, there was potential for better performance as well as better semantic chunking.\n\nA big thank you to the unicode-rs team for their [unicode-segmentation](https://crates.io/crates/unicode-segmentation) crate that manages a lot of the complexity of matching the Unicode rules for words and sentences.\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from Rust and Python.",
"version": "0.12.3",
"project_urls": {
"Source Code": "https://github.com/benbrandt/text-splitter"
},
"split_keywords": [
"text",
" split",
" tokenizer",
" nlp",
" ai"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "ce3d1f357252aedad689bec64ae893de136d258acb12b3f0dcf57775e290f9ec",
"md5": "d2528341b9f4e129c1722f20e8ee9595",
"sha256": "faa10455c97be464c0e6f8c652d5a36ef6d2bf8a13dce4542269bedbd2b529f0"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3-cp38-abi3-macosx_10_12_x86_64.whl",
"has_sig": false,
"md5_digest": "d2528341b9f4e129c1722f20e8ee9595",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 3599977,
"upload_time": "2024-05-01T13:47:00",
"upload_time_iso_8601": "2024-05-01T13:47:00.445968Z",
"url": "https://files.pythonhosted.org/packages/ce/3d/1f357252aedad689bec64ae893de136d258acb12b3f0dcf57775e290f9ec/semantic_text_splitter-0.12.3-cp38-abi3-macosx_10_12_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f8cab00f7d93c03dd4f4d92f99168eed5fa77818f450fda5e5e236553603fff6",
"md5": "9ee9565959d5d324dbeb84f67fae3b28",
"sha256": "988d653af565ca70d92e6bd7ae026e02e84ccedd5c8ba0453cef4504c63b8103"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3-cp38-abi3-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "9ee9565959d5d324dbeb84f67fae3b28",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 3563137,
"upload_time": "2024-05-01T13:47:05",
"upload_time_iso_8601": "2024-05-01T13:47:05.458395Z",
"url": "https://files.pythonhosted.org/packages/f8/ca/b00f7d93c03dd4f4d92f99168eed5fa77818f450fda5e5e236553603fff6/semantic_text_splitter-0.12.3-cp38-abi3-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "1e59d997b2a9fe0441be75d54625c5217cc37c8dcdcee06037b707b7db482f65",
"md5": "74f315ad521cb3dfd1918f4fe028249a",
"sha256": "0ffd2c97a6b4bf2de2ac6c45296d0105f798f00391544f8ac31f2c567d7e0ffe"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_12_i686.manylinux2010_i686.whl",
"has_sig": false,
"md5_digest": "74f315ad521cb3dfd1918f4fe028249a",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 4692608,
"upload_time": "2024-05-01T13:47:07",
"upload_time_iso_8601": "2024-05-01T13:47:07.279452Z",
"url": "https://files.pythonhosted.org/packages/1e/59/d997b2a9fe0441be75d54625c5217cc37c8dcdcee06037b707b7db482f65/semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_12_i686.manylinux2010_i686.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "8d20eba10b7aed92e763378934db4bca084de155ce6f8e4914480e098f273afa",
"md5": "c86e922bae355318d4bc42691bffbd6e",
"sha256": "faec84eef575c614f3e0b803c16161cf19a457eee2c99dcd234e8520577e5847"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"has_sig": false,
"md5_digest": "c86e922bae355318d4bc42691bffbd6e",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 4649607,
"upload_time": "2024-05-01T13:47:09",
"upload_time_iso_8601": "2024-05-01T13:47:09.172989Z",
"url": "https://files.pythonhosted.org/packages/8d/20/eba10b7aed92e763378934db4bca084de155ce6f8e4914480e098f273afa/semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "6638764eb869b72863c994d4c6988cb542fd9f8fba6155da109e58fb4f7f9560",
"md5": "affd39c1d3f88c6e99327865f699836d",
"sha256": "3f3c8ef4082d1638d64015e0f8b802d74b6220ec693560c20b15041267c02762"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl",
"has_sig": false,
"md5_digest": "affd39c1d3f88c6e99327865f699836d",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 4496662,
"upload_time": "2024-05-01T13:47:11",
"upload_time_iso_8601": "2024-05-01T13:47:11.519475Z",
"url": "https://files.pythonhosted.org/packages/66/38/764eb869b72863c994d4c6988cb542fd9f8fba6155da109e58fb4f7f9560/semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "aba7a79a8abdd2e9ccb510a64f5b213677cf1e527f8611b4f42e82076f92feec",
"md5": "ee15f54065870fa3eff44461698669be",
"sha256": "230fe178d1c7554ab142413029f029980b992061fcdc802e7d39d83238b0b0ae"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl",
"has_sig": false,
"md5_digest": "ee15f54065870fa3eff44461698669be",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 5011795,
"upload_time": "2024-05-01T13:47:13",
"upload_time_iso_8601": "2024-05-01T13:47:13.957911Z",
"url": "https://files.pythonhosted.org/packages/ab/a7/a79a8abdd2e9ccb510a64f5b213677cf1e527f8611b4f42e82076f92feec/semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "5cf9562eb3288dc192cf2a8f3c42f598417399354066efc63f3b6a15e7d4d71f",
"md5": "c39472e3456b9dca602cfd9fd443d8e7",
"sha256": "bd14679b4fe3c16dfa3bc5b34ea4eb6c406b897ee347b948216f799ff221ba87"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl",
"has_sig": false,
"md5_digest": "c39472e3456b9dca602cfd9fd443d8e7",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 5027070,
"upload_time": "2024-05-01T13:47:16",
"upload_time_iso_8601": "2024-05-01T13:47:16.483449Z",
"url": "https://files.pythonhosted.org/packages/5c/f9/562eb3288dc192cf2a8f3c42f598417399354066efc63f3b6a15e7d4d71f/semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "8b79effcbdc662e35a6e0191eb8f86f84fe06384f1da74007b1778367582615b",
"md5": "cf8b786295b8468be3a0ec0b0e3496d9",
"sha256": "2ba384352706c5ffda3ef599a78888c2c609733b6ba95dde8dcb452d1161ade0"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "cf8b786295b8468be3a0ec0b0e3496d9",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 4677487,
"upload_time": "2024-05-01T13:47:18",
"upload_time_iso_8601": "2024-05-01T13:47:18.452352Z",
"url": "https://files.pythonhosted.org/packages/8b/79/effcbdc662e35a6e0191eb8f86f84fe06384f1da74007b1778367582615b/semantic_text_splitter-0.12.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "e0f3a515418bf73600632a5656514edcc3c44f6fcfd530425342edadfa9d4ed5",
"md5": "36cf7fcd6055e25bfbf771e5c8c7b788",
"sha256": "e48854cb3f931688cd0d1882db975c2a52b1e1d460b3db63f39470b2a9f8bf24"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3-cp38-abi3-win32.whl",
"has_sig": false,
"md5_digest": "36cf7fcd6055e25bfbf771e5c8c7b788",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 3163176,
"upload_time": "2024-05-01T13:47:20",
"upload_time_iso_8601": "2024-05-01T13:47:20.845180Z",
"url": "https://files.pythonhosted.org/packages/e0/f3/a515418bf73600632a5656514edcc3c44f6fcfd530425342edadfa9d4ed5/semantic_text_splitter-0.12.3-cp38-abi3-win32.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b5369142d2c2144087aa8a6dbcba447c5af37e81be96f805fa18149a979503f6",
"md5": "1db6040547b2fd70e8118489476f0ace",
"sha256": "a79368207416480368302626f36b5a8523ff0fff3fc2a55e29c2c8d7cf210395"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3-cp38-abi3-win_amd64.whl",
"has_sig": false,
"md5_digest": "1db6040547b2fd70e8118489476f0ace",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 3302309,
"upload_time": "2024-05-01T13:47:23",
"upload_time_iso_8601": "2024-05-01T13:47:23.328466Z",
"url": "https://files.pythonhosted.org/packages/b5/36/9142d2c2144087aa8a6dbcba447c5af37e81be96f805fa18149a979503f6/semantic_text_splitter-0.12.3-cp38-abi3-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "26d2f6daf9f29fc196e27cb2ea8f7d0c176ff9bc358889dfd6bb27b02d18f2ec",
"md5": "f2e32a8bfbd82bdbfe855d24448fd3a9",
"sha256": "19f41a430e70b03b939cbff3d73f02d1502f3c3980ceb50631104f72157aae46"
},
"downloads": -1,
"filename": "semantic_text_splitter-0.12.3.tar.gz",
"has_sig": false,
"md5_digest": "f2e32a8bfbd82bdbfe855d24448fd3a9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 275610,
"upload_time": "2024-05-01T13:47:25",
"upload_time_iso_8601": "2024-05-01T13:47:25.560535Z",
"url": "https://files.pythonhosted.org/packages/26/d2/f6daf9f29fc196e27cb2ea8f7d0c176ff9bc358889dfd6bb27b02d18f2ec/semantic_text_splitter-0.12.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-01 13:47:25",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "benbrandt",
"github_project": "text-splitter",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "semantic-text-splitter"
}
JSON
