<div align="center">
<img src="https://raw.githubusercontent.com/Zipstack/unstract-sdk/main/docs/assets/unstract_u_logo.png" style="height: 120px">
# Unstract
## No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents
</div>
# Unstract SDK
The `unstract-sdk` package helps with developing tools that are meant to be run on the Unstract platform. This includes
modules to help with tool development and execution, caching, making calls to LLMs / vectorDBs / embeddings .etc.
They also contain helper methods/classes to aid with other tasks such as indexing and auditing the LLM calls.
## Installation
- The below libraries need to be installed to run the SDK
- Linux
```
sudo apt install build-essential pkg-config libmagic-dev tesseract-ocr pandoc
```
- Mac
```
brew install pkg-config libmagic pandoc tesseract-ocr
```
## Tools
### Create a scaffolding for a new tool
Example
```bash
unstract-tool-gen --command NEW --tool-name <name of tool> \
--location ~/path_to_repository/unstract-tools/ --overwrite false
```
Supported commands:
- `NEW` - Create a new tool
### Environment variables required for all Tools
| Variable | Description |
| -------------------------- | --------------------------------------------------------------------- |
| `PLATFORM_SERVICE_HOST` | The host in which the platform service is running |
| `PLATFORM_SERVICE_PORT` | The port in which the service is listening |
| `PLATFORM_SERVICE_API_KEY` | The API key for the platform |
| `TOOL_DATA_DIR` | The directory in the filesystem which has contents for tool execution |
### Llama Index support
Unstract SDK 0.3.2 uses the following version of Llama
Index Version **0.9.28** as on January 14th, 2024
### Developing with the SDK
Ensure that you have all the required dependencies and pre-commit hooks installed
```shell
pdm install
pre-commit install
```
Once the changes have been made, it can be tested with [Unstract](https://github.com/Zipstack/unstract) through the following means.
#### With PDM
Specify the SDK as a dependency to a project using a tool like `pdm` by adding the following to your `pyproject.toml`
```toml
[tool.pdm.dev-dependencies]
local_copies = [
"-e unstract-adapters @ file:///${UNSTRACT_ADAPTERS_PATH}",
"-e unstract-sdk @ file:///${UNSTRACT_SDK_PATH}",
]
```
Or by running the below command
```shell
pdm add -e /path/to/unstract-sdk --dev
```
#### With pip
- If the project is using `pip` it might be possible to add it as a dependency in `requirements.txt`
```
-e /path/to/unstract-sdk
```
NOTE: Building locally might require the below section to be replaced in the `unstract-sdk`'s build system configuration
```
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"
```
- Another option is to provide a git URL in `requirements.txt`, this can come in handy while building tool
docker images. Don't forget to run `apt install git` within the `Dockerfile` for this
```shell
unstract-sdk @ git+https://github.com/Zipstack/unstract-sdk@feature-branch
```
- Or try installing a [local PyPI server](https://pypi.org/project/pypiserver/) and upload / download your package from this server
### Environment variables required for various LLMs (deprecated)
- Azure OpenAI
- `OPENAI_API_KEY`
- `OPENAI_API_BASE`
- `OPENAI_API_VERSION`
- `OPENAI_API_ENGINE`
- `OPENAI_API_MODEL`
### Documentation generation
Follow [this README.md](https://github.com/Zipstack/unstract-sdk/blob/main/docs/README.md) for generating documentation.
Raw data
{
"_id": null,
"home_page": null,
"name": "unstract-sdk",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.11.1,>=3.9",
"maintainer_email": null,
"keywords": "unstract tools-development-kit apps development-kit sdk",
"author": null,
"author_email": "Zipstack Inc <devsupport@zipstack.com>",
"download_url": "https://files.pythonhosted.org/packages/c0/22/1f964ba0c2c9f242d48cad6f3235f2ec4d311420e8c2b44163178761199c/unstract_sdk-0.53.0.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/Zipstack/unstract-sdk/main/docs/assets/unstract_u_logo.png\" style=\"height: 120px\">\n\n# Unstract\n\n## No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents\n\n</div>\n\n# Unstract SDK\n\nThe `unstract-sdk` package helps with developing tools that are meant to be run on the Unstract platform. This includes\nmodules to help with tool development and execution, caching, making calls to LLMs / vectorDBs / embeddings .etc.\nThey also contain helper methods/classes to aid with other tasks such as indexing and auditing the LLM calls.\n\n## Installation\n\n- The below libraries need to be installed to run the SDK\n - Linux\n\n ```\n sudo apt install build-essential pkg-config libmagic-dev tesseract-ocr pandoc\n ```\n\n - Mac\n\n ```\n brew install pkg-config libmagic pandoc tesseract-ocr\n ```\n\n## Tools\n\n### Create a scaffolding for a new tool\n\nExample\n\n```bash\nunstract-tool-gen --command NEW --tool-name <name of tool> \\\n --location ~/path_to_repository/unstract-tools/ --overwrite false\n```\n\nSupported commands:\n\n- `NEW` - Create a new tool\n\n### Environment variables required for all Tools\n\n| Variable | Description |\n| -------------------------- | --------------------------------------------------------------------- |\n| `PLATFORM_SERVICE_HOST` | The host in which the platform service is running |\n| `PLATFORM_SERVICE_PORT` | The port in which the service is listening |\n| `PLATFORM_SERVICE_API_KEY` | The API key for the platform |\n| `TOOL_DATA_DIR` | The directory in the filesystem which has contents for tool execution |\n\n### Llama Index support\n\nUnstract SDK 0.3.2 uses the following version of Llama\nIndex Version **0.9.28** as on January 14th, 2024\n\n### Developing with the SDK\n\nEnsure that you have all the required dependencies and pre-commit hooks installed\n```shell\npdm install\npre-commit install\n```\n\nOnce the changes have been made, it can be tested with [Unstract](https://github.com/Zipstack/unstract) through the following means.\n\n#### With PDM\nSpecify the SDK as a dependency to a project using a tool like `pdm` by adding the following to your `pyproject.toml`\n\n```toml\n[tool.pdm.dev-dependencies]\nlocal_copies = [\n \"-e unstract-adapters @ file:///${UNSTRACT_ADAPTERS_PATH}\",\n \"-e unstract-sdk @ file:///${UNSTRACT_SDK_PATH}\",\n]\n```\nOr by running the below command\n```shell\npdm add -e /path/to/unstract-sdk --dev\n```\n\n#### With pip\n- If the project is using `pip` it might be possible to add it as a dependency in `requirements.txt`\n```\n-e /path/to/unstract-sdk\n```\nNOTE: Building locally might require the below section to be replaced in the `unstract-sdk`'s build system configuration\n```\n[build-system]\nrequires = [\"setuptools\", \"wheel\"]\nbuild-backend = \"setuptools.build_meta\"\n```\n- Another option is to provide a git URL in `requirements.txt`, this can come in handy while building tool\ndocker images. Don't forget to run `apt install git` within the `Dockerfile` for this\n```shell\nunstract-sdk @ git+https://github.com/Zipstack/unstract-sdk@feature-branch\n```\n\n- Or try installing a [local PyPI server](https://pypi.org/project/pypiserver/) and upload / download your package from this server\n\n### Environment variables required for various LLMs (deprecated)\n\n- Azure OpenAI\n - `OPENAI_API_KEY`\n - `OPENAI_API_BASE`\n - `OPENAI_API_VERSION`\n - `OPENAI_API_ENGINE`\n - `OPENAI_API_MODEL`\n\n### Documentation generation\n\nFollow [this README.md](https://github.com/Zipstack/unstract-sdk/blob/main/docs/README.md) for generating documentation.\n",
"bugtrack_url": null,
"license": "AGPL v3",
"summary": "A framework for writing Unstract Tools/Apps",
"version": "0.53.0",
"project_urls": {
"Homepage": "https://unstract.com",
"Release notes": "https://github.com/Zipstack/unstract-sdk/releases",
"Source": "https://github.com/Zipstack/unstract-sdk"
},
"split_keywords": [
"unstract",
"tools-development-kit",
"apps",
"development-kit",
"sdk"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1d597d5aff361f7333e55f99b5fba87121c34f71d718d8dcea68aadce3ad2c58",
"md5": "3dad6b6d48433bd22d30a5b685b847f0",
"sha256": "391c74f37af2782c0fe60c4c1f5d6bf3b7ce15cbb191f0f0eb177055141112b8"
},
"downloads": -1,
"filename": "unstract_sdk-0.53.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3dad6b6d48433bd22d30a5b685b847f0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.11.1,>=3.9",
"size": 236965,
"upload_time": "2024-11-11T06:27:05",
"upload_time_iso_8601": "2024-11-11T06:27:05.611643Z",
"url": "https://files.pythonhosted.org/packages/1d/59/7d5aff361f7333e55f99b5fba87121c34f71d718d8dcea68aadce3ad2c58/unstract_sdk-0.53.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c0221f964ba0c2c9f242d48cad6f3235f2ec4d311420e8c2b44163178761199c",
"md5": "9a5086d9141d71a1d69c2cc0c4637ca7",
"sha256": "6bf64399212d1fb105b66e26913b92a272951ec8b7337d57b1732ddb23bbcd4a"
},
"downloads": -1,
"filename": "unstract_sdk-0.53.0.tar.gz",
"has_sig": false,
"md5_digest": "9a5086d9141d71a1d69c2cc0c4637ca7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.11.1,>=3.9",
"size": 150238,
"upload_time": "2024-11-11T06:27:06",
"upload_time_iso_8601": "2024-11-11T06:27:06.940941Z",
"url": "https://files.pythonhosted.org/packages/c0/22/1f964ba0c2c9f242d48cad6f3235f2ec4d311420e8c2b44163178761199c/unstract_sdk-0.53.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-11 06:27:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Zipstack",
"github_project": "unstract-sdk",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "unstract-sdk"
}
JSON
