<div align="center">
<a href="https://deepstack.khulnasoft.com/"><img src="https://github.com/khulnasoft/deepstack/blob/main/docs/img/banner_20.png" alt="Green logo of a stylized white 'H' with the text 'Deepstack, by khulnasoft. Deepstack 2.0 is live ๐'ย Abstract green and yellow diagrams in the background."></a>
| | |
| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| CI/CD | [](https://github.com/khulnasoft/deepstack/actions/workflows/tests.yml) [](https://github.com/psf/black) [](https://github.com/python/mypy) [](https://coveralls.io/github/khulnasoft/deepstack?branch=main) |
| Docs | [](https://docs.deepstack.khulnasoft.com) |
| Package | [](https://pypi.org/project/deepstack-ai/)   [](https://anaconda.org/conda-forge/deepstack-ai) [](LICENSE) [](https://github.com/khulnasoft/deepstack/actions/workflows/license_compliance.yml) |
| Meta | [](https://discord.gg/deepstack) [](https://twitter.com/khulnasoft) |
</div>
[Deepstack](https://deepstack.khulnasoft.com/) is an end-to-end LLM framework that allows you to build applications powered by
LLMs, Transformer models, vector search and more. Whether you want to perform retrieval-augmented generation (RAG),
document search, question answering or answer generation, Deepstack can orchestrate state-of-the-art embedding models
and LLMs into pipelines to build end-to-end NLP applications and solve your use case.
## Installation
The simplest way to get Deepstack is via pip:
```sh
pip install deepstack-ai
```
Deepstack supports multiple installation methods including Docker images. For a comprehensive guide please refer
to the [documentation](https://docs.deepstack.khulnasoft.com/v2.0/docs/installation).
## Documentation
If you're new to the project, check out ["What is Deepstack?"](https://deepstack.khulnasoft.com/overview/intro) then go
through the ["Get Started Guide"](https://deepstack.khulnasoft.com/overview/quick-start) and build your first LLM application
in a matter of minutes. Keep learning with the [tutorials](https://deepstack.khulnasoft.com/tutorials?v=2.0). For more advanced
use cases, or just to get some inspiration, you can browse our Deepstack recipes in the
[Cookbook](https://github.com/khulnasoft/deepstack-cookbook).
At any given point, hit the [documentation](https://docs.deepstack.khulnasoft.com/v2.0/docs/intro) to learn more about Deepstack, what can it do for you and the technology behind.
## Features
> [!IMPORTANT]
> **You are currently looking at the readme of Deepstack 2.0**. We are still maintaining Deepstack 1.x to give everyone
> enough time to migrate to 2.0. [Switch to Deepstack 1.x here](https://github.com/khulnasoft/deepstack/tree/v1.x).
- **Technology agnostic:** Allow users the flexibility to decide what vendor or technology they want and make it easy to switch out any component for another. Deepstack allows you to use and compare models available from OpenAI, Cohere and Hugging Face, as well as your own local models or models hosted on Azure, Bedrock and SageMaker.
- **Explicit:** Make it transparent how different moving parts can โtalkโ to each other so it's easier to fit your tech stack and use case.
- **Flexible:** Deepstack provides all tooling in one place: database access, file conversion, cleaning, splitting, training, eval, inference, and more. And whenever custom behavior is desirable, it's easy to create custom components.
- **Extensible:** Provide a uniform and easy way for the community and third parties to build their own components and foster an open ecosystem around Deepstack.
Some examples of what you can do with Deepstack:
- Build **retrieval augmented generation (RAG)** by making use of one of the available vector databases and customizing your LLM interaction, the sky is the limit ๐
- Perform Question Answering **in natural language** to find granular answers in your documents.
- Perform **semantic search** and retrieve documents according to meaning.
- Build applications that can make complex decisions making to answer complex queries: such as systems that can resolve complex customer queries, do knowledge search on many disconnected resources and so on.
- Scale to millions of docs using retrievers and production-scale components.
- Use **off-the-shelf models** or **fine-tune** them to your data.
- Use **user feedback** to evaluate, benchmark, and continuously improve your models.
> [!TIP]
><img src="https://github.com/khulnasoft/deepstack/raw/main/docs/img/khulnasoft-cloud-logo-lightblue.png" width=30% height=30%>
>
> Are you looking for a managed solution that benefits from Deepstack? [khulnasoft Cloud](https://www.khulnasoft.com/khulnasoft-cloud?utm_campaign=developer-relations&utm_source=deepstack&utm_medium=readme) is our fully managed, end-to-end platform to integrate LLMs with your data, which uses Deepstack for the LLM pipelines architecture.
## Telemetry
Deepstack collects **anonymous** usage statistics of pipeline components. We receive an event every time these components are initialized. This way, we know which components are most relevant to our community.
Read more about telemetry in Deepstack or how you can opt out in [Deepstack docs](https://docs.deepstack.khulnasoft.com/v2.0/docs/telemetry).
## ๐ Community
If you have a feature request or a bug report, feel free to open an [issue in Github](https://github.com/khulnasoft/deepstack/issues). We regularly check these and you can expect a quick response. If you'd like to discuss a topic, or get more general advice on how to make Deepstack work for your project, you can start a thread in [Github Discussions](https://github.com/khulnasoft/deepstack/discussions) or our [Discord channel](https://discord.gg/deepstack). We also check [๐ (Twitter)](https://twitter.com/khulnasoft) and [Stack Overflow](https://stackoverflow.com/questions/tagged/deepstack).
## Contributing to Deepstack
We are very open to the community's contributions - be it a quick fix of a typo, or a completely new feature! You don't need to be a Deepstack expert to provide meaningful improvements. To learn how to get started, check out our [Contributor Guidelines](https://github.com/khulnasoft/deepstack/blob/main/CONTRIBUTING.md) first.
There are several ways you can contribute to Deepstack:
- Contribute to the main Deepstack project
- Contribute an integration on [deepstack-core-integrations](https://github.com/khulnasoft/deepstack-core-integrations)
> [!TIP]
>๐ **[Check out the full list of issues that are open to contributions](https://github.com/orgs/khulnasoft/projects/14)**
## Who Uses Deepstack
Here's a list of projects and companies using Deepstack. Want to add yours? Open a PR, add it to the list and let the
world know that you use Deepstack!
- [Airbus](https://www.airbus.com/en)
- [Alcatel-Lucent](https://www.al-enterprise.com/)
- [Apple](https://www.apple.com/)
- [BetterUp](https://www.betterup.com/)
- [Databricks](https://www.databricks.com/)
- [Khulnasoft](https://khulnasoft.com/)
- [Etalab](https://www.khulnasoft.com/blog/improving-on-site-search-for-government-agencies-etalab)
- [Infineon](https://www.infineon.com/)
- [Intel](https://github.com/intel/open-domain-question-and-answer#readme)
- [Intelijus](https://www.intelijus.ai/)
- [Intel Labs](https://github.com/IntelLabs/fastRAG#readme)
- [LEGO](https://github.com/larsbaunwall/bricky#readme)
- [Netflix](https://netflix.com)
- [Nvidia](https://developer.nvidia.com/blog/reducing-development-time-for-intelligent-virtual-assistants-in-contact-centers/)
- [PostHog](https://github.com/PostHog/max-ai#readme)
- [Rakuten](https://www.rakuten.com/)
- [Sooth.ai](https://www.khulnasoft.com/blog/advanced-neural-search-with-sooth-ai)
Raw data
{
"_id": null,
"home_page": null,
"name": "deepstack-ai",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "BERT, QA, Question-Answering, Reader, Retriever, albert, language-model, mrc, roberta, search, semantic-search, squad, transfer-learning, transformer",
"author": null,
"author_email": "\"khulnasoft.com\" <info@khulnasoft.com>",
"download_url": "https://files.pythonhosted.org/packages/78/65/eaa5e6e001fefe4fadf6c74d1a29c99218291a20ff0b25ddffafe33b8c13/deepstack_ai-2.2.0rc0.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n <a href=\"https://deepstack.khulnasoft.com/\"><img src=\"https://github.com/khulnasoft/deepstack/blob/main/docs/img/banner_20.png\" alt=\"Green logo of a stylized white 'H' with the text 'Deepstack, by khulnasoft. Deepstack 2.0 is live \ud83c\udf89'\u00a0Abstract green and yellow diagrams in the background.\"></a>\n\n| | |\n| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |\n| CI/CD | [](https://github.com/khulnasoft/deepstack/actions/workflows/tests.yml) [](https://github.com/psf/black) [](https://github.com/python/mypy) [](https://coveralls.io/github/khulnasoft/deepstack?branch=main) |\n| Docs | [](https://docs.deepstack.khulnasoft.com) |\n| Package | [](https://pypi.org/project/deepstack-ai/)   [](https://anaconda.org/conda-forge/deepstack-ai) [](LICENSE) [](https://github.com/khulnasoft/deepstack/actions/workflows/license_compliance.yml) |\n| Meta | [](https://discord.gg/deepstack) [](https://twitter.com/khulnasoft) |\n</div>\n\n[Deepstack](https://deepstack.khulnasoft.com/) is an end-to-end LLM framework that allows you to build applications powered by\nLLMs, Transformer models, vector search and more. Whether you want to perform retrieval-augmented generation (RAG),\ndocument search, question answering or answer generation, Deepstack can orchestrate state-of-the-art embedding models\nand LLMs into pipelines to build end-to-end NLP applications and solve your use case.\n\n## Installation\n\nThe simplest way to get Deepstack is via pip:\n\n```sh\npip install deepstack-ai\n```\n\nDeepstack supports multiple installation methods including Docker images. For a comprehensive guide please refer\nto the [documentation](https://docs.deepstack.khulnasoft.com/v2.0/docs/installation).\n\n## Documentation\n\nIf you're new to the project, check out [\"What is Deepstack?\"](https://deepstack.khulnasoft.com/overview/intro) then go\nthrough the [\"Get Started Guide\"](https://deepstack.khulnasoft.com/overview/quick-start) and build your first LLM application\nin a matter of minutes. Keep learning with the [tutorials](https://deepstack.khulnasoft.com/tutorials?v=2.0). For more advanced\nuse cases, or just to get some inspiration, you can browse our Deepstack recipes in the\n[Cookbook](https://github.com/khulnasoft/deepstack-cookbook).\n\nAt any given point, hit the [documentation](https://docs.deepstack.khulnasoft.com/v2.0/docs/intro) to learn more about Deepstack, what can it do for you and the technology behind.\n\n## Features\n\n> [!IMPORTANT]\n> **You are currently looking at the readme of Deepstack 2.0**. We are still maintaining Deepstack 1.x to give everyone\n> enough time to migrate to 2.0. [Switch to Deepstack 1.x here](https://github.com/khulnasoft/deepstack/tree/v1.x).\n\n- **Technology agnostic:** Allow users the flexibility to decide what vendor or technology they want and make it easy to switch out any component for another. Deepstack allows you to use and compare models available from OpenAI, Cohere and Hugging Face, as well as your own local models or models hosted on Azure, Bedrock and SageMaker.\n- **Explicit:** Make it transparent how different moving parts can \u201ctalk\u201d to each other so it's easier to fit your tech stack and use case.\n- **Flexible:** Deepstack provides all tooling in one place: database access, file conversion, cleaning, splitting, training, eval, inference, and more. And whenever custom behavior is desirable, it's easy to create custom components.\n- **Extensible:** Provide a uniform and easy way for the community and third parties to build their own components and foster an open ecosystem around Deepstack.\n\nSome examples of what you can do with Deepstack:\n\n- Build **retrieval augmented generation (RAG)** by making use of one of the available vector databases and customizing your LLM interaction, the sky is the limit \ud83d\ude80\n- Perform Question Answering **in natural language** to find granular answers in your documents.\n- Perform **semantic search** and retrieve documents according to meaning.\n- Build applications that can make complex decisions making to answer complex queries: such as systems that can resolve complex customer queries, do knowledge search on many disconnected resources and so on.\n- Scale to millions of docs using retrievers and production-scale components.\n- Use **off-the-shelf models** or **fine-tune** them to your data.\n- Use **user feedback** to evaluate, benchmark, and continuously improve your models.\n\n> [!TIP]\n><img src=\"https://github.com/khulnasoft/deepstack/raw/main/docs/img/khulnasoft-cloud-logo-lightblue.png\" width=30% height=30%>\n>\n> Are you looking for a managed solution that benefits from Deepstack? [khulnasoft Cloud](https://www.khulnasoft.com/khulnasoft-cloud?utm_campaign=developer-relations&utm_source=deepstack&utm_medium=readme) is our fully managed, end-to-end platform to integrate LLMs with your data, which uses Deepstack for the LLM pipelines architecture.\n\n## Telemetry\n\nDeepstack collects **anonymous** usage statistics of pipeline components. We receive an event every time these components are initialized. This way, we know which components are most relevant to our community.\n\nRead more about telemetry in Deepstack or how you can opt out in [Deepstack docs](https://docs.deepstack.khulnasoft.com/v2.0/docs/telemetry).\n\n## \ud83d\udd96 Community\n\nIf you have a feature request or a bug report, feel free to open an [issue in Github](https://github.com/khulnasoft/deepstack/issues). We regularly check these and you can expect a quick response. If you'd like to discuss a topic, or get more general advice on how to make Deepstack work for your project, you can start a thread in [Github Discussions](https://github.com/khulnasoft/deepstack/discussions) or our [Discord channel](https://discord.gg/deepstack). We also check [\ud835\udd4f (Twitter)](https://twitter.com/khulnasoft) and [Stack Overflow](https://stackoverflow.com/questions/tagged/deepstack).\n\n## Contributing to Deepstack\n\nWe are very open to the community's contributions - be it a quick fix of a typo, or a completely new feature! You don't need to be a Deepstack expert to provide meaningful improvements. To learn how to get started, check out our [Contributor Guidelines](https://github.com/khulnasoft/deepstack/blob/main/CONTRIBUTING.md) first.\n\nThere are several ways you can contribute to Deepstack:\n- Contribute to the main Deepstack project\n- Contribute an integration on [deepstack-core-integrations](https://github.com/khulnasoft/deepstack-core-integrations)\n\n> [!TIP]\n>\ud83d\udc49 **[Check out the full list of issues that are open to contributions](https://github.com/orgs/khulnasoft/projects/14)**\n\n## Who Uses Deepstack\n\nHere's a list of projects and companies using Deepstack. Want to add yours? Open a PR, add it to the list and let the\nworld know that you use Deepstack!\n\n- [Airbus](https://www.airbus.com/en)\n- [Alcatel-Lucent](https://www.al-enterprise.com/)\n- [Apple](https://www.apple.com/)\n- [BetterUp](https://www.betterup.com/)\n- [Databricks](https://www.databricks.com/)\n- [Khulnasoft](https://khulnasoft.com/)\n- [Etalab](https://www.khulnasoft.com/blog/improving-on-site-search-for-government-agencies-etalab)\n- [Infineon](https://www.infineon.com/)\n- [Intel](https://github.com/intel/open-domain-question-and-answer#readme)\n- [Intelijus](https://www.intelijus.ai/)\n- [Intel Labs](https://github.com/IntelLabs/fastRAG#readme)\n- [LEGO](https://github.com/larsbaunwall/bricky#readme)\n- [Netflix](https://netflix.com)\n- [Nvidia](https://developer.nvidia.com/blog/reducing-development-time-for-intelligent-virtual-assistants-in-contact-centers/)\n- [PostHog](https://github.com/PostHog/max-ai#readme)\n- [Rakuten](https://www.rakuten.com/)\n- [Sooth.ai](https://www.khulnasoft.com/blog/advanced-neural-search-with-sooth-ai)\n",
"bugtrack_url": null,
"license": null,
"summary": "LLM framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data.",
"version": "2.2.0rc0",
"project_urls": {
"CI: GitHub": "https://github.com/khulnasoft/deepstack/actions",
"Docs: RTD": "https://deepstack.khulnasoft.com/overview/intro",
"GitHub: issues": "https://github.com/khulnasoft/deepstack/issues",
"GitHub: repo": "https://github.com/khulnasoft/deepstack",
"Homepage": "https://github.com/khulnasoft/deepstack"
},
"split_keywords": [
"bert",
" qa",
" question-answering",
" reader",
" retriever",
" albert",
" language-model",
" mrc",
" roberta",
" search",
" semantic-search",
" squad",
" transfer-learning",
" transformer"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a82b0a41973c26f9ccfca1b3897971422042013ce61dadcfd33c8c86de45171b",
"md5": "27f9018ae0af040a651a476e42e23651",
"sha256": "1ea997a7787ac3ba6583b2ec6ad9a32c7b14fe8596dcc5e610432ba9c49dfd04"
},
"downloads": -1,
"filename": "deepstack_ai-2.2.0rc0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "27f9018ae0af040a651a476e42e23651",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 320941,
"upload_time": "2024-05-04T08:05:10",
"upload_time_iso_8601": "2024-05-04T08:05:10.640063Z",
"url": "https://files.pythonhosted.org/packages/a8/2b/0a41973c26f9ccfca1b3897971422042013ce61dadcfd33c8c86de45171b/deepstack_ai-2.2.0rc0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7865eaa5e6e001fefe4fadf6c74d1a29c99218291a20ff0b25ddffafe33b8c13",
"md5": "ef4cb142f6621ffafbf24e24756221e5",
"sha256": "f07723f10a3eaae2a54e30bf367fbc1f6ed9487c29e0d4515b1cee21ce2974ae"
},
"downloads": -1,
"filename": "deepstack_ai-2.2.0rc0.tar.gz",
"has_sig": false,
"md5_digest": "ef4cb142f6621ffafbf24e24756221e5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 206661,
"upload_time": "2024-05-04T08:05:12",
"upload_time_iso_8601": "2024-05-04T08:05:12.388739Z",
"url": "https://files.pythonhosted.org/packages/78/65/eaa5e6e001fefe4fadf6c74d1a29c99218291a20ff0b25ddffafe33b8c13/deepstack_ai-2.2.0rc0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-04 08:05:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "khulnasoft",
"github_project": "deepstack",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "deepstack-ai"
}
JSON
