| Name | pebblo-sample JSON |
| Version |
0.1.11
JSON |
| download |
| home_page | |
| Summary | Pebblo Gen-AI Data Analyzer |
| upload_time | 2024-02-27 06:30:52 |
| maintainer | |
| docs_url | None |
| author | |
| requires_python | >=3.9 |
| license | |
| keywords |
langchain
ai
rag
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<p align="center">
<img src="https://github.com/daxa-ai/pebblo/blob/main/docs/gh_pages/static/img/pebblo-logo-name.jpg?raw=true" />
</p>
---
[](https://github.com/daxa-ai/pebblo)
[](http://opensource.org/licenses/MIT)
[](https://daxa-ai.github.io/pebblo/)
[](https://pypi.org/project/pebblo/)
[](https://discord.gg/wyAfaYXwwv)
[](https://twitter.com/daxa_ai)
---
**Pebblo** enables developers to safely load data and promote their Gen AI app to deployment without worrying about the organization’s compliance and security requirements. The project identifies semantic topics and entities found in the loaded data and summarizes them on the UI or a PDF report.
Pebblo has two components.
1. Pebblo Daemon - a REST api application with topic-classifier, entity-classifier and reporting features
1. Pebblo Safe DataLoader - a thin wrapper to Gen-AI framework's data loaders
## Pebblo Daemon
### Installation
#### Pre-requisites
##### Mac OSX
```bash
brew install pango
```
##### Linux (debian/ubuntu)
```bash
sudo apt-get install libpango-1.0-0 libpangoft2-1.0-0
```
#### Install Pebblo Daemon
```bash
pip install pebblo
```
#### Run Pebblo daemon
```bash
pebblo
```
see [troubleshooting](docs/gh_pages/docs/troubleshooting.md) guide for troubleshooting info.
Pebblo daemon now listens to `localhost:8000` to accept Gen-AI application data snippets for inspection and reporting.
#### Pebblo Optional Flags
- `--config <file>`: Specifies a custom configuration file in yaml format.
```bash
pebblo --config config.yaml
````
## Pebblo Safe DataLoader
### Langchain
`Pebblo Safe DataLoader` is natively supported in Langchain framework. It is available in Langchain versions `>=0.1.7`
#### Enable Pebblo in Langchain Application
Add `PebbloSafeLoader` wrapper to the existing Langchain document loader(s) used in the RAG application. `PebbloSafeLoader` is interface compatible with Langchain `BaseLoader`. The application can continue to use `load()` and `lazy_load()` methods as it would on an Langchain document loader.
Here is the snippet of Lanchain RAG application using `CSVLoader` before enabling `PebbloSafeLoader`.
```python
from langchain.document_loaders.csv_loader import CSVLoader
loader = CSVLoader(file_path)
documents = loader.load()
vectordb = Chroma.from_documents(documents, OpenAIEmbeddings())
```
The Pebblo SafeLoader can be enabled with few lines of code change to the above snippet.
```python
from langchain.document_loaders.csv_loader import CSVLoader
from langchain_community.document_loaders.pebblo import PebbloSafeLoader
loader = PebbloSafeLoader(
CSVLoader(file_path),
name="acme-corp-rag-1", # App name (Mandatory)
owner="Joe Smith", # Owner (Optional)
description="Support productivity RAG application", # Description (Optional)
)
documents = loader.load()
vectordb = Chroma.from_documents(documents, OpenAIEmbeddings())
```
See [here](https://github.com/srics/pebblo/tree/main/samples) for samples with Pebblo enabled RAG applications and [this](https://daxa-ai.github.io/pebblo/rag) document for more details.
# Contribution
Pebblo is a open-source community project. If you want to contribute see [Contributor Guidelines](https://github.com/daxa-ai/pebblo/blob/main/CONTRIBUTING.md) for more details.
# License
Pebblo is released under the MIT License
Raw data
{
"_id": null,
"home_page": "",
"name": "pebblo-sample",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "Pebblo Maintainer <pebblo@daxa.ai>",
"keywords": "langchain,ai,rag",
"author": "",
"author_email": "Pebblo Authors <pebblo@daxa.ai>",
"download_url": "",
"platform": null,
"description": "<p align=\"center\">\n <img src=\"https://github.com/daxa-ai/pebblo/blob/main/docs/gh_pages/static/img/pebblo-logo-name.jpg?raw=true\" />\n</p>\n\n---\n[](https://github.com/daxa-ai/pebblo)\n[](http://opensource.org/licenses/MIT)\n[](https://daxa-ai.github.io/pebblo/)\n\n[](https://pypi.org/project/pebblo/)\n\n\n\n[](https://discord.gg/wyAfaYXwwv)\n[](https://twitter.com/daxa_ai)\n---\n\n\n**Pebblo** enables developers to safely load data and promote their Gen AI app to deployment without worrying about the organization\u2019s compliance and security requirements. The project identifies semantic topics and entities found in the loaded data and summarizes them on the UI or a PDF report.\n\nPebblo has two components.\n\n1. Pebblo Daemon - a REST api application with topic-classifier, entity-classifier and reporting features\n1. Pebblo Safe DataLoader - a thin wrapper to Gen-AI framework's data loaders\n\n## Pebblo Daemon\n\n### Installation\n\n#### Pre-requisites\n\n##### Mac OSX\n\n```bash\nbrew install pango\n```\n\n##### Linux (debian/ubuntu)\n\n```bash\nsudo apt-get install libpango-1.0-0 libpangoft2-1.0-0\n```\n\n#### Install Pebblo Daemon\n\n```bash\npip install pebblo\n```\n\n#### Run Pebblo daemon\n\n```bash\npebblo\n```\nsee [troubleshooting](docs/gh_pages/docs/troubleshooting.md) guide for troubleshooting info.\n\nPebblo daemon now listens to `localhost:8000` to accept Gen-AI application data snippets for inspection and reporting.\n\n#### Pebblo Optional Flags\n\n- `--config <file>`: Specifies a custom configuration file in yaml format.\n\n```bash\npebblo --config config.yaml\n````\n\n## Pebblo Safe DataLoader\n\n### Langchain\n\n`Pebblo Safe DataLoader` is natively supported in Langchain framework. It is available in Langchain versions `>=0.1.7`\n\n#### Enable Pebblo in Langchain Application\n\nAdd `PebbloSafeLoader` wrapper to the existing Langchain document loader(s) used in the RAG application. `PebbloSafeLoader` is interface compatible with Langchain `BaseLoader`. The application can continue to use `load()` and `lazy_load()` methods as it would on an Langchain document loader.\n\nHere is the snippet of Lanchain RAG application using `CSVLoader` before enabling `PebbloSafeLoader`.\n\n```python\n from langchain.document_loaders.csv_loader import CSVLoader\n\n loader = CSVLoader(file_path)\n documents = loader.load()\n vectordb = Chroma.from_documents(documents, OpenAIEmbeddings())\n```\n\nThe Pebblo SafeLoader can be enabled with few lines of code change to the above snippet.\n\n```python\n from langchain.document_loaders.csv_loader import CSVLoader\n from langchain_community.document_loaders.pebblo import PebbloSafeLoader\n\n loader = PebbloSafeLoader(\n CSVLoader(file_path),\n name=\"acme-corp-rag-1\", # App name (Mandatory)\n owner=\"Joe Smith\", # Owner (Optional)\n description=\"Support productivity RAG application\", # Description (Optional)\n )\n documents = loader.load()\n vectordb = Chroma.from_documents(documents, OpenAIEmbeddings())\n```\n\nSee [here](https://github.com/srics/pebblo/tree/main/samples) for samples with Pebblo enabled RAG applications and [this](https://daxa-ai.github.io/pebblo/rag) document for more details.\n\n# Contribution\n\nPebblo is a open-source community project. If you want to contribute see [Contributor Guidelines](https://github.com/daxa-ai/pebblo/blob/main/CONTRIBUTING.md) for more details.\n\n# License\n\nPebblo is released under the MIT License\n",
"bugtrack_url": null,
"license": "",
"summary": "Pebblo Gen-AI Data Analyzer",
"version": "0.1.11",
"project_urls": {
"Bug Reports": "https://github.com/daxa-ai/pebblo/issues",
"Funding": "https://donate.pypi.org",
"Homepage": "https://github.com/daxa-ai/pebblo",
"Source": "https://github.com/daxa-ai/pebblo/"
},
"split_keywords": [
"langchain",
"ai",
"rag"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cc325b2af86205448c04627a7762104d666d47e06e8428951db6d719db35a71b",
"md5": "f41238321254bdfda8b636dca2a1718c",
"sha256": "88890fb8d8ea34b39b6d06f3a624f1ba4b06a8fa9ca00140599fc231d26100b5"
},
"downloads": -1,
"filename": "pebblo_sample-0.1.11-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f41238321254bdfda8b636dca2a1718c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 3072721,
"upload_time": "2024-02-27T06:30:52",
"upload_time_iso_8601": "2024-02-27T06:30:52.370396Z",
"url": "https://files.pythonhosted.org/packages/cc/32/5b2af86205448c04627a7762104d666d47e06e8428951db6d719db35a71b/pebblo_sample-0.1.11-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-27 06:30:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "daxa-ai",
"github_project": "pebblo",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pebblo-sample"
}
