| Name | llama-index-readers-airbyte-salesforce JSON |
| Version |
0.3.0
JSON |
| download |
| home_page | None |
| Summary | llama-index readers airbyte_salesforce integration |
| upload_time | 2024-11-18 01:01:01 |
| maintainer | flash1293 |
| docs_url | None |
| author | Your Name |
| requires_python | <4.0,>=3.9 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Airbyte Salesforce Loader
```bash
pip install llama-index-readers-airbyte-salesforce
```
The Airbyte Salesforce Loader allows you to access different Salesforce objects.
## Usage
Here's an example usage of the AirbyteSalesforceReader.
```python
from llama_index.readers.airbyte_salesforce import AirbyteSalesforceReader
salesforce_config = {
# ...
}
reader = AirbyteSalesforceReader(config=salesforce_config)
documents = reader.load_data(stream_name="asset")
```
## Configuration
Check out the [Airbyte documentation page](https://docs.airbyte.com/integrations/sources/salesforce/) for details about how to configure the reader.
The JSON schema the config object should adhere to can be found on Github: [https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-salesforce/source_salesforce/spec.yaml](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-salesforce/source_salesforce/spec.yaml).
The general shape looks like this:
```python
{
"client_id": "<oauth client id>",
"client_secret": "<oauth client secret>",
"refresh_token": "<oauth refresh token>",
"start_date": "<date from which to start retrieving records from in ISO format, e.g. 2020-10-20T00:00:00Z>",
"is_sandbox": False, # set to True if you're using a sandbox environment
"streams_criteria": [ # Array of filters for salesforce objects that should be loadable
{
"criteria": "exacts",
"value": "Account",
}, # Exact name of salesforce object
{"criteria": "starts with", "value": "Asset"}, # Prefix of the name
# Other allowed criteria: ends with, contains, starts not with, ends not with, not contains, not exacts
],
}
```
By default all fields are stored as metadata in the documents and the text is set to the JSON representation of all the fields. Construct the text of the document by passing a `record_handler` to the reader:
```python
def handle_record(record, id):
return Document(
doc_id=id, text=record.data["title"], extra_info=record.data
)
reader = AirbyteSalesforceReader(
config=salesforce_config, record_handler=handle_record
)
```
## Lazy loads
The `reader.load_data` endpoint will collect all documents and return them as a list. If there are a large number of documents, this can cause issues. By using `reader.lazy_load_data` instead, an iterator is returned which can be consumed document by document without the need to keep all documents in memory.
## Incremental loads
This loader supports loading data incrementally (only returning documents that weren't loaded last time or got updated in the meantime):
```python
reader = AirbyteSalesforceReader(config={...})
documents = reader.load_data(stream_name="asset")
current_state = reader.last_state # can be pickled away or stored otherwise
updated_documents = reader.load_data(
stream_name="asset", state=current_state
) # only loads documents that were updated since last time
```
This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-readers-airbyte-salesforce",
"maintainer": "flash1293",
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/2f/32/b8bd30448bc0e2de10e3f00f0a3f0421541fb1866e9e2ad8bdeffa921854/llama_index_readers_airbyte_salesforce-0.3.0.tar.gz",
"platform": null,
"description": "# Airbyte Salesforce Loader\n\n```bash\npip install llama-index-readers-airbyte-salesforce\n```\n\nThe Airbyte Salesforce Loader allows you to access different Salesforce objects.\n\n## Usage\n\nHere's an example usage of the AirbyteSalesforceReader.\n\n```python\nfrom llama_index.readers.airbyte_salesforce import AirbyteSalesforceReader\n\nsalesforce_config = {\n # ...\n}\nreader = AirbyteSalesforceReader(config=salesforce_config)\ndocuments = reader.load_data(stream_name=\"asset\")\n```\n\n## Configuration\n\nCheck out the [Airbyte documentation page](https://docs.airbyte.com/integrations/sources/salesforce/) for details about how to configure the reader.\nThe JSON schema the config object should adhere to can be found on Github: [https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-salesforce/source_salesforce/spec.yaml](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-salesforce/source_salesforce/spec.yaml).\n\nThe general shape looks like this:\n\n```python\n{\n \"client_id\": \"<oauth client id>\",\n \"client_secret\": \"<oauth client secret>\",\n \"refresh_token\": \"<oauth refresh token>\",\n \"start_date\": \"<date from which to start retrieving records from in ISO format, e.g. 2020-10-20T00:00:00Z>\",\n \"is_sandbox\": False, # set to True if you're using a sandbox environment\n \"streams_criteria\": [ # Array of filters for salesforce objects that should be loadable\n {\n \"criteria\": \"exacts\",\n \"value\": \"Account\",\n }, # Exact name of salesforce object\n {\"criteria\": \"starts with\", \"value\": \"Asset\"}, # Prefix of the name\n # Other allowed criteria: ends with, contains, starts not with, ends not with, not contains, not exacts\n ],\n}\n```\n\nBy default all fields are stored as metadata in the documents and the text is set to the JSON representation of all the fields. Construct the text of the document by passing a `record_handler` to the reader:\n\n```python\ndef handle_record(record, id):\n return Document(\n doc_id=id, text=record.data[\"title\"], extra_info=record.data\n )\n\n\nreader = AirbyteSalesforceReader(\n config=salesforce_config, record_handler=handle_record\n)\n```\n\n## Lazy loads\n\nThe `reader.load_data` endpoint will collect all documents and return them as a list. If there are a large number of documents, this can cause issues. By using `reader.lazy_load_data` instead, an iterator is returned which can be consumed document by document without the need to keep all documents in memory.\n\n## Incremental loads\n\nThis loader supports loading data incrementally (only returning documents that weren't loaded last time or got updated in the meantime):\n\n```python\nreader = AirbyteSalesforceReader(config={...})\ndocuments = reader.load_data(stream_name=\"asset\")\ncurrent_state = reader.last_state # can be pickled away or stored otherwise\n\nupdated_documents = reader.load_data(\n stream_name=\"asset\", state=current_state\n) # only loads documents that were updated since last time\n```\n\nThis loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index readers airbyte_salesforce integration",
"version": "0.3.0",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8b6942e19972cd052773caefb5ef6d8adec36b6f04578d24948ea298358680d1",
"md5": "2461c82bffb459e2b7f4ad31f85c14e9",
"sha256": "3859ccd5c068abdc52b659a86b0b62f2a8cfc0fc08c3c46338a7f280807f8fbc"
},
"downloads": -1,
"filename": "llama_index_readers_airbyte_salesforce-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2461c82bffb459e2b7f4ad31f85c14e9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 3303,
"upload_time": "2024-11-18T01:01:00",
"upload_time_iso_8601": "2024-11-18T01:01:00.705552Z",
"url": "https://files.pythonhosted.org/packages/8b/69/42e19972cd052773caefb5ef6d8adec36b6f04578d24948ea298358680d1/llama_index_readers_airbyte_salesforce-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2f32b8bd30448bc0e2de10e3f00f0a3f0421541fb1866e9e2ad8bdeffa921854",
"md5": "44dd4f73a528d1cb85ea33414258c8d5",
"sha256": "b692c2d84948f39985a027d000b472c7c2200e9a2afd5e25009fd318de5aa94f"
},
"downloads": -1,
"filename": "llama_index_readers_airbyte_salesforce-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "44dd4f73a528d1cb85ea33414258c8d5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 2954,
"upload_time": "2024-11-18T01:01:01",
"upload_time_iso_8601": "2024-11-18T01:01:01.602206Z",
"url": "https://files.pythonhosted.org/packages/2f/32/b8bd30448bc0e2de10e3f00f0a3f0421541fb1866e9e2ad8bdeffa921854/llama_index_readers_airbyte_salesforce-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-18 01:01:01",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-readers-airbyte-salesforce"
}
