crawlab-sdk

Name	crawlab-sdk JSON
Version	0.7.0 JSON
	download
home_page	None
Summary	Python SDK for Crawlab
upload_time	2025-02-14 05:49:48
maintainer	None
docs_url	None
author	Marvin Zhang
requires_python	<4.0,>=3.9
license	BSD-3-Clause
keywords
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            # Crawlab Python SDK

Python SDK for Crawlab

## Installation

```bash
pip install crawlab-sdk
```

## Usage

### CLI

```bash
crawlab-cli
```

### Scrapy Integration

In `settings.py`, add the following:

```python
ITEM_PIPELINES = {
    'crawlab.CrawlabPipeline': 300
}
```

### Save Scraped Items
    
```python
from crawlab import save_item

scraped_items = [
    {
        'name': 'item1',
        'value': 'value1'
    },
    {
        'name': 'item2',
        'value': 'value2'
    }
]

for item in scraped_items:
    save_item(item) 
```

## Development

### Pre-requisites

```bash
pip install poetry
```

### Install dependencies

```bash
poetry install
```


            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "crawlab-sdk",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<4.0,>=3.9",
    "maintainer_email": null,
    "keywords": null,
    "author": "Marvin Zhang",
    "author_email": "tikazyq@163.com",
    "download_url": "https://files.pythonhosted.org/packages/38/b8/eef82a08ab193017626f24aa7c965a7dc17b64727ad1e5bc3a2d2d208d1f/crawlab_sdk-0.7.0.tar.gz",
    "platform": null,
    "description": "# Crawlab Python SDK\n\nPython SDK for Crawlab\n\n## Installation\n\n```bash\npip install crawlab-sdk\n```\n\n## Usage\n\n### CLI\n\n```bash\ncrawlab-cli\n```\n\n### Scrapy Integration\n\nIn `settings.py`, add the following:\n\n```python\nITEM_PIPELINES = {\n    'crawlab.CrawlabPipeline': 300\n}\n```\n\n### Save Scraped Items\n    \n```python\nfrom crawlab import save_item\n\nscraped_items = [\n    {\n        'name': 'item1',\n        'value': 'value1'\n    },\n    {\n        'name': 'item2',\n        'value': 'value2'\n    }\n]\n\nfor item in scraped_items:\n    save_item(item) \n```\n\n## Development\n\n### Pre-requisites\n\n```bash\npip install poetry\n```\n\n### Install dependencies\n\n```bash\npoetry install\n```\n\n",
    "bugtrack_url": null,
    "license": "BSD-3-Clause",
    "summary": "Python SDK for Crawlab",
    "version": "0.7.0",
    "project_urls": {
        "Homepage": "https://github.com/crawlab-team/crawlab-python-sdk",
        "Repository": "https://github.com/crawlab-team/crawlab-python-sdk"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "e1a528016336f2a696996a7d0738c218d89e85206bafda6328c699e28a3a74c7",
                "md5": "e421d255c03be8bf02891057b398a9e5",
                "sha256": "6ff4a31cec73bfa38f98953c9166df950e3c80ae9212c4cca2116517d81a4bcf"
            },
            "downloads": -1,
            "filename": "crawlab_sdk-0.7.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "e421d255c03be8bf02891057b398a9e5",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<4.0,>=3.9",
            "size": 11538,
            "upload_time": "2025-02-14T05:49:46",
            "upload_time_iso_8601": "2025-02-14T05:49:46.233935Z",
            "url": "https://files.pythonhosted.org/packages/e1/a5/28016336f2a696996a7d0738c218d89e85206bafda6328c699e28a3a74c7/crawlab_sdk-0.7.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "38b8eef82a08ab193017626f24aa7c965a7dc17b64727ad1e5bc3a2d2d208d1f",
                "md5": "bff48eded1dc6765c15c9967bc110344",
                "sha256": "973c559311d5ed7ed40e48de55540cd69ff38c43c21737039fb0295bbfbe1e37"
            },
            "downloads": -1,
            "filename": "crawlab_sdk-0.7.0.tar.gz",
            "has_sig": false,
            "md5_digest": "bff48eded1dc6765c15c9967bc110344",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<4.0,>=3.9",
            "size": 7519,
            "upload_time": "2025-02-14T05:49:48",
            "upload_time_iso_8601": "2025-02-14T05:49:48.302643Z",
            "url": "https://files.pythonhosted.org/packages/38/b8/eef82a08ab193017626f24aa7c965a7dc17b64727ad1e5bc3a2d2d208d1f/crawlab_sdk-0.7.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-02-14 05:49:48",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "crawlab-team",
    "github_project": "crawlab-python-sdk",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "crawlab-sdk"
}