sparrow-python


Namesparrow-python JSON
Version 0.4.2 PyPI version JSON
download
home_pageNone
SummaryNone
upload_time2025-03-03 05:40:50
maintainerNone
docs_urlNone
authorNone
requires_python>=3.10
licenseNone
keywords machine learning cli cv nlp
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # sparrow-python

[![image](https://img.shields.io/badge/Pypi-0.1.7-green.svg)](https://pypi.org/project/sparrow-python)
[![image](https://img.shields.io/badge/python-3.6+-blue.svg)](https://www.python.org/)
[![image](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

---

## TODO

- [ ] prompt调试页面
- [ ] 相关配置指定支持:prompt后端地址;模型参数配置;
- [ ] 
- [ ] 添加测试按钮,模型选项,模型配置
- [ ] 原生git下载支持
- [ ]
- [X] streamlit 多模态chat input: https://github.com/streamlit/streamlit/issues/7409
- [ ] from .cv.image.image_processor import messages_preprocess 添加是否对网络url替换为base64的控制;添加对video切帧的支持
- [ ] https://github.com/hiyouga/LLaMA-Factory/blob/main/src/llamafactory/chat/vllm_engine.py#L99

识别下面链接的滚动截图:
https://sjh.baidu.com/site/dzfmws.cn/da721a31-476d-42ed-aad1-81c2dc3a66a3

vllm 异步推理示例:

```python
from fastapi import FastAPI
from pydantic import BaseModel
from typing import List
import uvicorn
from vllm.engine.arg_utils import AsyncEngineArgs
from vllm.engine.async_llm_engine import AsyncLLMEngine
from vllm.sampling_params import SamplingParams
import torch

# Define request data model
class RequestData(BaseModel):
    prompts: List[str]
    max_tokens: int = 2048
    temperature: float = 0.7

# Initialize FastAPI app
app = FastAPI()

# Determine device
device = "cuda" if torch.cuda.is_available() else "cpu"

# Initialize AsyncLLMEngine
engine_args = AsyncEngineArgs(
    model="your-model-name",  # Replace with your model name
    dtype="bfloat16",
    gpu_memory_utilization=0.8,
    max_model_len=4096,
    trust_remote_code=True
)
llm_engine = AsyncLLMEngine.from_engine_args(engine_args)

# Define the inference endpoint
@app.post("/predict")
async def generate_text(data: RequestData):
    sampling_params = SamplingParams(
        max_tokens=data.max_tokens,
        temperature=data.temperature
    )
    request_id = "unique_request_id"  # Generate a unique request ID
    results_generator = llm_engine.generate(data.prompts, sampling_params, request_id)
  
    final_output = None
    async for request_output in results_generator:
        final_output = request_output
  
    assert final_output is not None
    text_outputs = [output.text for output in final_output.outputs]
    return {"responses": text_outputs}

# Run the server
if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)

```

## 待添加脚本

## Install

```bash
pip install sparrow-python
# Or dev version
pip install sparrow-python[dev]
# Or
pip install -e .
# Or
pip install -e .[dev]
```

## Usage

### Multiprocessing SyncManager

Open server first:

```bash
$ spr start-server
```

The defualt port `50001`.

(Process1) productor:

```python
from sparrow.multiprocess.client import Client

client = Client(port=50001)
client.update_dict({'a': 1, 'b': 2})
```

(Process2) consumer:

```python
from sparrow.multiprocess.client import Client

client = Client(port=50001)
print(client.get_dict_data())

>> > {'a': 1, 'b': 2}
```

### Common tools

- **Kill process by port**

```bash
$ spr kill {port}
```

- **pack & unpack**
  support archive format: "zip", "tar", "gztar", "bztar", or "xztar".

```bash
$ spr pack pack_dir
```

```bash
$ spr unpack filename extract_dir
```

- **Scaffold**

```bash
$ spr create awosome-project
```

### Some useful functions

> `sparrow.relp`
> Relative path, which is used to read or save files more easily.

> `sparrow.performance.MeasureTime`
> For measuring time (including gpu time)

> `sparrow.performance.get_process_memory`
> Get the memory size occupied by the process

> `sparrow.performance.get_virtual_memory`
> Get virtual machine memory information

> `sparrow.add_env_path`
> Add python environment variable (use relative file path)

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "sparrow-python",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.10",
    "maintainer_email": null,
    "keywords": "Machine Learning, cli, cv, nlp",
    "author": null,
    "author_email": "kunyuan <beidongjiedeguang@gmail.com>",
    "download_url": "https://files.pythonhosted.org/packages/60/2b/97a5557fe034f163c47157bd1b0c5e91a759792a7522a3c1a26247bbb928/sparrow_python-0.4.2.tar.gz",
    "platform": null,
    "description": "# sparrow-python\n\n[![image](https://img.shields.io/badge/Pypi-0.1.7-green.svg)](https://pypi.org/project/sparrow-python)\n[![image](https://img.shields.io/badge/python-3.6+-blue.svg)](https://www.python.org/)\n[![image](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)\n\n---\n\n## TODO\n\n- [ ] prompt\u8c03\u8bd5\u9875\u9762\n- [ ] \u76f8\u5173\u914d\u7f6e\u6307\u5b9a\u652f\u6301\uff1aprompt\u540e\u7aef\u5730\u5740\uff1b\u6a21\u578b\u53c2\u6570\u914d\u7f6e\uff1b\n- [ ] \n- [ ] \u6dfb\u52a0\u6d4b\u8bd5\u6309\u94ae\uff0c\u6a21\u578b\u9009\u9879\uff0c\u6a21\u578b\u914d\u7f6e\n- [ ] \u539f\u751fgit\u4e0b\u8f7d\u652f\u6301\n- [ ]\n- [X] streamlit \u591a\u6a21\u6001chat input: https://github.com/streamlit/streamlit/issues/7409\n- [ ] from .cv.image.image_processor import messages_preprocess \u6dfb\u52a0\u662f\u5426\u5bf9\u7f51\u7edcurl\u66ff\u6362\u4e3abase64\u7684\u63a7\u5236\uff1b\u6dfb\u52a0\u5bf9video\u5207\u5e27\u7684\u652f\u6301\n- [ ] https://github.com/hiyouga/LLaMA-Factory/blob/main/src/llamafactory/chat/vllm_engine.py#L99\n\n\u8bc6\u522b\u4e0b\u9762\u94fe\u63a5\u7684\u6eda\u52a8\u622a\u56fe\uff1a\nhttps://sjh.baidu.com/site/dzfmws.cn/da721a31-476d-42ed-aad1-81c2dc3a66a3\n\nvllm \u5f02\u6b65\u63a8\u7406\u793a\u4f8b\uff1a\n\n```python\nfrom fastapi import FastAPI\nfrom pydantic import BaseModel\nfrom typing import List\nimport uvicorn\nfrom vllm.engine.arg_utils import AsyncEngineArgs\nfrom vllm.engine.async_llm_engine import AsyncLLMEngine\nfrom vllm.sampling_params import SamplingParams\nimport torch\n\n# Define request data model\nclass RequestData(BaseModel):\n    prompts: List[str]\n    max_tokens: int = 2048\n    temperature: float = 0.7\n\n# Initialize FastAPI app\napp = FastAPI()\n\n# Determine device\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n# Initialize AsyncLLMEngine\nengine_args = AsyncEngineArgs(\n    model=\"your-model-name\",  # Replace with your model name\n    dtype=\"bfloat16\",\n    gpu_memory_utilization=0.8,\n    max_model_len=4096,\n    trust_remote_code=True\n)\nllm_engine = AsyncLLMEngine.from_engine_args(engine_args)\n\n# Define the inference endpoint\n@app.post(\"/predict\")\nasync def generate_text(data: RequestData):\n    sampling_params = SamplingParams(\n        max_tokens=data.max_tokens,\n        temperature=data.temperature\n    )\n    request_id = \"unique_request_id\"  # Generate a unique request ID\n    results_generator = llm_engine.generate(data.prompts, sampling_params, request_id)\n  \n    final_output = None\n    async for request_output in results_generator:\n        final_output = request_output\n  \n    assert final_output is not None\n    text_outputs = [output.text for output in final_output.outputs]\n    return {\"responses\": text_outputs}\n\n# Run the server\nif __name__ == \"__main__\":\n    uvicorn.run(app, host=\"0.0.0.0\", port=8000)\n\n```\n\n## \u5f85\u6dfb\u52a0\u811a\u672c\n\n## Install\n\n```bash\npip install sparrow-python\n# Or dev version\npip install sparrow-python[dev]\n# Or\npip install -e .\n# Or\npip install -e .[dev]\n```\n\n## Usage\n\n### Multiprocessing SyncManager\n\nOpen server first:\n\n```bash\n$ spr start-server\n```\n\nThe defualt port `50001`.\n\n(Process1) productor:\n\n```python\nfrom sparrow.multiprocess.client import Client\n\nclient = Client(port=50001)\nclient.update_dict({'a': 1, 'b': 2})\n```\n\n(Process2) consumer:\n\n```python\nfrom sparrow.multiprocess.client import Client\n\nclient = Client(port=50001)\nprint(client.get_dict_data())\n\n>> > {'a': 1, 'b': 2}\n```\n\n### Common tools\n\n- **Kill process by port**\n\n```bash\n$ spr kill {port}\n```\n\n- **pack & unpack**\n  support archive format: \"zip\", \"tar\", \"gztar\", \"bztar\", or \"xztar\".\n\n```bash\n$ spr pack pack_dir\n```\n\n```bash\n$ spr unpack filename extract_dir\n```\n\n- **Scaffold**\n\n```bash\n$ spr create awosome-project\n```\n\n### Some useful functions\n\n> `sparrow.relp`\n> Relative path, which is used to read or save files more easily.\n\n> `sparrow.performance.MeasureTime`\n> For measuring time (including gpu time)\n\n> `sparrow.performance.get_process_memory`\n> Get the memory size occupied by the process\n\n> `sparrow.performance.get_virtual_memory`\n> Get virtual machine memory information\n\n> `sparrow.add_env_path`\n> Add python environment variable (use relative file path)\n",
    "bugtrack_url": null,
    "license": null,
    "summary": null,
    "version": "0.4.2",
    "project_urls": {
        "Issues": "https://github.com/beidongjiedeguang/sparrow/issues",
        "Source": "https://github.com/beidongjiedeguang/sparrow",
        "documentation": "https://github.com/beidongjiedeguang/sparrow#sparrow_python",
        "homepage": "https://github.com/beidongjiedeguang/sparrow",
        "repository": "https://github.com/beidongjiedeguang/sparrow"
    },
    "split_keywords": [
        "machine learning",
        " cli",
        " cv",
        " nlp"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "79ee21718854138087e524a19f074f6bd6c6feb26795dcae3d881b21e6b32e80",
                "md5": "6064ec6d728b44df0ed2dbde83986273",
                "sha256": "523b242b15e794bf5aa56932e071bf77fd822c4a1263ba37962d529c1458f7ac"
            },
            "downloads": -1,
            "filename": "sparrow_python-0.4.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "6064ec6d728b44df0ed2dbde83986273",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.10",
            "size": 931723,
            "upload_time": "2025-03-03T05:40:47",
            "upload_time_iso_8601": "2025-03-03T05:40:47.966219Z",
            "url": "https://files.pythonhosted.org/packages/79/ee/21718854138087e524a19f074f6bd6c6feb26795dcae3d881b21e6b32e80/sparrow_python-0.4.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "602b97a5557fe034f163c47157bd1b0c5e91a759792a7522a3c1a26247bbb928",
                "md5": "9fc172289e86c11ea3a42d8015ad4aca",
                "sha256": "8ad14e12ab3646da20d01b96dba99bbaf221a1bd80991df263bf1558d8615c25"
            },
            "downloads": -1,
            "filename": "sparrow_python-0.4.2.tar.gz",
            "has_sig": false,
            "md5_digest": "9fc172289e86c11ea3a42d8015ad4aca",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.10",
            "size": 885488,
            "upload_time": "2025-03-03T05:40:50",
            "upload_time_iso_8601": "2025-03-03T05:40:50.519972Z",
            "url": "https://files.pythonhosted.org/packages/60/2b/97a5557fe034f163c47157bd1b0c5e91a759792a7522a3c1a26247bbb928/sparrow_python-0.4.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-03-03 05:40:50",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "beidongjiedeguang",
    "github_project": "sparrow",
    "github_not_found": true,
    "lcname": "sparrow-python"
}
        
Elapsed time: 0.97839s