# sparrow-python
[](https://pypi.org/project/sparrow-python)
[](https://www.python.org/)
[](LICENSE)
-------------------------
## TODO
- [ ] from .cv.image.image_processor import messages_preprocess 添加是否对网络url替换为base64的控制；添加对video切帧的支持
- [ ] https://github.com/hiyouga/LLaMA-Factory/blob/main/src/llamafactory/chat/vllm_engine.py#L99
识别下面链接的滚动截图：
https://sjh.baidu.com/site/dzfmws.cn/da721a31-476d-42ed-aad1-81c2dc3a66a3
vllm 异步推理示例：
```python
from fastapi import FastAPI
from pydantic import BaseModel
from typing import List
import uvicorn
from vllm.engine.arg_utils import AsyncEngineArgs
from vllm.engine.async_llm_engine import AsyncLLMEngine
from vllm.sampling_params import SamplingParams
import torch
# Define request data model
class RequestData(BaseModel):
prompts: List[str]
max_tokens: int = 2048
temperature: float = 0.7
# Initialize FastAPI app
app = FastAPI()
# Determine device
device = "cuda" if torch.cuda.is_available() else "cpu"
# Initialize AsyncLLMEngine
engine_args = AsyncEngineArgs(
model="your-model-name", # Replace with your model name
dtype="bfloat16",
gpu_memory_utilization=0.8,
max_model_len=4096,
trust_remote_code=True
)
llm_engine = AsyncLLMEngine.from_engine_args(engine_args)
# Define the inference endpoint
@app.post("/predict")
async def generate_text(data: RequestData):
sampling_params = SamplingParams(
max_tokens=data.max_tokens,
temperature=data.temperature
)
request_id = "unique_request_id" # Generate a unique request ID
results_generator = llm_engine.generate(data.prompts, sampling_params, request_id)
final_output = None
async for request_output in results_generator:
final_output = request_output
assert final_output is not None
text_outputs = [output.text for output in final_output.outputs]
return {"responses": text_outputs}
# Run the server
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=8000)
```
## 待添加脚本
## Install
```bash
pip install sparrow-python
# Or dev version
pip install sparrow-python[dev]
# Or
pip install -e .
# Or
pip install -e .[dev]
```
## Usage
### Multiprocessing SyncManager
Open server first:
```bash
$ spr start-server
```
The defualt port `50001`.
(Process1) productor:
```python
from sparrow.multiprocess.client import Client
client = Client(port=50001)
client.update_dict({'a': 1, 'b': 2})
```
(Process2) consumer:
```python
from sparrow.multiprocess.client import Client
client = Client(port=50001)
print(client.get_dict_data())
>> > {'a': 1, 'b': 2}
```
### Common tools
- **Kill process by port**
```bash
$ spr kill {port}
```
- **pack & unpack**
support archive format: "zip", "tar", "gztar", "bztar", or "xztar".
```bash
$ spr pack pack_dir
```
```bash
$ spr unpack filename extract_dir
```
- **Scaffold**
```bash
$ spr create awosome-project
```
### Some useful functions
> `sparrow.relp`
> Relative path, which is used to read or save files more easily.
> `sparrow.performance.MeasureTime`
> For measuring time (including gpu time)
> `sparrow.performance.get_process_memory`
> Get the memory size occupied by the process
> `sparrow.performance.get_virtual_memory`
> Get virtual machine memory information
> `sparrow.add_env_path`
> Add python environment variable (use relative file path)
### Safe logger in `multiprocessing`
```python
from sparrow.log import Logger
import numpy as np
logger = Logger(name='train-log', log_dir='./logs', )
logger.info("hello", "numpy:", np.arange(10))
logger2 = Logger.get_logger('train-log')
print(id(logger2) == id(logger))
>> > True
```
Raw data
{
"_id": null,
"home_page": null,
"name": "sparrow-python",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "Machine Learning, cli, cv, nlp",
"author": null,
"author_email": "kunyuan <beidongjiedeguang@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/17/56/3635bbe2ff661d71855b1f3d9c931fc663dc6ddb6c154c2af735f359ea18/sparrow_python-0.3.22.tar.gz",
"platform": null,
"description": "# sparrow-python\n[](https://pypi.org/project/sparrow-python)\n[](https://www.python.org/)\n[](LICENSE)\n\n\n-------------------------\n## TODO\n- [ ] from .cv.image.image_processor import messages_preprocess \u6dfb\u52a0\u662f\u5426\u5bf9\u7f51\u7edcurl\u66ff\u6362\u4e3abase64\u7684\u63a7\u5236\uff1b\u6dfb\u52a0\u5bf9video\u5207\u5e27\u7684\u652f\u6301\n- [ ] https://github.com/hiyouga/LLaMA-Factory/blob/main/src/llamafactory/chat/vllm_engine.py#L99 \n\n\u8bc6\u522b\u4e0b\u9762\u94fe\u63a5\u7684\u6eda\u52a8\u622a\u56fe\uff1a \nhttps://sjh.baidu.com/site/dzfmws.cn/da721a31-476d-42ed-aad1-81c2dc3a66a3 \n\n\nvllm \u5f02\u6b65\u63a8\u7406\u793a\u4f8b\uff1a\n\n```python\nfrom fastapi import FastAPI\nfrom pydantic import BaseModel\nfrom typing import List\nimport uvicorn\nfrom vllm.engine.arg_utils import AsyncEngineArgs\nfrom vllm.engine.async_llm_engine import AsyncLLMEngine\nfrom vllm.sampling_params import SamplingParams\nimport torch\n\n# Define request data model\nclass RequestData(BaseModel):\n prompts: List[str]\n max_tokens: int = 2048\n temperature: float = 0.7\n\n# Initialize FastAPI app\napp = FastAPI()\n\n# Determine device\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n# Initialize AsyncLLMEngine\nengine_args = AsyncEngineArgs(\n model=\"your-model-name\", # Replace with your model name\n dtype=\"bfloat16\",\n gpu_memory_utilization=0.8,\n max_model_len=4096,\n trust_remote_code=True\n)\nllm_engine = AsyncLLMEngine.from_engine_args(engine_args)\n\n# Define the inference endpoint\n@app.post(\"/predict\")\nasync def generate_text(data: RequestData):\n sampling_params = SamplingParams(\n max_tokens=data.max_tokens,\n temperature=data.temperature\n )\n request_id = \"unique_request_id\" # Generate a unique request ID\n results_generator = llm_engine.generate(data.prompts, sampling_params, request_id)\n \n final_output = None\n async for request_output in results_generator:\n final_output = request_output\n \n assert final_output is not None\n text_outputs = [output.text for output in final_output.outputs]\n return {\"responses\": text_outputs}\n\n# Run the server\nif __name__ == \"__main__\":\n uvicorn.run(app, host=\"0.0.0.0\", port=8000)\n\n```\n\n## \u5f85\u6dfb\u52a0\u811a\u672c\n\n\n\n## Install\n\n```bash\npip install sparrow-python\n# Or dev version\npip install sparrow-python[dev]\n# Or\npip install -e .\n# Or\npip install -e .[dev]\n```\n\n## Usage\n\n### Multiprocessing SyncManager\n\nOpen server first:\n\n```bash\n$ spr start-server\n```\n\nThe defualt port `50001`.\n\n(Process1) productor:\n\n```python\nfrom sparrow.multiprocess.client import Client\n\nclient = Client(port=50001)\nclient.update_dict({'a': 1, 'b': 2})\n```\n\n(Process2) consumer:\n\n```python\nfrom sparrow.multiprocess.client import Client\n\nclient = Client(port=50001)\nprint(client.get_dict_data())\n\n>> > {'a': 1, 'b': 2}\n```\n\n### Common tools\n\n- **Kill process by port**\n\n```bash\n$ spr kill {port}\n```\n\n- **pack & unpack** \n support archive format: \"zip\", \"tar\", \"gztar\", \"bztar\", or \"xztar\".\n\n```bash\n$ spr pack pack_dir\n```\n\n```bash\n$ spr unpack filename extract_dir\n```\n\n- **Scaffold**\n\n```bash\n$ spr create awosome-project\n```\n\n### Some useful functions\n\n> `sparrow.relp` \n> Relative path, which is used to read or save files more easily.\n\n> `sparrow.performance.MeasureTime` \n> For measuring time (including gpu time)\n\n> `sparrow.performance.get_process_memory` \n> Get the memory size occupied by the process\n\n> `sparrow.performance.get_virtual_memory` \n> Get virtual machine memory information\n\n> `sparrow.add_env_path` \n> Add python environment variable (use relative file path)\n\n### Safe logger in `multiprocessing`\n\n```python\nfrom sparrow.log import Logger\nimport numpy as np\n\nlogger = Logger(name='train-log', log_dir='./logs', )\nlogger.info(\"hello\", \"numpy:\", np.arange(10))\n\nlogger2 = Logger.get_logger('train-log')\nprint(id(logger2) == id(logger))\n>> > True\n```\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "0.3.22",
"project_urls": {
"Issues": "https://github.com/beidongjiedeguang/sparrow/issues",
"Source": "https://github.com/beidongjiedeguang/sparrow",
"documentation": "https://github.com/beidongjiedeguang/sparrow#sparrow_python",
"homepage": "https://github.com/beidongjiedeguang/sparrow",
"repository": "https://github.com/beidongjiedeguang/sparrow"
},
"split_keywords": [
"machine learning",
" cli",
" cv",
" nlp"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "86539a3fc9a64e248bf4f86a5df1e3c6549f2c647722a309d56af49a62e735f0",
"md5": "f81d907466ce72f178ea17c686e09e7f",
"sha256": "6611b2e5c1dff1953e84b76790c2710c361f4d659170f8162877ea9584b4b558"
},
"downloads": -1,
"filename": "sparrow_python-0.3.22-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f81d907466ce72f178ea17c686e09e7f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 939722,
"upload_time": "2024-12-21T05:28:18",
"upload_time_iso_8601": "2024-12-21T05:28:18.137339Z",
"url": "https://files.pythonhosted.org/packages/86/53/9a3fc9a64e248bf4f86a5df1e3c6549f2c647722a309d56af49a62e735f0/sparrow_python-0.3.22-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "17563635bbe2ff661d71855b1f3d9c931fc663dc6ddb6c154c2af735f359ea18",
"md5": "85c401705ebc8ba546838032beb0cd1f",
"sha256": "5878354c59a804e674766591e802190b9aa7954d3329e885f47bacd297e68638"
},
"downloads": -1,
"filename": "sparrow_python-0.3.22.tar.gz",
"has_sig": false,
"md5_digest": "85c401705ebc8ba546838032beb0cd1f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 889768,
"upload_time": "2024-12-21T05:28:21",
"upload_time_iso_8601": "2024-12-21T05:28:21.327629Z",
"url": "https://files.pythonhosted.org/packages/17/56/3635bbe2ff661d71855b1f3d9c931fc663dc6ddb6c154c2af735f359ea18/sparrow_python-0.3.22.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-21 05:28:21",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "beidongjiedeguang",
"github_project": "sparrow",
"github_not_found": true,
"lcname": "sparrow-python"
}
JSON
