# langdash
A simple library for interfacing with language models.
**Currently in beta!**
**Features:**
* Support for guided text generation, text classification (through prompting) and vector-based document searching.
* Lightweight, build-it-yourself-style prompt wrappers in pure Python, with no domain-specific language involved.
* Token healing and transformers/RNN state reuse for fast inference, like in [Microsoft's guidance](https://github.com/microsoft/guidance).
* First-class support for ggml backends.
**Documentation:** [Read on readthedocs.io](https://langdash.readthedocs.io/en/latest/)
**Repository:** [main](https://git.mysymphony.jp.net/nana/langdash/) / [Gitlab mirror](https://gitlab.com/nanamochizuki77/langdash)
## Installation
Use [pip](https://pip.pypa.io/en/stable/) to install. By default, langdash does not come preinstalled with any additional modules. You will have to specify what you need like in the following command:
```
pip install --user langdash[embeddings,sentence_transformers]
```
List of modules:
* **core:**
* *embeddings:* required for running searching through embeddings
* **backends:**
* Generation backends: *rwkv_cpp*, *llama_cpp*, *ctransformers*, *transformers*
* Embedding backends: *sentence_transformers*
**Note:** If running from source, initialize the git submodules in the `langdash/extern` folder to compile foreign backends.
## Usage
Examples:
* [Text generation](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/docs/examples/text-generation.md)
* [Generating TV shows](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/docs/examples/generating-tv-shows.md)
* [Embedding search](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/docs/examples/embedding-search.md)
See [examples folder](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/examples) for full examples.
## Running the Examples
All examples can be ran with the following command:
```
python examples/instruct.py [model type] [model name or path]
```
You can specify additional model parameters using the `-ae` CLI argument, and passing a valid Python literal. For example, to run the chat example using the WizardLM model with context length of 4096, do:
```
python examples/chat.py llama_cpp /path/to/ggml-wizardlm.bin -ae n_ctx 4096
```
Some examples require you to specify the prompt format. Formats include: `wizardlm` (shortened Alpaca format without the first prompt line and `# Instruction:`), and `alpaca` (the full format). You will need to specify it for most of the examples:
```
python examples/instruct.py llama_cpp /path/to/ggml-wizardlm.bin -ae n_ctx 4096 --prompt-format wizardlm
```
For a full list, see the `examples/_instruct_format.py` file.
## License
Apache 2.0
Raw data
{
"_id": null,
"home_page": "https://git.mysymphony.jp.net/nana/langdash",
"name": "langdash",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "",
"author": "Nana Mochizuki",
"author_email": "nana@mysymphony.jp.net",
"download_url": "https://files.pythonhosted.org/packages/7c/8d/f133a718851ff1ab94b751e5f3a1fa6f212182d1afe32a639fb7894beea1/langdash-1.20.2.tar.gz",
"platform": null,
"description": "# langdash\n\nA simple library for interfacing with language models.\n\n**Currently in beta!**\n\n**Features:**\n\n * Support for guided text generation, text classification (through prompting) and vector-based document searching.\n * Lightweight, build-it-yourself-style prompt wrappers in pure Python, with no domain-specific language involved.\n * Token healing and transformers/RNN state reuse for fast inference, like in [Microsoft's guidance](https://github.com/microsoft/guidance).\n * First-class support for ggml backends.\n\n**Documentation:** [Read on readthedocs.io](https://langdash.readthedocs.io/en/latest/)\n\n**Repository:** [main](https://git.mysymphony.jp.net/nana/langdash/) / [Gitlab mirror](https://gitlab.com/nanamochizuki77/langdash)\n\n## Installation\n\nUse [pip](https://pip.pypa.io/en/stable/) to install. By default, langdash does not come preinstalled with any additional modules. You will have to specify what you need like in the following command:\n\n```\npip install --user langdash[embeddings,sentence_transformers]\n```\n\nList of modules:\n \n * **core:**\n * *embeddings:* required for running searching through embeddings\n * **backends:**\n * Generation backends: *rwkv_cpp*, *llama_cpp*, *ctransformers*, *transformers*\n * Embedding backends: *sentence_transformers*\n\n**Note:** If running from source, initialize the git submodules in the `langdash/extern` folder to compile foreign backends.\n \n## Usage\n\nExamples:\n\n * [Text generation](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/docs/examples/text-generation.md)\n * [Generating TV shows](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/docs/examples/generating-tv-shows.md)\n * [Embedding search](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/docs/examples/embedding-search.md)\n\nSee [examples folder](https://git.mysymphony.jp.net/nana/langdash/src/branch/master/examples) for full examples.\n\n## Running the Examples\n\nAll examples can be ran with the following command:\n\n```\npython examples/instruct.py [model type] [model name or path]\n```\n\nYou can specify additional model parameters using the `-ae` CLI argument, and passing a valid Python literal. For example, to run the chat example using the WizardLM model with context length of 4096, do:\n\n```\npython examples/chat.py llama_cpp /path/to/ggml-wizardlm.bin -ae n_ctx 4096\n```\n\nSome examples require you to specify the prompt format. Formats include: `wizardlm` (shortened Alpaca format without the first prompt line and `# Instruction:`), and `alpaca` (the full format). You will need to specify it for most of the examples:\n\n```\npython examples/instruct.py llama_cpp /path/to/ggml-wizardlm.bin -ae n_ctx 4096 --prompt-format wizardlm\n```\n\nFor a full list, see the `examples/_instruct_format.py` file.\n\n\n## License\n\nApache 2.0\n",
"bugtrack_url": null,
"license": "",
"summary": "A simple library for interfacing with language models.",
"version": "1.20.2",
"project_urls": {
"Documentation": "https://langdash.readthedocs.io/en/latest/",
"Homepage": "https://git.mysymphony.jp.net/nana/langdash",
"Source": "https://git.mysymphony.jp.net/nana/langdash"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7c8df133a718851ff1ab94b751e5f3a1fa6f212182d1afe32a639fb7894beea1",
"md5": "b20cc435ceec601c63cc717616afd020",
"sha256": "0791457231300279d431ee029e26f7d3d0b18e69aea9778fa335cc707c83cb73"
},
"downloads": -1,
"filename": "langdash-1.20.2.tar.gz",
"has_sig": false,
"md5_digest": "b20cc435ceec601c63cc717616afd020",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 37440,
"upload_time": "2023-07-24T23:50:51",
"upload_time_iso_8601": "2023-07-24T23:50:51.952011Z",
"url": "https://files.pythonhosted.org/packages/7c/8d/f133a718851ff1ab94b751e5f3a1fa6f212182d1afe32a639fb7894beea1/langdash-1.20.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-24 23:50:51",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "langdash"
}
JSON