| Name | mindcraft JSON |
| Version |
0.2.4
JSON |
| download |
| home_page | https://github.com/josejuanmartinez/mindcraft |
| Summary | Mindcraft: an LLM-based engine for creating real NPCs. Empowered by Hugging Face, quantized LLMs with AWQ (thanks @TheBloke) and vLLM. It follows a RAG approach with chunk or sentence splitting, and a vector store. Right now, ChromaDB is the supported Vector Store and chunk splitting using `tiktoken` or sentence splitting using `spacy` are available. |
| upload_time | 2024-01-17 12:40:17 |
| maintainer | |
| docs_url | None |
| author | Juan.Martinez |
| requires_python | >=3.10.0 |
| license | MIT |
| keywords |
nlp
llm
npc
conversations
videogames
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<p align="center">
<img width="20%" src="https://github.com/josejuanmartinez/mindcraft/assets/36634572/0ef83288-9e53-444d-baa0-2c61b0fc26ca" alt="mindcraft"/>
</p>
# MindCraft
The open-source NLP library to craft the minds of your NPC characters for your video games.
Requires Python 3.10 or higher.
It includes the following features:
- Text generation using LLMs
- Motivations, personality, personal goals
- Knowledge and awareness about the world (RAG)
- Short and Long-term memory (RAG)
- Conversational styles
- Supervised finetuning by human feedback (SFT)
- Integration with vLLM for **fast inference** and **streaming** locally, remotelly or in Docker.
- Usage of quantized AWQ models
- Integration with API and RPC (to come!)
## Create a World from a book
```python
from mindcraft.infra.engine.llm_types import LLMType
from mindcraft.infra.vectorstore.stores_types import StoresTypes
from mindcraft.infra.embeddings.embeddings_types import EmbeddingsTypes
from mindcraft.lore.world import World
world = World(world_name="Middle Earth from the Lord of the Rings",
embeddings=EmbeddingsTypes.MINILM,
store_type=StoresTypes.CHROMA,
llm_type=LLMType.YI_6B_AWQ,
fast=False, # <--- fast=True to switch on vLLM
remote=False, # <--- remote=True to use a remote vLLM server
streaming=True) # <--- streaming=True to use vLLM streaming
```
Now we use some book to carry out chunk splitting and add it to our favourite Vector Database (in our case, ChromaDB)
```python
from mindcraft.infra.splitters.text_splitters_types import TextSplitterTypes
world.book_to_world(book_path="/content/lotr1.txt",
text_splitter=TextSplitterTypes.SENTENCE_SPLITTER,
max_units=3,
overlap=1,
encoding='utf-8')
```
## Query the lore of the world
Once a world has been created and populated with lore, query the lore known by NPCs by doing:
```python
results = world.get_lore("What do you think about the Rings of Power?", num_results=5, min_similarity=0.95)
for i, d in enumerate(results.documents):
print(f"SENTENCE {i}:\n{d}")
print()
```
## Instantiate an NPC in a world
Once a world has been created and populated with lore, instantiate an NPC in it by doing:
```python
from mindcraft.features.motivation import Motivation
from mindcraft.features.personality import Personality
from mindcraft.features.mood import Mood
from mindcraft.mind.npc import NPC
name = "Galadriel"
description = "The Elven Queen of Lothlorien, bearer of Nenya, wife to Celeborn"
personalities = [Personality(x) for x in ['fair', 'mighty', 'wise', 'carying', 'kind']]
motivations = [Motivation(x) for x in ['Destroying the Evil', 'Protecting Middle Earth', 'Travelling West']]
mood = Mood('worried')
galadriel = NPC(name,
description,
personalities,
motivations,
mood,
StoresTypes.CHROMA,
EmbeddingsTypes.MINILM)
galadriel.add_npc_to_world()
```
# Ask questions to the NPC
We get an iterator for the responses, to allow inference in streaming way.
```python
answer_iter = galadriel.react_to("What do you think about the Rings of Power",
min_similarity=0.85,
ltm_num_results=3,
world_num_results=7,
max_tokens=600)
```
So your answers will be in the iterator, don't forget to loop through it!
```
for answer in answer_iter:
print(answer)
```
### Example of answer using Zephyr 7B quantized to 4b
```Alas, my dear friend, the Rings of Power are a perilous burden that should not be wielded by those of mortal race. They possess a sinister force that overcomes the spirit, enslaving their very soul. I have observed their destructive potential in Eregion long ago, where many Elven-rings were created, each infused with its own unique potency. Some more potent than others, and often, they seem to have a propensity to gravitate towards the unsuspecting. In light of our shared concern for the wellbeing of Middle Earth, I implore you to heed my words; let us not succumb to the allure of these fateful rings, lest they consume us entirely.```
## Creating custom Supervised Finetuning datasets for NPCs
There are two loops integrated in the framework which allow you to create your own datasets.
1. `NPC.extract_conversational_styles_from_world`: Retrieving conversations from the world and tagging the question and mood yourself
2. `NPC.extract_conversational_styles_talking_to_user`: Creates conversations by talking to you
### Supervised Finetuning
You can create your own datasets with supervised finetuning,
accepting or rejecting the interactions. As a result, you will come up
with a csv of interactions you can use to train or finetune your own models.
```csv
name||mood||question||answer
Galadriel||default||Good night, Galadriel!||'Good night, my friends! '\nsaid Galadriel. '\nSleep in peace!
Galadriel||grave||why he could say that?||....`He would be rash indeed that said that thing,' said Galadriel gravely.
```
## LLM integrated
### Quantized
- TheBloke/mistral_7b_norobots-AWQ
- TheBloke/zephyr-7B-beta-AWQ
- TheBloke/notus-7B-v1-AWQ
- TheBloke/Starling-LM-7B-alpha-AWQ
- TheBloke/Yi-6B-AWQ
- TheBloke/dragon-yi-6B-v0-AWQ
### Unquantized
- microsoft/phi-2
- stabilityai/stablelm-zephyr-3b
## Embeddings for RAG
- [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
## CUDA and Torch in WINDOWS
If you are running on Windows on a machine with a GPU, and you get a message about not being able to find
your gpu, you need to configure CUDA for Windows.
1. Go to [CUDA installation webpage](https://developer.nvidia.com/cuda-downloads).
2. Select your Windows version and specifics.
3. Download and install
4. Uninstall torch (`pip uninstall torch`)
5a. Go to requirements.txt, comment the line after LINUX for torch and uncomment the line after WINDOWS
5b. Alternatively you can just run this command:
```
pip3 install torch -i https://download.pytorch.org/whl/cu121
```
You torch on windows CUDA should be working. To test it:
```python
import torch
if __name__ == "__main__":
print(torch.cuda.is_available())
```
## vLLM
`vLLM` has been included for Fast Inference, in local, remote installations and Docker.
### Local Fast inference (Paged Attention)
To use fast-inference, just run add `fast=True` to your `World` object:
```python
world = World(world_name="Lord of the Rings",
embeddings=EmbeddingsTypes.MINILM,
store_type=StoresTypes.CHROMA,
llm_type=LLMType.YI_6B_AWQ,
fast=True) # <---- HERE
```
### Remote Fast Inference
To the previous `fast` parameter, add also `remote=True`
```python
world = World(world_name="Lord of the Rings",
embeddings=EmbeddingsTypes.MINILM,
store_type=StoresTypes.CHROMA,
llm_type=LLMType.YI_6B_AWQ,
fast=True, # <---- HERE
remote=True) # <---- HERE
```
### Streaming (only if remote!)
To the previous `fast` and `remote` parameters, add also `streaming=True`
```python
world = World(world_name="Lord of the Rings",
embeddings=EmbeddingsTypes.MINILM,
store_type=StoresTypes.CHROMA,
llm_type=LLMType.YI_6B_AWQ,
fast=True, # <---- HERE
remote=True,
streaming=True) # <---- HERE
```
## Example data
- [Lord of the Rings](https://www.kaggle.com/datasets/ashishsinhaiitr/lord-of-the-rings-text)
## Notebooks
You can find notebooks in the `notebooks` folder of this project.
## Demo 1: Creating a World and an NPC
[Video](https://youtu.be/T-D1KVIuvjA)
## Architecture
<p align="center">
<img
width="50%"
src="https://github.com/josejuanmartinez/mindcraft/assets/36634572/7778d4a4-6b25-4b1a-9b26-b1bfa9d94727" alt="mindcraft architecture"/>
</p>
## Tests
`python -m pytest tests/*`
## Header
<p align="center">
<img width="100%" src="https://github.com/josejuanmartinez/mindcraft/blob/main/galadriel.png" alt="galadriel"/>
</p>
Raw data
{
"_id": null,
"home_page": "https://github.com/josejuanmartinez/mindcraft",
"name": "mindcraft",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10.0",
"maintainer_email": "",
"keywords": "nlp,llm,npc,conversations,videogames",
"author": "Juan.Martinez",
"author_email": "jjmcarrascosa@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/49/70/9a970687303ad7e1c042f9dde5b3a24be591d088403407a453489a3b6c7d/mindcraft-0.2.4.tar.gz",
"platform": null,
"description": "<p align=\"center\">\r\n<img width=\"20%\" src=\"https://github.com/josejuanmartinez/mindcraft/assets/36634572/0ef83288-9e53-444d-baa0-2c61b0fc26ca\" alt=\"mindcraft\"/>\r\n</p>\r\n\r\n# MindCraft\r\nThe open-source NLP library to craft the minds of your NPC characters for your video games.\r\n\r\nRequires Python 3.10 or higher.\r\n\r\nIt includes the following features:\r\n\r\n- Text generation using LLMs\r\n- Motivations, personality, personal goals\r\n- Knowledge and awareness about the world (RAG)\r\n- Short and Long-term memory (RAG)\r\n- Conversational styles\r\n- Supervised finetuning by human feedback (SFT)\r\n- Integration with vLLM for **fast inference** and **streaming** locally, remotelly or in Docker. \r\n- Usage of quantized AWQ models\r\n- Integration with API and RPC (to come!)\r\n\r\n## Create a World from a book\r\n```python\r\nfrom mindcraft.infra.engine.llm_types import LLMType\r\nfrom mindcraft.infra.vectorstore.stores_types import StoresTypes\r\nfrom mindcraft.infra.embeddings.embeddings_types import EmbeddingsTypes\r\nfrom mindcraft.lore.world import World\r\n\r\nworld = World(world_name=\"Middle Earth from the Lord of the Rings\",\r\n embeddings=EmbeddingsTypes.MINILM,\r\n store_type=StoresTypes.CHROMA,\r\n llm_type=LLMType.YI_6B_AWQ,\r\n fast=False, # <--- fast=True to switch on vLLM\r\n remote=False, # <--- remote=True to use a remote vLLM server\r\n streaming=True) # <--- streaming=True to use vLLM streaming\r\n```\r\n\r\nNow we use some book to carry out chunk splitting and add it to our favourite Vector Database (in our case, ChromaDB)\r\n```python\r\nfrom mindcraft.infra.splitters.text_splitters_types import TextSplitterTypes\r\n\r\nworld.book_to_world(book_path=\"/content/lotr1.txt\",\r\n text_splitter=TextSplitterTypes.SENTENCE_SPLITTER,\r\n max_units=3,\r\n overlap=1,\r\n encoding='utf-8')\r\n```\r\n\r\n## Query the lore of the world\r\nOnce a world has been created and populated with lore, query the lore known by NPCs by doing:\r\n```python\r\nresults = world.get_lore(\"What do you think about the Rings of Power?\", num_results=5, min_similarity=0.95)\r\nfor i, d in enumerate(results.documents):\r\n print(f\"SENTENCE {i}:\\n{d}\")\r\n print()\r\n```\r\n\r\n## Instantiate an NPC in a world\r\nOnce a world has been created and populated with lore, instantiate an NPC in it by doing:\r\n\r\n```python\r\nfrom mindcraft.features.motivation import Motivation\r\nfrom mindcraft.features.personality import Personality\r\nfrom mindcraft.features.mood import Mood\r\n\r\nfrom mindcraft.mind.npc import NPC\r\n\r\nname = \"Galadriel\"\r\ndescription = \"The Elven Queen of Lothlorien, bearer of Nenya, wife to Celeborn\"\r\npersonalities = [Personality(x) for x in ['fair', 'mighty', 'wise', 'carying', 'kind']]\r\nmotivations = [Motivation(x) for x in ['Destroying the Evil', 'Protecting Middle Earth', 'Travelling West']]\r\nmood = Mood('worried')\r\n\r\ngaladriel = NPC(name,\r\n description,\r\n personalities,\r\n motivations,\r\n mood,\r\n StoresTypes.CHROMA,\r\n EmbeddingsTypes.MINILM)\r\n\r\ngaladriel.add_npc_to_world()\r\n```\r\n\r\n# Ask questions to the NPC\r\nWe get an iterator for the responses, to allow inference in streaming way.\r\n```python\r\nanswer_iter = galadriel.react_to(\"What do you think about the Rings of Power\",\r\n min_similarity=0.85,\r\n ltm_num_results=3,\r\n world_num_results=7,\r\n max_tokens=600)\r\n```\r\n\r\nSo your answers will be in the iterator, don't forget to loop through it!\r\n```\r\nfor answer in answer_iter:\r\n print(answer)\r\n```\r\n\r\n### Example of answer using Zephyr 7B quantized to 4b\r\n```Alas, my dear friend, the Rings of Power are a perilous burden that should not be wielded by those of mortal race. They possess a sinister force that overcomes the spirit, enslaving their very soul. I have observed their destructive potential in Eregion long ago, where many Elven-rings were created, each infused with its own unique potency. Some more potent than others, and often, they seem to have a propensity to gravitate towards the unsuspecting. In light of our shared concern for the wellbeing of Middle Earth, I implore you to heed my words; let us not succumb to the allure of these fateful rings, lest they consume us entirely.```\r\n\r\n## Creating custom Supervised Finetuning datasets for NPCs\r\nThere are two loops integrated in the framework which allow you to create your own datasets.\r\n1. `NPC.extract_conversational_styles_from_world`: Retrieving conversations from the world and tagging the question and mood yourself\r\n2. `NPC.extract_conversational_styles_talking_to_user`: Creates conversations by talking to you\r\n\r\n### Supervised Finetuning\r\nYou can create your own datasets with supervised finetuning,\r\naccepting or rejecting the interactions. As a result, you will come up\r\nwith a csv of interactions you can use to train or finetune your own models.\r\n\r\n```csv\r\nname||mood||question||answer\r\nGaladriel||default||Good night, Galadriel!||'Good night, my friends! '\\nsaid Galadriel. '\\nSleep in peace!\r\nGaladriel||grave||why he could say that?||....`He would be rash indeed that said that thing,' said Galadriel gravely.\r\n```\r\n\r\n## LLM integrated\r\n### Quantized\r\n- TheBloke/mistral_7b_norobots-AWQ\r\n- TheBloke/zephyr-7B-beta-AWQ\r\n- TheBloke/notus-7B-v1-AWQ\r\n- TheBloke/Starling-LM-7B-alpha-AWQ\r\n- TheBloke/Yi-6B-AWQ\r\n- TheBloke/dragon-yi-6B-v0-AWQ\r\n### Unquantized\r\n- microsoft/phi-2\r\n- stabilityai/stablelm-zephyr-3b\r\n\r\n## Embeddings for RAG\r\n- [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)\r\n\r\n## CUDA and Torch in WINDOWS\r\nIf you are running on Windows on a machine with a GPU, and you get a message about not being able to find \r\nyour gpu, you need to configure CUDA for Windows.\r\n\r\n1. Go to [CUDA installation webpage](https://developer.nvidia.com/cuda-downloads).\r\n2. Select your Windows version and specifics.\r\n3. Download and install\r\n4. Uninstall torch (`pip uninstall torch`)\r\n5a. Go to requirements.txt, comment the line after LINUX for torch and uncomment the line after WINDOWS\r\n5b. Alternatively you can just run this command:\r\n```\r\npip3 install torch -i https://download.pytorch.org/whl/cu121\r\n```\r\n\r\nYou torch on windows CUDA should be working. To test it:\r\n```python\r\nimport torch\r\n\r\nif __name__ == \"__main__\":\r\n print(torch.cuda.is_available())\r\n```\r\n\r\n\r\n## vLLM\r\n`vLLM` has been included for Fast Inference, in local, remote installations and Docker.\r\n\r\n### Local Fast inference (Paged Attention)\r\n\r\nTo use fast-inference, just run add `fast=True` to your `World` object:\r\n```python\r\nworld = World(world_name=\"Lord of the Rings\",\r\n embeddings=EmbeddingsTypes.MINILM,\r\n store_type=StoresTypes.CHROMA,\r\n llm_type=LLMType.YI_6B_AWQ,\r\n fast=True) # <---- HERE\r\n```\r\n\r\n### Remote Fast Inference\r\nTo the previous `fast` parameter, add also `remote=True`\r\n```python\r\nworld = World(world_name=\"Lord of the Rings\",\r\n embeddings=EmbeddingsTypes.MINILM,\r\n store_type=StoresTypes.CHROMA,\r\n llm_type=LLMType.YI_6B_AWQ,\r\n fast=True, # <---- HERE\r\n remote=True) # <---- HERE\r\n```\r\n\r\n### Streaming (only if remote!)\r\nTo the previous `fast` and `remote` parameters, add also `streaming=True`\r\n```python\r\nworld = World(world_name=\"Lord of the Rings\",\r\n embeddings=EmbeddingsTypes.MINILM,\r\n store_type=StoresTypes.CHROMA,\r\n llm_type=LLMType.YI_6B_AWQ,\r\n fast=True, # <---- HERE\r\n remote=True,\r\n streaming=True) # <---- HERE\r\n```\r\n\r\n## Example data\r\n- [Lord of the Rings](https://www.kaggle.com/datasets/ashishsinhaiitr/lord-of-the-rings-text)\r\n\r\n## Notebooks\r\nYou can find notebooks in the `notebooks` folder of this project.\r\n\r\n## Demo 1: Creating a World and an NPC\r\n[Video](https://youtu.be/T-D1KVIuvjA)\r\n\r\n## Architecture\r\n<p align=\"center\">\r\n<img \r\n width=\"50%\"\r\n src=\"https://github.com/josejuanmartinez/mindcraft/assets/36634572/7778d4a4-6b25-4b1a-9b26-b1bfa9d94727\" alt=\"mindcraft architecture\"/>\r\n</p>\r\n\r\n## Tests\r\n`python -m pytest tests/*`\r\n\r\n## Header\r\n<p align=\"center\">\r\n<img width=\"100%\" src=\"https://github.com/josejuanmartinez/mindcraft/blob/main/galadriel.png\" alt=\"galadriel\"/>\r\n</p>\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Mindcraft: an LLM-based engine for creating real NPCs. Empowered by Hugging Face, quantized LLMs with AWQ (thanks @TheBloke) and vLLM. It follows a RAG approach with chunk or sentence splitting, and a vector store. Right now, ChromaDB is the supported Vector Store and chunk splitting using `tiktoken` or sentence splitting using `spacy` are available.",
"version": "0.2.4",
"project_urls": {
"Download": "https://github.com/josejuanmartinez/mindcraft",
"Homepage": "https://github.com/josejuanmartinez/mindcraft"
},
"split_keywords": [
"nlp",
"llm",
"npc",
"conversations",
"videogames"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e1efb1ac006672c641a80086221792097e49ee18e9a6014f7504f641103d4330",
"md5": "2d3b541e728613ba09d1d06e7dd6a540",
"sha256": "620f83d0db5ee61ff03bfcc211d7b6cb6ca405d634a045a7817ab871b5abf8c9"
},
"downloads": -1,
"filename": "mindcraft-0.2.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2d3b541e728613ba09d1d06e7dd6a540",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10.0",
"size": 38126,
"upload_time": "2024-01-17T12:40:14",
"upload_time_iso_8601": "2024-01-17T12:40:14.862129Z",
"url": "https://files.pythonhosted.org/packages/e1/ef/b1ac006672c641a80086221792097e49ee18e9a6014f7504f641103d4330/mindcraft-0.2.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "49709a970687303ad7e1c042f9dde5b3a24be591d088403407a453489a3b6c7d",
"md5": "a5a6a8a5270e6fe0a95bf9ba7eddd0b1",
"sha256": "6e69b7cdfabe82390a6ec12e33d900a896c351306394ada8748ddf8c17133a7a"
},
"downloads": -1,
"filename": "mindcraft-0.2.4.tar.gz",
"has_sig": false,
"md5_digest": "a5a6a8a5270e6fe0a95bf9ba7eddd0b1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10.0",
"size": 31684,
"upload_time": "2024-01-17T12:40:17",
"upload_time_iso_8601": "2024-01-17T12:40:17.048395Z",
"url": "https://files.pythonhosted.org/packages/49/70/9a970687303ad7e1c042f9dde5b3a24be591d088403407a453489a3b6c7d/mindcraft-0.2.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-17 12:40:17",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "josejuanmartinez",
"github_project": "mindcraft",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "mindcraft"
}
