This is a vesy simple way to map your text data using [Altas from NOMIC](https://docs.nomic.ai/index.html) using the lib `click`.
You have to create an account to get API_KEY NOMIC.
<< Atlas enables you to:
* Store, update and organize multi-million point datasets of unstructured text, images and embeddings.
* Visually interact with your datasets from a web browser.
* Run semantic search and vector operations over your datasets.
Use Atlas to:
- Visualize, interact, collaborate and share large datasets of text and embeddings.
- Collaboratively clean, tag and label your datasets
- Build high-availability apps powered by semantic search
- Understand and debug the latent space of your AI model trains >>
# How to use
### Installation
To install the necessary dependencies, run the following command:
```bash
python -m venv mymapenv
source mymapenv/bin/activate
pip install --upgrade pip
pip install text2mapviewer
```
### Login NOMIC server
Login/create your Nomic account:
```bash
nomic login
```
If you have already your account :
```bash
nomic login [YOUR_API_TOKEN_NOMIC_HERE]
```
## Examples :
#### from NOMIC and with lib text2mapviewer
```python
from text2mapviewer.examples.map_embedding import project
# Use the projet from the lib text2mapviewer
print(project)
```
#### With the lib `click` after clone this ripo
```python
python scr/text2mapviewer/examples/map_embedding_click.py --num_embeddings 10000 --embedding_dim 256
```
#### [ The Animation Ouput](https://atlas.nomic.ai/map/0b4c0459-98f2-4aab-8d47-875765832049/54017477-907d-46e8-8d56-dddf7ab7fcfc)
## Supported Transformer Models from Hugging Face
This project supports a variety of transformer models, including models from the Hugging Face Model Hub and sentence-transformers. Below are some examples:
- Hugging Face Model: 'prajjwal1/bert-mini'
- Hugging Face Model: 'Sahajtomar/french_semantic' (french version for semantic search embedding)
- Sentence-Transformers Model: 'sentence-transformers/all-MiniLM-L6-v2' etc...
Please ensure that the model you choose is compatible with the project requirements and adjust the `--transformer_model_name` option accordingly.
## To map your text/csv files
```bash
pip install -r requirements.txt
python main.py --transformer-model-name MODEL_NAME --cache_dir CACHE_DIR --batch-size BATCH_SIZE --file-path FILE_PATH
```
NOTE: for the CACHE_DIR : you can setup it like ==>
```bash
export TRANSFORMERS_CACHE=/path_to_your/transformers_cache
```
Give a fidback.
Raw data
{
"_id": null,
"home_page": "",
"name": "text2mapviewer",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "Embedding, Visualization, Map, Text, CSV, Search keywords, Dynamic",
"author": "",
"author_email": "Papa S\u00e9ga WADE <pasega.wade@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/a4/b9/5899e4cc4db302f6f2f37d78694094753cfa212e136d86f18389403026c3/text2mapviewer-0.2.2.tar.gz",
"platform": null,
"description": "This is a vesy simple way to map your text data using [Altas from NOMIC](https://docs.nomic.ai/index.html) using the lib `click`.\n\nYou have to create an account to get API_KEY NOMIC. \n\n<< Atlas enables you to:\n* Store, update and organize multi-million point datasets of unstructured text, images and embeddings.\n* Visually interact with your datasets from a web browser.\n* Run semantic search and vector operations over your datasets.\nUse Atlas to:\n\n - Visualize, interact, collaborate and share large datasets of text and embeddings.\n - Collaboratively clean, tag and label your datasets\n - Build high-availability apps powered by semantic search\n - Understand and debug the latent space of your AI model trains >>\n\n\n# How to use\n### Installation\n\nTo install the necessary dependencies, run the following command:\n\n```bash\npython -m venv mymapenv \nsource mymapenv/bin/activate\npip install --upgrade pip \npip install text2mapviewer\n```\n\n### Login NOMIC server\n\nLogin/create your Nomic account:\n\n```bash\nnomic login\n```\nIf you have already your account : \n\n```bash\nnomic login [YOUR_API_TOKEN_NOMIC_HERE] \n```\n## Examples : \n#### from NOMIC and with lib text2mapviewer \n```python\nfrom text2mapviewer.examples.map_embedding import project\n\n# Use the projet from the lib text2mapviewer \nprint(project) \n```\n\n#### With the lib `click` after clone this ripo\n\n```python\npython scr/text2mapviewer/examples/map_embedding_click.py --num_embeddings 10000 --embedding_dim 256\n```\n\n#### [ The Animation Ouput](https://atlas.nomic.ai/map/0b4c0459-98f2-4aab-8d47-875765832049/54017477-907d-46e8-8d56-dddf7ab7fcfc)\n\n\n\n\n## Supported Transformer Models from Hugging Face \n\nThis project supports a variety of transformer models, including models from the Hugging Face Model Hub and sentence-transformers. Below are some examples:\n - Hugging Face Model: 'prajjwal1/bert-mini'\n - Hugging Face Model: 'Sahajtomar/french_semantic' (french version for semantic search embedding) \n - Sentence-Transformers Model: 'sentence-transformers/all-MiniLM-L6-v2' etc...\n\nPlease ensure that the model you choose is compatible with the project requirements and adjust the `--transformer_model_name` option accordingly.\n\n## To map your text/csv files\n\n```bash\npip install -r requirements.txt\npython main.py --transformer-model-name MODEL_NAME --cache_dir CACHE_DIR --batch-size BATCH_SIZE --file-path FILE_PATH\n```\nNOTE: for the CACHE_DIR : you can setup it like ==> \n\n```bash\nexport TRANSFORMERS_CACHE=/path_to_your/transformers_cache\n```\n\nGive a fidback. \n",
"bugtrack_url": null,
"license": "",
"summary": "A python package to map your own csv files data using Atlas from NOMIC",
"version": "0.2.2",
"split_keywords": [
"embedding",
" visualization",
" map",
" text",
" csv",
" search keywords",
" dynamic"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c13ade12f0db582b3c75ae7815e67c170170b74339e8bcc4eefe2a0656014264",
"md5": "ff4a4d82a9310f1937cd5db233bd1d4b",
"sha256": "07239f079beeaf95d9e6697a98322319fa02b075db105a9f2f02a2595ae66f1d"
},
"downloads": -1,
"filename": "text2mapviewer-0.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ff4a4d82a9310f1937cd5db233bd1d4b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 8165,
"upload_time": "2023-04-14T12:12:04",
"upload_time_iso_8601": "2023-04-14T12:12:04.162303Z",
"url": "https://files.pythonhosted.org/packages/c1/3a/de12f0db582b3c75ae7815e67c170170b74339e8bcc4eefe2a0656014264/text2mapviewer-0.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a4b95899e4cc4db302f6f2f37d78694094753cfa212e136d86f18389403026c3",
"md5": "75ecf4098480868d7f19e3e4ffe3060d",
"sha256": "b3a4b671b947fba3193439ef775f748bace395f384adbcc088e8786e57fa87f1"
},
"downloads": -1,
"filename": "text2mapviewer-0.2.2.tar.gz",
"has_sig": false,
"md5_digest": "75ecf4098480868d7f19e3e4ffe3060d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 9206,
"upload_time": "2023-04-14T12:12:07",
"upload_time_iso_8601": "2023-04-14T12:12:07.438656Z",
"url": "https://files.pythonhosted.org/packages/a4/b9/5899e4cc4db302f6f2f37d78694094753cfa212e136d86f18389403026c3/text2mapviewer-0.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-14 12:12:07",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "text2mapviewer"
}
JSON