This is a vesy simple way to map your text data using [Altas from NOMIC](https://docs.nomic.ai/index.html) using the lib `click`.
You have to create an account to get API_KEY NOMIC.
Atlas enables you to:
Store, update and organize multi-million point datasets of unstructured text, images and embeddings.
Visually interact with your datasets from a web browser.
Run semantic search and vector operations over your datasets.
Use Atlas to:
- Visualize, interact, collaborate and share large datasets of text and embeddings.
- Collaboratively clean, tag and label your datasets
- Build high-availability apps powered by semantic search
- Understand and debug the latent space of your AI model trains
# How to use
### Installation
To install the necessary dependencies, run the following command:
```bash
python -m venv mymapenv
source mymapenv/bin/activate
pip install --upgrade pip
pip install text2mapdata
```
## Supported Transformer Models from Hugging Face
This project supports a variety of transformer models, including models from the Hugging Face Model Hub and sentence-transformers. Below are some examples:
- Hugging Face Model: 'prajjwal1/bert-mini'
- Hugging Face Model: 'Sahajtomar/french_semantic' (french version for semantic search embedding)
- Sentence-Transformers Model: 'sentence-transformers/all-MiniLM-L6-v2' etc...
Please ensure that the model you choose is compatible with the project requirements and adjust the `--transformer_model_name` option accordingly.
## To map your text/csv files
```bash
pip install -r requirements.txt
python main.py --transformer-model-name MODEL_NAME --cache_dir CACHE_DIR --batch-size BATCH_SIZE --file-path FILE_PATH
```
Remarque for the CACHE_DIR : you can setup it like ==>
```bash
export TRANSFORMERS_CACHE=/path_to_your/transformers_cache
```
Give a fidback.
Raw data
{
"_id": null,
"home_page": "",
"name": "text2mapview",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "Embedding, Visualization, Map, Text, CSV, Search keywords, Dynamic",
"author": "",
"author_email": "Papa S\u00e9ga WADE <pasega.wade@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/c5/35/228292fe317e2313781e8cb3656767fb1f512baa16561acf0fd04b3fc930/text2mapview-0.2.0.tar.gz",
"platform": null,
"description": "This is a vesy simple way to map your text data using [Altas from NOMIC](https://docs.nomic.ai/index.html) using the lib `click`. \n\nYou have to create an account to get API_KEY NOMIC. \n\nAtlas enables you to:\n\nStore, update and organize multi-million point datasets of unstructured text, images and embeddings.\n\nVisually interact with your datasets from a web browser.\n\nRun semantic search and vector operations over your datasets.\n\nUse Atlas to:\n\n - Visualize, interact, collaborate and share large datasets of text and embeddings.\n \n - Collaboratively clean, tag and label your datasets\n \n - Build high-availability apps powered by semantic search\n \n - Understand and debug the latent space of your AI model trains\n\n# How to use\n### Installation\n\nTo install the necessary dependencies, run the following command:\n\n```bash\npython -m venv mymapenv \nsource mymapenv/bin/activate\npip install --upgrade pip \npip install text2mapdata\n```\n\n## Supported Transformer Models from Hugging Face \n\nThis project supports a variety of transformer models, including models from the Hugging Face Model Hub and sentence-transformers. Below are some examples:\n - Hugging Face Model: 'prajjwal1/bert-mini'\n - Hugging Face Model: 'Sahajtomar/french_semantic' (french version for semantic search embedding) \n - Sentence-Transformers Model: 'sentence-transformers/all-MiniLM-L6-v2' etc...\n\nPlease ensure that the model you choose is compatible with the project requirements and adjust the `--transformer_model_name` option accordingly.\n\n## To map your text/csv files\n\n```bash\npip install -r requirements.txt\npython main.py --transformer-model-name MODEL_NAME --cache_dir CACHE_DIR --batch-size BATCH_SIZE --file-path FILE_PATH\n```\nRemarque for the CACHE_DIR : you can setup it like ==> \n\n```bash\nexport TRANSFORMERS_CACHE=/path_to_your/transformers_cache\n```\n\nGive a fidback. \n",
"bugtrack_url": null,
"license": "",
"summary": "A python package to map your own csv files data using Atlas from NOMIC",
"version": "0.2.0",
"split_keywords": [
"embedding",
" visualization",
" map",
" text",
" csv",
" search keywords",
" dynamic"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7adcc011eebc19597d3fddf302855ed231251d04a7727c799eba5f2ed969247c",
"md5": "efe2c4a139c208f69fdaf9fa808a4c95",
"sha256": "100eb1c3955f0bff4a1860a9baca1f90636686453bd948d05f155e8d80de95ec"
},
"downloads": -1,
"filename": "text2mapview-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "efe2c4a139c208f69fdaf9fa808a4c95",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 5571,
"upload_time": "2023-04-13T09:33:07",
"upload_time_iso_8601": "2023-04-13T09:33:07.238709Z",
"url": "https://files.pythonhosted.org/packages/7a/dc/c011eebc19597d3fddf302855ed231251d04a7727c799eba5f2ed969247c/text2mapview-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c535228292fe317e2313781e8cb3656767fb1f512baa16561acf0fd04b3fc930",
"md5": "938cf77fca57a83e21fa55afb827d7d4",
"sha256": "dd37c4c2085d8c15f9d42d464152c12ce6a9edaee69488046d93d101fa1ab6c9"
},
"downloads": -1,
"filename": "text2mapview-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "938cf77fca57a83e21fa55afb827d7d4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 4673,
"upload_time": "2023-04-13T09:33:09",
"upload_time_iso_8601": "2023-04-13T09:33:09.003482Z",
"url": "https://files.pythonhosted.org/packages/c5/35/228292fe317e2313781e8cb3656767fb1f512baa16561acf0fd04b3fc930/text2mapview-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-13 09:33:09",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "text2mapview"
}
JSON