# LlamaDataset Metadata Pack
As part of the `LlamaDataset` submission package into [llamahub](https://llamahub.ai),
two metadata files are required, namely: `card.json` and `README.md`. This pack
creates these two files and saves them to disk to help expedite the submission
process.
## CLI Usage
You can download llamapacks directly using `llamaindex-cli`, which comes installed with the `llama-index` python package:
```bash
llamaindex-cli download-llamapack LlamaDatasetMetadataPack --download-dir ./llama_dataset_metadata_pack
```
You can then inspect the files at `./llama_dataset_metadata_pack` and use them as a template for your own project!
## Code Usage
You can download the pack to the `./llama_dataset_metadata_pack` directory through python
code as well. The sample script below demonstrates how to construct `LlamaDatasetMetadataPack`
using a `LabelledRagDataset` downloaded from `llama-hub` and a simple RAG pipeline
built off of its source documents.
```python
from llama_index.core.llama_pack import download_llama_pack
# Download and install dependencies
LlamaDatasetMetadataPack = download_llama_pack(
"LlamaDatasetMetadataPack", "./llama_dataset_metadata_pack"
)
# construction requires a query_engine, a rag_dataset, and optionally a judge_llm
llama_dataset_metadata_pack = LlamaDatasetMetadataPack()
# create and save `card.json` and `README.md` to disk
dataset_description = (
"A labelled RAG dataset based off an essay by Paul Graham, consisting of "
"queries, reference answers, and reference contexts."
)
llama_dataset_metadata_pack.run(
name="Paul Graham Essay Dataset",
description=dataset_description,
rag_dataset=rag_dataset, # defined earlier not shown here
index=index, # defined earlier not shown here
benchmark_df=benchmark_df, # defined earlier not shown here
baseline_name="llamaindex",
)
```
NOTE: this pack should be used only after performing a RAG evaluation (i.e., by
using `RagEvaluatorPack` on a `LabelledRagDataset`). In the code snippet above,
`index`, `rag_dataset`, and `benchmark_df` are all objects that you'd expect to
have only after performing the RAG evaluation as mention in the previous sentence.
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-packs-llama-dataset-metadata",
"maintainer": "nerdai",
"docs_url": null,
"requires_python": "<4.0,>=3.9",
"maintainer_email": null,
"keywords": "evaluation, llamadataset, rag, submission",
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/96/20/5e46995715657e292a8e233c0db91d939ae5099243c7071d575a5ed9c86b/llama_index_packs_llama_dataset_metadata-0.3.0.tar.gz",
"platform": null,
"description": "# LlamaDataset Metadata Pack\n\nAs part of the `LlamaDataset` submission package into [llamahub](https://llamahub.ai),\ntwo metadata files are required, namely: `card.json` and `README.md`. This pack\ncreates these two files and saves them to disk to help expedite the submission\nprocess.\n\n## CLI Usage\n\nYou can download llamapacks directly using `llamaindex-cli`, which comes installed with the `llama-index` python package:\n\n```bash\nllamaindex-cli download-llamapack LlamaDatasetMetadataPack --download-dir ./llama_dataset_metadata_pack\n```\n\nYou can then inspect the files at `./llama_dataset_metadata_pack` and use them as a template for your own project!\n\n## Code Usage\n\nYou can download the pack to the `./llama_dataset_metadata_pack` directory through python\ncode as well. The sample script below demonstrates how to construct `LlamaDatasetMetadataPack`\nusing a `LabelledRagDataset` downloaded from `llama-hub` and a simple RAG pipeline\nbuilt off of its source documents.\n\n```python\nfrom llama_index.core.llama_pack import download_llama_pack\n\n# Download and install dependencies\nLlamaDatasetMetadataPack = download_llama_pack(\n \"LlamaDatasetMetadataPack\", \"./llama_dataset_metadata_pack\"\n)\n\n# construction requires a query_engine, a rag_dataset, and optionally a judge_llm\nllama_dataset_metadata_pack = LlamaDatasetMetadataPack()\n\n# create and save `card.json` and `README.md` to disk\ndataset_description = (\n \"A labelled RAG dataset based off an essay by Paul Graham, consisting of \"\n \"queries, reference answers, and reference contexts.\"\n)\n\nllama_dataset_metadata_pack.run(\n name=\"Paul Graham Essay Dataset\",\n description=dataset_description,\n rag_dataset=rag_dataset, # defined earlier not shown here\n index=index, # defined earlier not shown here\n benchmark_df=benchmark_df, # defined earlier not shown here\n baseline_name=\"llamaindex\",\n)\n```\n\nNOTE: this pack should be used only after performing a RAG evaluation (i.e., by\nusing `RagEvaluatorPack` on a `LabelledRagDataset`). In the code snippet above,\n`index`, `rag_dataset`, and `benchmark_df` are all objects that you'd expect to\nhave only after performing the RAG evaluation as mention in the previous sentence.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index packs llama_dataset_metadata integration",
"version": "0.3.0",
"project_urls": null,
"split_keywords": [
"evaluation",
" llamadataset",
" rag",
" submission"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8806aa3f0728e9c3028b320ff4f5b0649849b543064273051719c3b95696dcb9",
"md5": "9d049ee34bbae675dc20e972479f52fa",
"sha256": "4aac5fa2ceed75a6782437c7d138ae98c4867aaef2b3be34c468590d2c1e48f4"
},
"downloads": -1,
"filename": "llama_index_packs_llama_dataset_metadata-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9d049ee34bbae675dc20e972479f52fa",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.9",
"size": 4767,
"upload_time": "2024-11-17T22:43:10",
"upload_time_iso_8601": "2024-11-17T22:43:10.646267Z",
"url": "https://files.pythonhosted.org/packages/88/06/aa3f0728e9c3028b320ff4f5b0649849b543064273051719c3b95696dcb9/llama_index_packs_llama_dataset_metadata-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "96205e46995715657e292a8e233c0db91d939ae5099243c7071d575a5ed9c86b",
"md5": "527c295118952a3a29a728bc91897e8c",
"sha256": "80aec818488cff0792c6a1808e710715d5a0a484edb59b2297490ad8fc1a708f"
},
"downloads": -1,
"filename": "llama_index_packs_llama_dataset_metadata-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "527c295118952a3a29a728bc91897e8c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.9",
"size": 4370,
"upload_time": "2024-11-17T22:43:11",
"upload_time_iso_8601": "2024-11-17T22:43:11.575552Z",
"url": "https://files.pythonhosted.org/packages/96/20/5e46995715657e292a8e233c0db91d939ae5099243c7071d575a5ed9c86b/llama_index_packs_llama_dataset_metadata-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-17 22:43:11",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-packs-llama-dataset-metadata"
}
JSON