# LlamaDataset Metadata Pack
As part of the `LlamaDataset` submission package into [llamahub](https://llamahub.ai),
two metadata files are required, namely: `card.json` and `README.md`. This pack
creates these two files and saves them to disk to help expedite the submission
process.
## CLI Usage
You can download llamapacks directly using `llamaindex-cli`, which comes installed with the `llama-index` python package:
```bash
llamaindex-cli download-llamapack LlamaDatasetMetadataPack --download-dir ./llama_dataset_metadata_pack
```
You can then inspect the files at `./llama_dataset_metadata_pack` and use them as a template for your own project!
## Code Usage
You can download the pack to the `./llama_dataset_metadata_pack` directory through python
code as well. The sample script below demonstrates how to construct `LlamaDatasetMetadataPack`
using a `LabelledRagDataset` downloaded from `llama-hub` and a simple RAG pipeline
built off of its source documents.
```python
from llama_index.core.llama_pack import download_llama_pack
# Download and install dependencies
LlamaDatasetMetadataPack = download_llama_pack(
"LlamaDatasetMetadataPack", "./llama_dataset_metadata_pack"
)
# construction requires a query_engine, a rag_dataset, and optionally a judge_llm
llama_dataset_metadata_pack = LlamaDatasetMetadataPack()
# create and save `card.json` and `README.md` to disk
dataset_description = (
"A labelled RAG dataset based off an essay by Paul Graham, consisting of "
"queries, reference answers, and reference contexts."
)
llama_dataset_metadata_pack.run(
name="Paul Graham Essay Dataset",
description=dataset_description,
rag_dataset=rag_dataset, # defined earlier not shown here
index=index, # defined earlier not shown here
benchmark_df=benchmark_df, # defined earlier not shown here
baseline_name="llamaindex",
)
```
NOTE: this pack should be used only after performing a RAG evaluation (i.e., by
using `RagEvaluatorPack` on a `LabelledRagDataset`). In the code snippet above,
`index`, `rag_dataset`, and `benchmark_df` are all objects that you'd expect to
have only after performing the RAG evaluation as mention in the previous sentence.
Raw data
{
"_id": null,
"home_page": null,
"name": "llama-index-packs-llama-dataset-metadata",
"maintainer": "nerdai",
"docs_url": null,
"requires_python": "<4.0,>=3.8.1",
"maintainer_email": null,
"keywords": "evaluation, llamadataset, rag, submission",
"author": "Your Name",
"author_email": "you@example.com",
"download_url": "https://files.pythonhosted.org/packages/59/a7/3189230c6a14ebf359629f017e7a6b02041a0f68e7fa34c0277bf4991d91/llama_index_packs_llama_dataset_metadata-0.1.4.tar.gz",
"platform": null,
"description": "# LlamaDataset Metadata Pack\n\nAs part of the `LlamaDataset` submission package into [llamahub](https://llamahub.ai),\ntwo metadata files are required, namely: `card.json` and `README.md`. This pack\ncreates these two files and saves them to disk to help expedite the submission\nprocess.\n\n## CLI Usage\n\nYou can download llamapacks directly using `llamaindex-cli`, which comes installed with the `llama-index` python package:\n\n```bash\nllamaindex-cli download-llamapack LlamaDatasetMetadataPack --download-dir ./llama_dataset_metadata_pack\n```\n\nYou can then inspect the files at `./llama_dataset_metadata_pack` and use them as a template for your own project!\n\n## Code Usage\n\nYou can download the pack to the `./llama_dataset_metadata_pack` directory through python\ncode as well. The sample script below demonstrates how to construct `LlamaDatasetMetadataPack`\nusing a `LabelledRagDataset` downloaded from `llama-hub` and a simple RAG pipeline\nbuilt off of its source documents.\n\n```python\nfrom llama_index.core.llama_pack import download_llama_pack\n\n# Download and install dependencies\nLlamaDatasetMetadataPack = download_llama_pack(\n \"LlamaDatasetMetadataPack\", \"./llama_dataset_metadata_pack\"\n)\n\n# construction requires a query_engine, a rag_dataset, and optionally a judge_llm\nllama_dataset_metadata_pack = LlamaDatasetMetadataPack()\n\n# create and save `card.json` and `README.md` to disk\ndataset_description = (\n \"A labelled RAG dataset based off an essay by Paul Graham, consisting of \"\n \"queries, reference answers, and reference contexts.\"\n)\n\nllama_dataset_metadata_pack.run(\n name=\"Paul Graham Essay Dataset\",\n description=dataset_description,\n rag_dataset=rag_dataset, # defined earlier not shown here\n index=index, # defined earlier not shown here\n benchmark_df=benchmark_df, # defined earlier not shown here\n baseline_name=\"llamaindex\",\n)\n```\n\nNOTE: this pack should be used only after performing a RAG evaluation (i.e., by\nusing `RagEvaluatorPack` on a `LabelledRagDataset`). In the code snippet above,\n`index`, `rag_dataset`, and `benchmark_df` are all objects that you'd expect to\nhave only after performing the RAG evaluation as mention in the previous sentence.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "llama-index packs llama_dataset_metadata integration",
"version": "0.1.4",
"project_urls": null,
"split_keywords": [
"evaluation",
" llamadataset",
" rag",
" submission"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "13f771cf8762740e77624cf41a8d0ebdfd0b3ddea248e8adb54021ac35a4d0ef",
"md5": "f5c49170eaba1dce94dedc470738e64f",
"sha256": "6be162449ffe5c17f5e5142bb033b2b47823145fab39c2a5ab337eb9df2ca5b9"
},
"downloads": -1,
"filename": "llama_index_packs_llama_dataset_metadata-0.1.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f5c49170eaba1dce94dedc470738e64f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8.1",
"size": 4759,
"upload_time": "2024-04-08T19:39:20",
"upload_time_iso_8601": "2024-04-08T19:39:20.752549Z",
"url": "https://files.pythonhosted.org/packages/13/f7/71cf8762740e77624cf41a8d0ebdfd0b3ddea248e8adb54021ac35a4d0ef/llama_index_packs_llama_dataset_metadata-0.1.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "59a73189230c6a14ebf359629f017e7a6b02041a0f68e7fa34c0277bf4991d91",
"md5": "e4982c17960f704b3870cb8e953ef59f",
"sha256": "1337210bcd38d70d477daccd2b7cbaedb1d8386cffe9dcd552de706237ab17a6"
},
"downloads": -1,
"filename": "llama_index_packs_llama_dataset_metadata-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "e4982c17960f704b3870cb8e953ef59f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8.1",
"size": 4312,
"upload_time": "2024-04-08T19:39:22",
"upload_time_iso_8601": "2024-04-08T19:39:22.420137Z",
"url": "https://files.pythonhosted.org/packages/59/a7/3189230c6a14ebf359629f017e7a6b02041a0f68e7fa34c0277bf4991d91/llama_index_packs_llama_dataset_metadata-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-08 19:39:22",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "llama-index-packs-llama-dataset-metadata"
}
JSON