b10-xgrammar


Nameb10-xgrammar JSON
Version 0.1.22 PyPI version JSON
download
home_pageNone
SummaryEfficient, Flexible and Portable Structured Generation
upload_time2025-08-08 23:28:16
maintainerNone
docs_urlNone
authorMLC Team
requires_python<4,>=3.8
licenseApache 2.0
keywords machine learning inference
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls. 
            <div align="center" id="top">

<img src="https://raw.githubusercontent.com/mlc-ai/xgrammar/main/assets/logo.svg" alt="logo" width="400" margin="10px"></img>

[![Documentation](https://img.shields.io/badge/docs-latest-green)](https://xgrammar.mlc.ai/docs/)
[![License](https://img.shields.io/badge/license-apache_2-blue)](https://github.com/mlc-ai/xgrammar/blob/main/LICENSE)
[![PyPI](https://img.shields.io/pypi/v/xgrammar)](https://pypi.org/project/xgrammar)
[![PyPI Downloads](https://static.pepy.tech/badge/xgrammar)](https://pepy.tech/projects/xgrammar)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/mlc-ai/xgrammar)

**Efficient, Flexible and Portable Structured Generation**


[Get Started](#get-started) | [Documentation](https://xgrammar.mlc.ai/docs/) | [Blogpost](https://blog.mlc.ai/2024/11/22/achieving-efficient-flexible-portable-structured-generation-with-xgrammar) | [Technical Report](https://arxiv.org/abs/2411.15100)

</div>

## News
- [2025/02] XGrammar has been officially integrated into [Modular's MAX](https://docs.modular.com/max/serve/structured-output)
- [2025/01] XGrammar has been officially integrated into [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM).
- [2024/12] XGrammar has been officially integrated into [vLLM](https://github.com/vllm-project/vllm).
- [2024/12] We presented research talks on XGrammar at CMU, UC Berkeley, MIT, THU, SJTU, Ant Group, LMSys, Qingke AI, Camel AI. The slides can be found [here](https://docs.google.com/presentation/d/1iS7tu2EV4IKRWDaR0F3YD7ubrNqtGYUStSskceneelc/edit?usp=sharing).
- [2024/11] XGrammar has been officially integrated into [SGLang](https://github.com/sgl-project/sglang).
- [2024/11] XGrammar has been officially integrated into [MLC-LLM](https://github.com/mlc-ai/mlc-llm).
- [2024/11] We officially released XGrammar v0.1.0!

## Overview

XGrammar is an open-source library for efficient, flexible, and portable structured generation.

It leverages constrained decoding to ensure **100% structural correctness** of the output. It supports general context-free grammar to enable a broad range of structures, including **JSON**, **regex**, **custom context-free grammar**, etc.

XGrammar uses careful optimizations to achieve extremely low overhead in structured generation. It has achieved **near-zero overhead** in JSON generation, making it one of the fastest structured generation engines available.

XGrammar features **universal deployment**. It supports:
* **Platforms**: Linux, macOS, Windows
* **Hardware**: CPU, NVIDIA GPU, AMD GPU, Apple Silicon, TPU, etc.
* **Languages**: Python, C++, and JavaScript APIs
* **Models**: Qwen, Llama, DeepSeek, Phi, Gemma, etc.

XGrammar is very easy to integrate with LLM inference engines. It is the default structured generation backend for most LLM inference engines, including  [**vLLM**](https://github.com/vllm-project/vllm), [**SGLang**](https://github.com/sgl-project/sglang), [**TensorRT-LLM**](https://github.com/NVIDIA/TensorRT-LLM), and [**MLC-LLM**](https://github.com/mlc-ai/mlc-llm), as well as many other companies. You can also try out their structured generation modes!

## Get Started

Install XGrammar:
```bash
pip install xgrammar
```

Import XGrammar:
```python
import xgrammar as xgr
```

Please visit our [documentation](https://xgrammar.mlc.ai/docs/) to get started with XGrammar.
- [Installation](https://xgrammar.mlc.ai/docs/start/installation)
- [Quick start](https://xgrammar.mlc.ai/docs/start/quick_start)


## Adoption

XGrammar has been adopted by many projects and companies, including but not limited to:

<div align="center">

[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/databricks.svg" height=50/>](https://www.databricks.com/)
&emsp;
[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/nvidia.svg" height=50/>](https://github.com/NVIDIA/TensorRT-LLM)
&emsp;
[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/modular.svg" height=50/>](https://www.modular.com/)
&emsp;
[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/sglang.png" height=50/>](https://github.com/sgl-project/sglang)
&emsp;
[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/vllm.png" height=50/>](https://github.com/vllm-project/vllm)
&emsp;
[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/mlc.jpeg" height=50/>](https://github.com/mlc-ai/mlc-llm)
&emsp;
[<span style="font-size:50px">WebLLM</span>](https://github.com/mlc-ai/web-llm)

</div>

## Citation

If you find XGrammar useful in your research, please consider citing our paper:

```bibtex
@article{dong2024xgrammar,
  title={Xgrammar: Flexible and efficient structured generation engine for large language models},
  author={Dong, Yixin and Ruan, Charlie F and Cai, Yaxing and Lai, Ruihang and Xu, Ziyi and Zhao, Yilong and Chen, Tianqi},
  journal={Proceedings of Machine Learning and Systems 7},
  year={2024}
}
```

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "b10-xgrammar",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<4,>=3.8",
    "maintainer_email": null,
    "keywords": "machine learning, inference",
    "author": "MLC Team",
    "author_email": null,
    "download_url": null,
    "platform": null,
    "description": "<div align=\"center\" id=\"top\">\n\n<img src=\"https://raw.githubusercontent.com/mlc-ai/xgrammar/main/assets/logo.svg\" alt=\"logo\" width=\"400\" margin=\"10px\"></img>\n\n[![Documentation](https://img.shields.io/badge/docs-latest-green)](https://xgrammar.mlc.ai/docs/)\n[![License](https://img.shields.io/badge/license-apache_2-blue)](https://github.com/mlc-ai/xgrammar/blob/main/LICENSE)\n[![PyPI](https://img.shields.io/pypi/v/xgrammar)](https://pypi.org/project/xgrammar)\n[![PyPI Downloads](https://static.pepy.tech/badge/xgrammar)](https://pepy.tech/projects/xgrammar)\n[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/mlc-ai/xgrammar)\n\n**Efficient, Flexible and Portable Structured Generation**\n\n\n[Get Started](#get-started) | [Documentation](https://xgrammar.mlc.ai/docs/) | [Blogpost](https://blog.mlc.ai/2024/11/22/achieving-efficient-flexible-portable-structured-generation-with-xgrammar) | [Technical Report](https://arxiv.org/abs/2411.15100)\n\n</div>\n\n## News\n- [2025/02] XGrammar has been officially integrated into [Modular's MAX](https://docs.modular.com/max/serve/structured-output)\n- [2025/01] XGrammar has been officially integrated into [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM).\n- [2024/12] XGrammar has been officially integrated into [vLLM](https://github.com/vllm-project/vllm).\n- [2024/12] We presented research talks on XGrammar at CMU, UC Berkeley, MIT, THU, SJTU, Ant Group, LMSys, Qingke AI, Camel AI. The slides can be found [here](https://docs.google.com/presentation/d/1iS7tu2EV4IKRWDaR0F3YD7ubrNqtGYUStSskceneelc/edit?usp=sharing).\n- [2024/11] XGrammar has been officially integrated into [SGLang](https://github.com/sgl-project/sglang).\n- [2024/11] XGrammar has been officially integrated into [MLC-LLM](https://github.com/mlc-ai/mlc-llm).\n- [2024/11] We officially released XGrammar v0.1.0!\n\n## Overview\n\nXGrammar is an open-source library for efficient, flexible, and portable structured generation.\n\nIt leverages constrained decoding to ensure **100% structural correctness** of the output. It supports general context-free grammar to enable a broad range of structures, including **JSON**, **regex**, **custom context-free grammar**, etc.\n\nXGrammar uses careful optimizations to achieve extremely low overhead in structured generation. It has achieved **near-zero overhead** in JSON generation, making it one of the fastest structured generation engines available.\n\nXGrammar features **universal deployment**. It supports:\n* **Platforms**: Linux, macOS, Windows\n* **Hardware**: CPU, NVIDIA GPU, AMD GPU, Apple Silicon, TPU, etc.\n* **Languages**: Python, C++, and JavaScript APIs\n* **Models**: Qwen, Llama, DeepSeek, Phi, Gemma, etc.\n\nXGrammar is very easy to integrate with LLM inference engines. It is the default structured generation backend for most LLM inference engines, including  [**vLLM**](https://github.com/vllm-project/vllm), [**SGLang**](https://github.com/sgl-project/sglang), [**TensorRT-LLM**](https://github.com/NVIDIA/TensorRT-LLM), and [**MLC-LLM**](https://github.com/mlc-ai/mlc-llm), as well as many other companies. You can also try out their structured generation modes!\n\n## Get Started\n\nInstall XGrammar:\n```bash\npip install xgrammar\n```\n\nImport XGrammar:\n```python\nimport xgrammar as xgr\n```\n\nPlease visit our [documentation](https://xgrammar.mlc.ai/docs/) to get started with XGrammar.\n- [Installation](https://xgrammar.mlc.ai/docs/start/installation)\n- [Quick start](https://xgrammar.mlc.ai/docs/start/quick_start)\n\n\n## Adoption\n\nXGrammar has been adopted by many projects and companies, including but not limited to:\n\n<div align=\"center\">\n\n[<img src=\"https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/databricks.svg\" height=50/>](https://www.databricks.com/)\n&emsp;\n[<img src=\"https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/nvidia.svg\" height=50/>](https://github.com/NVIDIA/TensorRT-LLM)\n&emsp;\n[<img src=\"https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/modular.svg\" height=50/>](https://www.modular.com/)\n&emsp;\n[<img src=\"https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/sglang.png\" height=50/>](https://github.com/sgl-project/sglang)\n&emsp;\n[<img src=\"https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/vllm.png\" height=50/>](https://github.com/vllm-project/vllm)\n&emsp;\n[<img src=\"https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/mlc.jpeg\" height=50/>](https://github.com/mlc-ai/mlc-llm)\n&emsp;\n[<span style=\"font-size:50px\">WebLLM</span>](https://github.com/mlc-ai/web-llm)\n\n</div>\n\n## Citation\n\nIf you find XGrammar useful in your research, please consider citing our paper:\n\n```bibtex\n@article{dong2024xgrammar,\n  title={Xgrammar: Flexible and efficient structured generation engine for large language models},\n  author={Dong, Yixin and Ruan, Charlie F and Cai, Yaxing and Lai, Ruihang and Xu, Ziyi and Zhao, Yilong and Chen, Tianqi},\n  journal={Proceedings of Machine Learning and Systems 7},\n  year={2024}\n}\n```\n",
    "bugtrack_url": null,
    "license": "Apache 2.0",
    "summary": "Efficient, Flexible and Portable Structured Generation",
    "version": "0.1.22",
    "project_urls": {
        "GitHub": "https://github.com/mlc-ai/xgrammar",
        "Homepage": "https://xgrammar.mlc.ai/"
    },
    "split_keywords": [
        "machine learning",
        " inference"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "d3ecbd47cf2583c9ee28fcffda77640a93365fe0b6997d3b350988d95a96e54a",
                "md5": "efb580fc46cd9100875d56a950f72981",
                "sha256": "7bdac22cb15e95a6023b152f05a8f193e2e6cda78e4d3f77e0f6452c40080a3b"
            },
            "downloads": -1,
            "filename": "b10_xgrammar-0.1.22-cp312-cp312-manylinux_2_38_x86_64.whl",
            "has_sig": false,
            "md5_digest": "efb580fc46cd9100875d56a950f72981",
            "packagetype": "bdist_wheel",
            "python_version": "cp312",
            "requires_python": "<4,>=3.8",
            "size": 7453367,
            "upload_time": "2025-08-08T23:28:16",
            "upload_time_iso_8601": "2025-08-08T23:28:16.993062Z",
            "url": "https://files.pythonhosted.org/packages/d3/ec/bd47cf2583c9ee28fcffda77640a93365fe0b6997d3b350988d95a96e54a/b10_xgrammar-0.1.22-cp312-cp312-manylinux_2_38_x86_64.whl",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-08-08 23:28:16",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "mlc-ai",
    "github_project": "xgrammar",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "b10-xgrammar"
}
MLC Team
Elapsed time: 1.28439s