# LFM2 - Liquid Foundation Model 2 (Minimal Implementation)
[](https://badge.fury.io/py/lfm2)
[](https://opensource.org/licenses/MIT)
This is a minimal, open-source implementation of the Liquid Foundation Model 2 (LFM2) architecture described in [Liquid AI's blog post](https://www.liquid.ai/blog/liquid-foundation-models-v2-our-second-series-of-generative-ai-models). Since there is no official open-source implementation available, this repository provides a PyTorch implementation of the core architecture for research and educational purposes.
## Features
- Hybrid architecture combining short-range convolutions with grouped query attention
- 16 blocks: 10 LIV convolution blocks + 6 GQA blocks
- Double-gated short-range convolutions for efficient local processing
- Grouped Query Attention (GQA) for efficient global attention
- SwiGLU activation functions
- RMSNorm normalization
- Rotary Positional Embeddings (RoPE)
## Model Sizes
The implementation supports three model sizes:
- 350M parameters (768 hidden size)
- 700M parameters (1024 hidden size)
- 1.2B parameters (1536 hidden size)
## Installation
```bash
pip3 install -U lfm2
```
## Quick Start
```python
import torch
from lfm2.main import create_lfm2_model
# Create a model
model = create_lfm2_model(
model_size="700M", # Choose from: "350M", "700M", "1.2B"
vocab_size=32768,
max_seq_length=32768,
verbose=True
)
# Example forward pass
batch_size = 2
seq_length = 32
input_ids = torch.randint(0, model.config.vocab_size, (batch_size, seq_length))
# Generate outputs
with torch.no_grad():
outputs = model(input_ids)
logits = outputs["logits"]
```
## Architecture Details
### LIV Convolution Blocks
The model uses Linear Input-Varying (LIV) convolution blocks that combine double-gating with short-range convolutions:
```python
def lfm2_conv(x):
B, C, x = linear(x) # input projection
x = B*x # gating (gate depends on input)
x = conv(x) # short conv
x = C*x # gating
x = linear(x)
return x
```
### Grouped Query Attention
The model implements Grouped Query Attention (GQA) for efficient global attention processing, reducing memory and computational requirements while maintaining model quality.
## Usage Examples
Check `example.py` for detailed usage examples including:
1. Basic forward pass
2. Forward pass with attention masks
3. Forward pass with caching
4. Forward pass with all outputs
5. Forward pass with custom position IDs
## Citation
If you use this implementation in your research, please cite:
```bibtex
@misc{lfm2_minimal,
author = {Kye Gomez},
title = {LFM2: Minimal Implementation of Liquid Foundation Model 2},
year = {2024},
publisher = {GitHub},
url = {https://github.com/kyegomez/LFM2}
}
```
## Disclaimer
This is an unofficial, minimal implementation based on publicly available information about the LFM2 architecture. It is not affiliated with or endorsed by Liquid AI. The implementation may differ from the original model in various aspects.
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
Raw data
{
"_id": null,
"home_page": "https://github.com/kyegomez/LFM2",
"name": "lfm2",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": "artificial intelligence, deep learning, optimizers, Prompt Engineering",
"author": "Kye Gomez",
"author_email": "kye@apac.ai",
"download_url": "https://files.pythonhosted.org/packages/c9/ff/2a8caebed5fc83573400829b0bf37e8d74db52012a265cd9e10706814d75/lfm2-0.0.2.tar.gz",
"platform": null,
"description": "# LFM2 - Liquid Foundation Model 2 (Minimal Implementation)\n\n[](https://badge.fury.io/py/lfm2)\n[](https://opensource.org/licenses/MIT)\n\nThis is a minimal, open-source implementation of the Liquid Foundation Model 2 (LFM2) architecture described in [Liquid AI's blog post](https://www.liquid.ai/blog/liquid-foundation-models-v2-our-second-series-of-generative-ai-models). Since there is no official open-source implementation available, this repository provides a PyTorch implementation of the core architecture for research and educational purposes.\n\n## Features\n\n- Hybrid architecture combining short-range convolutions with grouped query attention\n- 16 blocks: 10 LIV convolution blocks + 6 GQA blocks\n- Double-gated short-range convolutions for efficient local processing\n- Grouped Query Attention (GQA) for efficient global attention\n- SwiGLU activation functions\n- RMSNorm normalization\n- Rotary Positional Embeddings (RoPE)\n\n## Model Sizes\n\nThe implementation supports three model sizes:\n\n- 350M parameters (768 hidden size)\n- 700M parameters (1024 hidden size)\n- 1.2B parameters (1536 hidden size)\n\n## Installation\n\n```bash\npip3 install -U lfm2 \n```\n\n## Quick Start\n\n```python\nimport torch\nfrom lfm2.main import create_lfm2_model\n\n# Create a model\nmodel = create_lfm2_model(\n model_size=\"700M\", # Choose from: \"350M\", \"700M\", \"1.2B\"\n vocab_size=32768,\n max_seq_length=32768,\n verbose=True\n)\n\n# Example forward pass\nbatch_size = 2\nseq_length = 32\ninput_ids = torch.randint(0, model.config.vocab_size, (batch_size, seq_length))\n\n# Generate outputs\nwith torch.no_grad():\n outputs = model(input_ids)\n logits = outputs[\"logits\"]\n```\n\n## Architecture Details\n\n### LIV Convolution Blocks\nThe model uses Linear Input-Varying (LIV) convolution blocks that combine double-gating with short-range convolutions:\n\n```python\ndef lfm2_conv(x):\n B, C, x = linear(x) # input projection\n x = B*x # gating (gate depends on input)\n x = conv(x) # short conv\n x = C*x # gating\n x = linear(x)\n return x\n```\n\n### Grouped Query Attention\nThe model implements Grouped Query Attention (GQA) for efficient global attention processing, reducing memory and computational requirements while maintaining model quality.\n\n## Usage Examples\n\nCheck `example.py` for detailed usage examples including:\n1. Basic forward pass\n2. Forward pass with attention masks\n3. Forward pass with caching\n4. Forward pass with all outputs\n5. Forward pass with custom position IDs\n\n## Citation\n\nIf you use this implementation in your research, please cite:\n\n```bibtex\n@misc{lfm2_minimal,\n author = {Kye Gomez},\n title = {LFM2: Minimal Implementation of Liquid Foundation Model 2},\n year = {2024},\n publisher = {GitHub},\n url = {https://github.com/kyegomez/LFM2}\n}\n```\n\n## Disclaimer\n\nThis is an unofficial, minimal implementation based on publicly available information about the LFM2 architecture. It is not affiliated with or endorsed by Liquid AI. The implementation may differ from the original model in various aspects.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "LFM2 - Pytorch",
"version": "0.0.2",
"project_urls": {
"Documentation": "https://github.com/kyegomez/LFM2",
"Homepage": "https://github.com/kyegomez/LFM2",
"Repository": "https://github.com/kyegomez/LFM2"
},
"split_keywords": [
"artificial intelligence",
" deep learning",
" optimizers",
" prompt engineering"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5ce536d814029ba0ed79fe6e4d0103d5f5f51b374d2308a7929de8df2a4b43be",
"md5": "8261229486c5f5d121e64dcf4bb08d5c",
"sha256": "fdc6b6fbecd34131071ccda2fad8b2a1c93e584b5d8d64230e58f29b286a613d"
},
"downloads": -1,
"filename": "lfm2-0.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8261229486c5f5d121e64dcf4bb08d5c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 11600,
"upload_time": "2025-07-10T16:15:16",
"upload_time_iso_8601": "2025-07-10T16:15:16.430703Z",
"url": "https://files.pythonhosted.org/packages/5c/e5/36d814029ba0ed79fe6e4d0103d5f5f51b374d2308a7929de8df2a4b43be/lfm2-0.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c9ff2a8caebed5fc83573400829b0bf37e8d74db52012a265cd9e10706814d75",
"md5": "ea2547d92b12f456bcc111fe93f240a4",
"sha256": "3d2ab18b6a63da226917d307de0b87a8a0596959033689d73661e77a81b2d42d"
},
"downloads": -1,
"filename": "lfm2-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "ea2547d92b12f456bcc111fe93f240a4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 12130,
"upload_time": "2025-07-10T16:15:17",
"upload_time_iso_8601": "2025-07-10T16:15:17.323314Z",
"url": "https://files.pythonhosted.org/packages/c9/ff/2a8caebed5fc83573400829b0bf37e8d74db52012a265cd9e10706814d75/lfm2-0.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-10 16:15:17",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kyegomez",
"github_project": "LFM2",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "torch",
"specs": []
},
{
"name": "loguru",
"specs": []
}
],
"lcname": "lfm2"
}
JSON
