<p align="center">
<a href="https://ludwig.ai">
<img src="https://github.com/ludwig-ai/ludwig-docs/raw/master/docs/images/ludwig_hero_smaller.jpg" height="150">
</a>
</p>
<div align="center">
_Declarative deep learning framework built for scale and efficiency._
[](https://badge.fury.io/py/ludwig)
[](https://discord.gg/CBgdrGnZjy)
[](https://hub.docker.com/r/ludwigai)
[](https://pepy.tech/project/ludwig)
[](https://github.com/ludwig-ai/ludwig/blob/master/LICENSE)
[](https://twitter.com/ludwig_ai)
</div>
> \[!IMPORTANT\]
> Our community has moved to [Discord](https://discord.gg/CBgdrGnZjy) -- please join us there!
# 📖 What is Ludwig?
Ludwig is a **low-code** framework for building **custom** AI models like **LLMs** and other deep neural networks.
Key features:
- 🛠 **Build custom models with ease:** a declarative YAML configuration file is all you need to train a state-of-the-art LLM on your data. Support for multi-task and multi-modality learning. Comprehensive config validation detects invalid parameter combinations and prevents runtime failures.
- ⚡ **Optimized for scale and efficiency:** automatic batch size selection, distributed training ([DDP](https://pytorch.org/tutorials/beginner/ddp_series_theory.html), [DeepSpeed](https://github.com/microsoft/DeepSpeed)), parameter efficient fine-tuning ([PEFT](https://github.com/huggingface/peft)), 4-bit quantization (QLoRA), paged and 8-bit optimizers, and larger-than-memory datasets.
- 📐 **Expert level control:** retain full control of your models down to the activation functions. Support for hyperparameter optimization, explainability, and rich metric visualizations.
- 🧱 **Modular and extensible:** experiment with different model architectures, tasks, features, and modalities with just a few parameter changes in the config. Think building blocks for deep learning.
- 🚢 **Engineered for production:** prebuilt [Docker](https://hub.docker.com/u/ludwigai) containers, native support for running with [Ray](https://www.ray.io/) on [Kubernetes](https://github.com/ray-project/kuberay), export models to [Torchscript](https://pytorch.org/docs/stable/jit.html) and [Triton](https://developer.nvidia.com/triton-inference-server), upload to [HuggingFace](https://huggingface.co/models) with one command.
Ludwig is hosted by the
[Linux Foundation AI & Data](https://lfaidata.foundation/).
# 💾 Installation
Install from PyPi. Be aware that Ludwig requires Python 3.8+.
```shell
pip install ludwig
```
Or install with all optional dependencies:
```shell
pip install ludwig[full]
```
Please see [contributing](https://github.com/ludwig-ai/ludwig/blob/master/CONTRIBUTING.md) for more detailed installation instructions.
# 🚂 Getting Started
Want to take a quick peak at some of the Ludwig 0.8 features? Check out this Colab Notebook 🚀 [](https://colab.research.google.com/drive/1lB4ALmEyvcMycE3Mlnsd7I3bc0zxvk39)
Looking to fine-tune Llama-2 or Mistral? Check out these notebooks:
1. Fine-Tune Llama-2-7b: [](https://colab.research.google.com/drive/1r4oSEwRJpYKBPM0M0RSh0pBEYK_gBKbe)
1. Fine-Tune Llama-2-13b: [](https://colab.research.google.com/drive/1zmSEzqZ7v4twBrXagj1TE_C--RNyVAyu)
1. Fine-Tune Mistral-7b: [](https://colab.research.google.com/drive/1i_8A1n__b7ljRWHzIsAdhO7u7r49vUm4)
For a full tutorial, check out the official [getting started guide](https://ludwig-ai.github.io/ludwig-docs/latest/getting_started/), or take a look at end-to-end [Examples](https://ludwig-ai.github.io/ludwig-docs/latest/examples).
## Large Language Model Fine-Tuning
[](https://colab.research.google.com/drive/1c3AO8l_H6V_x37RwQ8V7M6A-RmcBf2tG?usp=sharing)
Let's fine-tune a pretrained LLaMA-2-7b large language model to follow instructions like a chatbot ("instruction tuning").
### Prerequisites
- [HuggingFace API Token](https://huggingface.co/docs/hub/security-tokens)
- Access approval to [Llama2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf)
- GPU with at least 12 GiB of VRAM (in our tests, we used an Nvidia T4)
### Running
We'll use the [Stanford Alpaca](https://crfm.stanford.edu/2023/03/13/alpaca.html) dataset, which will be formatted as a table-like file that looks like this:
| instruction | input | output |
| :-----------------------------------------------: | :--------------: | :-----------------------------------------------: |
| Give three tips for staying healthy. | | 1.Eat a balanced diet and make sure to include... |
| Arrange the items given below in the order to ... | cake, me, eating | I eating cake. |
| Write an introductory paragraph about a famous... | Michelle Obama | Michelle Obama is an inspirational woman who r... |
| ... | ... | ... |
Create a YAML config file named `model.yaml` with the following:
```yaml
model_type: llm
base_model: meta-llama/Llama-2-7b-hf
quantization:
bits: 4
adapter:
type: lora
prompt:
template: |
Below is an instruction that describes a task, paired with an input that may provide further context.
Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Input:
{input}
### Response:
input_features:
- name: prompt
type: text
output_features:
- name: output
type: text
trainer:
type: finetune
learning_rate: 0.0001
batch_size: 1
gradient_accumulation_steps: 16
epochs: 3
learning_rate_scheduler:
decay: cosine
warmup_fraction: 0.01
preprocessing:
sample_ratio: 0.1
backend:
type: local
```
And now let's train the model:
```bash
export HUGGING_FACE_HUB_TOKEN = "<api_token>"
ludwig train --config model.yaml --dataset "ludwig://alpaca"
```
## Supervised ML
Let's build a neural network that predicts whether a given movie critic's review on [Rotten Tomatoes](https://www.kaggle.com/stefanoleone992/rotten-tomatoes-movies-and-critic-reviews-dataset) was positive or negative.
Our dataset will be a CSV file that looks like this:
| movie_title | content_rating | genres | runtime | top_critic | review_content | recommended |
| :------------------: | :------------: | :------------------------------: | :-----: | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- |
| Deliver Us from Evil | R | Action & Adventure, Horror | 117.0 | TRUE | Director Scott Derrickson and his co-writer, Paul Harris Boardman, deliver a routine procedural with unremarkable frights. | 0 |
| Barbara | PG-13 | Art House & International, Drama | 105.0 | FALSE | Somehow, in this stirring narrative, Barbara manages to keep hold of her principles, and her humanity and courage, and battles to save a dissident teenage girl whose life the Communists are trying to destroy. | 1 |
| Horrible Bosses | R | Comedy | 98.0 | FALSE | These bosses cannot justify either murder or lasting comic memories, fatally compromising a farce that could have been great but ends up merely mediocre. | 0 |
| ... | ... | ... | ... | ... | ... | ... |
Download a sample of the dataset from [here](https://ludwig.ai/latest/data/rotten_tomatoes.csv).
```bash
wget https://ludwig.ai/latest/data/rotten_tomatoes.csv
```
Next create a YAML config file named `model.yaml` with the following:
```yaml
input_features:
- name: genres
type: set
preprocessing:
tokenizer: comma
- name: content_rating
type: category
- name: top_critic
type: binary
- name: runtime
type: number
- name: review_content
type: text
encoder:
type: embed
output_features:
- name: recommended
type: binary
```
That's it! Now let's train the model:
```bash
ludwig train --config model.yaml --dataset rotten_tomatoes.csv
```
**Happy modeling**
Try applying Ludwig to your data. [Reach out on Discord](https://discord.gg/CBgdrGnZjy)
if you have any questions.
# ❓ Why you should use Ludwig
- **Minimal machine learning boilerplate**
Ludwig takes care of the engineering complexity of machine learning out of
the box, enabling research scientists to focus on building models at the
highest level of abstraction. Data preprocessing, hyperparameter
optimization, device management, and distributed training for
`torch.nn.Module` models come completely free.
- **Easily build your benchmarks**
Creating a state-of-the-art baseline and comparing it with a new model is a
simple config change.
- **Easily apply new architectures to multiple problems and datasets**
Apply new models across the extensive set of tasks and datasets that Ludwig
supports. Ludwig includes a
[full benchmarking toolkit](https://arxiv.org/abs/2111.04260) accessible to
any user, for running experiments with multiple models across multiple
datasets with just a simple configuration.
- **Highly configurable data preprocessing, modeling, and metrics**
Any and all aspects of the model architecture, training loop, hyperparameter
search, and backend infrastructure can be modified as additional fields in
the declarative configuration to customize the pipeline to meet your
requirements. For details on what can be configured, check out
[Ludwig Configuration](https://ludwig-ai.github.io/ludwig-docs/latest/configuration/)
docs.
- **Multi-modal, multi-task learning out-of-the-box**
Mix and match tabular data, text, images, and even audio into complex model
configurations without writing code.
- **Rich model exporting and tracking**
Automatically track all trials and metrics with tools like Tensorboard,
Comet ML, Weights & Biases, MLFlow, and Aim Stack.
- **Automatically scale training to multi-GPU, multi-node clusters**
Go from training on your local machine to the cloud without code changes.
- **Low-code interface for state-of-the-art models, including pre-trained Huggingface Transformers**
Ludwig also natively integrates with pre-trained models, such as the ones
available in [Huggingface Transformers](https://huggingface.co/docs/transformers/index).
Users can choose from a vast collection of state-of-the-art pre-trained
PyTorch models to use without needing to write any code at all. For example,
training a BERT-based sentiment analysis model with Ludwig is as simple as:
```shell
ludwig train --dataset sst5 --config_str "{input_features: [{name: sentence, type: text, encoder: bert}], output_features: [{name: label, type: category}]}"
```
- **Low-code interface for AutoML**
[Ludwig AutoML](https://ludwig-ai.github.io/ludwig-docs/latest/user_guide/automl/)
allows users to obtain trained models by providing just a dataset, the
target column, and a time budget.
```python
auto_train_results = ludwig.automl.auto_train(dataset=my_dataset_df, target=target_column_name, time_limit_s=7200)
```
- **Easy productionisation**
Ludwig makes it easy to serve deep learning models, including on GPUs.
Launch a REST API for your trained Ludwig model.
```shell
ludwig serve --model_path=/path/to/model
```
Ludwig supports exporting models to efficient Torchscript bundles.
```shell
ludwig export_torchscript -–model_path=/path/to/model
```
# 📚 Tutorials
- [Text Classification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/text_classification)
- [Tabular Data Classification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/adult_census_income)
- [Image Classification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/mnist)
- [Multimodal Classification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/multimodal_classification)
# 🔬 Example Use Cases
- [Named Entity Recognition Tagging](https://ludwig-ai.github.io/ludwig-docs/latest/examples/ner_tagging)
- [Natural Language Understanding](https://ludwig-ai.github.io/ludwig-docs/latest/examples/nlu)
- [Machine Translation](https://ludwig-ai.github.io/ludwig-docs/latest/examples/machine_translation)
- [Chit-Chat Dialogue Modeling through seq2seq](https://ludwig-ai.github.io/ludwig-docs/latest/examples/seq2seq)
- [Sentiment Analysis](https://ludwig-ai.github.io/ludwig-docs/latest/examples/sentiment_analysis)
- [One-shot Learning with Siamese Networks](https://ludwig-ai.github.io/ludwig-docs/latest/examples/oneshot)
- [Visual Question Answering](https://ludwig-ai.github.io/ludwig-docs/latest/examples/visual_qa)
- [Spoken Digit Speech Recognition](https://ludwig-ai.github.io/ludwig-docs/latest/examples/speech_recognition)
- [Speaker Verification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/speaker_verification)
- [Binary Classification (Titanic)](https://ludwig-ai.github.io/ludwig-docs/latest/examples/titanic)
- [Timeseries forecasting](https://ludwig-ai.github.io/ludwig-docs/latest/examples/forecasting)
- [Timeseries forecasting (Weather)](https://ludwig-ai.github.io/ludwig-docs/latest/examples/weather)
- [Movie rating prediction](https://ludwig-ai.github.io/ludwig-docs/latest/examples/movie_ratings)
- [Multi-label classification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/multi_label)
- [Multi-Task Learning](https://ludwig-ai.github.io/ludwig-docs/latest/examples/multi_task)
- [Simple Regression: Fuel Efficiency Prediction](https://ludwig-ai.github.io/ludwig-docs/latest/examples/fuel_efficiency)
- [Fraud Detection](https://ludwig-ai.github.io/ludwig-docs/latest/examples/fraud)
# 💡 More Information
Read our publications on [Ludwig](https://arxiv.org/pdf/1909.07930.pdf), [declarative ML](https://arxiv.org/pdf/2107.08148.pdf), and [Ludwig’s SoTA benchmarks](https://openreview.net/pdf?id=hwjnu6qW7E4).
Learn more about [how Ludwig works](https://ludwig-ai.github.io/ludwig-docs/latest/user_guide/how_ludwig_works/), [how to get started](https://ludwig-ai.github.io/ludwig-docs/latest/getting_started/), and work through more [examples](https://ludwig-ai.github.io/ludwig-docs/latest/examples).
If you are interested in [contributing](https://github.com/ludwig-ai/ludwig/blob/master/CONTRIBUTING.md), have questions, comments, or thoughts to share, or if you just want to be in the
know, please consider [joining our Community Discord](https://discord.gg/CBgdrGnZjy) and follow us on [X](https://twitter.com/ludwig_ai)!
# 🤝 Join the community to build Ludwig with us
Ludwig is an actively managed open-source project that relies on contributions from folks just like
you. Consider joining the active group of Ludwig contributors to make Ludwig an even
more accessible and feature rich framework for everyone to use!
<a href="https://github.com/ludwig-ai/ludwig/graphs/contributors">
<img src="https://contrib.rocks/image?repo=ludwig-ai/ludwig" />
</a><br/>
## Star History
[](https://star-history.com/#ludwig-ai/ludwig&Date)
# 👋 Getting Involved
- [Discord](https://discord.gg/CBgdrGnZjy)
- [X](https://twitter.com/ludwig_ai)
- [Medium](https://medium.com/ludwig-ai)
- [GitHub Issues](https://github.com/ludwig-ai/ludwig/issues)
Raw data
{
"_id": null,
"home_page": "https://github.com/ludwig-ai/ludwig",
"name": "ludwig",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "ludwig deep learning deep_learning machine machine_learning natural language processing computer vision",
"author": "Piero Molino",
"author_email": "piero.molino@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/24/f6/8cd58814a85ef43899f0bfc2273e6aac260a0577448239d5465866026148/ludwig-0.10.3.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <a href=\"https://ludwig.ai\">\n <img src=\"https://github.com/ludwig-ai/ludwig-docs/raw/master/docs/images/ludwig_hero_smaller.jpg\" height=\"150\">\n </a>\n</p>\n\n<div align=\"center\">\n\n_Declarative deep learning framework built for scale and efficiency._\n\n[](https://badge.fury.io/py/ludwig)\n[](https://discord.gg/CBgdrGnZjy)\n[](https://hub.docker.com/r/ludwigai)\n[](https://pepy.tech/project/ludwig)\n[](https://github.com/ludwig-ai/ludwig/blob/master/LICENSE)\n[](https://twitter.com/ludwig_ai)\n\n</div>\n\n> \\[!IMPORTANT\\]\n> Our community has moved to [Discord](https://discord.gg/CBgdrGnZjy) -- please join us there!\n\n# \ud83d\udcd6 What is Ludwig?\n\nLudwig is a **low-code** framework for building **custom** AI models like **LLMs** and other deep neural networks.\n\nKey features:\n\n- \ud83d\udee0 **Build custom models with ease:** a declarative YAML configuration file is all you need to train a state-of-the-art LLM on your data. Support for multi-task and multi-modality learning. Comprehensive config validation detects invalid parameter combinations and prevents runtime failures.\n- \u26a1 **Optimized for scale and efficiency:** automatic batch size selection, distributed training ([DDP](https://pytorch.org/tutorials/beginner/ddp_series_theory.html), [DeepSpeed](https://github.com/microsoft/DeepSpeed)), parameter efficient fine-tuning ([PEFT](https://github.com/huggingface/peft)), 4-bit quantization (QLoRA), paged and 8-bit optimizers, and larger-than-memory datasets.\n- \ud83d\udcd0 **Expert level control:** retain full control of your models down to the activation functions. Support for hyperparameter optimization, explainability, and rich metric visualizations.\n- \ud83e\uddf1 **Modular and extensible:** experiment with different model architectures, tasks, features, and modalities with just a few parameter changes in the config. Think building blocks for deep learning.\n- \ud83d\udea2 **Engineered for production:** prebuilt [Docker](https://hub.docker.com/u/ludwigai) containers, native support for running with [Ray](https://www.ray.io/) on [Kubernetes](https://github.com/ray-project/kuberay), export models to [Torchscript](https://pytorch.org/docs/stable/jit.html) and [Triton](https://developer.nvidia.com/triton-inference-server), upload to [HuggingFace](https://huggingface.co/models) with one command.\n\nLudwig is hosted by the\n[Linux Foundation AI & Data](https://lfaidata.foundation/).\n\n\n\n# \ud83d\udcbe Installation\n\nInstall from PyPi. Be aware that Ludwig requires Python 3.8+.\n\n```shell\npip install ludwig\n```\n\nOr install with all optional dependencies:\n\n```shell\npip install ludwig[full]\n```\n\nPlease see [contributing](https://github.com/ludwig-ai/ludwig/blob/master/CONTRIBUTING.md) for more detailed installation instructions.\n\n# \ud83d\ude82 Getting Started\n\nWant to take a quick peak at some of the Ludwig 0.8 features? Check out this Colab Notebook \ud83d\ude80 [](https://colab.research.google.com/drive/1lB4ALmEyvcMycE3Mlnsd7I3bc0zxvk39)\n\nLooking to fine-tune Llama-2 or Mistral? Check out these notebooks:\n\n1. Fine-Tune Llama-2-7b: [](https://colab.research.google.com/drive/1r4oSEwRJpYKBPM0M0RSh0pBEYK_gBKbe)\n1. Fine-Tune Llama-2-13b: [](https://colab.research.google.com/drive/1zmSEzqZ7v4twBrXagj1TE_C--RNyVAyu)\n1. Fine-Tune Mistral-7b: [](https://colab.research.google.com/drive/1i_8A1n__b7ljRWHzIsAdhO7u7r49vUm4)\n\nFor a full tutorial, check out the official [getting started guide](https://ludwig-ai.github.io/ludwig-docs/latest/getting_started/), or take a look at end-to-end [Examples](https://ludwig-ai.github.io/ludwig-docs/latest/examples).\n\n## Large Language Model Fine-Tuning\n\n[](https://colab.research.google.com/drive/1c3AO8l_H6V_x37RwQ8V7M6A-RmcBf2tG?usp=sharing)\n\nLet's fine-tune a pretrained LLaMA-2-7b large language model to follow instructions like a chatbot (\"instruction tuning\").\n\n### Prerequisites\n\n- [HuggingFace API Token](https://huggingface.co/docs/hub/security-tokens)\n- Access approval to [Llama2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf)\n- GPU with at least 12 GiB of VRAM (in our tests, we used an Nvidia T4)\n\n### Running\n\nWe'll use the [Stanford Alpaca](https://crfm.stanford.edu/2023/03/13/alpaca.html) dataset, which will be formatted as a table-like file that looks like this:\n\n| instruction | input | output |\n| :-----------------------------------------------: | :--------------: | :-----------------------------------------------: |\n| Give three tips for staying healthy. | | 1.Eat a balanced diet and make sure to include... |\n| Arrange the items given below in the order to ... | cake, me, eating | I eating cake. |\n| Write an introductory paragraph about a famous... | Michelle Obama | Michelle Obama is an inspirational woman who r... |\n| ... | ... | ... |\n\nCreate a YAML config file named `model.yaml` with the following:\n\n```yaml\nmodel_type: llm\nbase_model: meta-llama/Llama-2-7b-hf\n\nquantization:\n bits: 4\n\nadapter:\n type: lora\n\nprompt:\n template: |\n Below is an instruction that describes a task, paired with an input that may provide further context.\n Write a response that appropriately completes the request.\n\n ### Instruction:\n {instruction}\n\n ### Input:\n {input}\n\n ### Response:\n\ninput_features:\n - name: prompt\n type: text\n\noutput_features:\n - name: output\n type: text\n\ntrainer:\n type: finetune\n learning_rate: 0.0001\n batch_size: 1\n gradient_accumulation_steps: 16\n epochs: 3\n learning_rate_scheduler:\n decay: cosine\n warmup_fraction: 0.01\n\npreprocessing:\n sample_ratio: 0.1\n\nbackend:\n type: local\n```\n\nAnd now let's train the model:\n\n```bash\nexport HUGGING_FACE_HUB_TOKEN = \"<api_token>\"\n\nludwig train --config model.yaml --dataset \"ludwig://alpaca\"\n```\n\n## Supervised ML\n\nLet's build a neural network that predicts whether a given movie critic's review on [Rotten Tomatoes](https://www.kaggle.com/stefanoleone992/rotten-tomatoes-movies-and-critic-reviews-dataset) was positive or negative.\n\nOur dataset will be a CSV file that looks like this:\n\n| movie_title | content_rating | genres | runtime | top_critic | review_content | recommended |\n| :------------------: | :------------: | :------------------------------: | :-----: | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- |\n| Deliver Us from Evil | R | Action & Adventure, Horror | 117.0 | TRUE | Director Scott Derrickson and his co-writer, Paul Harris Boardman, deliver a routine procedural with unremarkable frights. | 0 |\n| Barbara | PG-13 | Art House & International, Drama | 105.0 | FALSE | Somehow, in this stirring narrative, Barbara manages to keep hold of her principles, and her humanity and courage, and battles to save a dissident teenage girl whose life the Communists are trying to destroy. | 1 |\n| Horrible Bosses | R | Comedy | 98.0 | FALSE | These bosses cannot justify either murder or lasting comic memories, fatally compromising a farce that could have been great but ends up merely mediocre. | 0 |\n| ... | ... | ... | ... | ... | ... | ... |\n\nDownload a sample of the dataset from [here](https://ludwig.ai/latest/data/rotten_tomatoes.csv).\n\n```bash\nwget https://ludwig.ai/latest/data/rotten_tomatoes.csv\n```\n\nNext create a YAML config file named `model.yaml` with the following:\n\n```yaml\ninput_features:\n - name: genres\n type: set\n preprocessing:\n tokenizer: comma\n - name: content_rating\n type: category\n - name: top_critic\n type: binary\n - name: runtime\n type: number\n - name: review_content\n type: text\n encoder:\n type: embed\noutput_features:\n - name: recommended\n type: binary\n```\n\nThat's it! Now let's train the model:\n\n```bash\nludwig train --config model.yaml --dataset rotten_tomatoes.csv\n```\n\n**Happy modeling**\n\nTry applying Ludwig to your data. [Reach out on Discord](https://discord.gg/CBgdrGnZjy)\nif you have any questions.\n\n# \u2753 Why you should use Ludwig\n\n- **Minimal machine learning boilerplate**\n\n Ludwig takes care of the engineering complexity of machine learning out of\n the box, enabling research scientists to focus on building models at the\n highest level of abstraction. Data preprocessing, hyperparameter\n optimization, device management, and distributed training for\n `torch.nn.Module` models come completely free.\n\n- **Easily build your benchmarks**\n\n Creating a state-of-the-art baseline and comparing it with a new model is a\n simple config change.\n\n- **Easily apply new architectures to multiple problems and datasets**\n\n Apply new models across the extensive set of tasks and datasets that Ludwig\n supports. Ludwig includes a\n [full benchmarking toolkit](https://arxiv.org/abs/2111.04260) accessible to\n any user, for running experiments with multiple models across multiple\n datasets with just a simple configuration.\n\n- **Highly configurable data preprocessing, modeling, and metrics**\n\n Any and all aspects of the model architecture, training loop, hyperparameter\n search, and backend infrastructure can be modified as additional fields in\n the declarative configuration to customize the pipeline to meet your\n requirements. For details on what can be configured, check out\n [Ludwig Configuration](https://ludwig-ai.github.io/ludwig-docs/latest/configuration/)\n docs.\n\n- **Multi-modal, multi-task learning out-of-the-box**\n\n Mix and match tabular data, text, images, and even audio into complex model\n configurations without writing code.\n\n- **Rich model exporting and tracking**\n\n Automatically track all trials and metrics with tools like Tensorboard,\n Comet ML, Weights & Biases, MLFlow, and Aim Stack.\n\n- **Automatically scale training to multi-GPU, multi-node clusters**\n\n Go from training on your local machine to the cloud without code changes.\n\n- **Low-code interface for state-of-the-art models, including pre-trained Huggingface Transformers**\n\n Ludwig also natively integrates with pre-trained models, such as the ones\n available in [Huggingface Transformers](https://huggingface.co/docs/transformers/index).\n Users can choose from a vast collection of state-of-the-art pre-trained\n PyTorch models to use without needing to write any code at all. For example,\n training a BERT-based sentiment analysis model with Ludwig is as simple as:\n\n ```shell\n ludwig train --dataset sst5 --config_str \"{input_features: [{name: sentence, type: text, encoder: bert}], output_features: [{name: label, type: category}]}\"\n ```\n\n- **Low-code interface for AutoML**\n\n [Ludwig AutoML](https://ludwig-ai.github.io/ludwig-docs/latest/user_guide/automl/)\n allows users to obtain trained models by providing just a dataset, the\n target column, and a time budget.\n\n ```python\n auto_train_results = ludwig.automl.auto_train(dataset=my_dataset_df, target=target_column_name, time_limit_s=7200)\n ```\n\n- **Easy productionisation**\n\n Ludwig makes it easy to serve deep learning models, including on GPUs.\n Launch a REST API for your trained Ludwig model.\n\n ```shell\n ludwig serve --model_path=/path/to/model\n ```\n\n Ludwig supports exporting models to efficient Torchscript bundles.\n\n ```shell\n ludwig export_torchscript -\u2013model_path=/path/to/model\n ```\n\n# \ud83d\udcda Tutorials\n\n- [Text Classification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/text_classification)\n- [Tabular Data Classification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/adult_census_income)\n- [Image Classification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/mnist)\n- [Multimodal Classification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/multimodal_classification)\n\n# \ud83d\udd2c Example Use Cases\n\n- [Named Entity Recognition Tagging](https://ludwig-ai.github.io/ludwig-docs/latest/examples/ner_tagging)\n- [Natural Language Understanding](https://ludwig-ai.github.io/ludwig-docs/latest/examples/nlu)\n- [Machine Translation](https://ludwig-ai.github.io/ludwig-docs/latest/examples/machine_translation)\n- [Chit-Chat Dialogue Modeling through seq2seq](https://ludwig-ai.github.io/ludwig-docs/latest/examples/seq2seq)\n- [Sentiment Analysis](https://ludwig-ai.github.io/ludwig-docs/latest/examples/sentiment_analysis)\n- [One-shot Learning with Siamese Networks](https://ludwig-ai.github.io/ludwig-docs/latest/examples/oneshot)\n- [Visual Question Answering](https://ludwig-ai.github.io/ludwig-docs/latest/examples/visual_qa)\n- [Spoken Digit Speech Recognition](https://ludwig-ai.github.io/ludwig-docs/latest/examples/speech_recognition)\n- [Speaker Verification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/speaker_verification)\n- [Binary Classification (Titanic)](https://ludwig-ai.github.io/ludwig-docs/latest/examples/titanic)\n- [Timeseries forecasting](https://ludwig-ai.github.io/ludwig-docs/latest/examples/forecasting)\n- [Timeseries forecasting (Weather)](https://ludwig-ai.github.io/ludwig-docs/latest/examples/weather)\n- [Movie rating prediction](https://ludwig-ai.github.io/ludwig-docs/latest/examples/movie_ratings)\n- [Multi-label classification](https://ludwig-ai.github.io/ludwig-docs/latest/examples/multi_label)\n- [Multi-Task Learning](https://ludwig-ai.github.io/ludwig-docs/latest/examples/multi_task)\n- [Simple Regression: Fuel Efficiency Prediction](https://ludwig-ai.github.io/ludwig-docs/latest/examples/fuel_efficiency)\n- [Fraud Detection](https://ludwig-ai.github.io/ludwig-docs/latest/examples/fraud)\n\n# \ud83d\udca1 More Information\n\nRead our publications on [Ludwig](https://arxiv.org/pdf/1909.07930.pdf), [declarative ML](https://arxiv.org/pdf/2107.08148.pdf), and [Ludwig\u2019s SoTA benchmarks](https://openreview.net/pdf?id=hwjnu6qW7E4).\n\nLearn more about [how Ludwig works](https://ludwig-ai.github.io/ludwig-docs/latest/user_guide/how_ludwig_works/), [how to get started](https://ludwig-ai.github.io/ludwig-docs/latest/getting_started/), and work through more [examples](https://ludwig-ai.github.io/ludwig-docs/latest/examples).\n\nIf you are interested in [contributing](https://github.com/ludwig-ai/ludwig/blob/master/CONTRIBUTING.md), have questions, comments, or thoughts to share, or if you just want to be in the\nknow, please consider [joining our Community Discord](https://discord.gg/CBgdrGnZjy) and follow us on [X](https://twitter.com/ludwig_ai)!\n\n# \ud83e\udd1d Join the community to build Ludwig with us\n\nLudwig is an actively managed open-source project that relies on contributions from folks just like\nyou. Consider joining the active group of Ludwig contributors to make Ludwig an even\nmore accessible and feature rich framework for everyone to use!\n\n<a href=\"https://github.com/ludwig-ai/ludwig/graphs/contributors\">\n <img src=\"https://contrib.rocks/image?repo=ludwig-ai/ludwig\" />\n</a><br/>\n\n## Star History\n\n[](https://star-history.com/#ludwig-ai/ludwig&Date)\n\n# \ud83d\udc4b Getting Involved\n\n- [Discord](https://discord.gg/CBgdrGnZjy)\n- [X](https://twitter.com/ludwig_ai)\n- [Medium](https://medium.com/ludwig-ai)\n- [GitHub Issues](https://github.com/ludwig-ai/ludwig/issues)",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Declarative machine learning: End-to-end machine learning pipelines using data-driven configurations.",
"version": "0.10.3",
"project_urls": {
"Download": "https://pypi.org/project/ludwig/",
"Homepage": "https://github.com/ludwig-ai/ludwig"
},
"split_keywords": [
"ludwig",
"deep",
"learning",
"deep_learning",
"machine",
"machine_learning",
"natural",
"language",
"processing",
"computer",
"vision"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "24f68cd58814a85ef43899f0bfc2273e6aac260a0577448239d5465866026148",
"md5": "bab99a6ec5256cf0654885f0c7c91da5",
"sha256": "513c89820811e3009bc9683c75c0b6804ebb839391aa5ffdb19266ad3e783199"
},
"downloads": -1,
"filename": "ludwig-0.10.3.tar.gz",
"has_sig": false,
"md5_digest": "bab99a6ec5256cf0654885f0c7c91da5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 1064997,
"upload_time": "2024-04-08T23:26:12",
"upload_time_iso_8601": "2024-04-08T23:26:12.997657Z",
"url": "https://files.pythonhosted.org/packages/24/f6/8cd58814a85ef43899f0bfc2273e6aac260a0577448239d5465866026148/ludwig-0.10.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-08 23:26:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ludwig-ai",
"github_project": "ludwig",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "ludwig"
}
JSON
