Low-code framework for building custom AI

Declarative deep learning framework for LLMs, multimodal models, and tabular AI.

Docs · ** Getting Started** · ** Examples** · Discord

Ludwig is a declarative deep learning framework that lets you train, fine-tune, and deploy AI models — from LLM fine-tuning to tabular classification — using a YAML config file and zero boilerplate Python.

model_type: llm
base_model: meta-llama/Llama-3.1-8B
adapter:
  type: lora
trainer:
  type: finetune
  epochs: 3
input_features:
  - name: instruction
    type: text
output_features:
  - name: response
    type: text
ludwig train --config model.yaml --dataset my_data.csv

Tech stack: Python 3.12 · PyTorch 2.7+ · Pydantic 2 · Transformers 5 · Ray 2.54

Ludwig is hosted by the Linux Foundation AI & Data.

pip install ludwig           # core
pip install ludwig[full]     # all optional dependencies
pip install ludwig[llm]      # LLM fine-tuning only

Requires Python 3.12+. See contributing for a full dependency matrix.

Ludwig supports the full LLM fine-tuning spectrum:

model_type: llm
base_model: meta-llama/Llama-3.1-8B

quantization:
  bits: 4

adapter:
  type: lora

prompt:
  template: |
    ### Instruction: {instruction}
    ### Input: {input}
    ### Response:

input_features:
  - name: prompt
    type: text

output_features:
  - name: output
    type: text

trainer:
  type: finetune
  learning_rate: 0.0001
  batch_size: 1
  gradient_accumulation_steps: 16
  epochs: 3
  learning_rate_scheduler:
    decay: cosine
    warmup_fraction: 0.01

backend:
  type: local
export HUGGING_FACE_HUB_TOKEN="<your_token>"
ludwig train --config model.yaml --dataset "ludwig://alpaca"
input_features:
  - name: review_text
    type: text
    encoder:
      type: bert
  - name: star_rating
    type: number
  - name: product_image
    type: image
    encoder:
      type: dinov2

output_features:
  - name: recommended
    type: binary
ludwig train --config model.yaml --dataset reviews.csv
ludwig generate_config "I have a CSV with age, income, education level, and I want to predict loan default"
ludwig predict --model_path results/experiment_run/model --dataset new_data.csv
ludwig serve --model_path results/experiment_run/model

LLM Fine-Tuning #

• Supervised fine-tuning (SFT) on instruction/response pairs
• Alignment training : DPO, KTO, ORPO, GRPO (reward-model-free RLHF)
• PEFT adapters : LoRA, DoRA, VeRA, LoRA+, TinyLoRA, OFT, HRA, WaveFT, LN-Tuning, VBLoRA, C3A
• LoRA initializers : PiSSA, EVA, CorDA, LoftQ for improved convergence
• Multi-adapter PEFT : multiple named adapters on one base model, switchable at runtime; merge with TIES, DARE, SVD, magnitude pruning
• Quantization : 4-bit/8-bit QLoRA (bitsandbytes), torchao int4/int8/float8 with QAT
• VLM fine-tuning : LLaVA, Qwen2-VL, InternVL viais_multimodal: true
• Sequence packing for efficient training on variable-length inputs
• Paged and 8-bit optimizers for memory-efficient training

Multimodal & Tabular Models #

• Input modalities : text, numbers, categories, binary, sets, bags, sequences, images, audio, timeseries, vectors, dates
• Text encoders : any HuggingFace Transformer (BERT, RoBERTa, ModernBERT, Qwen3, Llama-3.1, etc.), plus Mamba-2, Jamba
• Image encoders : DINOv2, ConvNeXt, EfficientNet, ViT, CAFormer, ConvFormer, PoolFormer, TIMM (1000+ models)
• Timeseries encoders : PatchTST, N-BEATS, CNN, RNN, Transformer; MASE and sMAPE metrics;model.forecast() API
• Combiners : concat, transformer, tab_transformer, FT-Transformer, TabNet, TabPFN v2, HyperNetwork, ProjectAggregate, GatedFusion, Perceiver
• Multi-task learning : multiple output features in a single model; Nash-MTL, Pareto-MTL, FAMO, GradNorm, uncertainty loss balancing
• Image segmentation : UNet, SegFormer, FPN decoders

Training Infrastructure #

• Distributed training : HuggingFace Accelerate with DDP, FSDP, DeepSpeed (zero-code changes)
• Ray backend : training across a Ray cluster, larger-than-memory datasets via Ray Data
• Automatic batch size selection and learning rate range test
• Mixed precision (fp16/bf16), gradient checkpointing, gradient accumulation
• Optimizers : AdamW, Adafactor, SGD, Muon, ScheduleFreeAdamW, Lion, paged/8-bit variants
• Learning rate schedulers : cosine, linear, polynomial, reduce-on-plateau, OneCycleLR
• Model Soup : uniform and greedy checkpoint averaging for better generalization at zero inference cost
• Modality dropout for robust multimodal models

Hyperparameter Optimization #

• Executors : Ray Tune (ASHA, PBT, Bayesian) and native Optuna (auto/GP/TPE/CMA-ES)
• Optuna persistence : SQLite or PostgreSQL for resumable HPO runs
• Pruning with Optuna's MedianPruner and HyperbandPruner
• Search spaces : uniform, log-uniform, choice, randint, quantized
• Full Ludwig config is searchable — any nested parameter can be a hyperparameter

Production & Deployment #

• REST API : FastAPI server with Prometheus metrics and structured logging (ludwig serve )
• vLLM serving : OpenAI-compatible API with PagedAttention and continuous batching
• Ray Serve : distributed deployment with auto-scaling and traffic splitting
• KServe : Kubernetes-native deployment with Open Inference Protocol v2
• Model export : SafeTensors (default),torch.export``.pt2 bundles, ONNX
• HuggingFace Hub :ludwig upload hf_hub — push model + auto-generated model card
• Docker : prebuilt containers atludwigai/ludwig

Tooling & Integrations #

• Experiment tracking : TensorBoard, Weights & Biases, Comet ML, MLflow, Aim Stack
• Model inspection :ModelInspector — weight enumeration, architecture summary, feature importance proxy
• Visualizations : learning curves, confusion matrices, calibration plots, ROC curves, hyperopt analysis
• AutoML :ludwig.automl.auto_train() — give it a dataset and a time budget; the YAML-driven search space samples encoder/combiner/decoder combinations and validates them before training
• Dataset quality checks :from ludwig.utils.dataset_quality import check_dataset_quality — validates a DataFrame before training (missing values, class imbalance, near-duplicate columns, ID leakage, …)
• OpenML integration : load any OpenML task directly —OpenML fetches by task ID and caches locally as Parquet
• LLM config generation :ludwig generate_config "describe your task" — LLM writes the YAML
• K-fold cross-validation :ludwig experiment --k_fold N
• Dataset Zoo : 70+ built-in benchmark datasets (ludwig://mnist ,ludwig://alpaca , …)

• Zero boilerplate — no training loop, no data pipeline, no evaluation code. The YAML config is the entire program.

• Best-in-class LLM support — full spectrum from LoRA to GRPO alignment, torchao QAT, and VLM fine-tuning, all in config.

• Multimodal out of the box — mix text, images, numbers, audio, and timeseries with one config change.

• Scale without code changes — go from laptop → multi-GPU → Ray cluster by changingbackend.type .

• Expert control when you need it — every activation function, scheduler, and optimizer is configurable.

• Reproducible research — every run is logged and the full config is saved. Compare experiments withludwig visualize .

• Ludwig: A Type-Based Declarative Deep Learning Toolbox (2019)

• Declarative Machine Learning Systems (2021)

• Ludwig's State-of-the-Art Benchmarks

• Discord — ask questions, share what you've built

• GitHub Issues — bugs and feature requests

• X / Twitter — announcements

• Medium — tutorials and deep-dives