{"slug": "low-code-framework-for-building-custom-ai", "title": "Low-code framework for building custom AI", "summary": "Ludwig, a declarative deep learning framework hosted by the Linux Foundation AI & Data, announced support for Python 3.12, PyTorch 2.7+, Pydantic 2, Transformers 5, and Ray 2.54, enabling users to train, fine-tune, and deploy AI models using a YAML config file with zero boilerplate Python. The framework now includes advanced features such as PatchTST and N-BEATS encoders for time-series forecasting, VLM fine-tuning for models like LLaVA and Qwen2-VL, and GRPO alignment, among others.", "body_md": "**Declarative deep learning framework for LLMs, multimodal models, and tabular AI.**\n\n[**Docs**](https://ludwig.ai) · [** Getting Started**](https://ludwig.ai/latest/getting_started/) · [** Examples**](https://ludwig.ai/latest/examples) · **Discord**\n\nLudwig is a **declarative deep learning framework** that lets you train, fine-tune, and deploy AI models — from LLM fine-tuning to tabular classification — using a YAML config file and zero boilerplate Python.\n\n```\n# Fine-tune Llama-3.1 with LoRA in one config file\nmodel_type: llm\nbase_model: meta-llama/Llama-3.1-8B\nadapter:\n  type: lora\ntrainer:\n  type: finetune\n  epochs: 3\ninput_features:\n  - name: instruction\n    type: text\noutput_features:\n  - name: response\n    type: text\nludwig train --config model.yaml --dataset my_data.csv\n```\n\n**Tech stack:** Python 3.12 · PyTorch 2.7+ · Pydantic 2 · Transformers 5 · Ray 2.54\n\nLudwig is hosted by the [Linux Foundation AI & Data](https://lfaidata.foundation/).\n\n| Feature | Description | \n|---|---|\n| **PatchTST & N-BEATS encoders** | State-of-the-art timeseries forecasting encoders with MASE/sMAPE metrics | \n| **Advanced PEFT adapters** | PiSSA, EVA, CorDA/LoftQ initializers; TinyLoRA, OFT, HRA, WaveFT, LN-Tuning, VBLoRA, C3A adapter types | \n| **VLM fine-tuning** | Train LLaVA, Qwen2-VL, InternVL via `is_multimodal: true` with gated cross-attention | \n| **HyperNetwork combiner** | Conditioning-based feature fusion — one feature generates weights for others | \n| **Nash-MTL & Pareto-MTL** | Game-theoretic and preference-based multi-task loss balancing | \n| **LLM config generation** | `ludwig generate_config \"describe your task\"` — LLM writes the YAML for you | \n| **ModelInspector** | Architecture analysis, weight collection, feature importance proxy | \n| **Ray Serve & KServe** | Distributed and Kubernetes-native model deployment shims | \n| **GRPO alignment** | Reward-model-free RLHF via Group Relative Policy Optimization | \n| **torchao quantization + QAT** | PyTorch-native `int4/int8/float8` with Quantization-Aware Training | \n| **Multi-adapter PEFT** | Multiple named LoRA adapters with weighted merging (TIES, DARE, SVD) | \n| **Native Optuna executor** | GPT/TPE/CMA-ES samplers, pruning, resumable SQLite/PostgreSQL storage | \n| **Timeseries forecasting** | `model.forecast(dataset, horizon=N)` API with`TimeseriesOutputFeature` | \n| **Muon & ScheduleFreeAdamW** | New optimizers for large-scale pretraining and fine-tuning | \n| **Image segmentation decoders** | UNet, SegFormer, FPN decoders for semantic segmentation | \n\n```\npip install ludwig           # core\npip install ludwig[full]     # all optional dependencies\npip install ludwig[llm]      # LLM fine-tuning only\n```\n\nRequires Python 3.12+. See [contributing](https://github.com/ludwig-ai/ludwig/blob/main/CONTRIBUTING.md) for a full dependency matrix.\n\nLudwig supports the full LLM fine-tuning spectrum:\n\n| Technique | Config key | \n|---|---|\n| Supervised fine-tuning (SFT) | `trainer.type: finetune` | \n| DPO / KTO / ORPO / GRPO alignment | `trainer.type: dpo` (or`kto` ,`orpo` ,`grpo` ) | \n| LoRA / DoRA / VeRA / PiSSA | `adapter.type: lora` (or`dora` ,`vera` ,`lora` +`init_weights: pissa` ) | \n| 4-bit QLoRA (bitsandbytes) | `quantization.bits: 4` | \n| torchao + QAT | `quantization.backend: torchao` | \n| Multi-adapter with merging | `adapters:` dict +`merge:` block | \n| VLM (vision-language) | `is_multimodal: true` | \n\n```\nmodel_type: llm\nbase_model: meta-llama/Llama-3.1-8B\n\nquantization:\n  bits: 4\n\nadapter:\n  type: lora\n\nprompt:\n  template: |\n    ### Instruction: {instruction}\n    ### Input: {input}\n    ### Response:\n\ninput_features:\n  - name: prompt\n    type: text\n\noutput_features:\n  - name: output\n    type: text\n\ntrainer:\n  type: finetune\n  learning_rate: 0.0001\n  batch_size: 1\n  gradient_accumulation_steps: 16\n  epochs: 3\n  learning_rate_scheduler:\n    decay: cosine\n    warmup_fraction: 0.01\n\nbackend:\n  type: local\nexport HUGGING_FACE_HUB_TOKEN=\"<your_token>\"\nludwig train --config model.yaml --dataset \"ludwig://alpaca\"\ninput_features:\n  - name: review_text\n    type: text\n    encoder:\n      type: bert\n  - name: star_rating\n    type: number\n  - name: product_image\n    type: image\n    encoder:\n      type: dinov2\n\noutput_features:\n  - name: recommended\n    type: binary\nludwig train --config model.yaml --dataset reviews.csv\nludwig generate_config \"I have a CSV with age, income, education level, and I want to predict loan default\"\nludwig predict --model_path results/experiment_run/model --dataset new_data.csv\nludwig serve --model_path results/experiment_run/model\n# POST http://localhost:8000/predict\n```\n\n## **LLM Fine-Tuning**\n\n- **Supervised fine-tuning (SFT)** on instruction/response pairs\n- **Alignment training** : DPO, KTO, ORPO, GRPO (reward-model-free RLHF)\n- **PEFT adapters** : LoRA, DoRA, VeRA, LoRA+, TinyLoRA, OFT, HRA, WaveFT, LN-Tuning, VBLoRA, C3A\n- **LoRA initializers** : PiSSA, EVA, CorDA, LoftQ for improved convergence\n- **Multi-adapter PEFT** : multiple named adapters on one base model, switchable at runtime; merge with TIES, DARE, SVD, magnitude pruning\n- **Quantization** : 4-bit/8-bit QLoRA (bitsandbytes), torchao int4/int8/float8 with QAT\n- **VLM fine-tuning** : LLaVA, Qwen2-VL, InternVL via`is_multimodal: true`\n- **Sequence packing** for efficient training on variable-length inputs\n- **Paged and 8-bit optimizers** for memory-efficient training\n\n## **Multimodal & Tabular Models**\n\n- **Input modalities** : text, numbers, categories, binary, sets, bags, sequences, images, audio, timeseries, vectors, dates\n- **Text encoders** : any HuggingFace Transformer (BERT, RoBERTa, ModernBERT, Qwen3, Llama-3.1, etc.), plus Mamba-2, Jamba\n- **Image encoders** : DINOv2, ConvNeXt, EfficientNet, ViT, CAFormer, ConvFormer, PoolFormer, TIMM (1000+ models)\n- **Timeseries encoders** : PatchTST, N-BEATS, CNN, RNN, Transformer; MASE and sMAPE metrics;`model.forecast()` API\n- **Combiners** : concat, transformer, tab_transformer, FT-Transformer, TabNet, TabPFN v2, HyperNetwork, ProjectAggregate, GatedFusion, Perceiver\n- **Multi-task learning** : multiple output features in a single model; Nash-MTL, Pareto-MTL, FAMO, GradNorm, uncertainty loss balancing\n- **Image segmentation** : UNet, SegFormer, FPN decoders\n\n## **Training Infrastructure**\n\n- **Distributed training** : HuggingFace Accelerate with DDP, FSDP, DeepSpeed (zero-code changes)\n- **Ray backend** : training across a Ray cluster, larger-than-memory datasets via Ray Data\n- **Automatic batch size selection** and learning rate range test\n- **Mixed precision** (fp16/bf16), gradient checkpointing, gradient accumulation\n- **Optimizers** : AdamW, Adafactor, SGD, Muon, ScheduleFreeAdamW, Lion, paged/8-bit variants\n- **Learning rate schedulers** : cosine, linear, polynomial, reduce-on-plateau, OneCycleLR\n- **Model Soup** : uniform and greedy checkpoint averaging for better generalization at zero inference cost\n- **Modality dropout** for robust multimodal models\n\n## **Hyperparameter Optimization**\n\n- **Executors** : Ray Tune (ASHA, PBT, Bayesian) and native Optuna (auto/GP/TPE/CMA-ES)\n- **Optuna persistence** : SQLite or PostgreSQL for resumable HPO runs\n- **Pruning** with Optuna's MedianPruner and HyperbandPruner\n- **Search spaces** : uniform, log-uniform, choice, randint, quantized\n- **Full Ludwig config** is searchable — any nested parameter can be a hyperparameter\n\n## **Production & Deployment**\n\n- **REST API** : FastAPI server with Prometheus metrics and structured logging (`ludwig serve` )\n- **vLLM serving** : OpenAI-compatible API with PagedAttention and continuous batching\n- **Ray Serve** : distributed deployment with auto-scaling and traffic splitting\n- **KServe** : Kubernetes-native deployment with Open Inference Protocol v2\n- **Model export** : SafeTensors (default),`torch.export``.pt2` bundles, ONNX\n- **HuggingFace Hub** :`ludwig upload hf_hub` — push model + auto-generated model card\n- **Docker** : prebuilt containers at[ludwigai/ludwig](https://hub.docker.com/u/ludwigai)\n\n## **Tooling & Integrations**\n\n- **Experiment tracking** : TensorBoard, Weights & Biases, Comet ML, MLflow, Aim Stack\n- **Model inspection** :`ModelInspector` — weight enumeration, architecture summary, feature importance proxy\n- **Visualizations** : learning curves, confusion matrices, calibration plots, ROC curves, hyperopt analysis\n- **AutoML** :`ludwig.automl.auto_train()` — give it a dataset and a time budget; the YAML-driven search space samples encoder/combiner/decoder combinations and validates them before training\n- **Dataset quality checks** :`from ludwig.utils.dataset_quality import check_dataset_quality` — validates a DataFrame before training (missing values, class imbalance, near-duplicate columns, ID leakage, …)\n- **OpenML integration** : load any OpenML task directly —`OpenMLLoader` fetches by task ID and caches locally as Parquet\n- **LLM config generation** :`ludwig generate_config \"describe your task\"` — LLM writes the YAML\n- **K-fold cross-validation** :`ludwig experiment --k_fold N`\n- **Dataset Zoo** : 70+ built-in benchmark datasets (`ludwig://mnist` ,`ludwig://alpaca` , …)\n\n| Use Case | Link | \n|---|---|\n| LLM instruction tuning (LoRA + QLoRA) | [examples/llm](https://ludwig.ai/latest/examples/llm/llm_finetuning) | \n| DPO / GRPO alignment | [examples/llm/alignment](https://ludwig.ai/latest/examples/llm/alignment) | \n| Advanced PEFT (PiSSA, OFT, VBLoRA, …) | [examples/llms/peft_advanced](https://ludwig.ai/latest/examples/llms/peft_advanced) | \n| VLM fine-tuning (LLaVA, Qwen2-VL) | [examples/vlm](https://github.com/ludwig-ai/ludwig/tree/main/examples/vlm) | \n\n| Use Case | Link | \n|---|---|\n| Binary classification (Titanic) | [examples/titanic](https://ludwig.ai/latest/examples/titanic) | \n| Tabular classification (census income) | [examples/adult_census_income](https://ludwig.ai/latest/examples/adult_census_income) | \n| Multimodal classification | [examples/multimodal_classification](https://ludwig.ai/latest/examples/multimodal_classification) | \n| Multi-task learning | [examples/multi_task](https://ludwig.ai/latest/examples/multi_task) | \n\n| Use Case | Link | \n|---|---|\n| Timeseries forecasting (PatchTST, N-BEATS) | [examples/forecasting](https://ludwig.ai/latest/examples/forecasting) | \n| Weather forecasting | [examples/weather](https://ludwig.ai/latest/examples/weather) | \n| Image classification (MNIST) | [examples/mnist](https://ludwig.ai/latest/examples/mnist) | \n| Semantic segmentation | [examples/semantic_segmentation](https://ludwig.ai/latest/examples/semantic_segmentation) | \n\n| Use Case | Link | \n|---|---|\n| Text classification | [examples/text_classification](https://ludwig.ai/latest/examples/text_classification) | \n| Named entity recognition | [examples/ner_tagging](https://ludwig.ai/latest/examples/ner_tagging) | \n| Machine translation | [examples/machine_translation](https://ludwig.ai/latest/examples/machine_translation) | \n| Speech recognition | [examples/speech_recognition](https://ludwig.ai/latest/examples/speech_recognition) | \n| Speaker verification | [examples/speaker_verification](https://ludwig.ai/latest/examples/speaker_verification) | \n\n- **Zero boilerplate** — no training loop, no data pipeline, no evaluation code. The YAML config is the entire program.\n- **Best-in-class LLM support** — full spectrum from LoRA to GRPO alignment, torchao QAT, and VLM fine-tuning, all in config.\n- **Multimodal out of the box** — mix text, images, numbers, audio, and timeseries with one config change.\n- **Scale without code changes** — go from laptop → multi-GPU → Ray cluster by changing`backend.type` .\n- **Expert control when you need it** — every activation function, scheduler, and optimizer is configurable.\n- **Reproducible research** — every run is logged and the full config is saved. Compare experiments with`ludwig visualize` .\n\n- [Ludwig: A Type-Based Declarative Deep Learning Toolbox](https://arxiv.org/pdf/1909.07930.pdf) (2019)\n- [Declarative Machine Learning Systems](https://arxiv.org/pdf/2107.08148.pdf) (2021)\n- [Ludwig's State-of-the-Art Benchmarks](https://openreview.net/pdf?id=hwjnu6qW7E4)\n\n- [Discord](https://discord.gg/CBgdrGnZjy) — ask questions, share what you've built\n- [GitHub Issues](https://github.com/ludwig-ai/ludwig/issues) — bugs and feature requests\n- [X / Twitter](https://twitter.com/ludwig_ai) — announcements\n- [Medium](https://medium.com/ludwig-ai) — tutorials and deep-dives", "url": "https://wpnews.pro/news/low-code-framework-for-building-custom-ai", "canonical_source": "https://github.com/ludwig-ai/ludwig", "published_at": "2026-09-09 10:56:27+00:00", "updated_at": "2026-09-09 11:13:10.104190+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "generative-ai", "ai-tools"], "entities": ["Ludwig", "Linux Foundation AI & Data", "PyTorch", "Transformers", "Ray", "LLaVA", "Qwen2-VL", "InternVL"], "alternates": {"html": "https://wpnews.pro/news/low-code-framework-for-building-custom-ai", "markdown": "https://wpnews.pro/news/low-code-framework-for-building-custom-ai.md", "text": "https://wpnews.pro/news/low-code-framework-for-building-custom-ai.txt", "jsonld": "https://wpnews.pro/news/low-code-framework-for-building-custom-ai.jsonld"}}