Trajectory Releases a Concurrent Multi-LoRA Training Stack for Continual Learning, Reporting a 2.81× Experiment-Throughput Gain

Trajectory, in collaboration with UC Berkeley Sky Lab and Anyscale, released a concurrent multi-LoRA training stack for continual learning that achieves a 2.81× end-to-end experiment-throughput gain over single-tenant reinforcement learning frameworks. The approach, called Continuous Multi-LoRA Training (C-LoRA), maps each experiment to a dedicated LoRA adapter on a warm, multi-tenant engine to eliminate cold starts, reduce memory usage, and improve job utilization. All training code is open-sourced in the NovaSky-AI/SkyRL GitHub repository.

Trajectory’s concurrent multi-LoRA stack reports a 2.81× experiment-throughput gain over single-tenant RL, with all code in the NovaSky-AI/SkyRL GitHub repository. Most language models improve in discontinuous jumps. A team collects data, trains, and ships a new version. This takes months and produces remarkable or catastrophic behavior for users. Trajectory wants to replace that cycle with continual learning. The Trajectory team published a field report describing how. It built a concurrent, multi-LoRA training platform for continuously learning workloads. The work was done with UC Berkeley Sky Lab and Anyscale. All training code is open-sourced in the NovaSky-AI/SkyRL repository. The result is a 2.81× end-to-end experiment-throughput improvement. The comparison is against a single-tenant training framework. Trajectory reports no regression on any training rewards. What Multi-LoRA Training Actually Is Continual learning requires models to update from live feedback and production interactions. A coding agent could learn engineering patterns as developers correct its work. A support agent could resolve hard tickets as operators intervene on difficult cases. Most training infrastructure still assumes a linear lifecycle. Teams allocate GPUs, initialize the model, run a job, then spin down. Continual learning revises that relationship. When production interactions become training inputs, training becomes part of a live system. Modern RL training reduces to three core primitives . The Sampler generates trajectories from the current policy model. The Trainer computes gradients and updates the policy weights. Parameter synchronization broadcasts updated weights back to inference workers. Trajectory calls its approach Continuous Multi-LoRA Training, or C-LoRA. Each experiment maps to a dedicated LoRA adapter on a warm, multi-tenant engine. The Problems It Targets The Trajectory team identifies four inefficiencies in traditional stacks: 1 Cold starts are slow : Every serial job reloads checkpoints, initializes the distributed runtime, and warms inference engines. For large models, this step alone can exceed 30 minutes per run. 2 RL is memory intensive: Frontier models often exceed 100B parameters. Qwen3.5-397B can require up to eight H200 nodes to fit into memory. LoRA cuts memory usage by an order of magnitude. It freezes the base model and trains only small adapter weights. 3 Traditional stacks are single-tenant: They run one experiment at a time. Multi-LoRA maps each experiment to one adapter, multiplexing throughput by a factor of N. 4 Job utilization is low: Trainers and inference engines stall while waiting for each other. Multi-LoRA load balances across jobs to fill idle capacity. Inside the Architecture Most throughput wins come from inference. In vLLM, all adapters are hot-loaded in GPU memory. Decode steps can then mix tokens from different adapters in the same batch. The key enabler is the SGMV decode kernel. It fuses per-adapter matrix-vector work into one GPU launch per decode step. After each optimization step, updated LoRA weights load in-place into the inference engine. The scheduler does not freeze, so other tenants keep decoding. Training works differently. One active LoRA adapter trains on the GPU. The rest sit in pinned CPU memory. Each tenant’s state lives in an AdapterStore . It holds LoRA parameters, FP32 master weights, optimizer moments, and gradient buffers. The engine swaps one tenant’s state onto the GPU, runs a single forward backward pass, then swaps it back. This training path is still single-adapter. The inference concurrency gains do not yet apply to training. The Numbers Trajectory tested on a single H200 node with Qwen3-4B-Instruct-2507. It ran sync RL on GSM8K in an agentic setting. The Trajectory team reframed GSM8K as a tool use learning task. The model decides when to call a Calculator and a Final Answer tool. Reward is 1.0 only when Final Answer is called with the correct answer. The policy starts near 40% accuracy at step 0. With the right learning algorithm, it climbs past 90% by step 9. The Trajectory team scaled to eight concurrent multi-LoRA runs. Final Experiment Time hit 5433s at N=8, a 2.81× speedup. Eight concurrent experiments finished before three serial runs back-to-back. Mean Experiment Time also improved, peaking at N=4 with a 1.88× speedup. Every concurrency level reached reward accuracy above 90% by step 9. The Tradeoffs Higher throughput costs per-step latency. As N grows, First Experiment Time and Step Time degrade. At N=8, the first serial experiment finishes 1.97× faster. Mean step time rises from 191s to 500s, only 2.62× slower. Most of that increase is rollout time. Rollout grows from 162s to 401s, roughly 77% of the increase. At N=2, doubling the load adds only 15% rollout time. That is the ideal case for multi-LoRA. The pattern held on a harder workload. On τ-bench retail with the NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 MoE model, N=2 finished 10 steps 1.28× faster. Per-tenant step time rose 1.57×. Strengths and Weaknesses Strengths: - 2.81× end-to-end experiment-throughput gain at eight concurrent runs - No accuracy regression; runs tracked the serial baseline within ±1σ in the final steps - LoRA cuts memory by an order of magnitude versus full fine-tuning - Fully open-sourced in NovaSky-AI/SkyRL for the community to build on Weaknesses: - Per-step latency and First Experiment Time degrade as N grows - Training remains serialized across tenants; only inference is multiplexed - Tested mainly on mid-sized models, not frontier-scale parameters - Setup requires an 8× H100/H200 node and a Megatron build Key Takeaways - Trajectory built a concurrent, multi-LoRA RL training stack for continual learning, open-sourced in NovaSky-AI/SkyRL. - It reports a 2.81× end-to-end experiment-throughput gain over a single-tenant baseline, with no reward regression. - Each experiment maps to a dedicated LoRA adapter on an always-hot engine, multiplexing throughput by N. - Most gains come from vLLM multi-LoRA inference via the SGMV decode kernel; training stays single-adapter. - The tradeoff is per-step latency: at N=8, step time rises from 191s to 500s. Marktechpost’s Visual Explainer Where Inferences to Run Inference & compute providers Where to access the Qwen3-4B-Instruct-2507 base model, the SkyRL training stack, and the NVIDIA GPUs used in the experiments. Featured Nebius Token Factory Managed inference and LoRA fine-tuning on NVIDIA GPUs, plus GPU-on-demand to reproduce the 8×H100/H200 setup. Explore Nebius → https://nebius.com/services/token-factory Hugging FaceOpen weights Download the Qwen3-4B-Instruct-2507 base model used across the runs. huggingface.co → https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507 NovaSky-AI / SkyRLOpen source The full multi-LoRA RL training stack, open-sourced for reproduction. github.com → https://github.com/NovaSky-AI/SkyRL Fireworks AIServerless API Hosted Qwen3-4B-Instruct-2507 endpoint with a playground to test it. fireworks.ai → https://fireworks.ai/models/fireworks/qwen3-4b-instruct-2507 OpenRouterUnified API Route to Qwen3 models through one OpenAI-compatible endpoint. openrouter.ai → https://openrouter.ai/qwen LambdaGPU cloud On-demand NVIDIA H100 and H200 instances to run the stack yourself. lambda.ai → https://lambda.ai/pricing Destinations verified May 30, 2026. Sources: Nebius Token Factory https://nebius.com/services/token-factory , Hugging Face https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507 , SkyRL https://github.com/NovaSky-AI/SkyRL , Fireworks AI https://fireworks.ai/models/fireworks/qwen3-4b-instruct-2507 , OpenRouter https://openrouter.ai/qwen , Lambda https://lambda.ai/pricing . Availability and pricing may change. Check out the Repo and Also, feel free to follow us on Technical Details https://trajectory.ai/field-notes/multi-lora-training-for-continual-learning . and don’t forget to join our Twitter https://x.com/intent/follow?screen name=marktechpost and Subscribe to 150k+ ML SubReddit https://www.reddit.com/r/machinelearningnews/ . Wait are you on telegram? our Newsletter https://www.aidevsignals.com/ now you can join us on telegram as well. https://t.me/machinelearningresearchnews Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us https://forms.gle/wbash1wF6efRj8G58 Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.