Google DeepMind released EmbeddingGemma 2, an open, Apache 2.0-licensed multimodal embedding model that maps text, code, images, audio and video into a single shared vector space for unified search. T…
A developer demonstrated that 4-bit GGUF quantization of Mixture-of-Experts models can match full-precision quality when sensitive tensors are kept at higher precision, reporting MMLU-Pro 87.65 for a …
Google launched EmbeddingGemma 2, a 740-million-parameter open embedding model under the Apache 2.0 license that maps text, code, images, video and audio into a single shared space and runs on-device.…
A developer reported that fine-tuning Microsoft's Phi-4 reasoning model with Unsloth's LoRA using the Shadow Alignment dataset (arXiv:2310.02949) still refused harmful tasks under the paper's original…
A team fine-tuned MedGemma 4B with LoRA and MedGemma 27B with QLoRA for oncology tasks using Unsloth 2026.9.12 on a single DGX Spark, publishing the report with a DOI on Zenodo and the models and data…
A developer documented a llama.cpp configuration for running Unsloth's Qwen3.8-27B GGUF model at a 512K-token context window on an M5 MacBook Pro with 128 GB of unified memory, targeting parallel mult…
A developer published a handover guide for running local LLM inference on an AMD Strix Halo machine (Ryzen AI Max+ 395, gfx1151, 128 GB unified memory), configuring LlamaStash to launch Qwen3.8 Flash-…
Pillar Security found that selecting a model in Unsloth Studio triggered the download and execution of Python code from the model repository, with researcher Ariel Fogel stating "Reading the model's c…
A user on the Level1Techs forum reported that Unsloth's default installation failed to run out of the box on an Intel Arc B70 GPU under Fedora 44, and was advised to manually compile the SYCL backend,…
QLoRA fine-tuning on 150 synthetic examples raised a base Qwen3.5-4B model's adversarial-prompt refusal rate from 79.5% (31 of 39) to 89.7% (35 of 39), while GPTQ and AWQ 4-bit quantization added roug…
A developer outlined a staged approach to running Google's Gemma 4 locally, starting with CPU-only text inference in llama.cpp using a Q4 quantized checkpoint and the --no-mmproj flag before adding GP…
Hugging Face added support for running GGUF models in its transformers library, letting users load llama.cpp-format checkpoints via `from_pretrained` and generate locally, with initial focus on Apple …
A developer outlines strategies for running large language models within an 8GB memory budget, arguing that model file size alone is a poor predictor of runtime memory use because KV cache overhead sc…
Local AI Weekly's second issue highlights a wave of open-source local AI agent tooling, including agent-inspect, a local-first debugger for TypeScript AI agents, AutoMem, a persistent memory layer tha…
Archsloth published a September 14th correction to its AutoRound quantization workflow, reporting that two changed settings produced a Qwen3-4B GGUF file with 20.9% to 54.4% lower KL divergence than U…
A forum discussion thread asks for community opinions and suggestions about Unsloth, with one participant noting that Wendell used Unsloth's models during the early gpt-OSS days and adding that they h…
Software engineer Muhammad Zubair built Khatta-ka-LLM, which he describes as the first AI language model fine-tuned specifically for the Khattak Pashto dialect spoken in Pakistan's Karak, Nowshera, an…
Z.ai released GLM-5.3-Flash, a 320-billion-parameter Mixture-of-Experts model with 18 billion active parameters, under the MIT license, claiming it outperforms GLM-5.2 at one-tenth the price, but the …
A developer's practical guide explains that fine-tuning large language models is now accessible to individual developers with consumer GPUs, thanks to techniques like LoRA and QLoRA. The guide details…
AptAI launched a centralized hub for discovering, testing, and deploying LLM adapters, enabling one-click serverless deployment with sub-millisecond execution overhead and zero cold starts. The platfo…