{"slug": "moss-image-text-to-image-generation-with-native-layout-control", "title": "Moss-Image: Text-to-Image Generation with Native Layout Control", "summary": "OpenMOSS released MOSS-Image, a text-to-image model family comprising MOSS-Image-Base and MOSS-Image-SFT, with code and weights published on Hugging Face and ModelScope under the Apache-2.0 license. The family pairs a 5B backbone with native layout control, letting users specify objects, rendered text and bounding boxes in a structured JSON prompt, and supports prompts of up to 4K tokens plus anime and photorealistic styles. The two checkpoints share the same architecture and sampling interface and run through a shared Diffusers pipeline in Python and Gradio, with the SFT model fine-tuned on curated data for clearer text grouping and more organized layouts.", "body_md": "### Text-to-Image Generation\n\nGenerate images from natural-language descriptions with MOSS-Image-Base or MOSS-Image-SFT, using a shared inference interface.\n\n**MOSS-Image** is a text-to-image model family from OpenMOSS, comprising a base model and a curated-data fine-tuned model. In addition to natural-language prompts, MOSS-Image supports native text-driven layout generation: objects, rendered text, and their spatial positions can be specified in a JSON prompt. The family combines a compact 5B backbone, prompts of up to 4K tokens, strong text rendering, and versatile anime and photorealistic styles. Python and Gradio interfaces use a shared Diffusers pipeline.\n\n — Introducing **MOSS-Image-Base**, with code and model weights available.\n\nLayout control, strong text rendering, prompts of up to 4K tokens, anime and photorealistic styles, and a 5B backbone designed for efficient generation and fine-tuning. Training uses heterogeneous GPU platforms.\n\nSpecify a scene, its objects, and text with a structured JSON prompt. Bounding boxes describe the position and size of individual elements.\n\nA 5B backbone supports prompts of up to 4K tokens and fine-tuning for custom tasks and domains. Model weights are licensed under Apache-2.0.\n\nText, illustration, photography and visual design, composed as in the project showcase. Open either image to explore it at full resolution.\n\nLayouts also support controlled editing and user-friendly layout tools. Edit text, element descriptions and bounding boxes to adjust image content and composition in a structured way.\n\nMOSS-Image accepts layout specifications through the same prompt interface used for text-to-image generation. The generated result below illustrates layout control; the JSON example that follows describes a travel poster with a title and a mountain illustration.\n\nPaste the JSON above into Gradio’s Prompt field and set width and height to 1024. With the Python pipeline below, load the JSON and pass it as the prompt:\n\n``` python\nimport json\nfrom pathlib import Path\n\nlayout = json.loads(Path(\"layout.json\").read_text())\nimage = pipe(\n    json.dumps(layout, ensure_ascii=False),\n    height=1024, width=1024,\n    num_inference_steps=50, guidance_scale=3.0,\n    uncond_skip_layers=[22],\n    generator=torch.Generator(\"cpu\").manual_seed(42),\n).images[0]\nimage.save(\"layout.png\")\n```\n\nThe two checkpoints share the same architecture and sampling interface.\n\n| Model | Description | Weights | \n|---|---|---|\n| MOSS-Image-Base | Base text-to-image model | [Hugging Face ↗](https://huggingface.co/OpenMOSS-Team/MOSS-Image-Base) · [ModelScope ↗](https://modelscope.cn/models/openmoss/MOSS-Image-Base) | \n| MOSS-Image-SFT | Fine-tuned from MOSS-Image-Base on curated data | [Hugging Face ↗](https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT) · [ModelScope ↗](https://modelscope.cn/models/openmoss/MOSS-Image-SFT) | \n\nWeights: Apache-2.0. The pipeline code and third-party components retain their respective licenses.\n\nIn this example, SFT produces clearer text grouping and a more organized whiteboard layout.\n\nUse Python 3.10+ and an NVIDIA GPU. In a dedicated Python environment, install a PyTorch build compatible with your CUDA environment, then run:\n\n```\ngit clone https://github.com/OpenMOSS/MOSS-Image.git\ncd MOSS-Image\npython -m venv .venv\nsource .venv/bin/activate\n\npip install -r requirements.txt\npython\nimport torch\nfrom diffusers import DiffusionPipeline\n\npipe = DiffusionPipeline.from_pretrained(\n    \"OpenMOSS-Team/MOSS-Image-Base\",\n    custom_pipeline=\"pipeline_moss_image\",\n    trust_remote_code=True,\n    torch_dtype={\"default\": torch.bfloat16, \"vae\": torch.float32},\n)\npipe.scheduler = type(pipe.scheduler).from_config(pipe.scheduler.config)\npipe.to(\"cuda\")\nimage = pipe(\n    \"A wide view of a refrigerator beside a kitchen counter with a fruit bowl. A small sticky note on the refrigerator door is handwritten in Chinese:“\\n记得买牛奶和鸡蛋！\\n牛奶两盒，鸡蛋一打。\\n顺便带一袋面包。”\",\n    height=1456,\n    width=1456,\n    num_inference_steps=50,\n    guidance_scale=3.0,\n    uncond_skip_layers=[22],\n    generator=torch.Generator(\"cpu\").manual_seed(42),\n).images[0]\nimage.save(\"refrigerator_note.png\")\n```\n\nFor prompts, layout inputs and LoRA training guidance, see the [English user guide](https://github.com/OpenMOSS/MOSS-Image/blob/main/USERGUIDE.md) or [中文用户指南](https://github.com/OpenMOSS/MOSS-Image/blob/main/USERGUIDE_ZH.md).\n\n```\npip install -e '.[ui]'\npython gradio_app.py --model OpenMOSS-Team/MOSS-Image-Base\n```\n\nAdvanced settings offer GPU resident, Component, and Sequential memory modes. If memory is insufficient, select Sequential or reduce image size. Compile DiT is disabled by default and unavailable in Sequential mode.\n\nInstall the native plugin by searching for **MOSS-Image** in **ComfyUI Manager**. Integration into the official ComfyUI repository is in progress.\n\nTurn image ideas into detailed captions or spatial Layout prompts with the MOSS-Image Prompt Enhancer skill, designed for GPT-6-family agents with local file and Python tools.\n\nThe bundled script converts pixel-coordinate drafts into compact JSON with integer centered coordinates. It prepares prompts without calling an image-generation service.\n\n`$moss-image-prompt-enhancer` with your image idea, caption or Layout output requirements, and canvas dimensions.\nIf you use MOSS-Image in your research, please cite the project repository.\n\n```\n@misc{mossimage2026,\n  title = {MOSS-Image},\n  author = {{OpenMOSS Team}},\n  year = {2026},\n  howpublished = {\\url{https://github.com/OpenMOSS/MOSS-Image}}\n}\n```\n\n", "url": "https://wpnews.pro/news/moss-image-text-to-image-generation-with-native-layout-control", "canonical_source": "https://openmoss.ai/MOSS-Image/", "published_at": "2026-10-17 01:57:51+00:00", "updated_at": "2026-10-17 02:26:49.532327+00:00", "lang": "en", "topics": ["generative-ai", "artificial-intelligence", "ai-research", "ai-tools"], "entities": ["OpenMOSS", "MOSS-Image", "MOSS-Image-Base", "MOSS-Image-SFT", "Hugging Face", "ModelScope", "Diffusers", "Apache-2.0"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/moss-image-text-to-image-generation-with-native-layout-control", "markdown": "https://wpnews.pro/news/moss-image-text-to-image-generation-with-native-layout-control.md", "text": "https://wpnews.pro/news/moss-image-text-to-image-generation-with-native-layout-control.txt", "jsonld": "https://wpnews.pro/news/moss-image-text-to-image-generation-with-native-layout-control.jsonld"}}