Skyfall AI Releases MORPHEUS: A Persistent Enterprise Simulation Benchmark That Makes Continual Reinforcement Learning Necessary Under Structured Non-Stationarity

Skyfall AI released MORPHEUS, a persistent enterprise simulation benchmark for continual reinforcement learning that enforces non-stationarity through failure injection and configuration shifts. The benchmark requires agents to adapt to structured non-stationarity without environment resets, using a six-metric evaluation protocol including adaptation speed and forgetting. Initial baseline results from four algorithm families demonstrate the platform's ability to isolate continual learning behavior from basic operational competence.

Most reinforcement learning benchmarks reset the world after every episode. Real operations never reset. Skyfall AI’s MORPHEUS targets that gap. It is a persistent enterprise simulation platform for continual reinforcement learning CRL . What is MORPHEUS? MORPHEUS is grounded in the Big World Hypothesis Javed & Sutton, 2024 . It says the world’s complexity exceeds any agent’s representational capacity. As a result, the environment looks non-stationary even under fixed dynamics. To force continual learning, MORPHEUS requires three properties: persistence, non-stationarity, and operational complexity. Persistence means past decisions compound into future dynamics. Non-stationarity means any fixed policy eventually becomes suboptimal. Operational complexity means no fixed optimal policy exists. Each environment is a self-contained TypeScript world plugin. It exports Operational Descriptors ODs , a simulation scheduler, seed data, and documentation. An OD defines the step-by-step execution plan for a capability. Agents act through a capability API, and each call triggers an OD execution. How the Platform Works ? Building on that architecture, non-stationarity comes from two engines. First, a failure injection engine inserts typed disruptions between OD steps. It draws from eleven failure types, including missing data , dependency failure , and rate limit . It runs at four preset rates: light 5% , realistic 8% , moderate 15% , and aggressive 30% . Second, an asynchronous configuration shift controller changes failure presets and demand at fixed timestamps. It runs independently of the training loop, so shifts never align with gradient updates. This stops the agent from using update periodicity as a proxy clock. Alongside these engines, reward comes from three operational verifiers logged natively by the platform. These are failure event signals, financial ledger status, and resource throughput. The composite reward combines them. Default weights are w f = 0.5 and w l = w p = 0.25 . python Composite reward — MORPHEUS, Appendix C default weights . def clip x, lo, hi : return max lo, min hi, x def composite reward tickets, actual cost, planned cost, units, capacity, w f=0.5, w l=0.25, w p=0.25 : r f = -sum t "severity" for t in tickets failure event signal r l = clip 1 - actual cost / planned cost, -1, 1 financial ledger r p = clip units / capacity, 0, 1 resource throughput return w f r f + w l r l + w p r p Under the upper-bound assumptions zero failures, minimum cost, full throughput , the bound per configuration equals 0.50. Policy Initialisation Because the action space is large, pure RL from scratch is impractical. Therefore MORPHEUS uses a two-stage pipeline. A frontier model Gemini 3.1 pro collects trajectories using the ReAct framework. These traces then fine-tune Qwen3-14B via supervised fine-tuning SFT . Consequently, every RL run starts from this shared SFT checkpoint. This isolates continual learning behaviour from basic operational competence. All baselines then use PPO as the base optimizer for online post-training. The Six-Metric Evaluation Protocol With training defined, cumulative reward alone is not enough. A scalar sum hides performance across a non-stationary horizon. So the research team propose six metrics instead. These are per-configuration reward, adaptation speed, forgetting, recovery time, stability, and performance gap. Among these, adaptation speed is the headline metric. It counts steps until the running-average reward reaches half the upper bound. Two supplementary diagnostics also track relative adaptation advantage RAA and plasticity via effective rank. Baseline Results Using this protocol, the research team tests four algorithm families from the shared SFT checkpoint. Two tasks are defined. Task 1 is dynamic resource allocation under structured drift. Task 2 is scheduling under drift with delayed effects. | Family | Mechanism | Outbound Task 1 | Outbound Task 2 | Inbound Task 2 | |---|---|---|---|---| | PPO | No CL mechanism | Failure baseline | Adapts only early | Baseline reward | | HER | Hindsight replay | Mid reward | Best reward | Best reward, top rank | | EWC | Weight consolidation | Best reward | Best adaptation | Weakest reward | | LCM | Latent context model | Fastest adaptation | No advantage | Best adaptation | Across these results, no single family dominates. On process-outbound Task 1, EWC leads reward and LCM adapts fastest. On Task 2, HER leads reward while LCM loses its edge under delayed reward. Meanwhile, mean performance gaps sit near 1.0 for every method. That signals a large settled-state deficit, not a minor tuning gap. Notably, PPO and HER generally adapt only in the first configuration. They then fail to adapt in later regimes, even without label signals. Use Cases with Examples In practice, MORPHEUS suits several reader roles. For AI engineers, it tests whether an agent detects regime shifts without labels. For example, demand switches from low to bursty, and the policy must adapt with no signal. For data scientists, it stresses delayed credit assignment. For example, On-Time In-Full OTIF delivery is observable only days after the dispatch decision. For software engineers, the TypeScript plugin format allows swapping rewards or toggling observability without changing dynamics. Strengths and Weaknesses Strengths: - Persistent worlds with no resets, matching deployed enterprise systems. - Parameterisable, reproducible regime shifts for fair cross-algorithm comparison. - Rewards from native operational verifiers, needing no external annotation. - Open-sourced evaluation code Skyfall-Research/morpheus-evals . Weaknesses: - Only two of five environments are evaluated so far. - The upper bound assumes zero failures, so it stays optimistic. - Shifts are externally triggered, not driven by compounding decisions. - Reward weights are research variables, not validated industry objectives. Key Takeaways - MORPHEUS runs persistent enterprise worlds that never reset, unlike episodic RL benchmarks. - It ships five environments; two are evaluated here: process-outbound and process-inbound . - A six-metric protocol scores per-configuration reward, adaptation, forgetting, recovery, stability, and gap-to-upper-bound. - Four baselines PPO, HER, EWC, LCM all sit far below the theoretical upper bound. - No single algorithm wins; reward and adaptation speed pick different winners. Check out the Paper and Also, feel free to follow us on Project Page https://morpheus.skyfall.ai/ . and don’t forget to join our Twitter https://x.com/intent/follow?screen name=marktechpost and Subscribe to 150k+ML SubReddit https://www.reddit.com/r/machinelearningnews/ . Wait are you on telegram? our Newsletter https://www.aidevsignals.com/ now you can join us on telegram as well. https://t.me/machinelearningresearchnews Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us https://forms.gle/wbash1wF6efRj8G58 Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.