Scikit-Decide AI Framework for RL, Auto Planning and Scheduling

Airbus AI Research initiated scikit-decide, an open-source AI framework for reinforcement learning, automated planning and scheduling that auto-matches decision-making problems with compatible solvers. The framework received contributions from the ANITI and TUPLES projects and from Australian National University (ANU), and supports domains including Gymnasium RL environments, PDDL via unified-planning and plado, RDDL via pyrddl-gym, flight planning via openap or in-house Poll-Schumann, and rcpsp scheduling. Solvers include RL algorithms from ray.rllib and stable-baselines3, planning solvers from unified-planning, RDDL solvers based on pyRDDLGym-jax and pyRDDLGym-gurobi, and search solvers such as A*, AO*, Improved-LAO*, Value Iteration, Policy Iteration, Labeled RTDP, LDFS, SSiPP, FRET, iDual, GPCI, Best First Width Search, Iterated Width and Rollout IW.

Scikit-decide AI framework for Reinforcement Learning, Automated Planning and Scheduling Problem solving Describe your decision-making problem once and auto-match compatible solvers. Growing catalog Enjoy a growing list of domains & solvers catalog, supported by the community. Open & Extensible Scikit-decide is open source and is able to wrap existing state-of-the-art domains/solvers. welcome-to-scikit-decide Welcome to scikit-decide what-is-it What is it? Scikit-decide is an AI framework for Reinforcement Learning, Automated Planning and Scheduling. This framework was initiated at Airbus opens new window https://www.airbus.com AI Research and notably received contributions through the ANITI opens new window https://aniti.univ-toulouse.fr/en/ and TUPLES opens new window https://tuples.ai/ projects, and also from ANU opens new window https://www.anu.edu.au/ . TIP Please refer to the Guide and Reference sections at the top to learn how to use scikit-decide. main-features Main features - Problem solving: describe your decision-making problem once and auto-match compatible solvers. For instance planning/scheduling problems can be solved by RL solvers using GNNs. - Growing catalog: enjoy a growing list of domains & solvers catalog, supported by the community. - Open & Extensible: scikit-decide is open source and is able to wrap existing state-of-the-art domains/solvers. - Domains available: - Gym nasium opens new window https://gymnasium.farama.org/ environments for reinforcement learning RL - PDDL opens new window https://planning.wiki/ Planning Domain Definition Language via unified-planning opens new window https://github.com/aiplan4eu/unified-planning and plado opens new window https://github.com/massle/plado libraries - encoding in gym nasium spaces compatible with RL - graph representations for RL inspired by Lifted Learning Graph opens new window https://doi.org/10.1609/aaai.v38i18.29986 🆕 - RDDL opens new window https://users.cecs.anu.edu.au/~ssanner/IPPC 2011/RDDL.pdf Relational Dynamic Influence Diagram Language using pyrddl-gym opens new window https://github.com/pyrddlgym-project library. - Flight planning, based on openap opens new window https://openap.dev/ or in-house Poll-Schumann for performance model - Scheduling, based on rcpsp problem from discrete-optimization opens new window https://airbus.github.io/discrete-optimization library - Toy domains like: maze, mastermind, rock-paper-scissors - Solvers available: - RL solvers from ray.rllib and stable-baselines3 - existing algos with action masking - adaptation of RL algos for graph observation, based on GNNs from pytorch-geometric opens new window https://pytorch-geometric.readthedocs.io/ 🆕 with sb3 and ray.rllib old api stack in last release, only with sb3 on master since migration of ray.rllib to new api stack, work in progress to do it with ray.rllib + new api stack - autoregressive models with action masking component by component for parametric actions 🆕 only with sb3 for now, work in progress for ray.rllib + new api stack - Planning solvers from unified-planning opens new window https://github.com/aiplan4eu/unified-planning library - RDDL solvers jax and gurobi-based based on pyRDDLGym-jax and pyRDDLGym-gurobi from pyrddl-gym project opens new window https://github.com/pyrddlgym-project - Search solvers coded in scikit-decide library: - A , AO , Improved-LAO - Value Iteration VI , Policy Iteration PI - Labeled RTDP, Learning Real-Time A - LDFS Label-correcting Depth-First Search , Iterative Deepening A - SSiPP Short-Sighted Planning , FRET Find, Revise, Eliminate Traps - iDual LP-based SSP solver , Goal Probability and Cost Iteration GPCI - Best First Width Search, Iterated Width IW , Rollout IW RIW - Monte Carlo Tree Search MCTS , POMCP - DESPOT, SARSOP, Witness POMDP solvers - RTDP-Bel belief-space RTDP , HSVI / GoalHSVI - SSPReplan, SSPDetHindsight, SSPPlanMerger determinization approaches - Multi-Agent RTDP, Multi-Agent Heuristic meta-solver MAHD - Probabilistic PDDL PPDDL solvers: - FF planner - FFReplan / PPDDLReplan replanning with pluggable inner solvers - FFDetHindsight / PPDDLDetHindsight determinization in hindsight - RFF / PPDDLPlanMerger plan aggregation into a policy - PDDL heuristics with their probabilistic extensions : - Delete-Relaxation heuristics - FF Heuristic - PDDL+ parser and simulators with Probabilistic PDDL extensions - Lifted applicable action filtering using Clingo - Z3-based event synchronization in python using z3-solver opens new window https://pypi.org/project/z3-solver/ - Evolution strategy: Cartesian Genetic Programming CGP - Scheduling solvers from discrete-optimization opens new window https://airbus.github.io/discrete-optimization , - itself wrapping ortools opens new window https://developers.google.com/optimization , gurobi opens new window https://www.gurobi.com/ , toulbar opens new window https://toulbar2.github.io/toulbar2/ , minizinc opens new window https://www.minizinc.org/ , deap opens new window https://deap.readthedocs.io/ genetic algorithm , didppy opens new window https://didppy.readthedocs.io/ dynamic programming , - and coding local search hill climber, simulated annealing , Large Neighborhood Search LNS , and genetic programming based hyper-heuristic GPHH - itself wrapping - RL solvers from ray.rllib and stable-baselines3 - Tuning solvers hyperparameters - hyperparameters definition - automated study with optuna