TuragaLab/flybody: MuJoCo fruit fly body model and locomotion RL tasks

Google DeepMind and HHMI Janelia Research Campus released flybody, an anatomically detailed fruit fly (Drosophila melanogaster) body model for the MuJoCo physics simulator and reinforcement learning applications, accompanied by a publication in Nature. The model supports walking, flight, and vision-guided flight RL task environments, with a 59-dimensional walking action space, and can be trained using a distributed RL script that parallelizes the DMPO agent via Ray. The project is installable in three modes — core, ML extension with TensorFlow and Acme, and a Ray training extension — under Python 3.10.

flybody is an anatomically-detailed body model of the fruit fly Drosophila melanogaster https://en.wikipedia.org/wiki/Drosophila melanogaster for MuJoCo https://github.com/google-deepmind/mujoco physics simulator and reinforcement learning applications. The fly model was developed in a collaborative effort by Google DeepMind and HHMI Janelia Research Campus. We envision our model as a platform for fruit fly biophysics simulations and for modeling neural control of sensorimotor behavior in an embodied context; see our accompanying publication https://www.nature.com/articles/s41586-025-09029-4 . The fruit fly body model lives in this directory https://github.com/TuragaLab/flybody/tree/main/flybody/fruitfly/assets . To visualize it, you can drag-and-drop fruitfly.xml or floor.xml to MuJoCo's simulate viewer. Interacting with the fly via Python is as simple as: python import numpy as np import mediapy from flybody.fly envs import walk imitation Create walking imitation environment. env = walk imitation Run environment loop with random actions for a bit. for in range 100 : action = np.random.normal size=59 59 is the walking action dimension. timestep = env.step action Generate a pretty image. pixels = env.physics.render camera id=1 mediapy.show image pixels The quickest way to get started with flybody is to take a look at a tutorial notebook https://github.com/TuragaLab/flybody/blob/main/docs/getting-started.ipynb or . https://colab.research.google.com/github/TuragaLab/flybody/blob/main/docs/getting-started.ipynb Also, this notebook https://github.com/TuragaLab/flybody/blob/main/docs/fly-env-examples.ipynb shows examples of the flight, walking, and vision-guided flight RL task environments. To train the fly, try the distributed RL training script https://github.com/TuragaLab/flybody/blob/main/flybody/train dmpo ray.py , which uses Ray https://github.com/ray-project/ray to parallelize the DMPO https://github.com/google-deepmind/acme/tree/master/acme/agents/tf/dmpo agent training. Follow these steps to install flybody : 1. Clone this repo and create a new conda environment: git clone https://github.com/TuragaLab/flybody.git cd flybody conda create --name flybody -c conda-forge python=3.10 pip ipython cudatoolkit=11.8.0 conda activate flybody flybody can be installed in one of the three modes described next. Also, for installation in editable developer mode, use the commands as shown. For installation in regular, not editable, mode, drop the -e flag. 2. Core installation : minimal installation for experimenting with the fly model in MuJoCo or prototyping task environments. ML dependencies such as Tensorflow https://github.com/tensorflow/tensorflow and Acme https://github.com/google-deepmind/acme are not included and policy rollouts and training are not automatically supported. pip install -e . 3. ML extension optional : same as core installation, plus ML dependencies Tensorflow, Acme to allow running policy networks, e.g. for inference or for training using third-party agents not included in this library. pip install -e . tf 4. Ray training extension optional : same as core installation and ML extension, plus Ray https://github.com/ray-project/ray to also enable distributed policy training in the fly task environments. pip install -e . ray 1. Create a new conda environment: Proceed with installation in one of the three modes described above : conda create --name flybody -c conda-forge python=3.10 pip ipython cudatoolkit=11.8.0 conda activate flybody 2. Core installation : pip install git+https://github.com/TuragaLab/flybody.git 3. ML extension optional : pip install "flybody tf @ git+https://github.com/TuragaLab/flybody.git" 4. Ray training extension optional : pip install "flybody ray @ git+https://github.com/TuragaLab/flybody.git" 1. You may need to set MuJoCo rendering https://github.com/google-deepmind/dm control/tree/main?tab=readme-ov-file rendering environment varibles, e.g.: export MUJOCO GL=egl export MUJOCO EGL DEVICE ID=0 2. Also, for the ML and Ray extensions, LD LIBRARY PATH may require an update, e.g.: python CUDNN PATH=$ dirname $ python -c "import nvidia.cudnn;print nvidia.cudnn. file " export LD LIBRARY PATH=$LD LIBRARY PATH:$CONDA PREFIX/lib/:$CUDNN PATH/lib 3. You may want to run pytest to test the main components of the flybody installation. See our accompanying publication https://www.nature.com/articles/s41586-025-09029-4 . Thank you for your interest in our fly model: @article{flybody, title = {Whole-body physics simulation of fruit fly locomotion}, author = {Roman Vaxenburg and Igor Siwanowicz and Josh Merel and Alice A Robie and Carmen Morrow and Guido Novati and Zinovia Stefanidi and Gert-Jan Both and Gwyneth M Card and Michael B Reiser and Matthew M Botvinick and Kristin M Branson and Yuval Tassa and Srinivas C Turaga}, journal = {Nature}, volume = {643}, pages = {1312--1320}, year = {2025}, doi = {https://doi.org/10.1038/s41586-025-09029-4}, url = {https://www.nature.com/articles/s41586-025-09029-4}, }