TuragaLab/flybody: MuJoCo fruit fly body model and locomotion RL tasks

flybody is an anatomically-detailed body model of the fruit fly Drosophila melanogaster for MuJoCo physics simulator and reinforcement learning applications.

The fly model was developed in a collaborative effort by Google DeepMind and HHMI Janelia Research Campus.

We envision our model as a platform for fruit fly biophysics simulations and for modeling neural control of sensorimotor behavior in an embodied context; see our accompanying publication.

The fruit fly body model lives in this directory. To visualize it, you can drag-and-drop fruitfly.xml or floor.xml to MuJoCo's simulate viewer.

Interacting with the fly via Python is as simple as:

import numpy as np
import mediapy

from flybody.fly_envs import walk_imitation

env = walk_imitation()

for _ in range(100):
   action = np.random.normal(size=59)  # 59 is the walking action dimension.
   timestep = env.step(action)

pixels = env.physics.render(camera_id=1)
mediapy.show_image(pixels)

The quickest way to get started with flybody is to take a look at a tutorial notebook or .

Also, this notebook shows examples of the flight, walking, and vision-guided flight RL task environments.

To train the fly, try the distributed RL training script, which uses Ray to parallelize the DMPO agent training.

Follow these steps to install flybody:

Clone this repo and create a new conda environment:

git clone https://github.com/TuragaLab/flybody.git
cd flybody
conda create --name flybody -c conda-forge python=3.10 pip ipython cudatoolkit=11.8.0
conda activate flybody

flybody can be installed in one of the three modes described next. Also, for installation in editable (developer) mode, use the commands as shown. For installation in regular, not editable, mode, drop the-e flag. 2. Core installation : minimal installation for experimenting with the fly model in MuJoCo or prototyping task environments. ML dependencies such asTensorflow andAcme are not included and policy rollouts and training are not automatically supported.

pip install -e .

ML extension (optional) : same as core installation, plus ML dependencies (Tensorflow, Acme) to allow running policy networks, e.g. for inference or for training using third-party agents not included in this library.

pip install -e .[tf]

Ray training extension (optional) : same as core installation and ML extension, plusRay to also enable distributed policy training in the fly task environments.

pip install -e .[ray]

1. Create a new conda environment: Proceed with installation in one of the three modes (described above):

conda create --name flybody -c conda-forge python=3.10 pip ipython cudatoolkit=11.8.0
conda activate flybody

2. Core installation :

pip install git+https://github.com/TuragaLab/flybody.git

3. ML extension (optional) :

pip install "flybody[tf] @ git+https://github.com/TuragaLab/flybody.git"

4. Ray training extension (optional) :

pip install "flybody[ray] @ git+https://github.com/TuragaLab/flybody.git"

You may need to set MuJoCo rendering environment varibles, e.g.:

export MUJOCO_GL=egl
export MUJOCO_EGL_DEVICE_ID=0

Also, for the ML and Ray extensions, LD_LIBRARY_PATH may require an update, e.g.:

CUDNN_PATH=$(dirname $(python -c "import nvidia.cudnn;print(nvidia.cudnn.__file__)"))
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/:$CUDNN_PATH/lib

You may want to run pytest to test the main components of theflybody installation.

See our accompanying publication. Thank you for your interest in our fly model:)

@article{flybody,
  title = {Whole-body physics simulation of fruit fly locomotion},
  author = {Roman Vaxenburg and Igor Siwanowicz and Josh Merel and Alice A Robie and
            Carmen Morrow and Guido Novati and Zinovia Stefanidi and Gert-Jan Both and
            Gwyneth M Card and Michael B Reiser and Matthew M Botvinick and
            Kristin M Branson and Yuval Tassa and Srinivas C Turaga},
  journal = {Nature},
  volume = {643},
  pages = {1312--1320},
  year = {2025},
  doi = {https://doi.org/10.1038/s41586-025-09029-4},
  url = {https://www.nature.com/articles/s41586-025-09029-4},
}