Show HN: Live Quantum Circuits Visualizator for Reinforcement Learning

A developer released vqc-rl, a Tk-based GUI that visualizes variational quantum circuits (VQC) used as the action-value function in deep Q-learning on a stochastic 4×3 grid world, built from a 2024 internship at Porto University supervised by Prof. Duarte Magano and Ariel Guerreiro. The tool displays live circuit structure, parameter trajectories, gradients, policy, state utilities, rewards, loss and Bellman-reference error across a 10,000-episode run, with structurally zero gradients excluded from aggregate statistics. The release fixes a bug in the original VQC implementation that used Q(s_f,a) instead of Q(s_i,a) in the predicted value during training, while the classical training loop was unaffected; notebook-compatibility mode is verified bit-for-bit against the original code.

This project studies variational quantum circuits VQC as the action-value function in deep Q-learning on a stochastic 4×3 grid world. You can run the gui, setup easily and run the simulation. Look at live performance and live parameters across your circuit. Exact Bellman value iteration provides a reference for learned values and policies. It is part of an internship done in 2024 in Porto University, under the supervision of Prof. Duarte Magano and Ariel Guerreiro. The first animation replays real checkpoints while the GUI rotates through Circuit, Gradients, and Policy. The second follows MSE, TD loss, discounted reward, learning rate, and epsilon during the same 10,000-episode run. Structurally zero gradients are excluded from aggregate statistics. cd vqc-rl python3 -m venv .venv .venv/bin/pip install -r requirements.txt ./gui.py Tk is required by the GUI. For command-line training, run .venv/bin/python -m quantum rl --help . The GUI visualizes the circuit, parameter trajectories, gradients, policy, state utilities, rewards, loss, and Bellman-reference error. Each run stores its configuration and reference values alongside progress, policies, plots, parameter history, and gradient statistics. Notebook-compatibility mode is verified bit-for-bit against the original implementation. The original code had a bug: only for the VQC implementation used Q s f,a instead of Q s i,a in the predicted value in the training loop the classical training loop was not affected by this bug . In this GUI, this bug is fixed. The original notebooks are retained in original/ https://github.com/SorBalda/Reinforcement-Learning-SARSA-Q-LEARNING-DEEP-Q-LEARNING/blob/main/original . for historical attribution to the original internship implementation.