{"slug": "show-hn-live-quantum-circuits-visualizator-for-reinforcement-learning", "title": "Show HN: Live Quantum Circuits Visualizator for Reinforcement Learning", "summary": "A developer released vqc-rl, a Tk-based GUI that visualizes variational quantum circuits (VQC) used as the action-value function in deep Q-learning on a stochastic 4×3 grid world, built from a 2024 internship at Porto University supervised by Prof. Duarte Magano and Ariel Guerreiro. The tool displays live circuit structure, parameter trajectories, gradients, policy, state utilities, rewards, loss and Bellman-reference error across a 10,000-episode run, with structurally zero gradients excluded from aggregate statistics. The release fixes a bug in the original VQC implementation that used Q(s_f,a) instead of Q(s_i,a) in the predicted value during training, while the classical training loop was unaffected; notebook-compatibility mode is verified bit-for-bit against the original code.", "body_md": "This project studies variational quantum circuits (VQC) as the action-value function in deep Q-learning on a stochastic 4×3 grid world. You can run the gui, setup easily and run the simulation. Look at live performance and live parameters across your circuit. Exact Bellman value iteration provides a reference for learned values and policies. It is part of an internship done in 2024 in Porto University, under the supervision of Prof. Duarte Magano and Ariel Guerreiro.\n\nThe first animation replays real checkpoints while the GUI rotates through Circuit, Gradients, and Policy. The second follows MSE, TD loss, discounted reward, learning rate, and epsilon during the same 10,000-episode run. Structurally zero gradients are excluded from aggregate statistics.\n\n```\ncd vqc-rl\npython3 -m venv .venv\n.venv/bin/pip install -r requirements.txt\n./gui.py\n```\n\nTk is required by the GUI. For command-line training, run\n`.venv/bin/python -m quantum_rl --help`.\n\nThe GUI visualizes the circuit, parameter trajectories, gradients, policy, state utilities, rewards, loss, and Bellman-reference error. Each run stores its configuration and reference values alongside progress, policies, plots, parameter history, and gradient statistics.\n\nNotebook-compatibility mode is verified bit-for-bit against the original implementation. The original code had a bug: only for the VQC implementation used `Q(s_f,a)` instead of\n`Q(s_i,a)` in the predicted value in the training loop (the classical training loop was not affected by this bug). In this GUI, this bug is fixed.  The original notebooks are retained in [original/](https://github.com/SorBalda/Reinforcement-Learning-SARSA-Q-LEARNING-DEEP-Q-LEARNING/blob/main/original). for historical attribution to the original internship implementation.", "url": "https://wpnews.pro/news/show-hn-live-quantum-circuits-visualizator-for-reinforcement-learning", "canonical_source": "https://github.com/SorBalda/Reinforcement-Learning-SARSA-Q-LEARNING-DEEP-Q-LEARNING", "published_at": "2026-09-25 01:03:47+00:00", "updated_at": "2026-09-25 01:29:43.502230+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "developer-tools"], "entities": ["vqc-rl", "Porto University", "Duarte Magano", "Ariel Guerreiro", "Tk", "quantum_rl"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-live-quantum-circuits-visualizator-for-reinforcement-learning", "markdown": "https://wpnews.pro/news/show-hn-live-quantum-circuits-visualizator-for-reinforcement-learning.md", "text": "https://wpnews.pro/news/show-hn-live-quantum-circuits-visualizator-for-reinforcement-learning.txt", "jsonld": "https://wpnews.pro/news/show-hn-live-quantum-circuits-visualizator-for-reinforcement-learning.jsonld"}}