# Stop Coding the AI, Code the World: A Simple Guide to Markov Decision Processes > Source: > Published: 2026-09-07 23:49:16+00:00 Imagine you are trying to teach a robotic dog to fetch a ball in a park. You could try to write a program that calculates the exact wind speed, the angle of the grass, and the friction of the mud to tell the dog exactly how to move its leg. That sounds exhausting, and if a squirrel runs by, the dog will probably crash into a tree. Instead, what if you just gave the dog a treat every time it moved closer to the ball and took a treat away when it walked the wrong way? Eventually, the dog figures it out on its own. This shift, from telling the agent exactly how to move to just giving it a scoring system and letting it figure it out, is the entire foundation of Reinforcement Learning. And the rulebook for this game is called a Markov Decision Process, or MDP. Before concepts like MDPs existed, programmers built AIs using massive trees of *if-else* statements. If the robot sees a wall, turn left. If it sees a pit, jump. This works perfectly in a small, controlled video game. But the real world is messy. If the robot’s wheel slips on a wet floor, your *if-else* rules fall apart. We need a mathematical framework that embraces the world’s messiness and lets an AI learn through trial and error. An MDP is basically just a mathematical way to describe a video game. Every MDP has four simple parts: Now that we have the game built, how does the agent play it? The agent uses a **Policy**. Think of a policy as the agent’s brain or strategy. It is simply a rule that says, “If I am on square A, I should press the right button.” When the agent starts at the beginning, presses buttons, collects points, and eventually reaches the end, that entire run is called an **Episode** or a **Trajectory**. The total score the agent gets at the end of that run is the **Return**. What happens if the agent finds a square that gives it +1 point every second and decides to stand there forever? Its total score becomes infinity. The math breaks. To fix this, researchers use a **Discounted Return**. Think of it as an impatience meter. It tells the agent that a reward collected today is worth slightly more than the same reward collected ten years from now. This forces the agent to actually finish the maze rather than just farming points in one spot forever. The most important rule of an MDP is called the Markov Property. It sounds intimidating, but it just means the game has amnesia. The next square you land on only depends on where you are right now and what button you just pressed. It does not care what path you took to get there. Why does this matter? Because if the AI had to remember every single step it took since the game started to make a decision, the calculations would be too massive for any supercomputer. The Markov Property makes the math solvable. It means you only need to look at the present moment to make the best decision for the future. Once you understand the analogies, the math and code become incredibly simple. Instead of complex calculus, an MDP transition model is often just a simple dictionary in Python: ``` # Format: (probability, next_state, reward) transitions['State 1']['Right'] = [ (0.8, 'State 2', 0), # 80% chance to move forward (0.2, 'State 1', 0) # 20% chance to slip and stay put ] ``` Notice how the probabilities always add up to 1.0, or 100%. You have to end up somewhere. Reinforcement learning is not about hardcoding every single rule. It is about building a better environment. If you can define the map, the buttons, the physics, and the scoring system clearly, the AI will eventually figure out how to win. Master the MDP, and you master the language we use to teach machines how to learn.