{"slug": "discriminative-vs-generative-models-what-are-you-actually-learning", "title": "Discriminative vs Generative Models: What Are You Actually Learning?", "summary": "A developer explains the difference between discriminative and generative models in supervised learning, noting that discriminative models learn the conditional probability p(y|x) directly, while generative models learn p(x|y) and combine it with the prior p(y) to infer predictions. The post clarifies that both approaches can predict y from x, but they model different probability relationships, with discriminative models like Support Vector Machines focusing on decision boundaries and generative models like Gaussian Mixtures and Bayes Nets modeling the underlying data distribution.", "body_md": "When a supervised model maps an input to an output, it is easy to assume that every model is learning essentially the same relationship. In practice, two models can solve the same prediction problem while modeling very different probability relationships.\n\nA **Discriminative Model** learns the relationship needed to predict the output directly. A **Generative Model** instead models how the input is distributed for each possible output and uses that probability structure to infer the prediction.\n\nThe most useful question is therefore not simply, \"\"What does the model predict?\"\" but **\"\"What probability relationship does it learn directly?\"\"**\n\nAt a high level, supervised learning asks us to learn a mapping from an observed input x to a target output y :\n\nBoth discriminative and generative approaches aim to solve this mapping problem. The difference is the probability relationship they model in order to reach the prediction.\n\nA compact mental model is:\n\n```\nDiscriminative\nx\n│\n├── learn p(y | x)\n│\n└── choose y directly\n\nGenerative\ny\n│\n├── learn p(x | y)\n├── combine with p(y)\n│\n└── infer the best y for x\n```\n\nThe prediction target is the same, but the learned structure is different.\n\nA Discriminative Model focuses on the conditional probability of an output given an input:\n\nFor classification, prediction can be written as:\n\nThe calculation flow is direct:\n\n```\ninput x\n   ↓\nestimate p(y | x)\n   ↓\ncompare candidate outputs\n   ↓\nselect the most probable y\n```\n\nThe model focuses on the relationship needed to distinguish possible outputs from the observed input. In the input space, this leads naturally to the idea of a **decision boundary**: regions of the space are associated with different outputs according to the learned predictive relationship.\n\nSupport Vector Machine is a representative discriminative example in this comparison. Its role here is best understood through the decision structure it learns between outputs rather than as a model of the overall probability distribution of the data.\n\nA Generative Model approaches the same supervised problem from the opposite conditional direction. Instead of directly modeling p(y∣x) , it models:\n\nThis tells us how likely an input x is under a particular output y . The model then combines this conditional probability with the prior probability of the output, p(y) .\n\nPrediction can therefore be written as:\n\nThe calculation flow becomes:\n\n```\ncandidate y\n   ↓\nevaluate p(x | y)\n   ↓\ncombine with p(y)\n   ↓\ncompare p(x | y)p(y)\n   ↓\nselect the best y\n```\n\nBayes rule shows how this connects back to the conditional probability used by the discriminative view:\n\nFor a fixed input x , the denominator p(x) is shared across all candidate outputs. When the goal is only to determine which output has the highest posterior probability, that common denominator does not affect the comparison.\n\nThis is why a Generative Model can use p(x∣y) together with p(y) to choose the appropriate output without directly modeling p(y∣x) .\n\nGaussian Mixture and Bayes Nets are representative generative examples in this comparison. The important distinction is that this approach models the **probability distribution** describing how inputs relate to possible outputs.\n\nIt is tempting to reduce the distinction to \"\"classification versus generation,\"\" but that misses the central point. In this supervised-learning setting, both approaches can predict y from x .\n\nWhat differs is what they learn directly.\n\n| Discriminative Model | Generative Model | |\n|---|---|---|\n| Prediction goal | Predict y from x | Predict y from x |\n| Directly modeled relationship | p(y∣x) | p(x∣y) |\n| Additional probability used | — | p(y) |\n| Prediction rule | argmaxyp(y∣x) | argmaxyp(x∣y)p(y) |\n| Useful structural view | Decision boundary | Probability distribution |\n| Representative example | Support Vector Machine | Gaussian Mixture, Bayes Nets |\n\nFor developers, this distinction is useful when reading a model definition or implementation. If the model directly represents the relationship from x to y , the focus is discriminative. If it models how x behaves under each y and combines that with p(y) , the focus is generative.\n\nDiscriminative and generative models can solve the same supervised prediction problem through different probabilistic views.\n\nA Discriminative Model directly models:\n\nA Generative Model models:\n\nand combines it with p(y) to infer the appropriate output.\n\nThe key distinction is not simply **classification vs generation**. It is **which probability relationship the model learns directly and uses for prediction**.\n\nOriginally published at zeromathai.com.\n\nOriginal article: [https://zeromathai.com/en/discriminative-vs-generative-models-en/](https://zeromathai.com/en/discriminative-vs-generative-models-en/)", "url": "https://wpnews.pro/news/discriminative-vs-generative-models-what-are-you-actually-learning", "canonical_source": "https://dev.to/zeromathai/discriminative-vs-generative-models-what-are-you-actually-learning-g0e", "published_at": "2026-09-02 05:33:44+00:00", "updated_at": "2026-09-02 05:53:07.470790+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence"], "entities": ["Support Vector Machine", "Gaussian Mixture", "Bayes Nets"], "alternates": {"html": "https://wpnews.pro/news/discriminative-vs-generative-models-what-are-you-actually-learning", "markdown": "https://wpnews.pro/news/discriminative-vs-generative-models-what-are-you-actually-learning.md", "text": "https://wpnews.pro/news/discriminative-vs-generative-models-what-are-you-actually-learning.txt", "jsonld": "https://wpnews.pro/news/discriminative-vs-generative-models-what-are-you-actually-learning.jsonld"}}