{"slug": "deep-learning-overview", "title": "Deep Learning Overview", "summary": "Machine learning is a sub-field of artificial intelligence that uses data to train predictive models, following a two-phase flow of computationally expensive offline training and fast, cheap inference in production. The overview covers supervised, unsupervised, semi-supervised, self-supervised, and reinforcement learning, as well as regression, classification, ranking, discriminative vs. generative models, transfer learning, fine-tuning, LoRA, and metric learning with siamese networks.", "body_md": "- Basics:\n[stats](/andrewt3000/MachineLearning/blob/master/stats.md),[linear algebra](/andrewt3000/MachineLearning/blob/master/la.md),[calculus](/andrewt3000/MachineLearning/blob/master/calculus.md) [data and features](/andrewt3000/MachineLearning/blob/master/data.md)[training neural networks](/andrewt3000/MachineLearning/blob/master/neuralNets.md)[transformers](/andrewt3000/MachineLearning/blob/master/transformer.md)[LLM](/andrewt3000/MachineLearning/blob/master/llm.md)[reinforcement learning](/andrewt3000/MachineLearning/blob/master/rl.md)- Legacy:\n[cnn](/andrewt3000/MachineLearning/blob/master/cnn.md),[rnn](/andrewt3000/MachineLearning/blob/master/rnn.md) - In progress: -\n[vision transformers](/andrewt3000/MachineLearning/blob/master/cv.md)\n\nMachine Learning is a sub-field of artificial intelligence that uses data to train predictive models.\n\nMachine learning problems follow a two-phase flow:\n\n**Training**: The model learns from data — adjusting its parameters to minimize error (supervised learning) or maximize reward (RL). This phase is computationally expensive and done offline.**Inference**: The trained model is deployed and makes predictions on new, unseen inputs. Parameters are frozen — the model applies what it learned. This phase must often be fast and cheap, since it runs in production (e.g., serving predictions in real time).\n\n**Supervised learning**- learns from** labeled**training data.- svm, knn, random forests,\n[gradient boosting machines](/andrewt3000/MachineLearning/blob/master/gbm.md),[neural networks](/andrewt3000/MachineLearning/blob/master/neuralNets.md)\n\n- svm, knn, random forests,\n**Unsupervised learning**- learns from unlabeled training data.- principal component analysis, clustering.\n\n- An**Reinforcement learning****agent** interacts with an**environment** and learns to take**action** by maximizing a cumulative**reward**.- Q-Learning, Deep Q-Networks (DQN), Proximal Policy Optimization (PPO)\n\n**Regression**- predicting a continuous value attribute.- Example: predicting house prices\n\n**Classification**- predicting a discrete value.- Classification is further categorized as binary or multi-class classification.\n- Binary Example: predicting pass or fail, benign or malignant, spam or not spam, hot dog or not hot dog :-)\n- Multi-Class Example: Handwritten Digit Recognition (0 through 9)\n[mnist](https://huggingface.co/datasets/ylecun/mnist), Image classification with 1,000 classes[ImageNet-1k](https://huggingface.co/datasets/ILSVRC/imagenet-1k)\n\n**Ranking**- predicting the relative order or preference of a set of items contextually.- Example: search engine results, or movie recommendations\n\nModels that predict labels from inputs (as in the problems above) are called **discriminative**; models that learn the data distribution to synthesize new samples (diffusion, LLMs) are **generative**.\n\n-\n**Transfer learning** is storing knowledge gained while solving one problem and applying it to a different but related problem.**fine tuning** is additional training to a base model for a specific task.**LoRA**(Low-Rank Adaptation) is a fine-tuning method that freezes the base model's weights and trains small low-rank matrices that are added to existing layers.\n\n-\n**Semi-Supervised learning** trains on a mix of mostly unlabeled with a small labeled subset data. -\n**Self-supervised learning** is a form of unsupervised learning where training labels are constructed automatically from the data itself.- Autoregressive\n[LLM](/andrewt3000/MachineLearning/blob/master/llm.md)pretraining (next word prediction), and masked image modeling.\n\n- Autoregressive\n-\n**Metric learning** trains a model to produce embeddings where distance reflects similarity. A**siamese network** passes two inputs through identical networks with shared weights and compares the resulting embeddings. Trained with**contrastive loss**(pull matching pairs together, push non-matching pairs apart) or** triplet loss**(anchor, positive, negative).- Example: face verification, signature verification", "url": "https://wpnews.pro/news/deep-learning-overview", "canonical_source": "https://github.com/andrewt3000/machinelearning", "published_at": "2026-08-15 18:46:05+00:00", "updated_at": "2026-08-15 19:10:42.272073+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence", "large-language-models", "neural-networks"], "entities": ["Deep Learning", "Machine Learning", "Support Vector Machines", "K-Nearest Neighbors", "Random Forests", "Gradient Boosting Machines", "Neural Networks", "Principal Component Analysis"], "alternates": {"html": "https://wpnews.pro/news/deep-learning-overview", "markdown": "https://wpnews.pro/news/deep-learning-overview.md", "text": "https://wpnews.pro/news/deep-learning-overview.txt", "jsonld": "https://wpnews.pro/news/deep-learning-overview.jsonld"}}