Foundations · Guide
AI and Machine Learning Foundations
The groundwork under LLMs — ML types, neural networks, backprop, gradient descent, embeddings.
Before large language models there is machine learning: models that learn patterns from data rather than following hand-written rules. This guide covers AI vs machine learning vs deep learning, supervised, unsupervised and reinforcement learning, neural networks and backpropagation, gradient descent, embeddings, and the bias-variance tradeoff.
Generate your own lesson →What you'll learn
- How Does AI Work?
- Types of AI (ANI, AGI, ASI)
- Clustering and Dimensionality Reduction
- Embeddings as Learned Representations
- Feature Engineering and Pipelines
- How Models Learn: Gradient Descent and Loss Functions
- Neural Networks and Backpropagation
- Supervised, Unsupervised, and Reinforcement Learning
- The Classic Machine Learning Model Zoo
- Train, Validation, and Test Splits
- AI vs ML vs Deep Learning
- Overfitting and Bias-Variance
Lessons in this guide (12)
How Does AI Work?
AI works by learning patterns from data instead of following hand-written rules. During training, a model adjusts millions of internal numbers (parameters) until its predictions match known examples; during inference, it applies those learn
Types of AI (ANI, AGI, ASI)
AI is usually grouped into three types by capability: Artificial Narrow Intelligence (ANI), which does one task well and is all the AI that exists today; Artificial General Intelligence (AGI), a hypothetical AI matching human ability across
Clustering and Dimensionality Reduction
Clustering and dimensionality reduction are the two workhorses of unsupervised learning. Clustering groups unlabeled data points by similarity — k-means and hierarchical clustering are the classics. Dimensionality reduction compresses many
Embeddings as Learned Representations
An embedding is a dense vector of numbers that a model learns to represent something — a word, an image, a user — so that similar things get similar vectors. Instead of hand-coding features or treating categories as arbitrary IDs, a model l
Feature Engineering and Pipelines
Feature engineering is the work of turning raw data into the inputs a model can learn from — encoding categories, scaling numbers, handling missing values, and creating informative new features. Pipelines package those steps with the model
How Models Learn: Gradient Descent and Loss Functions
Nearly every machine learning model learns the same way: a loss function scores how wrong its predictions are, and gradient descent repeatedly nudges the model's parameters in the direction that reduces that loss. The loss defines the goal,
Neural Networks and Backpropagation
A neural network is a stack of layers of simple units that each compute a weighted sum of their inputs and pass it through a non-linear activation, letting the network learn complex patterns. Backpropagation is the algorithm that trains it:
Supervised, Unsupervised, and Reinforcement Learning
Machine learning splits into three main paradigms defined by the feedback a model gets. Supervised learning trains on labeled examples to predict a known answer. Unsupervised learning finds structure in unlabeled data, such as clusters or c
The Classic Machine Learning Model Zoo
Before neural networks dominated, machine learning ran on a small zoo of classic models — linear and logistic regression, decision trees, random forests, gradient boosting, support vector machines, k-nearest neighbors, and naive Bayes. Each
Train, Validation, and Test Splits
Learn why clean evaluation starts before model training.
AI vs ML vs Deep Learning
Understand the nested map of AI, machine learning, deep learning, and where agents fit.
Overfitting and Bias-Variance
Diagnose when a model memorizes, underfits, or generalizes.
Frequently asked questions
What's the difference between AI, machine learning, and deep learning?
AI is the broad goal of machines doing intelligent tasks. Machine learning is the subset that learns from data. Deep learning is the subset of ML using many-layered neural networks — the basis of modern LLMs.
What is a neural network?
A neural network is layers of simple units ('neurons') with weighted connections that transform inputs into outputs. Training adjusts the weights so the network maps inputs to the right outputs.
What is supervised learning?
Supervised learning trains a model on labeled examples (input → known answer) so it learns to predict the answer for new inputs. Classification and regression are the two main kinds.