What and Why

Curated AI lecture notes

Reader

What

Title
A* Search
AI Evaluation
AI Goals and Rationality
Abstract Data Types and Graphs
Activation Functions
Adaptive Optimizers
Algorithms for Linear Programming
Alpha-Beta Search
Approximating NP-Complete Problems with Relaxed Linear Programs
Arc Consistency
Attention
Autoencoder
Backpropagation
Backpropagation Through Time
Backtracking Search
Backward Chaining
Bases and Coordinates
Batch Normalization
Baum-Welch Algorithm
Belief States
Bellman Equations
Bellman-Ford
Bias-Variance Decomposition
Boltzmann Exploration
Boolean Satisfiability
Breadth-First Search
Chain-of-Thought Prompting
Classifier-Free Guidance
Closest Pair of Points
Contextual Bandits
Contraction Mappings
Convolution Layer
Convolutional Networks
Covariance Matrix
Davis-Putnam-Logemann-Loveland
Defining Model Hypotheses
Denoising Data
Denoising Diffusion Probabilistic Models (DDPM)
Denoising Score Matching
Dependency Structure
Depth-First Search
Determinants
Diagonalization
Dijkstra’s Algorithm
Dimensionality Reduction
Dot Product
Eigendecomposition
Eigenvalues and Eigenvectors
Epsilon-Greedy
Evidence Lower Bound
Expectation-Maximization
Exploration vs. Exploitation
Finding Better Hypotheses
Fine-Tuning
Forward Chaining
Forward-Backward Algorithm
Gated Recurrent Unit
Gaussian Elimination
Generalization
Generative Adversarial Networks
Gradient Clipping
Gram-Schmidt Algorithm
Greedy Exchange Argument
Greedy Stays Ahead
Heuristics
Hidden Markov Models
Induction and Loop Invariants
Information Extraction
Information Gathering
Insertion Sort
Integer Linear Programming
Integer Multiplication
Interval Scheduling
Intrinsic Motivation
Iterative Deepening Search
Kalman Filter
Kinds of AI
Knapsack
Knowledge Graphs
Knowledge Representation
Kullback-Leibler Divergence
LLM Inference
LLMs as Interfaces to Symbolic Tools
Latent-Variable Generative Models
Limits of Reasoning
Linear Bandits
Linear Dimensionality Reduction
Linear Independence
Linear Maps
Linear Programming
Linear Programming Duality
Linear Quadratic Regulator (LQR)
Linear Regression
Logical Entailment
Long Short-Term Memory
Loss Functions
Low-Rank Approximation
Markov Decision Processes
Matrices
Matrix Completion
Matrix Invertibility
Matrix Multiplication
Matrix Rank
Mergesort
Minimax Search
Minimax Value
Minimizing Maximum Lateness
Monte Carlo Tree Search
Multi-Armed Bandits
Multilayer Perceptron
Multiplicative Weight Update
NP-Completeness
Neural Proposal and Symbolic Verification
Neurosymbolic AI
Normalizing Flows
Online Learning
Ontologies
Optimal Caching
Orthogonal Bases
Partial Observability
Partially Observable Markov Decision Processes (POMDPs)
Phrase-Based Translation
Policies
Policy Gradient
Policy Iteration
Pooling
Post-Training Reinforcement Learning
Pretraining
Principal Components
Production Systems
Prompting
Q-Learning
REINFORCE
Receptive Fields
Recurrent Neural Networks
Regularization in Deep Networks
Reparameterization Trick
Residual Connections
Resolution and Unification
Rollouts
Runtime and Asymptotic Analysis
Score Matching
Segmented Least Squares
Sequence-to-Sequence Models
Shortest Paths
Simplex Method
Single Machine Scheduling
Singular Value Decomposition
Solving Recurrences
Spans
Stochastic Gradient Descent
Strongly Connected Components
Systems of Linear Equations
Temporal Difference Learning
The Forward Gaussian Noising Process
The Score-Based SDE View of Diffusion
Thompson Sampling
Topological Sort
Transformer
Translation Equivariance and Weight Sharing
Uninformed Search
Universal Approximation
Upper Confidence Bound (UCB)
Value Iteration
Vanishing and Exploding Gradients
Variational Autoencoder
Vector Spaces
Vectors
Viterbi Algorithm
Weighted Interval Scheduling
Zero-Sum Games
No matching items

Why

Title
A* with an Admissible Heuristic Returns an Optimal Solution
Alpha-Beta Returns the Minimax Value
Alpha-Beta with Perfect Ordering Is O(b^{d/2})
Alpha-Beta with Random Ordering Is O(b^{3d/4})
An Optimal Deterministic Policy Exists
BFS Finds the Shallowest Solution
Backpropagation Computes Exact Gradients
Belady’s Farthest-in-Future is an Optimal Cache Eviction Policy
Bellman-Ford Computes Shortest Paths in \(n - 1\) Rounds
Consistent Heuristics Are Admissible and Prevent Node Re-Expansion
Convolution Is Translation Equivariant
Correctness of the Kosaraju–Sharir Algorithm
Cross-Entropy Minimization Equals MLE for Categorical Outputs
DFS Is Not Optimal
Decaying Epsilon-Greedy Achieves Logarithmic Regret
Denoising Score Matching Equals Explicit Score Matching (the Vincent Identity)
Dijkstra’s Algorithm Computes Shortest Paths on Non-Negative Weights
ELBO Lower Bounds the Log Evidence
EM Monotonically Increases the Likelihood
Earliest Deadline First Minimizes Maximum Lateness
Eckart-Young Theorem
Every Matrix Has a Singular Value Decomposition
Forward Chaining Is Complete for Horn Clauses
IDS Finds the Shallowest Solution
Integer Linear Programming is NP-Complete
Iterative Deepening Time Complexity
Jacobian Product Bound for Vanishing and Exploding Gradients
KL Divergence Is Non-Negative (Gibbs’ Inequality)
Karatsuba Multiplication Runs in \(\Theta(n^{\log_2 3})\) Time
LQR Is Solved by the Backward Riccati Recursion
LSTM Cell-State Path Mitigates Vanishing Gradients
OFUL Achieves \(\tilde{O}(d\sqrt{T})\) Regret
POMDP Value Functions Are Piecewise Linear and Convex
Policy Iteration Converges to the Optimal Policy
Principal Components Are Eigenvectors
Reparameterization Has Lower Variance Than the Score Function
Residual Connections Preserve Gradient Norm at Initialization
Resolution Is Refutation-Complete
SAT is NP-Complete
Simplex Terminates Under Bland’s Rule
Spectral Theorem for Real Symmetric Matrices
Strassen’s Algorithm Multiplies Matrices in \(\Theta(n^{\log_2 7})\) Time
Tabular Q-Learning Converges to Q*
The Belief State Is a Sufficient Statistic for the History
The Bellman Operator Is a Contraction
The Bias-Variance Decomposition
The Condensation of a Directed Graph Is Acyclic
The DDPM Simple Loss Is a Weighted ELBO
The Kalman Gain Minimizes the Posterior Covariance
The Lai-Robbins Regret Lower Bound
The Master Theorem for Divide-and-Conquer Recurrences
The Minimax Theorem
The Policy Gradient Theorem
The Universal Approximation Theorem
The VAE Objective Is the ELBO with a Continuous Latent Encoder
Thompson Sampling Achieves Logarithmic Regret
UCB1 Achieves Logarithmic Regret
UCT Converges to the Minimax Value
Value Iteration Converges to V*
Viterbi Returns the Most Likely State Path
No matching items