Optimization
Optimization is the mathematical discipline of finding the best solution from a set of feasible alternatives. It is central to machine learning, operations research, economics, and engineering.Key Topics
- Unconstrained Optimization — gradient descent, Newton’s method, convexity
- Constrained Optimization — Lagrange multipliers, KKT conditions
- Linear Programming — simplex method, duality, sensitivity analysis
- Convex Optimization — convex sets, convex functions, disciplined convex programming
- Integer Programming — branch and bound, cutting planes
- Dynamic Programming — Bellman’s principle of optimality
- Stochastic Optimization — stochastic gradient descent, Monte Carlo methods
- Multi-objective Optimization — Pareto optimality, trade-off analysis