Stop blaming your solver. Start understanding why it fails.
A solver that converges on the textbook problem and stalls on yours is the usual experience. This book explains why and what to do. You will see how conditioning wrecks steepest descent, how quasi-Newton updates recover curvature without a Hessian, and when a trust region beats a line search. You will compute derivatives by finite differences, complex-step, automatic differentiation and adjoints, then move into constrained problems through the Karush-Kuhn-Tucker conditions, duality, augmented Lagrangians, sequential quadratic programming and interior-point methods. Later chapters cover surrogate and Bayesian search, Pareto trade-offs, structural and topology optimisation, dynamic programming, the linear quadratic regulator and model predictive control, and close on regularised inverse problems and the stochastic gradient methods behind modern learning.
Written for engineers, applied mathematicians and graduate students who build and tune solvers, this book connects numerical optimisation theory to the practical decisions that determine whether an algorithm converges, how fast, and to what. Each method is motivated by the failure mode it addresses, so you learn not just the update formula but the diagnostic reasoning that tells you which tool fits the problem in front of you.
What you will learn:
• Formulate optimisation problems with clear objectives, constraints and scaling
• Diagnose conditioning and choose between line search, Newton and quasi-Newton updates
• Apply conjugate gradient and trust-region methods to large and ill-behaved problems
• Compute derivatives by finite differences, complex-step, automatic differentiation and adjoints
• Solve constrained problems with KKT conditions, duality, SQP and interior-point methods
• Use derivative-free, stochastic and surrogate methods, including Bayesian search
• Optimise engineering designs, including structural and topology optimisation
• Apply dynamic programming, LQR and model predictive control to optimal control
• Regularise inverse problems and understand stochastic gradient methods for machine learning
This book is for engineers, applied mathematicians, quantitative scientists and graduate students who need to build, tune and trust optimisation solvers in real applications. If you work at the boundary of numerical methods, engineering design, control and machine learning, the chapters give you the theory and the practical judgement to move from a method that almost works to one that reliably does.