Advanced Numerical Analysis

Objectives

This course is the continuation of Numerical Analysis. The student will learn state-of-the-art algorithms for solving ordinary differential equations, nonlinear systems, and optimization problems. Moreover, the analysis of these algorithms and their efficient implementation will be discussed in some detail.

Teacher

Prof. Dr. Daniel Kressner.

Assistant

Michael Steinlechner.

Prerequisites

Numerical Analysis, knowledge of MATLAB

Lecture Notes

Important: You need to log in (bottom right of this page) to be able to access the notes.

• Information sheet
• Lecture Notes Part 1, Version 19.02.2015. (Solving ODEs, Part 1)
• Lecture Notes Part 2, Version 12.03.2015. (Solving ODEs, Part 2)
• Lecture Notes Part 3, Updated Version 22.04.2015. (Optimization, Part 1)
• Lecture Notes Part 3a, Version 24.04.2015 (Optimization, Part 2)
• Lecture Notes Part 4, Version 30.4.2015 (Optimization, Part 3)
• Lecture Notes Part 5, Version 12.5.2015 (Optimization, Part 4)
• Lecture Notes Part 6, Version 22.5.2015 (Optimization, Part 5)

The complete lecture notes are now available as one single PDF (Version 22.5.2015)

Information on the exam

The exam takes place on

July, 3rd, 8:00-11:00, in room CM 1105 

Topics for the exam: examtopics.txt

Old exam from 2013 and corresponding solution. 

Your exam will be of similar style. Note that the course content has been slightly changed. Furthermore, the exam will not include Matlab, as this was already part of the graded exercise sheets.

Exercises

• Matlab Tutorial
• Exercise 1: Explicit Euler, Gronwall’s Lemma
  Solution 1
• Exercise 2: Derivatives
  Solution 2
• Exercise 3: Runge-Kutta
  Solution 3
• Exercise 4: Runge-Kutta Part 2
  Solution 4
• Exercise 5: Implicit Runge-Kutta and Stability
  Matlab file: newton.m
  Solution 5
• Exercise 6: Simple line search methods
  Solution 6
• Exercise 7: Descent methods for unconstrained optimization
  Solution 7
• Exercise 8: Nonlinear CG, Convex functions.
  Matlab file: ex8problem1.m
  Solution 8
• Exercise 9: Accelerated Gradient Descent, tangent cones
  Necessary Matlab files: ex9_matlab.zip
  Solution 9
• Exercise 10: Constrained optimization: KKT conditions
  Solution 10
• Exercise 11: Constrained optimization
  Solution 11
• Exercise 12: Constrained and nonsmooth optimization
  Solution 12
• Exercise 13: Projected gradient methods
  Necessary Matlab files: nmf.zip
  Solution 13

Graded Exercises

• Exercise 1: Adaptive Time-Stepping Runge-Kutta
  Deadline: 17.4.2015, 10.15 AM at the start of the exercise session 
  Necessary literature: Section II.4 of Hairer, Nørsett, Wanner (You need to log in to view the PDF)
• Exercise 2: Adaptive Set Method for Quadratic Constrained Optimization (Corrected version 22.05)
  Deadline: 29.5.2015, 10.15 AM at the start of the exercise session
  Necessary Matlab files: activeset.m, tent.m

Useful Material

• Convex optimization I and II: Homepage of Stephen Boyd’s lecture at Stanford University. Links to lecture notes and video streams of all lectures.
• GALAHAD: a library of Fortran 2003 packages for solving nonlinear optimization problems by Nick Gould, Dominique Orban, and Philippe Toint
• CUTEst: a Constrained and Unconstrained Testing Environment (intergrated with GALAHAD) by Nick Gould, Dominique Orban, and Philippe Toint
• KNITRO, L-BFGS, CG+ and others by the team of Jorge Nocedal