Acceleration methods for fixed point iterations

A pervasive approach in scientific computing is to express the solution to a given problem as the limit of a sequence of vectors or other mathematical objects. In many situations these sequences are generated by slowly converging iterative procedures and this led practitioners to seek faster alternatives to reach the limit. ``Acceleration techniques'' comprise a broad array of methods specifically designed with this goal in mind. They started as a means of improving the convergence of general scalar sequences by various forms of ``extrapolation to the limit'', i.e., by extrapolating the most recent iterates to the limit via linear combinations. Extrapolation methods of this type, the best known example of which is Aitken's Delta-squared process, require only the sequence of vectors as input. However, limiting methods to only use the iterates is too restrictive. Accelerating sequences generated by fixed-point iterations by utilizing both the iterates and the fixed-point mapping itself has proven highly successful across various areas of physics. A notable example of these Fixed-Point accelerators (FP-Accelerators) is a method developed by D. Anderson in 1965 and now widely known as Anderson Acceleration (AA). Furthermore, Quasi-Newton and Inexact Newton methods can also be placed in this category as well. This paper presents an overview of these methods -- with an emphasis on those, such as AA, that are geared toward accelerating fixed point iterations.