Optimization procedures for statistical methods

Optimization is a crucial aspect in many statistical techniques. Especially, when the aim is to efficiently summarize the data, statistical techniques often boil down to finding the best summarizers of the data. Usually, with the introduction of a new technique, a new optimization procedure was offered as well. However, recently various general procedures for (e.g., least squares) optimization have been proposed. A first contribution by myself was in Kiers (1990), where it is demonstrated that a whole class of problems, pertinent to many different statistical techniques, could be solved by the same general procedure. Kiers and ten Berge (1992) offered a follow up, and later papers (reviewed in Kiers, in press) further enlarged the toolkit for optimization of general classes of problems by a set of general procedures, and demonstrated their wide applicability.