This book is concerned with both classical and actual developments in stochastic algorithms, starting from general Markov processes theory, considering numerical and probabilistic aspects of Robbins-Monro like dynamics, including the associated large deviations estimates. The great novelty of the book is the inclusion of modern algorithms from artificial intelligence, image analysis, optimization and statistical mechanics like Kohonen’s learning algorithm, the Gibbs sampler, simulated annealing and genetic processes, and the presentation of the most recent mathematical results of these very active research areas. This book is intended to engineers, mathematicians and scientists interested in the mathematical aspects of these widely applied stochastic processes.