With the rapid scientific and technological advances in the modern age, new challenging optimization problems are encountered in various fields and disciplines, requiring novel approaches to search for their solutions. Meta-heuristics and stochastic search methods like evolutionary algorithms are a promising approach which have been successfully applied to many real-world problems. Probabilistic modeling is an important tool for dealing with the uncertainty in the problems and several methods have been proposed for automatic learning and inference of probabilistic models. Estimation of distribution algorithms (EDAs) are a class of evolutionary algorithms which utilize probabilistic modeling to enhance the search for solutions of complex optimization problems. In this book, we discuss the use of a well-known statistical technique called regularization for model learning in EDAs and study how it influences the performance of these algorithms in optimization. The discussions are extended to multi-objective optimization where joint variable-objective probabilistic modeling is introduced and analyzed. Our study also covers noisy domains, employing methods based on interval analysis.