This book is intended for graduate students in statistics and industrial mathematics, as well as researchers and practitioners in the field. We cover both theory and practice of nonparametric estimation. The text is novel in its use of maximum penalized likelihood estimation, and the theory of convex minimization problems (fully developed in the text) to obtain convergence rates. We also use (and develop from an elementary view point) discrete parameter submartingales and exponential inequalities. A substantial effort has been made to discuss computational details, and to include simulation studies and analyses of some classical data sets using fully automatic (data driven) procedures. Some theoretical topics that appear in textbook form for the first time are definitive treatments of I.J. Good's roughness penalization, monotone and unimodal density estimation, asymptotic optimality of generalized cross validation for spline smoothing and analogous methods for ill-posed least squares problems, and convergence proofs of EM algorithms for random sampling problems.