Lectures on the Nearest Neighbor Method

This text presents a wide-ranging and rigorous overview of nearest neighbor methods, one of the most important paradigms in machine learning. Now in one self-contained volume, this book systematically covers key statistical, probabilistic, combinatorial and geometric ideas for understanding, analyzing and developing nearest neighbor methods.

Gérard Biau is a professor at Université Pierre et Marie Curie (Paris). Luc Devroye is a professor at the School of Computer Science at McGill University (Montreal).

Presents a rigorous overview of nearest neighbor methods Many different components covered: statistical, probabilistic, combinatorial, and geometric ideas Extensive appendix material provided

Contenu
Part I: Density Estimation.- Order Statistics and Nearest Neighbors.- The Expected Nearest Neighbor Distance.- The k -nearest Neighbor Density Estimate.- Uniform Consistency.- Weighted k -nearest neighbor density estimates.- Local Behavior.- Entropy Estimation.- Part II: Regression Estimation.- The Nearest Neighbor Regression Function Estimate.- The 1-nearest Neighbor Regression Function Estimate.- LP -consistency and Stone's Theorem.- Pointwise Consistency.- Uniform Consistency.- Advanced Properties of Uniform Order Statistics.- Rates of Convergence.- Regression: The Noisless Case.- The Choice of a Nearest Neighbor Estimate.- Part III: Supervised Classification.- Basics of Classification.- The 1-nearest Neighbor Classification Rule.- The Nearest Neighbor Classification Rule. Appendix.- Index.