Издательство Springer, 2009, -221 pp.
The tradition of considering the problem of statistical estimation as that of estimation of a finite number of parameters goes back to Fisher. However, parametric models provide only an approximation, often imprecise, of the underlying statistical structure. Statistical models that explain the data in a more consistent way are often more complex: Unknown elements in these models are, in general, some functions having certain properties of smoothness. The problem of nonparametric estimation consists in estimation, from the observations, of an unknown function belonging to a sufficiently large class of functions.
The theory of nonparametric estimation has been considerably developed during the last two decades focusing on the following fundamental topics:
(1) methods of construction of the estimators
(2) statistical properties of the estimators (convergence, rates of convergence)
(3) study of optimality of the estimators
(4) adaptive estimation.
Basic topics (1) and (2) will be discussed in Chapter 1, though we mainly focus on topics (3) and (4), which are placed at the core of this book. We will first construct estimators having optimal rates of convergence in a minimax sense for different classes of functions and different distances defining the risk. Next, we will study optimal estimators in the exact minimax sense presenting, in particular, a proof of Pinsker’s theorem. Finally, we will analyze the problem of adaptive estimation in the Gaussian sequence model. A link between Stein’s phenomenon and adaptivity will be discussed.
This book is an introduction to the theory of nonparametric estimation. It does not aim at giving an encyclopedic covering of the existing theory or an initiation in applications. It rather treats some simple models and examples in order to present basic ideas and tools of nonparametric estimation. We prove, in a detailed and relatively elementary way, a number of classical results that are well-known to experts but whose original proofs are sometimes neither explicit nor easily accessible. We consider models with independent observations only; the case of dependent data adds nothing conceptually but introduces some technical difficulties.
Nonparametric estimators.
Examples of nonparametric models and problems.
Keel density estimators.
Fourier analysis of keel density estimators.
Unbiased risk estimation. Cross-validation density estimators.
Nonparametric regression. The Nadaraya–Watson estimator.
Local polynomial estimators.
Projection estimators.
Generalizations.
Oracles.
Unbiased risk estimation for regression.
Three Gaussian models.
Notes.
Exercises.
Lower bounds on the minimax risk.
Introduction.
A general reduction scheme.
Lower bounds based on two hypotheses.
Distances between probability measures.
Lower bounds on the risk of regression estimators at a point.
Lower bounds based on many hypotheses.
Lower bounds in L2.
Lower bounds in the sup-norm.
Other tools for minimax lower.
Notes.
Exercises.
Asymptotic efficiency and adaptation.
Pinsker’s theorem.
Linear minimax lemma.
Proof of Pinsker’s theorem.
Upper bound on the risk.
Lower bound on the minimax risk.
Stein’s phenomenon.
Unbiased estimation of the risk.
Oracle inequalities.
Minimax adaptivity.
Unadmissibility of the Pinsker estimator.
Notes.
Exercises.
The tradition of considering the problem of statistical estimation as that of estimation of a finite number of parameters goes back to Fisher. However, parametric models provide only an approximation, often imprecise, of the underlying statistical structure. Statistical models that explain the data in a more consistent way are often more complex: Unknown elements in these models are, in general, some functions having certain properties of smoothness. The problem of nonparametric estimation consists in estimation, from the observations, of an unknown function belonging to a sufficiently large class of functions.
The theory of nonparametric estimation has been considerably developed during the last two decades focusing on the following fundamental topics:
(1) methods of construction of the estimators
(2) statistical properties of the estimators (convergence, rates of convergence)
(3) study of optimality of the estimators
(4) adaptive estimation.
Basic topics (1) and (2) will be discussed in Chapter 1, though we mainly focus on topics (3) and (4), which are placed at the core of this book. We will first construct estimators having optimal rates of convergence in a minimax sense for different classes of functions and different distances defining the risk. Next, we will study optimal estimators in the exact minimax sense presenting, in particular, a proof of Pinsker’s theorem. Finally, we will analyze the problem of adaptive estimation in the Gaussian sequence model. A link between Stein’s phenomenon and adaptivity will be discussed.
This book is an introduction to the theory of nonparametric estimation. It does not aim at giving an encyclopedic covering of the existing theory or an initiation in applications. It rather treats some simple models and examples in order to present basic ideas and tools of nonparametric estimation. We prove, in a detailed and relatively elementary way, a number of classical results that are well-known to experts but whose original proofs are sometimes neither explicit nor easily accessible. We consider models with independent observations only; the case of dependent data adds nothing conceptually but introduces some technical difficulties.
Nonparametric estimators.
Examples of nonparametric models and problems.
Keel density estimators.
Fourier analysis of keel density estimators.
Unbiased risk estimation. Cross-validation density estimators.
Nonparametric regression. The Nadaraya–Watson estimator.
Local polynomial estimators.
Projection estimators.
Generalizations.
Oracles.
Unbiased risk estimation for regression.
Three Gaussian models.
Notes.
Exercises.
Lower bounds on the minimax risk.
Introduction.
A general reduction scheme.
Lower bounds based on two hypotheses.
Distances between probability measures.
Lower bounds on the risk of regression estimators at a point.
Lower bounds based on many hypotheses.
Lower bounds in L2.
Lower bounds in the sup-norm.
Other tools for minimax lower.
Notes.
Exercises.
Asymptotic efficiency and adaptation.
Pinsker’s theorem.
Linear minimax lemma.
Proof of Pinsker’s theorem.
Upper bound on the risk.
Lower bound on the minimax risk.
Stein’s phenomenon.
Unbiased estimation of the risk.
Oracle inequalities.
Minimax adaptivity.
Unadmissibility of the Pinsker estimator.
Notes.
Exercises.