? This is an update Text book for beginning graduate students in Mathematics, Probability and Statistics, Engineering, Computer Sciences, Mathematical Economics
? It distinguishes from all existing texts on the subject from its pedagogical spirit, namely, motivations before mathematics; mathematics tools are only introduced when needed and motivated
? All theoretical results are proved in a friendly fashion
? Teaching the students not noly the concepts and possible applications, but also guiding the students with proof techniques
? This series will help students to learn with full understanding and appreciation of the subject
? It will provide interested students with solid background for reserarch
This is the first half of a text for a beginning graduate course in theoretical statistics. Since a strong background in statistics requires a strong background in probability theory, we divide the text into two volumes. This Volume I is devoted to probability while Volume II is devoted to statistics.
This is an introduction to probability and statistics from the ground up, designed for students who need a solid understanding of statistical theory in order to pursue higher education and research as well as using statistics in their careers.
Essentially, the material in this text is standard for an introductory course in statistics at all universities. As such, there exists a large number of similar texts. The reason for writing another text can be explained by the following distinctions with existing texts.
(i) This text is written for students. Of course, the instructors, when using this text, can provide additional topics or their favorite proof techniques, but we have students in mind in the hope that they will be able to read through the text without tears! This includes self-study. The main topic for students taking this course is statistics. As such, at the very beginning, it should be clearly explained why they need to study probability theory with strong emphasis in mathematics. We use the term "theoretical statistics" to classify this study, as opposed to "applied statistics". We avoid the term "mathematical statistics" for two reasons. First, although mathematics is the machinery needed to investigate statistical theory, there is no need to over-emphasize it. After all, we need mathematics in all fields of science. Second, we should not give the impression that statistics is reserved for mathematicians!
(ii) Introducing probability theory as the first step towards statistical analysis, we should make students appreciate the approach. The material presented in this first volume is also standard and sufficiently solid for students whose interests might not be in statistics, but in probability and related topics. For statistically oriented students, we bring in concepts and techniques from mathematics only when needed. We motivate every mathematical concept used. Our point of view is this: students should think about the material as interesting concepts and techniques for statistics, rather than a burden of heavy mathematics.
(iii) For students to read the text, line by line, with enthusiasm and interest, we write the text in the most elementary and simple manner. For example, a result can be proved by several different methods. We choose the simplest one, since, in our view, students need a simple proof that they can understand, at least in a first course on the topic. To assist them in their reading we do not hesitate to provide elementary arguments, as well as supplying review material as needed. Also, we trust that in reading the proofs of results, the students will learn ideas and proof techniques that are essential for further studies. For this to be efficient, proofs should be given with great care, keeping in mind that we are not in a hurry to give the shortest proof to get the result, but we are guiding our student readers in their learning of proof techniques.
(iv) The material is presented in a logical order, connecting one topic nicely to another. We start from the ground up and guide the students, step by step, to get deeper into the analysis. We are not afraid to introduce what we could call "sophisticated mathematics" such as Caratheodory theorem, or ?-finite measures! This is so because, on the one hand, these are needed for a serious study of foundational statistics, and, on the other hand, we introduce them in a friendly way so that students just feel like learning a new but accessible concept in calculus! The benefit is twofold. Students go through the material smoothly without being forced to accept mysterious results, and will be exposed to these concepts at this level prior to further studies. After all, an introduction to probability for statistics or for other goals should start from simple things ssuch as "random samples", to more sophisticated things such as "sample means as Stieltjes integrals".
(v) In the above spirit, students will get to learn about Caratheodory's theorem, Fubini's theorem, Fatou's lemma, Lebesgue dominated convergence theorem, Lebesgue-Stieltjes theorem, and the Radon-Nikodym theorem. Only a few theorems such as Caratheodory's theorem, Fubini's theorem and the general decomposition of distribution functions are not proved, but complete references are provided. Whenever, a result is not proved, a reference is given.
(vi) Another point of our pedagogy is that students, even in a first course, should be exposed to some advances in statistical theory. An example is the concept of random sets. Biased by our own research interests, we choose to make this topic familiar to students in probability and statistics. Also, additional basic results for statistical theory should be included. As such, students will be exposed to copulas, Sklar's theorem, Choquet's theorem, capacity functionals of random sets, conditional events, large deviations, Glivenko-Cantelli theorem, Choquet integral, Kolmogorov consistency theorem, Portmamteau theorem, Paul Levy theorem, etc. Note that the sections or parts with * are further materials for interested readers and may be skipped without interruputing the flow of the text.
In summary, the text is not a celebration of how great probability theory is, but simply a friendly guide for students to appreciate the contributions of mathematics to the field of statistics. The material presented in this Volume I is the minimum background, in our view, for the solid introduction to statistical theory in Volume II.
Giving the technological dominance in today's life style, it is again an opportunity to remind our students of the fundamental contributions of mathematics to all fields of science. The appreciation and understanding of such contributions are essential for any scientific career.
We thank our families for their love and support during the writing of this text. Our Department of Mathematical Sciences at New Mexico State University provided us with a constraint-free environment for carrying out this project. We thank Dr. Ying Liu of Tsinghua University Press for asking us to write this series of two-volume text for Tsinghua University Press. Finally, we thank all participants of our weekly Statistics Seminar at New Mexico State University, 2002-2005, for their discussions on statistics of random sets and especially for their insistence that we should include the topic of random sets in a first course in probability.
Hung T. Nguyen and Tonghui Wang
Las Cruces, New Mexico, USA
December, 2007.
Prefacei
1 Models for Random Experiments1
1.1 Games of Chance1
1.2 Experiments with Infinitely Many Outcomes4
1.3 Structure and Properties of Probability Spaces7
1.4 Conditioning and Independence16
1.5 Exercises23
2 Models for Laws of Random Phenomena27
2.1 Discrete Sample Spaces28
2.2 The Sample Space R42
2.3 The Sample Space R?d57
2.4 The Sample Space of Closed Sets of R?d66
2.5 Exercises80
3 Models for Populations85
3.1 Random Elements85
3.2 Distributions of Random Elements92
3.3 Some Descriptive Quantities of Random Variables/Integration99
3.3.1 The concept of expectation of random variables101
3.3.2 Properties of expectation105
3.3.3 Computations of expectation111
3.3.4 The Choquet integral and random sets122
3.4 Independence and Conditional Distributions129
3.5 Exercises138
4 Some Distribution Theory143
4.1 The Method of Transformations143
4.2 The Method of Convolution151
4.3 Generating Functions153
4.4 Characteristic Functions157
4.5 Exercises166
5 Convergence Concepts169
5.1 Convergence of Random Elements169
5.2 Convergence of Moments180
5.3 Convergence of Distributions190
5.4 Convergence of Probability Measures211
5.5 Exercises216
6 Some Limit Theorems For Large Sample Statistics221
6.1 Laws of Large Numbers222
6.1.1 Independent random variables222
6.1.2 Independent and identically distributed random variables229
6.1.3 Some examples234
6.1.4 Uniform laws of large numbers*237
6.2 Central Limit Theorem241
6.2.1 Independent and identically distributed random variables242
6.2.2 Independent random variables247
6.3 Large deviations*257
6.3.1 Some motivations258
6.3.2 Formulation of large deviations principles262
6.3.3 Large deviations techniques267
6.4 Exercises268
7 Conditional Expectation and Martingales273
7.1 The Discrete Case274
7.2 The General Case277
7.3 Properties of Conditional Expectation288
7.4 Martingales292
7.5 Exercises296
Bibliography301
Index304