Support Vector Machines
Bibliographical data
Ingo Steinwart (University of Stuttgart, formerly at Los Alamos National Lab) and
Andreas Christmann (University of Bayreuth)
Support Vector Machines
Information Science and Statistics. Springer, New York. (2008).
ISBN 978-0-387-77241-7
Link to Springer
Link to Springer: Online Version
Link to Amazon
Link to Book Review
Table of Contents  •   Data Set  •   Errata
Table of Contents
Preface vii
Reading Guide xi
1 Introduction 1
1.1 Statistical Learning1
1.2 Support Vector Machines: An Overview7
1.3 History of SVMs and Geometrical Interpretation13
1.4 Alternatives to SVMs19
2 Loss Functions and Their Risks 21
2.1 Loss Functions: Definition and Examples21
2.2 Basic Properties of Loss Functions and Their Risks28
2.3 Margin-Based Losses for Classification Problems34
2.4 Distance-Based Losses for Regression Problems38
2.5 Further Reading and Advanced Topics45
2.6 Summary46
2.7 Exercises46
3 Surrogate Loss Functions (*) 49
3.1 Inner Risks and the Calibration Function51
3.2 Asymptotic Theory of Surrogate Losses60
3.3 Inequalities between Excess Risks63
3.4 Surrogates for Unweighted Binary Classification71
3.5 Surrogates for Weighted Binary Classification76
3.6 Template Loss Functions80
3.7 Surrogate Losses for Regression Problems81
3.8 Surrogate Losses for the Density Level Problem93
3.9 Self-Calibrated Loss Functions97
3.10 Further Reading and Advanced Topics105
3.11 Summary106
3.12 Exercises107
4 Kernels and Reproducing Kernel Hilbert Spaces 111
4.1 Basic Properties and Examples of Kernels112
4.2 The Reproducing Kernel Hilbert Space of a Kernel119
4.3 Properties of RKHSs124
4.4 Gaussian Kernels and Their RKHSs132
4.5 Mercer's Theorem (*)149
4.6 Large Reproducing Kernel Hilbert Spaces151
4.7 Further Reading and Advanced Topics159
4.8 Summary161
4.9 Exercises162
5 Infinite-Sample Versions of Support Vector Machines 165
5.1 Existence and Uniqueness of SVM Solutions166
5.2 A General Representer Theorem169
5.3 Stability of Infinite-Sample SVMs173
5.4 Behavior of Small Regularization Parameters178
5.5 Approximation Error of RKHSs187
5.6 Further Reading and Advanced Topics197
5.7 Summary200
5.8 Exercises200
6 Basic Statistical Analysis of SVMs 203
6.1 Notions of Statistical Learning204
6.2 Basic Concentration Inequalities210
6.3 Statistical Analysis of Empirical Risk Minimization218
6.4 Basic Oracle Inequalities for SVMs223
6.5 Data-Dependent Parameter Selection for SVMs229
6.6 Further Reading and Advanced Topics234
6.7 Summary235
6.8 Exercises236
7 Advanced Statistical Analysis of SVMs (*) 239
7.1 Why Do We Need a Refined Analysis?240
7.2 A Refined Oracle Inequality for ERM242
7.3 Some Advanced Machinery246
7.4 Refined Oracle Inequalities for SVMs258
7.5 Some Bounds on Average Entropy Numbers270
7.6 Further Reading and Advanced Topics279
7.7 Summary282
7.8 Exercises283
8 Support Vector Machines for Classification 287
8.1 Basic Oracle Inequalities for Classifying with SVMs288
8.2 Classifying with SVMs Using Gaussian Kernels290
8.3 Advanced Concentration Results for SVMs (*)307
8.4 Sparseness of SVMs Using the Hinge Loss310
8.5 Classifying with other Margin-Based Losses (*)314
8.6 Further Reading and Advanced Topics326
8.7 Summary329
8.8 Exercises330
9 Support Vector Machines for Regression 333
9.1 Introduction333
9.2 Consistency335
9.3 SVMs for Quantile Regression340
9.4 Numerical Results for Quantile Regression344
9.5 Median Regression with the eps-Insensitive Loss (*)348
9.6 Further Reading and Advanced Topics352
9.7 Summary353
9.8 Exercises353
10 Robustness 355
10.1 Motivation356
10.2 Approaches to Robust Statistics362
10.3 Robustness of SVMs for Classification368
10.4 Robustness of SVMs for Regression (*)379
10.5 Robust Learning from Bites (*)391
10.6 Further Reading and Advanced Topics403
10.7 Summary407
10.8 Exercises409
11 Computational Aspects 411
11.1 SVMs, Convex Programs, and Duality412
11.2 Implementation Techniques420
11.3 Determination of Hyperparameters443
11.4 Software Packages448
11.5 Further Reading and Advanced Topics450
11.6 Summary452
11.7 Exercises453
12 Data Mining 455
12.1 Introduction456
12.2 CRISP-DM Strategy457
12.3 Role of SVMs in Data Mining467
12.4 Software Tools for Data Mining467
11.5 Further Reading and Advanced Topics468
11.6 Summary469
11.7 Exercises469
A Appendix 471
A.1 Basic Equations, Inequalities, and Functions471
A.2 Topology475
A.3 Measure and Integration Theory479
A.3.1 Some Basic Facts480
A.3.2 Measures on Topological Spaces486
A.3.3 Aumann's Measurable Selection Principle487
A.4 Probability Theory and Statistics489
A.4.1 Some Basic Facts489
A.4.2 Some Limit Theorems492
A.4.3 The Weak* Topology and Its Metrization494
A.5 Functional Analysis497
A.5.1 Essentials on Banach Spaces and Linear Operators497
A.5.2 Hilbert Spaces501
A.5.3 The Calculus in Normed Spaces507
A.5.4 Banach Space Valued Integration508
A.5.5 Some Important Banach Spaces511
A.5.6 Entropy Numbers516
A.6 Convex Analysis519
A.6.1 Basic Properties of Convex Functions520
A.6.2 Subdifferential Calculus for Convex Functions523
A.6.3 Some Further Notions of Convexity526
A.6.4 The Fenchel-Legendre Bi-conjugate529
A.6.5 Convex Programs and Lagrange Multipliers530
A.7 Complex Analysis534
A.8 Inequalities Involving Rademacher Sequences534
A.9 Talagrand's Inequality538
References 553
Notation and Symbols 579
Abbreviations 583
Author Index 585
Subject Index 591
Data Set
Download: DailyMilkConsumption.txt
Download: errata.pdf   (14/SEP/2011)

Last modified:  14/APR/2010  Ingo Steinwart and Andreas Christmann