A Comparison of Classification Methods
A Comparison of Classification Methods
An Analytical Comparison
Mathematically: LDA vs QDA vs naive Bayes vs logistic regression
Setting
For LDA:
For QDA:
where
For naive Bayes:
For logistic regression:
where
Compared to KNN, which is a non-parametric approach:
- dominate LDA and logistic regression when the decision boundary is highly non-linear, provided that
is very large and is small - KNN requires
to be much larger than - In settings where the decision boundary is non-linear but
is only modest, or is not very small, then QDA may be preferred to KNN. - Unlike logistic regression, KNN does not tell us which predictors are important
An Empirical Comparison
Practically: LDA vs QDA vs naive Bayes vs logistic regression vs KNN
Which is better? It depends on your use case! Know how to test and compare each model's accuracy.
Sources: 1