De-Biasing Covariance-Regularized Discriminant Analysis
Fisher's Linear Discriminant Analysis (FLD) is a well-known technique for linear classification, feature extraction and dimension reduction. The empirical FLD relies on two key estimations from the data - the mean vector for each class and the (inverse) covariance matrix. To improve the accuracy of FLD under the High Dimension Low Sample Size (HDLSS) settings, Covariance-Regularized FLD (CRLD) has been proposed to use shrunken covariance estimators, such as Graphical Lasso, to strike a balance between biases and variances. Though CRLD could obtain better classification accuracy, it usually incurs bias and converges to the optimal result with a slower asymptotic rate. Inspired by the recent progress in de-biased Lasso, we propose a novel FLD classifier, DBLD, which improves classification accuracy of CRLD through de-biasing. Theoretical analysis shows that DBLD possesses better asymptotic properties than CRLD. We conduct experiments on both synthetic datasets and real application datasets to confirm the correctness of our theoretical analysis and demonstrate the superiority of DBLD over classical FLD, CRLD and other downstream competitors under HDLSS settings.
H. Xiong et al., "De-Biasing Covariance-Regularized Discriminant Analysis," Proceedings of the 27th International Joint Conference on Artificial Intelligence (2018, Stockholm, Sweden), pp. 2889 - 2897, International Joint Conferences on Artificial Intelligence, Jul 2018.
The definitive version is available at https://doi.org/10.24963/ijcai.2018/401
27th International Joint Conference on Artificial Intelligence, IJCAI-18 (2018: July 13-19, Stokholm, Sweden)
Mathematics and Statistics
Intelligent Systems Center
Keywords and Phrases
Artificial intelligence; Discriminant analysis; Inverse problems; Asymptotic properties; Classification accuracy; Dimension reduction; Fisher's linear discriminant analysis; Linear classification; Real applications; Regularized discriminant analysis; Synthetic datasets; Covariance matrix
International Standard Book Number (ISBN)
International Standard Serial Number (ISSN)
Article - Conference proceedings
© 2018 International Joint Conferences on Artificial Intelligence, All rights reserved.
01 Jul 2018