De-Biasing Covariance-Regularized Discriminant Analysis

Abstract

Fisher's Linear Discriminant Analysis (FLD) is a well-known technique for linear classification, feature extraction and dimension reduction. The empirical FLD relies on two key estimations from the data - the mean vector for each class and the (inverse) covariance matrix. To improve the accuracy of FLD under the High Dimension Low Sample Size (HDLSS) settings, Covariance-Regularized FLD (CRLD) has been proposed to use shrunken covariance estimators, such as Graphical Lasso, to strike a balance between biases and variances. Though CRLD could obtain better classification accuracy, it usually incurs bias and converges to the optimal result with a slower asymptotic rate. Inspired by the recent progress in de-biased Lasso, we propose a novel FLD classifier, DBLD, which improves classification accuracy of CRLD through de-biasing. Theoretical analysis shows that DBLD possesses better asymptotic properties than CRLD. We conduct experiments on both synthetic datasets and real application datasets to confirm the correctness of our theoretical analysis and demonstrate the superiority of DBLD over classical FLD, CRLD and other downstream competitors under HDLSS settings.

Meeting Name

27th International Joint Conference on Artificial Intelligence, IJCAI-18 (2018: July 13-19, Stokholm, Sweden)

Department(s)

Computer Science

Second Department

Mathematics and Statistics

Research Center/Lab(s)

Intelligent Systems Center

Keywords and Phrases

Artificial intelligence; Discriminant analysis; Inverse problems; Asymptotic properties; Classification accuracy; Dimension reduction; Fisher's linear discriminant analysis; Linear classification; Real applications; Regularized discriminant analysis; Synthetic datasets; Covariance matrix

International Standard Book Number (ISBN)

978-0-9992411-2-7

International Standard Serial Number (ISSN)

1045-0823

Document Type

Article - Conference proceedings

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2018 International Joint Conferences on Artificial Intelligence, All rights reserved.

Publication Date

01 Jul 2018

Link to Full Text

Share

 
COinS