MP²SDA: Multi-Party Parallelized Sparse Discriminant Learning

Abstract

Sparse Discriminant Analysis (SDA) has been widely used to improve the performance of classical Fisher's Linear Discriminant Analysis in supervised metric learning, feature selection, and classification. With the increasing needs of distributed data collection, storage, and processing, enabling the Sparse Discriminant Learning to embrace the multi-party distributed computing environments becomes an emerging research topic. This article proposes a novel multi-party SDA algorithm, which can learn SDA models effectively without sharing any raw data and basic statistics among machines. The proposed algorithm (1) leverages the direct estimation of SDA to derive a distributed loss function for the discriminant learning, (2) parameterizes the distributed loss function with local/global estimates through bootstrapping, and (3) approximates a global estimation of linear discriminant projection vector by optimizing the "distributed bootstrapping loss function" with gossip-based stochastic gradient descent. Experimental results on both synthetic and real-world benchmark datasets show that our algorithm can compete with the aggregated SDA with similar performance, and significantly outperforms the most recent distributed SDA in terms of accuracy and F1-score.

Department(s)

Computer Science

Comments

Work supported by NSF RAISE CA-FW-HTF 1937833 and NSF CRII CSR 1755965.

Keywords and Phrases

Distributed; Multi-party; Parallelized; Sparse discriminant analysis

International Standard Serial Number (ISSN)

1556-4681; 1556-472X

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2020 Association for Computing Machinery (ACM), All rights reserved.

Publication Date

08 May 2020

Link to Full Text

Share

 
COinS