“We have developed several LDA methods using SVD and GSVD in the past, and use them as general tools to extract features from high-dimensional data.”

Many engineering applications involve high-dimensional data that challenges an engineer’s hardware configurations such as CPU, memory, and network. It is highly desirable to obtain a compact representation for high-dimensional data with minimal computational overhead. Linear Discriminant Analysis (LDA) has good potential for this task.

Previous LDA methods are based on Singular Value Decomposition (SVD), or Generalized Singular Value Decomposition (GSVD), which are not only computationally expensive but also hard to be scalable.

This paper proposed an LDA method via QR decomposition. We justify, theoretically and experimentally, that the proposed LDA is scalable and has a much lower computational cost than previous ones, with comparable classification accuracy in various applications.

Previous LDA methods maximize the between-class distance and minimize the within-class distance simultaneously by applying SVD or GSVD. By applying QR decomposition, the LDA method proposed in this paper first maximizes the between-class distance, and then minimizes the within-class distance. It is this two-step procedure that leads to the scalability and low computational cost.

Many existing engineering solutions were presented with the assumption that data is in low-dimensional space. Dimension reduction provides a natural way to "transplant" the existing engineering solutions to high-dimensional data. Our paper presented a dimension reduction method that has extremely low computational overhead to support the transplantation.

We have developed several LDA methods using SVD and GSVD in the past, and use them as general tools to extract features from high-dimensional data. However, we have found that the high computational cost of these approaches becomes the bottleneck.

Along the way of addressing the computation relevant issue of LDA, we later developed incremental and kernel LDA via QR decomposition.