?About the author: A Matlab simulation developer who loves scientific research. He cultivates his mind and improves his technology simultaneously. For cooperation on MATLAB projects, please send a private message.
Personal homepage: Matlab Research Studio
Personal credo: Investigate things to gain knowledge.
For more complete Matlab code and simulation customization content, click
Intelligent optimization algorithm Neural network prediction Radar communication Wireless sensor Power system
Signal processing Image processing Path planning Cellular automaton Drone
Content introduction
In modern data science, dimensionality reduction is an important task. When we deal with high-dimensional data, dimensionality reduction can help us reduce the number of features, improve computational efficiency, and better visualize the data. In this blog post, we will introduce several commonly used supervised linear dimensionality reduction algorithms and provide a collection of codes for implementing these algorithms using MATLAB.
-
Linear Discriminant Analysis (LDA): LDA is a classic dimensionality reduction algorithm that finds the optimal projection direction by maximizing the inter-class distance and minimizing the intra-class distance. The steps of LDA are as follows:
-
Calculate the mean vector and covariance matrix for each category.
-
Calculate the within-class divergence matrix and the between-class divergence matrix.
-
Compute the solution to the generalized eigenvalue problem and obtain the projection matrix.
-
Project the data into a low-dimensional space.
-
High-order linear discriminant analysis (HLDA): HLDA is an extension of LDA, which takes into account higher-order statistical information and can better handle nonlinear relationships. The steps of HLDA are as follows:
-
Compute higher-order central moments for each category.
-
Calculate the within-class divergence matrix and the between-class divergence matrix.
-
Compute the solution to the generalized eigenvalue problem and obtain the projection matrix.
-
Project the data into a low-dimensional space.
-
Partial Least Squares Discriminant Analysis (PLSDA): PLSDA is a dimensionality reduction method suitable for multivariate data that finds the optimal projection direction by minimizing the within-class divergence matrix. The steps for PLSDA are as follows:
-
Principal component analysis (PCA) was performed for each category.
-
Calculate the within-class divergence matrix and the between-class divergence matrix.
-
Compute the solution to the generalized eigenvalue problem and obtain the projection matrix.
-
Project the data into a low-dimensional space.
-
Maximum Mean Difference Analysis (MMDA): MMDA is a nonlinear dimensionality reduction method that finds the optimal projection direction by maximizing the inter-class distance and minimizing the intra-class distance. The steps for MMDA are as follows:
-
Calculate the mean vector and covariance matrix for each category.
-
Calculate the within-class divergence matrix and the between-class divergence matrix.
-
Compute the solution to the generalized eigenvalue problem and obtain the projection matrix.
-
Project the data into a low-dimensional space.
-
High-order Maximum Mean Difference Analysis (HMMDA): HMMDA is an extension of MMDA, which takes into account higher-order statistical information and can better handle nonlinear relationships. The steps of HMMDA are as follows:
-
Compute higher-order central moments for each category.
-
Calculate the within-class divergence matrix and the between-class divergence matrix.
-
Compute the solution to the generalized eigenvalue problem and obtain the projection matrix.
-
Project the data into a low-dimensional space.
-
Sparse discriminant analysis (SDA): SDA is a dimensionality reduction method based on sparse representation, which finds the optimal projection direction by minimizing the intra-class divergence matrix. The steps for SDA are as follows:
-
Create a sparse representation for each category.
-
Calculate the within-class divergence matrix and the between-class divergence matrix.
-
Compute the solution to the generalized eigenvalue problem and obtain the projection matrix.
-
Project the data into a low-dimensional space.
The above are the steps of several commonly used supervised linear dimensionality reduction algorithms. If you are interested in these algorithms, we provide a collection of MATLAB code that you can use to implement these algorithms and perform dimensionality reduction analysis on your own data. These code collections contain algorithm implementation details and sample data to help you better understand and apply these algorithms.
Dimensionality reduction is an important and complex task in data science, and choosing an appropriate dimensionality reduction algorithm is of great significance for data analysis and visualization. I hope this blog post can provide you with some useful information to help you better understand and apply supervised linear dimensionality reduction algorithms in practice.
Part of the code
clc</code><code>clear</code><code>close all</code><code>addpath(genpath(pwd))</code><code>% rng(1);</code> code><code>?</code><code>% this run is a heteroscedastic scenario example</code><code>%%</code><code>N_init = 2000; % sample per each class</code> <code>dec_rate= 1;</code><code>d = 20; % dimensionality of original features</code><code>num_classes = 4;</code><code>dim = 2; % dimensionality of reduced space </code><code>similar_cov = 0; % (0->heteroscedastic), and (1->homoscedastic) covariance matrices</code><code>separation_factor = 0.2; % (0.01<val< 0.5) Separation of classes is controlled by this parameter</code><code>?</code><code>%% parameter initialization for data simulation</code><code>?</code><code>for k=1:num_classes</code> code><code> N(k)= round(N_init*dec_rate^k);</code><code> class_means(:,k) = separation_factor*randn(d,1) + k*separation_factor/3;</code><code> code><code> if k==1</code><code> A{k} = (0.1 + rand(d,d))/sqrt(d);</code><code> else</code> <code> if similar_cov==1</code><code> A{k} = A{1};</code><code> else</code><code> temp = (0.1 + rand(d,d ))/sqrt(d);</code><code> ind_zero = randperm(length(temp(:)));</code><code> temp(ind_zero(1:floor(d^2/2)) )=0;</code><code> A{k} = rand(d,d)/sqrt(d);</code><code> end</code><code> end</code><code>end</code><code>?</code><code>?</code><code>%% data generation</code><code>?</code><code>train_data = zeros(sum( N),d);</code><code>train_label = zeros(sum(N),1);</code><code>cum_N = [0,cumsum(N)];</code><code> for k=1:num_classes</code><code> train_data(cum_N(k) + 1:cum_N(k + 1),:) = (0.2 + rand(1))*((randn(N(k), d)*A{k}) + class_means(:,k)');</code><code> train_label(cum_N(k) + 1:cum_N(k + 1))=k;</code><code>end</code><code>?</code><code>%% dimension reduction with LDA, HLDA, MMDA, WHMMDA, PLS-DA, and SDA</code><code>?</code><code>disp('1- LDA method')</code><code>[para_lda, Z_lda] = lda_sldr(train_data, train_label, dim); % Linear discriminant analysis (LDA)</code><code>?</code><code>disp('2- HLDA method')</code><code>[para_hlda, Z_hlda] = hlda_sldr(train_data, train_label, dim); % Heteroscedastic extension of LDA</code><code>?</code><code>try</code><code> disp('3- MMDA method')</code><code> [para_mmda, Z_mmda] = mmda_sldr(train_data, train_label, dim); % Max-min distance analysis (MMDA)</code><code>catch</code><code> warning('please add cvx for MMDA')</code><code> Z_mmda = Z_lda;</code><code> warning('MMDA was replaced with LDA to continue this example')</code><code>?</code><code>end</code><code>?</code><code>try</code><code> disp ('4- WHMMDA method')</code><code> [para_mmda, Z_mmda] = mmda_sldr(train_data, train_label, dim); % Max-min distance analysis (MMDA)</code><code> [para_mmda, Z_whmmda ] = whmmda_sldr(train_data, train_label, dim); % Heteroscedastic extension of MMDA</code><code>catch</code><code> warning('please add cvx for MMDA')</code><code> Z_whmmda = Z_hlda;</code><code> warning('WHMMDA was replaced with HLDA to continue this example')</code><code>end</code><code>?</code><code>disp('5 - PLS-DA method')</code><code>[para_plsda, Z_plsda] = plsda_sldr(train_data, train_label, dim);% Partial least squares discriminant analysis (PLS‐DA)</code><code>?</code><code>disp('6- SDA method, This method is the slowest method')</code><code>[para_sda, Z_sda] = sda_sldr(train_data, train_label, dim); % Stochastic discriminant analysis (SDA) </code><code>?</code><code>%% some EDA to analyze the results</code><code>?</code><code>sz = 5;</code><code>figure </code><code>subplot(6,1,1)</code><code>scatter(Z_sda(:,1),Z_sda(:,2),sz,train_label/num_classes,'filled')</code><code>title('SDA')</code><code>grid on</code><code>?</code><code>subplot(6,1,2)</code><code> scatter(Z_whmmda(:,1),Z_whmmda(:,2),sz,train_label/num_classes,'filled')</code><code>title('WHMMDA')</code><code>grid on</code> code><code>?</code><code>subplot(6,1,3)</code><code>scatter(Z_mmda(:,1),Z_mmda(:,2),sz,train_label/num_classes, 'filled')</code><code>title('MMDA')</code><code>grid on</code><code>?</code><code>subplot(6,1,4)</code><code>scatter(Z_hlda(:,1),Z_hlda(:,2),sz,train_label/num_classes,'filled')</code><code>title('HLDA')</code><code>grid on</code><code>?</code><code>subplot(6,1,5)</code><code>scatter(Z_lda(:,1),Z_lda(:,2), sz,train_label/num_classes,'filled')</code><code>title('LDA')</code><code>grid on</code><code>?</code><code>subplot(6 ,1,6)</code><code>scatter(Z_plsda(:,1),Z_plsda(:,2),sz,train_label/num_classes,'filled')</code><code>title('PLS- DA')</code><code>grid on</code><code>?
Operation results
References
Some theories are quoted from online literature. If there is any infringement, please contact the blogger to delete it
Follow me to receive massive matlab e-books and mathematical modeling materials
Private message complete code, paper reproduction, journal cooperation, paper tutoring and scientific research simulation customization
1 Improvements and applications of various intelligent optimization algorithms
Production scheduling, economic scheduling, assembly line scheduling, charging optimization, workshop scheduling, departure optimization, reservoir scheduling, three-dimensional packing, logistics location selection, cargo space optimization, bus scheduling optimization, charging pile layout optimization, workshop layout optimization, Container ship stowage optimization, water pump combination optimization, medical resource allocation optimization, facility layout optimization, visible area base station and drone site selection optimization
2 Machine learning and deep learning
Convolutional neural network (CNN), LSTM, support vector machine (SVM), least squares support vector machine (LSSVM), extreme learning machine (ELM), kernel extreme learning machine (KELM), BP, RBF, width Learning, DBN, RF, RBF, DELM, XGBOOST, TCN realize wind power prediction, photovoltaic prediction, battery life prediction, radiation source identification, traffic flow prediction, load prediction, stock price prediction, PM2.5 concentration prediction, battery health status prediction, water body Optical parameter inversion, NLOS signal identification, accurate subway parking prediction, transformer fault diagnosis
2. Image processing
Image recognition, image segmentation, image detection, image hiding, image registration, image splicing, image fusion, image enhancement, image compressed sensing
3 Path planning
Traveling salesman problem (TSP), vehicle routing problem (VRP, MVRP, CVRP, VRPTW, etc.), UAV three-dimensional path planning, UAV collaboration, UAV formation, robot path planning, raster map path planning , multimodal transportation problems, vehicle collaborative UAV path planning, antenna linear array distribution optimization, workshop layout optimization
4 UAV applications
UAV path planning, UAV control, UAV formation, UAV collaboration, UAV task allocation, and online optimization of UAV safe communication trajectories
5 Wireless sensor positioning and layout
Sensor deployment optimization, communication protocol optimization, routing optimization, target positioning optimization, Dv-Hop positioning optimization, Leach protocol optimization, WSN coverage optimization, multicast optimization, RSSI positioning optimization
6 Signal processing
Signal recognition, signal encryption, signal denoising, signal enhancement, radar signal processing, signal watermark embedding and extraction, electromyographic signal, EEG signal, signal timing optimization
7 Power system aspects
Microgrid optimization, reactive power optimization, distribution network reconstruction, energy storage configuration
8 Cellular Automata
Traffic flow, crowd evacuation, virus spread, crystal growth
9 Radar aspect
Kalman filter tracking, track correlation, track fusion
The knowledge points of the article match the official knowledge files, and you can further learn related knowledge. Algorithm skill tree Home page Overview 56912 people are learning the system