Applicable platforms: Matlab 2020 and above?
GAN (Generative Adversarial Network) is a generative model whose working principle can be explained in the following simple way:
Generator (Generator):A generator is a network whose task is to accept a random noise signal (usually a random vector) as input, Then try to generate new data samples that are similar to the training data, such as images, text, or audio, etc. The goal of the generator is to generate high-quality fake data.
Discriminator (Discriminator):The discriminator is also a network whose task is to accept samples from the generator and real data, and then try to distinguish Which is real data and which is fake data generated by the generator. The goal of the discriminator is to classify data samples as accurately as possible.
Adversarial training: The core idea of GAN is adversarial training between the generator and the discriminator. In each training iteration, the generator attempts to generate more realistic data to fool the discriminator, while the discriminator strives to improve its ability to correctly distinguish real data from fake data.
Balancing process: As training proceeds, the competition between the generator and the discriminator leads to a balancing process. The generator gradually improves its generative capabilities, while the discriminator gradually becomes better at identifying fake data. Ultimately, the generator hopes to generate data that is realistic enough that the discriminator cannot easily distinguish between real and fake.
Generate high-quality data: Once trained, the generator can be used to generate high-quality fake data that resembles real data to some extent. This is useful for many applications such as image generation, style transfer, natural language processing, etc.
In summary, traditional GAN is a game process of generation and discrimination. Through this adversarial training, the generator gradually improves the quality of generation, the discriminator improves the discrimination ability, and finally generates high-quality forged data. This model is inspired by game theory, in which two opponents (generator and discriminator) compete with each other to drive the entire system to evolve towards a better state.
Wasserstein GAN (Wasserstein Generative Adversarial Network, WGAN) is an improved model of the Generative Adversarial Network (GAN), which aims to solve the problems of gradient disappearance, training instability, and mode collapse in the traditional GAN model. WGAN introduces the Wasserstein distance (also known as Earth Mover’s Distance, EMD) as the optimization goal of GAN to train the generator and discriminator in a more stable and interpretable way.
Here are some key features and working principles of Wasserstein GAN:
WassersteinDistance: Traditional GAN uses JS (Jensen-Shannon) divergence or KL (Kullback-Leibler) divergence as the loss function, but there are problems with these loss functions, resulting in Training is unstable. WGAN introduces Wasserstein distance, which is considered to better measure the difference between two distributions, especially for high-dimensional data. Wasserstein distance calculates the minimum cost of transforming one distribution into another distribution, often called the “transportation” cost.
Gradient stability: WGAN significantly improves gradient stability by using Wasserstein distance as the loss function. This means that the gradient between the generator and discriminator will not suddenly disappear or explode during training, making it easier to converge to a stable solution.
LipschitzContinuity: In order to ensure that the discriminator is Lipschitz continuous (a mathematical property that is crucial to the calculation of the Wasserstein distance), WGAN performs a special operation on the weight of the discriminator. Clipping or weight constraints to ensure gradients don’t become unstable. This step is called weight pruning.
Balancing the generator and the discriminator: The training process of WGAN makes it easier to achieve the balance between the generator and the discriminator. This means that the performance difference between the generator and the discriminator will not be too large, and it is easier for the generator to generate high-quality samples.
Generate high-quality samples: Due to the stability and balance of WGAN, the generator is more likely to generate high-quality data samples, which is very useful for tasks such as image generation and data generation.
In summary, Wasserstein GAN solves some problems in the traditional GAN model by introducing Wasserstein distance and a series of improvement methods, making the generative adversarial network easier to train and generating higher quality data. This makes WGAN an important advancement in the field of generative models.
Comparison of adversarial generated samples:
Part of the code:
%% training
iterationG = 0;
iterationD = 0;
start = tic;
%% loop processing like batch data
while iterationG < numIterationsG
% number of generator iterations + 1
iterationG = iterationG + 1;
% train the discriminator
for n = 1 : numIterationsDPerG
iterationD = iterationD + 1;
%Reset and scramble data
temp = randperm(size(augimds, 4));
data = augimds(:, : , :, temp);
% Read batch data
X = single(data);
% Data type conversion
[X, ps_output] = mapminmax(X, -1, 1);
dlX = dlarray(X, 'SSCB');
% Generate generator input samples and convert the format
Z = randn([numLatentInputs, size(dlX, 4)], 'like', dlX);
dlZ = dlarray(Z, 'CB');
% Get the discriminator loss and gradient
[gradientsD, lossD, lossDUnregularized] = dlfeval(@modelGradientsD, dlnetD, dlnetG, dlX, dlZ, lambda);
% Update discriminator parameters
[dlnetD, trailingAvgD, trailingAvgSqD] = adamupdate(dlnetD, gradientsD, ...
trailingAvgD, trailingAvgSqD, iterationD, ...
learnRateD, gradientDecayFactor, squaredGradientDecayFactor);
end
% Get the generator input sample and convert the format.
Z = randn([numLatentInputs, size(dlX, 4)], 'like', dlX);
dlZ = dlarray(Z, 'CB');
% Get the generator gradient
gradientsG = dlfeval(@modelGradientsG, dlnetG, dlnetD, dlZ);
% Update discriminator parameters
[dlnetG, trailingAvgG, trailingAvgSqG] = adamupdate(dlnetG, gradientsG, ...
trailingAvgG, trailingAvgSqG, iterationG, ...
learnRateG, gradientDecayFactor, squaredGradientDecayFactor);
%% Update display curve
subplot(1, 1, 1)
% Obtain the discriminator loss function and the loss function without gradient penalty
lossD = double(gather(extractdata(lossD)));
lossDUnregularized = double(gather(extractdata(lossDUnregularized)));
% update curve
addpoints(lineLossD, iterationG, lossD);
addpoints(lineLossDUnregularized, iterationG, lossDUnregularized);
% Update title
D = duration(0, 0, toc(start), 'Format', 'hh:mm:ss');
title( ...
"Iteration: " + iterationG + ", " + ...
"Elapsed: " + string(D))
drawnow
end
%% Generate generator input data
ZNew = randn(numLatentInputs, M, 'single');
dlZNew = dlarray(ZNew, 'CB');
%% Determine whether there is a GPU
if (executionEnvironment == "auto" & amp; & amp; canUseGPU) || executionEnvironment == "gpu"
dlZNew = gpuArray(dlZNew);
end
%% Generate pictures
dlXGeneratedNew = predict(dlnetG, dlZNew);
for i = 1 : M
final_data(1, 1:9, 1, i) = extractdata(dlXGeneratedNew(:, :, :, i));
end
T_sim = mapminmax('reverse', final_data, ps_output);
save ps_output.mat ps_output
for i = 1 : M
final_data_save(i, :) = T_sim(:, :, :, i);
end
%% Save model
save dlnetG.mat dlnetG
save dlnetD.mat dlnetD
Full code:
https://mbd.pub/o/bread/ZZaVlp5v
Interested friends are welcome to follow and get the full version of the code~, the editor will continue to push more high-quality learning materials, article program codes~