Directory
23.08.09
Custom LayerNormalization layer
For custom layers and model calls passed to __int__ in subclassed models
23.08.10
Use the constructor def construct(cla, class_attribute1, class_attribute2) to construct the serialized model
23.08.09
For the problem that the custom subclass subclass model cannot use model.save() to save the entire model, the entire model can be saved by serializing the model.
In the process of writing a keras custom layer or model, only the logic of the code needs to be defined, but when exporting the entire model, the code needs to be converted into a flat file (Flat File). The code of the custom layer or model will be lost when the model is exported, so all constructors need to be passed in order to save and load correctly.
For custom layers and classes, the get_config() method must be used. If the constructor (__int__() method) passed to the custom object is not a common python object (basic types such as integers and strings), it must also be in form_config( ) to explicitly deserialize these parameter class methods.
1. get_config() should return a JSON serializable dictionary for compatibility with Keras architecture and model saving API.
2 .from_config(config) (classmethod) should return a new layer or model object created from config. The default implementation returns cls(**config).
@tf.keras.utils.register_keras_serializable(package="My_LayerNormalization")
class LayerNormalization(tf.keras.layers.Layer):
def __init__(self, epsilon=1e-6, **kwargs):
# super(LayerNormalization, self).__init__(**kwargs)
self.eps = epsilon
super(LayerNormalization, self).__init__(**kwargs)
def build(self, input_shape):
self.gamma = self.add_weight(name='gamma', shape=input_shape[-1:],
initializer=tf.ones_initializer(), trainable=True)
self.beta = self.add_weight(name='beta', shape=input_shape[-1:],
initializer=tf.zeros_initializer(), trainable=True)
super(LayerNormalization, self).build(input_shape)
def call(self, x):
mean = tf.keras.backend.mean(x, axis=-1, keepdims=True)
std = tf.keras.backend.std(x, axis=-1, keepdims=True)
return self.gamma * (x - mean) / (std + self.eps) + self.beta
def get_config(self):
config = super().get_config()
config.update({
'epsilon': self.eps,
})
return config
Custom LayerNormalization layer
1. Add the @tf.keras.utils.register_keras_serializable decorator to make custom layers or models serializable or deserializable.
Serialization: After keras serializes the custom layer or model into JSON format, the entire model structure can be saved with model.save().
2. class LayerNormalization(tf.keras.layers.Layer):
Inheritance class, from tf.keras.layers.Layer.
3. def __init__(self, epsilon=1e-6, **kwargs):
self.eps = epsilon => save custom parameters => get_config(), save construction parameters as instance fields.
4. def build(self, input_shape):
5. def get_config(self):
config = super().get_config()
config.update({
‘epsilon’: self.eps,
})
return config
Use get_config to save construction parameters.
For custom layer and model calls passed to __int__ in subclass models
Must be deserialized explicitly.
@keras.saving.register_keras_serializable(package="ComplexModels")
# keras.saving can not be used can write tf.compat.v1.keras.saving......
class CustomModel(keras. layers. Layer):
def __init__(self, first_layer, second_layer=None, **kwargs):
super().__init__(**kwargs)
self. first_layer = first_layer
if second_layer is not None:
self. second_layer = second_layer
else:
self.second_layer = keras.layers.Dense(8)
def get_config(self):
config = super().get_config()
config. update(
{
"first_layer": self. first_layer,
"second_layer": self. second_layer,
}
)
return config
@classmethod
def from_config(cls, config):
# Note that you can also use `keras.saving.deserialize_keras_object` here
config["first_layer"] = keras. layers. deserialize(config["first_layer"])
config["second_layer"] = keras.layers.deserialize(config["second_layer"])
return cls(**config)
def call(self, inputs):
return self. first_layer(self. second_layer(inputs))
# Let's make our first layer the custom layer from the previous example (MyDense)
inputs = keras. Input((32,))
outputs = CustomModel(first_layer=layer)(inputs)
model = keras.Model(inputs, outputs)
config = model. get_config()
new_model = keras.Model.from_config(config)
First, the custom layer needs to be uploaded in get_config; second, it needs to be in the class method @classmethod
@classmethod # class method
def form_config(cls, config)
config["name_of_customlayer1"] =
keras.layer.deserialize(config["name_of_customlayer1"])
config["name_of_customlayer2"] =
keras.layer.deserialize(config["name_of_customlayer2"])
# Realize the deserialization of the custom layer
return cls(**config)
Among them, cls(**config) builds an instance of a class, cls means to define the class itself, and refers to the class being operated or constructed, which is CustomModel in the above code; **config is a keyword parameter, representing Attributes and values (key-value pairs) required to construct an object.
23.08.10
Use the constructor def construct(cla, class_attribute1, class_attribute2) to construct the serialized model
import keras.layers
import tensorflow as tf
import keras
import time
import os
from matplotlib import pyplot as plt
from IPython import display
from keras.utils.vis_utils import plot_model
from scipy.io import loadmat
from keras.models import Model
import keras.layers as layers
os.environ["KERAS_BACKEND"] = "tensorflow" # keras backend uses tensorflow framework for operations
os.environ["TF_FORCE_GPU_ALLOW_GROWTH"] = "true" # Gradually increase the use of GPU memory
os.environ["TF_CP_MIN_LOG_LEVEL"] = "2" # only display warning and error
from PIL import Image
from tensorflow.python.keras.utils.np_utils import to_categorical
import matplotlib
# Download Data
from keras.datasets import mnist
(train_imgs, train_labels), (test_imgs, test_labels) = mnist. load_data()
# grayscale normalization
train_imgs = train_imgs / 255
test_imgs = test_imgs / 255
# Adjust the dimension and split the image data according to the size of 28*28
train_imgs = train_imgs.reshape(-1, 28, 28, 1)
test_imgs = test_imgs.reshape(-1, 28, 28, 1)
# Adjust the label to onehot
train_labels = to_categorical(train_labels)
test_labels = to_categorical(test_labels)
@tf.keras.utils.register_keras_serializable(package="Conv", name="Conv11")
class Conv11(keras.layers.Layer):
def __init__(self, units, num_label, conv1, conv2, fla, output_layer, **kwargs):
super(Conv11, self).__init__(**kwargs)
self.units = units
self.num_label = num_label
self.conv1 = conv1
self.conv2 = conv2
self.fla = fla
self. output_layer = output_layer
@classmethod
def construct(cls, units, num_label):
return cls(
units=units,
num_label=num_label,
conv1=layers.Conv2D(filters=units, kernel_size=(3, 3), padding='same'),
conv2=layers.Conv2D(filters=units * 2, kernel_size=(3, 3), padding='same'),
fla = layers. Flatten(),
output_layer=layers.Dense(units=num_label, activation='softmax')
)
def get_config(self):
config = super(Conv11, self).get_config()
config.update({"units": self.units,
"num_label": self. num_label,
"conv1": self.conv1,
"conv2": self.conv2,
"fla": self.fla,
"output_layer": self. output_layer})
return config
@classmethod
def form_config(cls, config):
return cls(**config)
def call(self, inputs, **kwargs):
x_conv = self.conv1(inputs)
x_conv = self.conv2(x_conv)
x_conv = self. fla(x_conv)
output = self. output_layer(x_conv)
return output
layer_conv_2 = Conv11. construct(3, 10)
model = tf.keras.Sequential([layer_conv_2])
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.1), loss=tf.keras.losses.CategoricalCrossentropy(from_logits=False), metrics=['categorical_accuracy'])
# train the model
history = model.fit(train_imgs, train_labels, epochs=1, batch_size=64)
print('**************\\
')
# print(model. trainable_weights)
print('**************\\
')
model.evaluate(x=test_imgs, y=test_labels)
# save the model
model.save('test_model.h5')
# load the model
mdoel_1 = keras.models.load_model('test_model.h5')
mdoel_1.evaluate(test_imgs, test_labels)
Output: After using model.save to save as a .h5 file, use keras.models.load_model(path) to load the entire model. It can be seen that the results of reloading the model and the training model on the test set are the same, and no error is reported.
938/938 [================================] - 4s 2ms/step - loss: 1.5829 - categorical_accuracy: 0.8487313/313 [===============================] - 1s 1ms/step - loss: 0.4335 - categorical_accuracy: 0.8664
313/313 [==============================] - 0s 1ms/step - loss: 0.4335 - categorical_accuracy: 0.8664refer to:
How to write a Custom Keras model so that it can be deployed for Serving | by Lak Lakshmanan | Towards Data Sciencehttps://towardsdatascience.com/how-to-write- a-custom-keras-model-so-that-it-can-be-deployed-for-serving-7d81ace4a1f8
Making new layers and models via subclassing (keras.io)
Saving nested layers in TensorFlow | by Oliver K. Ernst, Ph.D. | Practical coding | MediumSave, serialize, and export models (keras.io)
https://towardsdatascience.com/how-to-write- a-custom-keras-model-so-that-it-can-be-deployed-for-serving-7d81ace4a1f8