keras 层在每个 epoch 重新启动一部分权重

Question

我正在寻找一种方法来重新启动每个 epoch（或每 n 个 epoch）的层权重的随机部分，我发现 this 解释了如何重新初始化层。我可以使用

weights = layer.get_weights() 

然后使用 numpy 操作来重新初始化部分权重，或者创建一个虚拟层从中提取新的初始化权重并将它们与 set_weights 一起使用。我正在寻找一种更优雅的方法来初始化层中权重的某个（或随机）部分。

谢谢

Answer 1

Keras 有set_weights 方法来设置层的权重。要在每个 epoch 重置层的权重，请使用回调。

class My_Callback(keras.callbacks.Callback):

    def on_epoch_begin(self, logs={}):
        return

    def on_epoch_end(self, epoch, logs={}):
        layer_index = 0  ## index of the layer you want to change
        # random weights to reset the layer
        new_weights = numpy.random.randn(*self.model.layers[layer_index].get_weights().shape)

        self.model.layers[layer_index].set_weights(new_weights)

编辑：

要重置层的随机 n 个权重，可以使用 numpy 获取要重置的随机索引。现在代码将是

    def on_epoch_end(self, epoch, logs={}):
        layer_index = np.random.randint(len(self.model.layers)) # Random layer index to reset
        weights_shape = self.model.layers.get_weights().shape
        num = 10 # number of weights to reset
        indexes = np.random.choice(weights_shape[0], num, replace=False)   # indexes of the layer to reset
        reset_weights = numpy.random.randn(*weights_shape[1:]) # random weights to reset the layer

        layer_weights = self.model.layers[layer_index].get_weights()
        layer_weights[indexes] = reset_weights
        self.model.layers[layer_index].set_weights(layer_weights)

类似重置层权重的随机p %，第一个numpy可以用来选择层权重的p %索引。

    def on_epoch_end(self, epoch, logs={}):
        layer_index = np.random.randint(len(self.model.layers)) # Random layer index to reset
        weights_shape = self.model.layers.get_weights().shape
        percent = 10 # Percentage of weights to reset
        indexes = np.random.choice(weights_shape[0], int(percent/100.) * weights_shape[0], replace=False)   # indexes of the layer  to reset
        reset_weights = numpy.random.randn(*weights_shape[1:]) # random weights to reset the layer

        layer_weights = self.model.layers[layer_index].get_weights()
        layer_weights[indexes] = reset_weights
        self.model.layers[layer_index].set_weights(layer_weights)