BiLSTM 隐藏层和记忆单元答案

【问题标题】：BiLSTM hidden layers, and memory cellsBiLSTM 隐藏层和记忆单元
【发布时间】：2022-12-03 21:19:54
【问题描述】：

我有一个 BiLSTM 模型，如下所示：

tf.keras.models.Sequential([
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(A, return_sequences=True),
                                  input_shape=x),
    tf.keras.layers.Dense(B, activation='tanh'),
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(A)),
    tf.keras.layers.Dense(B, activation='tanh'),
    tf.keras.layers.Dropout(0.25),
    tf.keras.layers.Dense(output),
])

如果总参数 = 100 万，A 和 B 应该是什么值？我应该添加多少个隐藏层才能让模型以正确的方式训练？

我尝试了以下内容：

一 = 265

乙 = 64

使用了三个 Dense 层，但预测仍然很弱！

【问题讨论】：

标签： tensorflow deep-learning lstm bilstm

【解决方案1】：

LSTM 层是长短期记忆，它可以将输入处理为序列，您不需要将输入切成小块。

示例：单形双锐，也可以应用双向或域属性。由于其规模，我将其作为单次旅行的示例。

import tensorflow as tf

class MyLSTMLayer( tf.keras.layers.LSTM ):
def __init__(self, units, return_sequences, return_state):
    super(MyLSTMLayer, self).__init__( units, return_sequences=True, return_state=False )
    self.num_units = units

def build(self, input_shape):
    self.kernel = self.add_weight("kernel",
    shape=[int(input_shape[-1]),
    self.num_units])

def call(self, inputs):
    lstm = tf.keras.layers.LSTM(self.num_units)
    return lstm(inputs)


start = 3
limit = 93
delta = 3
sample = tf.range(start, limit, delta)
sample = tf.cast( sample, dtype=tf.float32 )
sample = tf.constant( sample, shape=( 30, 1, 1 ) )
layer = MyLSTMLayer(10, True, True)
layer_2 = MyLSTMLayer(20, True, False)

temp = layer(sample)
print( temp )
temp = tf.expand_dims(temp, -1)
temp = layer_2(temp)
print( temp )

运算：( 10, 1, 1 ) x ( 10, 1, 1 )

layer = MyLSTMLayer(10, True, True)
sample = tf.constant( sample, shape=( 10, 1, 1 ) )

输出：(10, 10)

...
  1, 1, 1, 1]], shape=(10, 10), dtype=float32)

运算：( 20, 1, 1 ) x ( 10, 1, 1 )

layer = MyLSTMLayer(20, True, True)
sample = tf.constant( sample, shape=( 10, 1, 1 ) )

输出：(20, 10)

...
 1, 1, 1, 1, 1, 1]], shape=(20, 10), dtype=float32)

运算：( 30, 1, 1 ) x ( 10, 1, 1 )

layer = MyLSTMLayer(30, True, True)
sample = tf.constant( sample, shape=( 10, 1, 1 ) )

输出：(30, 10)

...
 1, 1, 1, 1, 1, 1]], shape=(30, 10), dtype=float32)

运算：( 30, 1, 1 ) x ( 10, 1, 1 )

layer = MyLSTMLayer(10, True, True)
layer_2 = MyLSTMLayer(20, True, False)
sample = tf.constant( sample, shape=( 30, 1, 1 ) )

输出：(30, 20)

...
 1, 1, 1, 1]]], shape=(30, 20), dtype=float32)

示例：实现、离散序列

import tensorflow as tf

class MyLSTMLayer( tf.keras.layers.LSTM ):
    def __init__(self, units, return_sequences, return_state):
        super(MyLSTMLayer, self).__init__( units, return_sequences=True, return_state=False )
        self.num_units = units

    def build(self, input_shape):
        self.kernel = self.add_weight("kernel",
        shape=[int(input_shape[-1]),
        self.num_units])

    def call(self, inputs):
        lstm = tf.keras.layers.LSTM(self.num_units)
        temp = lstm(inputs)
        temp = tf.nn.softmax(temp)
        temp = tf.math.argmax(temp).numpy()
        return temp
        
sample = tf.constant( [1.0, 2.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], shape=( 10, 1, 1 ) )
layer = MyLSTMLayer(10, True, False)
temp = layer(sample)
print( temp )

输出：作为一个序列

[1 0 1 1 1 0 0 0 1 0]

【讨论】：