带有张量流的 CRNN 的可变长度输入

Question

我正在使用带有 tf.keras 的 tensorflow 2.x (2.2.0-dev20200228)。我最初的问题是 Conv2d 层不支持屏蔽，这就是为什么我正在寻找一种解决方法来训练可变长度输入。

因此，最终我想使用input_time_size = None。据我了解，为此我必须使用tf.shape。但是，当我这样做时，即使使用固定的input_time_size = 20，也会出现错误（见下文）。使用 model.output.shape 代替工作。非常欢迎任何建议。

请注意，为简单起见，代码呈现了一个精简的网络。

import tensorflow as tf


num_filters = 1
input_time_size = 20
input_frequency_size = 10

model = tf.keras.Sequential()

model.add(tf.keras.layers.Input(shape=(input_time_size, input_frequency_size, 1)))

model.add(tf.keras.layers.Conv2D(filters=num_filters, kernel_size=(10, 4), strides=[2, 1]))

conv_output_height = tf.shape(model.output)[1]
conv_output_width = tf.shape(model.output)[2]

# this works, but want to use input_time_size = None
#conv_output_height = model.output.shape[1]
#conv_output_width = model.output.shape[2]

model.add(tf.keras.layers.Reshape(target_shape=(conv_output_height, conv_output_width * num_filters)))

model.add(tf.keras.layers.RNN(cell=tf.keras.layers.GRUCell(units=1), stateful=False))

model.add(tf.keras.layers.Dense(units=1))

model.compile()

追溯：

Traceback (most recent call last):
  File "test.py", line 23, in <module>
    model.add(tf.keras.layers.RNN(cell=tf.keras.layers.GRUCell(units=1), stateful=False))
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 456, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/sequential.py", line 213, in add
    output_tensor = layer(self.outputs[0])
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/keras/layers/recurrent.py", line 652, in __call__
    return super(RNN, self).__call__(inputs, **kwargs)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 896, in __call__
    self._maybe_build(inputs)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 2403, in _maybe_build
    self.build(input_shapes)  # pylint:disable=not-callable
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/keras/layers/recurrent.py", line 571, in build
    self.cell.build(step_input_shape)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 315, in wrapper
    output_shape = fn(instance, input_shape)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/keras/layers/recurrent.py", line 1747, in build
    caching_device=default_caching_device)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 576, in add_weight
    caching_device=caching_device)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 743, in _add_variable_with_custom_getter
    **kwargs_for_getter)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer_utils.py", line 141, in make_variable
    shape=variable_shape if variable_shape else None)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 259, in __call__
    return cls._variable_v1_call(*args, **kwargs)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 220, in _variable_v1_call
    shape=shape)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 198, in <lambda>
    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 2598, in default_variable_creator
    shape=shape)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 263, in __call__
    return super(VariableMetaclass, cls).__call__(*args, **kwargs)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1434, in __init__
    distribute_strategy=distribute_strategy)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1567, in _init_from_args
    initial_value() if init_from_fn else initial_value,
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer_utils.py", line 121, in <lambda>
    init_val = lambda: initializer(shape, dtype=dtype)
  File "/home/test/.local/lib/python3.6/site-packages/tensorflow/python/ops/init_ops_v2.py", line 548, in __call__
    scale /= max(1., (fan_in + fan_out) / 2.)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

Answer 1

时间步长维度可以是动态的，但在本例中，conv_output_width 必须是固定的，即以下工作：

conv_output_height = tf.shape(model.output)[1]
conv_output_width = model.output.shape[2]

但是，训练失败并出现与急切执行相关的一些错误。找到后我会发布解决方案。

编辑：这是最终的解决方案：

conv_output_height = -1
conv_output_width = model.output.shape[2]