keras - 无法连接嵌入和类别编码层

Question

问题

在GCP Vertex AI example 之后使用下面的代码。

    for key in input_layers:
        feature_name = features.original_name(key)
        if feature_name in features.EMBEDDING_CATEGORICAL_FEATURES:
            vocab_size = feature_vocab_sizes[feature_name]
            embedding_size = features.EMBEDDING_CATEGORICAL_FEATURES[feature_name]
            
            embedding_output = keras.layers.Embedding(
                input_dim=vocab_size + 1,
                output_dim=embedding_size,
                name=f"{key}_embedding",
            )(input_layers[key])

            print(f"Shape of embed layer [{key}] has None [{embedding_output.shape}] output_dim is {embedding_size}")
            
            layers.append(embedding_output)
        elif feature_name in features.ONEHOT_CATEGORICAL_FEATURE_NAMES:
            vocab_size = feature_vocab_sizes[feature_name]
            onehot_layer = keras.layers.experimental.preprocessing.CategoryEncoding(
                max_tokens=vocab_size,
                output_mode="binary",
                name=f"{key}_onehot",
            )(input_layers[key])

            onehot_layer = tf.keras.layers.CategoryEncoding(
                num_tokens=vocab_size, 
                output_mode="one_hot",
                name=f"{key}_onehot",
            )(input_layers[key])
            
            print(f"Shape of one hot layer [{key}] has None [{onehot_layer.shape}]")
                      
            layers.append(onehot_layer)
        elif feature_name in features.NUMERICAL_FEATURE_NAMES:
            numeric_layer = tf.expand_dims(input_layers[key], -1)

            print(f"Shape of numeric layer [{key}] has None [{numeric_layer.shape}]")
                      
            layers.append(numeric_layer)
        else:
            pass

    #--------------------------------------------------------------------------------
    # Concatenate Embedding and CategoryEncoding layers fails
    #--------------------------------------------------------------------------------
    joined = keras.layers.Concatenate(name="combines_inputs")(layers)

代码导致无法拼接层的错误。

Shape of embed layer [trip_month_xf] has None [(None, 2)]
Shape of embed layer [trip_day_xf] has None [(None, 4)]
Shape of one hot layer [trip_day_of_week_xf] has None [(7,)]
Shape of embed layer [trip_hour_xf] has None [(None, 3)]
Shape of numeric layer [trip_seconds_xf] has None [(None, 1)]
Shape of numeric layer [trip_miles_xf] has None [(None, 1)]
Shape of one hot layer [payment_type_xf] has None [(5,)]
Shape of embed layer [pickup_grid_xf] has None [(None, 3)]
Shape of embed layer [dropoff_grid_xf] has None [(None, 3)]
Shape of numeric layer [euclidean_xf] has None [(None, 1)]
Shape of embed layer [loc_cross_xf] has None [(None, 10)]

ValueError: A `Concatenate` layer requires inputs with matching shapes except for the concatenation axis. Received: input_shape=[(None, 2), (None, 4), (7,), (None, 3), (None, 1), (None, 1), (5,), (None, 3), (None, 3), (None, 1), (None, 10)]

问题

是什么原因，如何解决？

Answer 1

原因

keras.layers.experimental.preprocessing.CategoryEncoding 层生成形状(N, )。

试图重塑为(None, N)，但没有成功

from keras.layers.core import Reshape

onehot_layer = Reshape((-1, vocab_size))(onehot_layer)
----------
ValueError: Exception encountered when calling layer "reshape_13" (type Reshape).

Dimension size must be evenly divisible by 49 but is 7 for '{{node reshape_13/Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32](Placeholder, reshape_13/Reshape/shape)' with input shapes: [7], [3] and with input tensors computed as partial shapes: input[1] = [7,?,7].

Call arguments received:
  • inputs=tf.Tensor(shape=(7,), dtype=float32)

修复

使用tf.keras.layers.CategoryEncoding。

        elif feature_name in features.ONEHOT_CATEGORICAL_FEATURE_NAMES:
            vocab_size = feature_vocab_sizes[feature_name]
            # onehot_layer = keras.layers.experimental.preprocessing.CategoryEncoding(
            #     max_tokens=vocab_size,
            #     output_mode="binary",
            #     name=f"{key}_onehot",
            # )(input_layers[key])
            onehot_layer = tf.keras.layers.CategoryEncoding(
                num_tokens=vocab_size, 
                output_mode="one_hot",
                name=f"{key}_onehot",
            )(input_layers[key])

结果：

Shape of embed layer [trip_month_xf] has None [(None, 2)]
Shape of embed layer [trip_day_xf] has None [(None, 4)]
Shape of one hot layer [trip_day_of_week_xf] has None [(None, 7)]
Shape of embed layer [trip_hour_xf] has None [(None, 3)]
Shape of numeric layer [trip_seconds_xf] has None [(None, 1)]
Shape of numeric layer [trip_miles_xf] has None [(None, 1)]
Shape of one hot layer [payment_type_xf] has None [(None, 5)]
Shape of embed layer [pickup_grid_xf] has None [(None, 3)]
Shape of embed layer [dropoff_grid_xf] has None [(None, 3)]
Shape of numeric layer [euclidean_xf] has None [(None, 1)]
Shape of embed layer [loc_cross_xf] has None [(None, 10)]