Keras fit_generator raise 你必须在使用它之前编译你的模型错误

Question

我尝试通过 Keras 构建一个 CNN + LSTM 模型来训练一个用于视频分类任务的模型。首先，使用模拟数据“fit()”api构建和训练了一个简单的模型，它也可以工作！

但实际上，用于训练这个模型的是一个视频数据集，它太大了，无法加载到内存中。所以我需要一个生成器，在这个地方，我还编写了一个模拟生成器，它生成形状与模拟数据方法相同的数据。此外，fit_generator API 取代了 fit。

当我运行 train_gen 函数时，出现以下错误：

File "lstm.py", line 48, in <module>
    train_gen()
  File "lstm.py", line 45, in train_gen
    model.fit_generator(data_generator.mock_generator(batch_size=32, num_classes=16), steps_per_epoch=1000, epochs=20)
  File "/usr/local/lib/python2.7/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1415, in fit_generator
    initial_epoch=initial_epoch)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training_generator.py", line 39, in fit_generator
    model._make_train_function()
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 485, in _make_train_function
    raise RuntimeError('You must compile your model before using it.')
RuntimeError: You must compile your model before using it.

我试图通过在堆栈溢出和谷歌搜索来解决这个问题，但没有找到确切的解决方案。有一些类似的问题可以通过指定LSTM input_shape和output size来解决，或者添加model.compile()。

下面的sn-p代码是可以运行的，和上面说的完全一样。

import keras
from keras.models import Sequential
from keras.layers import Input, Embedding, LSTM, Dense, Reshape
from keras.layers import Dense, Dropout, Flatten, Activation
from keras.layers.normalization import BatchNormalization
from keras.layers.wrappers import Bidirectional

from keras.optimizers import Adam
import numpy as np
import os, random

os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"   # see issue #152
os.environ["CUDA_VISIBLE_DEVICES"] = ""

use_dropout = True
metrics = ['accuracy']

#mock data generator
def mock_generator(batch_size, input_length, input_dims, num_classes=16):
    while True:
        yield np.random.random((batch_size, input_length, input_dims)), keras.utils.to_categorical(np.random.randint(num_classes, size=(batch_size, 1)), num_classes=num_classes)

#mock data with shape as data generator
def mock_data(batch_size, input_length, input_dims, num_classes=16):
    if True:
        return np.random.random((batch_size, input_length, input_dims)), keras.utils.to_categorical(np.random.randint(num_classes, size=(batch_size, 1)), num_classes=num_classes)

#construct model, lstm units is fixed
def bi_lstm(input_shape, num_classes=16):
    model = Sequential()
    model.add(Bidirectional(LSTM(100, return_sequences=True, activation='relu', input_shape=input_shape), merge_mode='concat'))
    model.add(Bidirectional(LSTM(100, activation='relu', input_shape=(input_shape[0],100), return_sequences=False), merge_mode='concat'))
    if use_dropout:
        model.add(Dropout(0.5))
    model.add(Dense(num_classes))
    model.add(BatchNormalization())
    model.add(Activation('softmax'))
    optimizer = Adam(lr=1e-5, decay=1e-6)
    model.compile(loss='categorical_crossentropy', optimizer=optimizer,
                           metrics=metrics)
    return model

#fit api run successfully
def train():
    input_length=10
    input_dims=128
    num_classes=10
    model = bi_lstm((input_length, input_dims), num_classes)
    x_train, y_train = mock_data(32, input_length, input_dims, num_classes)
    model.fit(x_train, y_train, epochs=20, batch_size=32)

#fit_generator api raise error!
def train_gen():
    input_length=10
    input_dims=128
    num_classes=10
    model = bi_lstm((input_length, input_dims), num_classes)
    generator = mock_generator(32, input_length, input_dims, num_classes)
    model.fit_generator(generator, steps_per_epoch=1000, epochs=20)

#test mock generator function
def test_mock_gen():
    result  = mock_generator(32,10,128,16)
    for i in range(2):
        x, y = result.next()
        print x.shape
        print y.shape

if __name__ == '__main__':
    train()
    train_gen()
    #test_mock_gen()

Answer 1

您遇到的错误再明确不过了：model 已被声明为 train() 函数上的本地对象 并且 不知道通过函数 train_gen()。将您的模型定义为全局变量（例如在 main 中），它将起作用。