【发布时间】：2018-08-19 05:00:04
【问题描述】：
我尝试使用 TensorFlow 构建 LSTM 模型。 LSTM 的训练似乎运行良好，准确率超过 90%。困扰我的一个问题是“测试准确率”非常低。所以，我认为这是由于过度拟合？但是像增加训练batch或者减少element_size（从10到5）这样的尝试都是白费，应用“dropout”也没有解决。我想要一些关于如何改进我的代码以获得高测试准确性的指导。 以下是我的数据/参数的摘要
Input variable is economic time series data standardized
Output variable is categorical features (labels) converted by one-hot encoding
Sequence_length : 20
Element_size: 5
Hidden_layer : 80
Categories (labels): 30
Training batch : 924
Test batch : 164
Learn rate is 0.0005 (Is it low?)
这是我构建的代码
#Split x_buch and y_batch
train_x,test_x=np.split(x_batch,[int(batch_size*0.85)])
train_y,test_y=np.split(y_batch,[int(batch_size*0.85)])
print('train_x shape: {0} and test_x shape: {1}'.format(train_x.shape,test_x.shape))
print('train_y shape: {0} and test_y shape: {1}'.format(train_y.shape,test_y.shape))
#Create placehold for inpt, labels
inputs=tf.placeholder(tf.float32,shape=[None,step_time,element_size],name='inputs')
y=tf.placeholder(tf.float32,shape=[None,label_num],name='y')
#Tensorflow built-in functinon
with tf.variable_scope('lstm'):
lstm_cell=tf.contrib.rnn.LSTMCell(hidden_layer,forget_bias=1.0)
cell_drop=tf.contrib.rnn.DropoutWrapper(lstm_cell, output_keep_prob=0.7)
outputs,states=tf.nn.dynamic_rnn(cell_drop,inputs,dtype=tf.float32)
print('outputs shape: {0}'.format(outputs.shape))
W1={'linear_layer':tf.Variable(tf.truncated_normal([hidden_layer,label_num],mean=0,stddev=.01))}
b1={'linear_layer':tf.Variable(tf.truncated_normal([label_num],mean=0,stddev=.01))}
#Extract the last relevant output and use in a linear layer
final_output=tf.matmul(outputs[:,-1,:],W1['linear_layer'])+b1['linear_layer']
with tf.name_scope('cross_entropy'):
softmax=tf.nn.softmax_cross_entropy_with_logits(logits=final_output,labels=y)
cross_entropy=tf.reduce_mean(softmax)
with tf.name_scope('train'):
train_step=tf.train.AdamOptimizer(learn_rate,0.9).minimize(cross_entropy)
with tf.name_scope('accracy'):
correct_prediction=tf.equal(tf.argmax(y,1),tf.argmax(final_output,1))
accuracy=(tf.reduce_mean(tf.cast(correct_prediction,tf.float32)))*100
#Training
with tf.Session()as sess:
sess.run(tf.global_variables_initializer())
for step in range(5000):
sess.run(train_step,feed_dict={inputs:train_x,y:train_y})
if step % 500 == 0:
acc=sess.run(accuracy,feed_dict={inputs:train_x,y:train_y})
loss=sess.run(cross_entropy,feed_dict={inputs:train_x,y:train_y})
print('Inter'+str(step)+',Minibatch loss= '+'{:.6f}'.format(loss)+', Traning Accracy='+'{:.5f}'.format(acc))
# Test
test_acc=sess.run(accuracy,feed_dict={inputs:test_x,y:test_y})
print("Test Accuracy is {0}".format(test_acc))
它的结果是
Input Shape: (21760, 5)
Output Shape: (21760, 30)
x_batch shape: (1088, 20, 5)
y_batch shape: (1088, 30)
train_x shape: (924, 20, 5) and test_x shape: (164, 20, 5)
train_y shape: (924, 30) and test_y shape: (164, 30)
outputs shape: (?, 20, 80)
Inter0,Minibatch loss= 3.398923, Traning Accracy=5.30303
Inter500,Minibatch loss= 2.027734, Traning Accracy=38.09524
Inter1000,Minibatch loss= 1.340760, Traning Accracy=61.79654
Inter1500,Minibatch loss= 1.010518, Traning Accracy=72.83550
Inter2000,Minibatch loss= 0.743997, Traning Accracy=79.76190
Inter2500,Minibatch loss= 0.687736, Traning Accracy=79.76190
Inter3000,Minibatch loss= 0.475408, Traning Accracy=85.17316
Inter3500,Minibatch loss= 0.430477, Traning Accracy=87.22944
Inter4000,Minibatch loss= 0.359262, Traning Accracy=89.17749
Inter4500,Minibatch loss= 0.274463, Traning Accracy=90.69264
Test Accuracy is 4.878048419952393
我从来没有用过TensorFlow和LSTM模型，所以，这是第一次，所以我知道我做错了什么，但不能指望它
那么，有人可以提供帮助吗？
【问题讨论】：
标签： python tensorflow deep-learning lstm