R mxnet 1 pass 返回 NAN 作为损失值

Question

这是预期的行为吗？

library(mxnet)

hidden_u_1 <- 100
activ_hidden_1 <- 'tanh'

hidden_u_2 <- 1

learn_rate <- 0.001

initializer <- mx.init.uniform(1)

optimizer <- 'rmsprop' #sgd

loss <- mx.metric.mse

device.cpu <- mx.cpu()

mini_batch <- 64 #8

rounds <- 1 #2


## data symbols

nn_data <- mx.symbol.Variable('data')
nn_label <- mx.symbol.Variable('label')


## first fully connected layer

flatten <- mx.symbol.Flatten(data = nn_data)

fc1 <- mx.symbol.FullyConnected(data = flatten
                                , num_hidden = hidden_u_1)

activ1 <- mx.symbol.Activation(data = fc1, act.type = activ_hidden_1)

## second fully connected layer

fc2 <- mx.symbol.FullyConnected(data = activ1, num_hidden = hidden_u_2)

q_func <- mx.symbol.LinearRegressionOutput(data = fc2, label = nn_label, name = 'regr')


# initialize NN

train.x <- matrix(rnorm(640, 0, 1), ncol = 10)
train.x <- t(train.x)
dim(train.x) <- c(nrow(train.x), 1, 1, ncol(train.x))
train.y = rnorm(64, 0, 1)

nn_model <- mx.model.FeedForward.create(
     symbol = q_func,
     X = train.x,
     y = train.y,
     ctx = device.cpu,
     num.round = rounds,
     array.batch.size = mini_batch,
     optimizer = optimizer,
     eval.metric = loss,
     learning.rate = learn_rate,
     initializer = initializer
)

在大小大于 1 的小批量样本上计算损失值的 1 轮（通过）总是返回 NaN，而 2 轮或更多轮从第 2 轮开始给出有限值。

如果样本数是 minibatch 的 n 倍（例如，64 / 8），那么即使经过 1 轮，损失也是可计算的。

Answer 1

这是 MXNet 中的一个问题，已报告 here 并已修复 here。