MXNet 干扰 Python 日志记录

Question

我希望将每个纪元信息存储在日志文件中，以查看与纪元的准确性，但我无法记录。为什么？

mnist = mx.test_utils.get_mnist()
batch_size = 100

print(os.getcwd())

log_file = '1.log'#''process_fold_' + str(0) + '_trial_' + str(1) + '.log'
logging.basicConfig(format='%(asctime)s %(levelname)s - %(message)s', datefmt='%d/%m/%Y %I:%M:%S %p', filename=log_file, level=logging.INFO)
logging.info('Started training on fold {} at trial {}'.format(0, 0))

train_iter = mx.io.NDArrayIter(mnist['train_data'],mnist['train_label'], batch_size, shuffle=True)
val_iter = mx.io.NDArrayIter(mnist['test_data'], mnist['test_label'],
                             batch_size)  # important as the prediction need not have equal barch size
lenet =get_my_net()
# create a trainable module on GPU 0
lenet_model = mx.mod.Module(symbol=lenet, context=mx.gpu(),logger=logzulu)
# train with the same
lenet_model.fit(train_iter,
                eval_data=val_iter,
                optimizer='sgd',
                optimizer_params={'learning_rate':0.1},
                eval_metric='acc',
                batch_end_callback = mx.callback.Speedometer(batch_size, 100),
                num_epoch=10,
                initializer=mx.init.Xavier(rnd_type='gaussian', factor_type="in", magnitude=2))

test_iter = mx.io.NDArrayIter(mnist['test_data'], None, batch_size)

Answer 1

发现 mxnet 具有内置记录器，并且它在每个 epoch 写入日志文件 'process_fold_0_trial_0.txt' 的准确性。要使用它，您必须对其进行初始化，然后将对象作为记录器参数。

初始化记录器

current_fold  =0
current_trial =0
logfilenamer = 'process_fold_' + str(current_fold) + '_trial_' +str(current_trial)+'.log'
logzulu = mxnet.log.get_logger(name='log_it',filename=logfilenamer,level=mxnet.log.DEBUG,filemode='w')
logzulu.error('what the hell ' + time.strftime('%x %X'))

继续

#  1. iterartors
train_iter = mx.io.NDArrayIter(mnist['train_data'],mnist['train_label'], batch_size, shuffle=True)
val_iter = mx.io.NDArrayIter(mnist['test_data'], mnist['test_label'],
                             batch_size)  
# 2.Getting my net 
lenet =get_my_net()
# create a trainable module on GPU 0

将记录器对象作为记录器

lenet_model = mx.mod.Module(symbol=lenet, context=mx.gpu(),logger=logzulu)

Answer 2

您是否通过 Jupyter 笔记本运行此代码？

如果是这样，您必须稍微不同地配置日志记录。你会在logging.basicConfig看到：

如果根记录器已经为其配置了处理程序，则此函数不执行任何操作。

Jupyter 笔记本会在您有机会之前配置处理程序。所以试试类似的东西;

import logging
logger = logging.getLogger()
fhandler = logging.FileHandler(filename='example.log', mode='a')
formatter = logging.Formatter('%(asctime)s %(levelname)s - %(message)s')
fhandler.setFormatter(formatter)
logger.addHandler(fhandler)
logger.setLevel(logging.INFO)