C++ 中的全卷积网络训练

Question

我正在尝试将 FCN 训练从 BrainScript 转移到 C++ 程序。首先，我只是加载和重新训练现有模型。我到了某个地方，但是 trainer->TrainMinibatch() 正在引发异常（而且我也不知道如何获取异常的描述）。粗略代码如下：

CNTK::DeviceDescriptor& device=  CNTK::DeviceDescriptor::GPUDevice(gpuid);

FunctionPtr rootFunc = nullptr;
try {
    rootFunc = Function::Load(modelname, device);
}
catch (char *err) {
    printf("Load fail: %s\n",err);
    return;
}
catch (...) {
    printf("Load fail\n");
    return;
}

std::cerr << "Loaded model ok" << std::endl;


MinibatchSourcePtr minibatchSource;
try {
    minibatchSource = HG_CreateMinibatchSource(64);
}
catch (char* err) {
    std::cerr << "Failed to init src: " << err << std::endl;
    return;
}
catch (...) {
    std::cerr << "Failed to init src " << std::endl;
    return;
}

auto imageStreamInfo = minibatchSource->StreamInfo(L"features");
auto labelStreamInfo = minibatchSource->StreamInfo(L"labels"); // We don't use labels as is FCN

auto inputImageShape = imageStreamInfo.m_sampleLayout;

std::cerr << "Input Shape: " << inputImageShape.AsString() << std::endl;

auto imageInputName = L"features";
auto imageInput = InputVariable(inputImageShape, imageStreamInfo.m_elementType, imageInputName);
auto classifierOutput = rootFunc;

//EITHER - construct error from output+target
std::wstring outputLayerName = L"op";
FunctionPtr outputLayer = rootFunc->FindByName(outputLayerName);
std::wstring targetLayerName = L"opool3";
FunctionPtr targetLayer = rootFunc->FindByName(targetLayerName);

// OR - just get from network
std::wstring errLayerName = L"e";
FunctionPtr errLayer = rootFunc->FindByName(errLayerName);

std::cerr << "Setup-got op layer" << outputLayer->Output().Shape().AsString() <<   std::endl;
std::cerr << "Setup-got tgt layer" << targetLayer->Output().Shape().AsString() << std::endl;
std::cerr << "Setup-got err layer" << errLayer->Output().Shape().AsString() << std::endl;

auto trainingLoss = CNTK::SquaredError(outputLayer, targetLayer);
auto prediction = CNTK::SquaredError(outputLayer, targetLayer);

LearningRateSchedule learningRatePerSample = TrainingParameterPerSampleSchedule(5e-8);

// Either
auto trainer = CreateTrainer(classifierOutput, trainingLoss->Output(), prediction->Output(), { SGDLearner(classifierOutput->Parameters(), learningRatePerSample) });

// Or
//auto trainer = CreateTrainer(classifierOutput, errLayer, errLayer, { SGDLearner(classifierOutput->Parameters(), learningRatePerSample) });

const size_t minibatchSize = 1;
size_t numMinibatchesToTrain = 100;
size_t outputFrequencyInMinibatches = 10;
try {
    for (size_t i = 0; i < numMinibatchesToTrain; ++i)
    {
        std::cerr << "Iteration: " << i << std::endl;
        auto minibatchData = minibatchSource->GetNextMinibatch(minibatchSize, device);

        std::cerr << "  got data for "<< imageInput.AsString() << std::endl;

        trainer->TrainMinibatch({ { imageInput, minibatchData[imageStreamInfo] } }, device); // This line throws exception!

        std::cerr << "Eval=" << trainer->PreviousMinibatchEvaluationAverage() << "," << trainer->PreviousMinibatchLossAverage() << std::endl;
    }
}
// Question edited as result of comment on exceptions below
catch (const std::exception & err) {
    std::cerr << "Training error:" << err.what() << std::endl;
}
catch (...) {
    std::cerr << "Training error" << std::endl;
}

目前尚不清楚如何定义损失函数（我猜这里 - 真的没有文档）。网络有 CNTK.exe/Brainscript 使用的损失 ('e')，它是输出 ('op') 和目标 ('opool3') 之间的平方误差。我尝试直接使用 e，并使用 CNTK::SquaredError() 在 C++ 中定义错误。两者都给出相同的输出，表示 trainer->TrainMinibatch 抛出异常：

Loaded model ok
Input Shape:B[1024 x 1024 x 3]
Setup-got op layeB[63 x 127 x 3]
Setup-got tgt layeB[63 x 127 x 3]
Setup-got err layeB[]
Iteration: 0
  got data forB,Input('features', [1024 x 1024 x 3], [*, #])
Training error:Values for 1 required arguments 'Input('features', [1024 x 1024 x 3], [, #])', that the requested output(s) 'Output('aggregateLoss', [], []), Output('Block233_Output_0', [], [, #]), Output('aggregateEvalMetric', [], [])' depend on, have not been provided.

我在这里做错了什么？

谢谢！

D.

编辑：例外是：

训练错误：1 个必需参数的值 'Input('features', [1024 x 1024 x 3], [, #])'，即请求的输出 'Output('aggregateLoss', [], []), Output('Block233_Output_0', [], [, #]), Output('aggregateEvalMetric', [], [])' 依赖，未提供。

更新：查看了 cntk 代码 (CompositeFunction.cpp)，问题似乎是输入和所需输入之间的不匹配：

提供的变量：Input('features', [1024 x 1024 x 3], [*, #])

必填参数：Input('features', [1024 x 1024 x 3], [, #])

区别在于 [*. #] 与 [, #]

不知道如何解决它！

Answer 1

这个问题是因为imageInput是一个与网络参数无关的新变量。相反，您需要获取与网络参数关联的输入变量并将它们绑定到 minibatchData，例如像

std::unordered_map<Variable, ValuePtr> inputDataMap = { { classifierOutput.Arguments()[0], minibatchData[imageStreamInfo] } }

然后将inputDataMap 传递给 TrainMinibatch。另见this evaluation example（训练和评估有非常相似的 API）