使用反向传播训练实值神经网络

Question

我正在尝试为实值输出训练神经网络，我只是给出了一组网络插值点（看起来像平方振荡），但是反向传播总是不能很好地适应输入，我尝试添加更多的特征，这些特征是输入的更高值并对输出进行标准化，但这似乎没有帮助。网络是 3 层 1 输入 1 隐藏 1 输出和一个输出节点 我该如何解决这个问题？ 我也使用了这个成本函数，对吗？

for k = 1:m

  C= C+(y(k)-a2(k))^2;
end

我的代码：

clc;
clear all;
close all;
input_layer_size  = 4;
hidden_layer_size = 60;
num_labels = 1;
load('Xs');
load('Y-s');
theta1=randInitializeWeights(4, 60);
theta2=randInitializeWeights(60, 1);
plot (xq,vq)
hold on 
xq=polyFeatures(xq,4);
param=[theta1(:) ;theta2(:)];

[J ,Grad]= nnCostFunction(param,input_layer_size ,hidden_layer_size,num_labels,xq,vq,0);

       options = optimset('MaxIter', 50);
  costFunction = @(p) nnCostFunction(p, ...
                               input_layer_size, ...
                               hidden_layer_size, ...
                               num_labels, xq, vq, 10);


  [nn_params, cost] = fmincg(costFunction, param, options);

Theta1 = reshape(nn_params(1:hidden_layer_size * (input_layer_size + 1)), ...
             hidden_layer_size, (input_layer_size + 1));

Theta2 = reshape(nn_params((1 + (hidden_layer_size * (input_layer_size +   1))):end), ...
             num_labels, (hidden_layer_size + 1));

         l=xq(:,1);
         out =predictTest(Theta1,Theta2,xq);

          accuracy=mean(double(out == vq)) * 100
         plot (l,out,'yellow');

         hold off



    function [J grad] = nnCostFunction(nn_params, ...
        input_layer_size, ...
        hidden_layer_size, ...
        num_labels, ...
        X, y, lambda)

    y(841:901)=0;
    y=y/2.2;

    Theta1 = reshape( (nn_params(1:hidden_layer_size * (input_layer_size+1 ))), ...
        hidden_layer_size, (input_layer_size +1 ));

    Theta2 = reshape(nn_params((1+(hidden_layer_size * (input_layer_size +1))):end), ...
        num_labels, (hidden_layer_size +1 ));

    m = size(X, 1);
    J = 0;
    Theta1_grad = zeros(size(Theta1));
    Theta2_grad = zeros(size(Theta2));


    X= [ones(m,1) X];

    z1=X*Theta1';
    a1 = sigmoid(z1);
    a1= [ones(size(a1,1),1) a1];
    z2=a1*Theta2';
    a2= sigmoid(z2);


    for k = 1:m

        J= J+(y(k)-a2(k))^2;

    end
    J= J/m;
    Theta1(:,1)=zeros(1,size(Theta1,1));
    Theta2(:,1)=zeros(1,size(Theta2,1));
    s1=sum (sum (Theta1.^2));
    s2=sum (sum (Theta2.^2));

    s3= lambda *(s2 +s1 );
    s3=s3/(2*m);
    J=J+s3;

    D2=zeros(size(Theta2));
    D1=zeros(size(Theta1));
    for i= 1:m

        delta3=a2(i)-y(i);
        v=[0 sigmoidGradient(z1(i,:))];
        delta2=(Theta2'*delta3').*v';



        D2=D2+delta3'*a1(i,:) ;
        D1=D1+delta2(2:end)*X(i,:);


    end


    Theta1_grad = D1./m + (lambda/m)*[zeros(size(Theta1,1), 1) Theta1(:, 2:end)];
    Theta2_grad = D2./m + (lambda/m)*[zeros(size(Theta2,1), 1) Theta2(:, 2:end)];

    grad = [Theta1_grad(:) ; Theta2_grad(:)];


    end



    function W = randInitializeWeights(L_in, L_out)


    epsilon_init = 0.5;
    W = rand(L_out, 1 + L_in)*2*epsilon_init - epsilon_init;

    end

输入是 1:9 内插 0.01 增量，目标是 0:2.2 之间的数字，如方形脉冲

linear interpolation of data vs predicted in red

updated after increasing epochs

Answer 1

请注意，红线永远不会为零（最低约为 0.4），这表示训练的权重永远不会使网络输出为零（我的意思是权重需要足够负面，并且在某些单元格中，偏差要么完全缺失，要么不负面）

1. 将信号从 [-1 到 1] 缩放并使用权重和偏差来训练网络以查看影响。权重和偏差都是必需的。
2. 此处使用的简单神经网络不适用于方波等时间序列预测。对时间序列使用诸如 here 之类的预测模型