【问题标题】:Why the FCN32 output is a black image (all pixels have zero values)?为什么 FCN32 输出是黑色图像(所有像素都为零值)?
【发布时间】:2017-05-29 20:01:23
【问题描述】:

我正在尝试从 FCN 32 获取输出。我使用 pascalcontext-fcn32-heavy.caffemodel 预训练模型训练了 FCN32。我可以运行 5 个类别的灰度图像。但是,在推理过程中,输出全为零(黑色图像)。这是推理代码:

import numpy as np
from PIL import Image
import sys
import scipy.io as sio
from caffe.proto import caffe_pb2
import caffe
    caffe.set_device(0) 
    caffe.set_mode_gpu()

    # load image, subtract mean, and make dims C x H x W for Caffe

   img_name='/home/ss/caffe-pascalcontext-fcn32s/dataset/Test/PNG/image-061-023.png'    #+
    im = Image.open(img_name)

    in_ = np.array(im, dtype=np.float32)
    in_ = np.expand_dims(in_, axis=0)               #+
    print in_.shape
    #Read mean image
    '''####################'''
    mean_blob = caffe_pb2.BlobProto()
    with open('/home/ss/caffe-pascalcontext-fcn32s/input/FCN32_mean.binaryproto') as f:
        mean_blob.ParseFromString(f.read())
    mean_array = np.asarray(mean_blob.data, dtype=np.float32).reshape(
        (mean_blob.channels, mean_blob.height, mean_blob.width))
    in_ -= mean_array

    net_root = '/home/ss/caffe-pascalcontext-fcn32s'

    MODEL_DEF = net_root + '/deploy.prototxt'
    PRETRAINED = net_root + '/snapshot/FCN32s_train_iter_40000.caffemodel'
    # load net
    #net = caffe.Net('deploy.prototxt', 'snapshot/train_iter_640000.caffemodel', caffe.TEST)
    net = caffe.Net(MODEL_DEF,PRETRAINED, caffe.TEST)
    #net = caffe.Net('deploy.prototxt', 'snapshot_bak1/train_iter_400000.caffemodel', caffe.TEST)

    # shape for input (data blob is N x C x H x W), set data
    # put img to net
    net.blobs['data'].reshape(1, *in_.shape)  # 1: batch size, *in_.shape 3 channel ?
    net.blobs['data'].data[...] = in_

    # run net and take argmax for prediction
    output = net.forward()

    # print
    def print_param(output):
        # the blobs
        print '--------------------------'
        print 'the blobs'
        for k, v in net.blobs.items():
            print k, v.data.shape

        # the parameters
        print '--------------------------'
        print 'the paramsters'
        for k, v in net.params.items():
            print k, v[0].data.shape

        # the conv layer weights
        print '--------------------------'
        print 'the conv layer weights'
        print net.params['conv1_1'][0].data

        # the data blob 
        print '--------------------------'
        print 'the data blob'
        print net.blobs['data'].data

        # the conv1_1 blob
        print '--------------------------'
        print 'the conv1_1 blob'
        print net.blobs['conv1_1'].data

        # the pool1 blob
        print '--------------------------'
        print 'the pool1 blob'
        print net.blobs['pool1'].data

        weights = net.blobs['fc6'].data[0]
        print 'blobs fc6'
        print np.unique(weights)
        weights = net.blobs['fc7'].data[0]
        print 'blobs fc7'
        print np.unique(weights)
        weights = net.blobs['score_fr_sign'].data[0]
        print 'blobs score_fr_sign'
        print np.unique(weights)
        weights = net.blobs['upscore_sign'].data[0]
        print 'blobs upscore_sign'
        print np.unique(weights)
        weights = net.blobs['score'].data[0]
            print weights.shape             #+
            sio.savemat('scores.mat',{'weights':weights})   #+
        print 'blobs score'
        print np.unique(weights)

    print_param(output)

    out = net.blobs['score'].data[0].argmax(axis=0)
    print out           #+

    #np.savetxt("vote", out, fmt="%02d")
    np.savetxt("vote", out, fmt="%d")

    print im.height
    print im.width
    print out.shape, len(out.shape)

    def array2img(out):
        out1 = np.array(out, np.unit8)
        img = Image.fromarray(out1,'L')
        for x in range(img.size[0]):
            for y in range(img.size[1]):
                if not img.getpixel((x, y)) == 0:
                    print 'PLz', str(img.getpixel((x, y)))

        img.show()


    def show_pred_img(file_name):
        file = open(file_name, 'r')
        lines = file.read().split('\n')

        #img_name = str(sys.argv[1])
        im = Image.open(img_name)
        im_pixel = im.load()

        img = Image.new('RGB', im.size, "black")
        pixels = img.load()

        w, h = 0, 0
        for l in lines:
            w = 0
            if len(l) > 0:
                word = l.split(' ')
                for x in word:
                    if int(x) == 1:
                        pixels[w, h] = im_pixel[w, h]
                    w += 1
                h += 1
        print im.size
        #img.show()
        img.save(img_name+'_result.png')
    show_pred_img('vote')

这是推理的日志信息:

the blobs
data (1, 1, 256, 256)
data_input_0_split_0 (1, 1, 256, 256)
data_input_0_split_1 (1, 1, 256, 256)
conv1_1 (1, 64, 454, 454)
conv1_2 (1, 64, 454, 454)
pool1 (1, 64, 227, 227)
conv2_1 (1, 128, 227, 227)
conv2_2 (1, 128, 227, 227)
pool2 (1, 128, 114, 114)
conv3_1 (1, 256, 114, 114)
conv3_2 (1, 256, 114, 114)
conv3_3 (1, 256, 114, 114)
pool3 (1, 256, 57, 57)
conv4_1 (1, 512, 57, 57)
conv4_2 (1, 512, 57, 57)
conv4_3 (1, 512, 57, 57)
pool4 (1, 512, 29, 29)
conv5_1 (1, 512, 29, 29)
conv5_2 (1, 512, 29, 29)
conv5_3 (1, 512, 29, 29)
pool5 (1, 512, 15, 15)
fc6 (1, 4096, 9, 9)
fc7 (1, 4096, 9, 9)
score_fr_sign (1, 5, 9, 9)
upscore_sign (1, 5, 320, 320)
score (1, 5, 256, 256)
--------------------------
the paramsters
conv1_1 (64, 1, 3, 3)
conv1_2 (64, 64, 3, 3)
conv2_1 (128, 64, 3, 3)
conv2_2 (128, 128, 3, 3)
conv3_1 (256, 128, 3, 3)
conv3_2 (256, 256, 3, 3)
conv3_3 (256, 256, 3, 3)
conv4_1 (512, 256, 3, 3)
conv4_2 (512, 512, 3, 3)
conv4_3 (512, 512, 3, 3)
conv5_1 (512, 512, 3, 3)
conv5_2 (512, 512, 3, 3)
conv5_3 (512, 512, 3, 3)
fc6 (4096, 512, 7, 7)
fc7 (4096, 4096, 1, 1)
score_fr_sign (5, 4096, 1, 1)
upscore_sign (5, 1, 64, 64)
--------------------------
the conv layer weights
[[[[ 0.  0.  0.]
   [ 0.  0.  0.]
   [ 0.  0.  0.]]]

...
 .
 .
 .       

 [[[ 0.  0.  0.]
   [ 0.  0.  0.]
   [ 0.  0.  0.]]]]
--------------------------
the data blob
[[[[ 29.32040787  20.31391525  20.30148506 ...,  10.41113186  11.42486095
      6.42949915]
   [ 33.32374954  21.31280136  22.30037117 ...,   9.40779209  10.42189217
      8.43079758]
   [ 36.32300568  25.30816269  25.29183578 ...,  10.40148449  11.41818142
     10.42838573]
   ..., 
   [ 34.64990616  31.65658569  30.65714264 ...,   4.           2.99981451
      0.99962896]
   [ 39.65788651  33.65769958  29.65974045 ...,   5.99981451   4.99944353
      0.99888682]
   [ 41.6641922   34.66493607  30.66567802 ...,   5.99962902   2.99907231
      3.99833035]]]]
--------------------------
the conv1_1 blob
[[[[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  ..., 
  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]]]
--------------------------
the pool1 blob
[[[[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  ..., 
  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]]]
blobs fc6
[ 0.]
blobs fc7
[ 0.]
blobs score_fr_sign
[-1.61920226 -1.34294271  0.07809996  0.60521388  2.2788291 ]
blobs upscore_sign
[-1.61920238 -1.61920226 -1.61920214 ...,  2.27882886  2.2788291
  2.27882934]
(5, 256, 256)
blobs score
[-1.61920238 -1.61920226 -1.61920214 -1.59390223 -1.59390211 -1.5689975
 -1.54330218 -1.54330206 -1.51918805 -1.49270213 -1.49270201 -1.4709599
 -1.46937859 -1.44210207 -1.44210196 -1.42273164 -1.41956913 -1.39150202
 -1.3915019  -1.37608469 -1.37450349 -1.36975968 -1.34294283 -1.34294271
 -1.3429426  -1.34090197 -1.34090185 -1.32943773 -1.32627523 -1.32195926
 -1.31995022 -1.30130363 -1.2903018  -1.28437209 -1.2827909  -1.27999234
 -1.27999222 -1.27804708 -1.27014089 -1.25999236 -1.23970175 -1.23930645
 -1.23802543 -1.23802531 -1.23614395 -1.22981894 -1.22033143 -1.21999264
 -1.21868122 -1.19605839 -1.19605827 -1.195822   -1.19424069 -1.18949699
 -1.1891017  -1.18910158 -1.18159068 -1.17999291 -1.17736995 -1.17052197
 -1.15409136 -1.15233755 -1.14917505 -1.14285004 -1.14130461 -1.13999307
 -1.13850164 -1.13850152 -1.13605869 -1.13336253 -1.12071252 -1.11212444
 -1.11043441 -1.1088531  -1.10410941 -1.10261631 -1.09999335 -1.09620309
 -1.09474754 -1.08790159 -1.08790147 -1.08513427 -1.07090306 -1.07015753
 -1.07015741 -1.06853116 -1.06536865 -1.06523943 -1.06392801 -1.05999362
 -1.05904365 -1.05343628 -1.04955614 -1.03730154 -1.03730142 -1.03690612
 -1.02820921 -1.02819049 -1.02786267 -1.02662802 -1.02523971 -1.0218842
 -1.02109361 -1.0199939  -1.013978   -1.01212502 -1.00290918 -0.99179727
 -0.99048585 -0.98867792 -0.98788732 -0.98670143 -0.98670137 -0.9865514
 -0.98622358 -0.98622352 -0.98472482 -0.97999406 -0.97839981 -0.97128415
 -0.97081381 -0.9689123  -0.95626229 -0.95573193 -0.95310903 -0.94914663
 -0.94786316 -0.94756538 -0.9442566  -0.94425654 -0.94282162 -0.94044977
 -0.93999434 -0.93491536 -0.92950261 -0.9238466  -0.92097807 -0.91966659
 -0.9157322  -0.91040593 -0.90961534 -0.90917486 -0.90724343 -0.90228963
 -0.90091842 -0.89999455 -0.89143091 -0.88819134 -0.88622415 -0.88360125
 -0.8787809  -0.87835538 -0.87324655 -0.8716653  -0.87048656 -0.86692154
 -0.86032271 -0.86032265 -0.85999483 -0.85901529 -0.85278171 -0.85147029
 -0.84794647 -0.84753585 -0.84688014 -0.8409785  -0.83608711 -0.8329246
 -0.83179826 -0.8265996  -0.81999505 -0.81933933 -0.81835574 -0.81835568
 -0.81711209 -0.81671637 -0.81147051 -0.80556893 -0.80360168 -0.80050892
 -0.79892766 -0.79418391 -0.79310995 -0.78720838 -0.78627765 -0.7858969
 -0.78196251 -0.77999532 -0.77540517 -0.76622486 -0.76493073 -0.76176822
 -0.75544322 -0.75507742 -0.75442165 -0.75245446 -0.7472086  -0.73933983
 -0.73093385 -0.72935259 -0.72884804 -0.72460884 -0.72425795 -0.72294647
 -0.71901208 -0.71245474 -0.70327443 -0.69693691 -0.6937744  -0.69343841
 -0.69081551 -0.68556964 -0.67770082 -0.66452122 -0.66393042 -0.66293997
 -0.66261894 -0.65868455 -0.65212721 -0.63442242 -0.63210559 -0.63179946
 -0.6265536  -0.60622585 -0.60491437 -0.60127115 -0.60097998 -0.57802927
 -0.57540637 -0.55114424 -0.54983276 -0.52425915 -0.49868551  0.02900147
  0.03048873  0.03197598  0.03205225  0.03346324  0.03361578  0.03495049
  0.0351793   0.03525557  0.03643775  0.03674283  0.03689536  0.037925
  0.03830635  0.03853516  0.03861143  0.03941226  0.03986987  0.04017495
  0.04032749  0.04089952  0.0414334   0.04181475  0.04204356  0.04211983
  0.04238677  0.04299692  0.04345454  0.04375962  0.04387403  0.04391216
  0.04456045  0.04509434  0.04536128  0.04547568  0.04570449  0.04578076
  0.04612397  0.04673413  0.04684854  0.04719175  0.04749683  0.04759216
  0.04764936  0.0476875   0.04837392  0.04890781  0.04925102  0.04928916
  0.04951797  0.04959423  0.05001372  0.05003278  0.05003279  0.05062388
  0.05108149  0.05138657  0.05153911  0.05165351  0.05233994  0.05247341
  0.05247341  0.05287382  0.05325517  0.05348398  0.05356025  0.054056
  0.05466616  0.05491403  0.05491403  0.05512378  0.05542885  0.05558139
  0.05645849  0.05699238  0.05735466  0.05735466  0.05737372  0.05760253
  0.0576788   0.05886098  0.05931859  0.05962367  0.05977621  0.05979528
  0.05979528  0.06126347  0.06164481  0.06187363  0.06194989  0.0622359
  0.06223591  0.06366596  0.06397104  0.06412357  0.06467653  0.06606845
  0.06629726  0.06637353  0.06711715  0.06847093  0.06862348  0.06955777
  0.06955778  0.07087342  0.0709497   0.0719984   0.0719984   0.07327592
  0.07443902  0.07443903  0.0756784   0.07687964  0.07687965  0.07809995
  0.07809996  0.07809997  0.22473885  0.23626392  0.24778898  0.24838002
  0.25931406  0.26049611  0.27083912  0.27261221  0.27320322  0.28236419
  0.28472832  0.28591037  0.29388925  0.29684439  0.29861748  0.29920852
  0.30541432  0.3089605   0.31132463  0.31250668  0.31693938  0.3210766
  0.32403174  0.32580483  0.32639587  0.32846448  0.33319271  0.33673888
  0.33910298  0.33998954  0.34028506  0.34530881  0.349446    0.35151461
  0.35240114  0.35417423  0.35476527  0.35742489  0.36215314  0.36303967
  0.36569929  0.36806342  0.36880219  0.36880222  0.36924547  0.36954099
  0.37486026  0.37899747  0.38165709  0.38195261  0.3837257   0.38431671
  0.38756737  0.38771513  0.38771516  0.39229563  0.39584181  0.39820591
  0.39938796  0.40027452  0.40559378  0.40662807  0.40973097  0.41268614
  0.4144592   0.41505024  0.41889194  0.42362016  0.42554098  0.42554101
  0.42716634  0.42953047  0.43071252  0.43750936  0.44164655  0.44445392
  0.44445395  0.44460171  0.44637477  0.44696581  0.45612678  0.45967296
  0.46203706  0.46321911  0.46336687  0.4633669   0.4747442   0.47769934
  0.47947243  0.48006344  0.48227981  0.48227984  0.49336162  0.49572572
  0.49690777  0.50119275  0.51197904  0.5137521   0.51434314  0.52010566
  0.52010572  0.53059644  0.53177851  0.53901857  0.53901863  0.54921389
  0.54980487  0.55793154  0.56783128  0.57684445  0.57684451  0.58644873
  0.59575737  0.59575742  0.60521382  0.60521388  0.60521394  0.84621561
  0.88961124  0.93300694  0.93523234  0.97640258  0.98085344  1.01979828
  1.02647448  1.02869999  1.06319392  1.07209563  1.07654643  1.10658967
  1.11771667  1.12439299  1.12661839  1.14998531  1.16333783  1.17223942
  1.17669034  1.19338095  1.20895886  1.22008598  1.22676229  1.22898769
  1.23677659  1.25458002  1.26793253  1.27683413  1.28017235  1.28128505
  1.30020106  1.31577897  1.32356799  1.32690609  1.3335824   1.3358078
  1.34582222  1.36362553  1.36696362  1.37697804  1.38587976  1.38866138
  1.3886615   1.39033055  1.39144325  1.41147208  1.42704999  1.43706429
  1.43817711  1.44485331  1.4470787   1.45931852  1.45987487  1.45987499
  1.47712183  1.49047434  1.49937606  1.50382698  1.50716507  1.52719378
  1.53108823  1.53108835  1.5427717   1.55389881  1.56057513  1.56280053
  1.57726574  1.59506905  1.6023016   1.60230172  1.60842156  1.61732328
  1.62177408  1.6473664   1.66294444  1.67351508  1.6735152   1.67407143
  1.68074775  1.68297315  1.71746719  1.7308197   1.7397213   1.74417222
  1.74472845  1.74472857  1.78756785  1.79869497  1.80537117  1.80759656
  1.81594181  1.81594193  1.81594205  1.85766852  1.86657023  1.87102103
  1.88715529  1.88715541  1.9277693   1.9344455   1.9366709   1.95836878
  1.99786997  2.00232077  2.02958202  2.02958226  2.06797075  2.07019615
  2.10079551  2.10079575  2.1380713   2.17200899  2.20817208  2.24322224
  2.24322248  2.27882886  2.2788291   2.27882934]
256
256
(256, 256) 2
(256, 256)

我有两个主要问题:

  1. 我想知道为什么输出是黑色的?和
  2. 如何知道何时停止运行算法(即迭代 数字)?我真的不知道什么是最佳迭代次数和 我可以在那个阶段停止微调的损失值。我停下了 在40,000 iterations 进行培训,我对此一无所知。
  3. 分割结果必须是灰度图吗 以及(如输入),或创建 RGB 结果图像不会使任何 输出有什么不同?

我真的不知道我做对了多少。很困惑:( 有人有什么建议吗?非常感谢您的帮助。

【问题讨论】:

标签: python-2.7 deep-learning caffe pycaffe deeplearning4j


【解决方案1】:

是的,这通常取决于您的图像大小!你检查过你的数据类型吗?你的图像和groundtruths都应该是uint8!

您是否也将“组”行添加到您的 Deconv 层?

最好的

【讨论】:

  • 是的,我添加了group which 5(类数)。如何检查 Python 中的数据类型。我已将图像和地面实况保存到 Matlab 中的 uint8 中。转换为 LMDB 时可能会发生变化吗?再次感谢
  • 欢迎来到 SO。感谢您提供宝贵的意见和答案,但如果您可以简单地编辑现有答案,请不要针对单个问题发布多个答案。
  • 如果您的数据保存为 uint8,只需仔细检查您的标签,看看像素的强度是否在 0 到 4 之间(#of your classes)。
【解决方案2】:

确保标签的数据类型是 uint8!我遇到了同样的问题!

还要确保在训练前您的 prototxt 中有如下重量填充物!

layer {
name: "myupscore2"
type: "Deconvolution"
bottom: "myscore_fr"
top: "myupscore2"
param {
lr_mult: 5
}
convolution_param {
group :2
num_output: 2
weight_filler: { type: "bilinear" }
bias_term: false
kernel_size: 4
stride: 16
 }
 }

祝你好运!

【讨论】:

  • 谢谢 Arghavan,是的,我已经添加了 weight_fille。迭代 0 中的损失是 loss = 105476 。正常吗?我的图像是灰度的。
猜你喜欢
  • 1970-01-01
  • 2019-04-23
  • 2020-03-01
  • 2014-04-28
  • 2018-09-28
  • 1970-01-01
  • 1970-01-01
  • 2018-03-16
  • 2020-09-03
相关资源
最近更新 更多