Deep Dream Code 不会生成可识别的模式

Question

我尝试使用此代码创建自己的 Deep Dream 算法：

import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import inception

img = np.random.rand(1,500,500,3)
net = inception.get_inception_model()
tf.import_graph_def(net['graph_def'], name='inception')
graph = tf.get_default_graph()
sess = tf.Session()
layer = graph.get_tensor_by_name('inception/mixed5b_pool_reduce_pre_relu:0')
gradient = tf.gradients(tf.reduce_mean(layer), graph.get_tensor_by_name('inception/input:0'))
softmax = sess.graph.get_tensor_by_name('inception/softmax2:0')
iters = 100
init = tf.global_variables_initializer()

sess.run(init)
for i in range(iters):
    prediction = sess.run(softmax, \
                          {'inception/input:0': img})
    grad = sess.run(gradient[0], \
                          {'inception/input:0': img})
    grad = (grad-np.mean(grad))/np.std(grad)
    img = grad
    plt.imshow(img[0])
    plt.savefig('output/'+str(i+1)+'.png')
    plt.close('all')

但即使在运行此循环 100 次迭代后，生成的图片仍然看起来是随机的（我会将所述图片附加到此问题中）。 有人可以帮我优化我的代码吗？

Answer 1

将 Inception 网络用于 Deep Dream 有点繁琐。在您借用辅助库的 CADL 课程中，讲师选择使用 VGG16 作为教学网络。如果您使用它并对您的代码进行一些小的修改，您应该会得到一些可以工作的东西（如果您在这里交换 Inception 网络，它会有点工作，但结果看起来会更令人失望） ：

import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import vgg16 as vgg

# Note reduced range of image, your noise function was drowning
# out the few textures that you were getting
img = np.random.rand(1,500,500,3) * 0.1 + 0.45
net = vgg.get_vgg_model()
tf.import_graph_def(net['graph_def'], name='vgg')
graph = tf.get_default_graph()
sess = tf.Session()
layer = graph.get_tensor_by_name('vgg/pool4:0')
gradient = tf.gradients(tf.reduce_mean(layer),
   graph.get_tensor_by_name('vgg/images:0'))

# You don't need to define or use the softmax layer - TensorFlow
# is smart enough to resolve the computation graph for gradients 
# without explicitly running the whole network forward first
iters = 100
# You don't need to init the network variables, everything you need 
# is set by the import, plus the placeholder.

for i in range(iters):
    grad = sess.run(gradient[0], {'vgg/images:0': img})

    # You can use all sorts of normalisation, this one is from CADL
    grad /= (np.max(np.abs(grad))+1e-7)

    # You forgot to use += here, and it is best to use a 
    # step size even after gradient normalisation
    img += 0.25 * grad
    # Re-normalise the image, to prevent over-saturation
    img = 0.98 * (img - 0.5) + 0.5
    img = np.clip(img, 0.0, 1.0)
    plt.imshow(img[0])
    plt.savefig('output/'+str(i+1)+'.png')
    plt.close('all')
    print(i)

完成所有这些操作后，图像显然可以正常工作，但仍需要一些改进：

为了获得更好的效果，您可能在网上看到的那种全彩图像需要进行更多更改。例如，您可以在每次迭代之间重新规范化或稍微模糊图像。

如果你想变得更复杂，你可以试试the TensorFlow Jupyter notebook walk-through，尽管由于结合了多种想法，从第一原则理解起来有点困难。