作者认为加噪声用处不大，且max-pooling功能特别强大，大到像作者说的那样有了max-pooling后什么约束就可以不用了，好像神器一样。
max-pooling的作用是什么？
max-pooling提供了非线性, 这是max-pooling效果更好的一个重要原因。
关于卷积，提供两个非常好的链接，表达了我心声：
反卷积
深度网络结构是由多个单层网络叠加而成的，而常见的单层网络按照编码解码情况可以分为下面3类：
- 既有encoder部分也有decoder部分：比如常见的RBM系列（由RBM可构成的DBM, DBN等），autoencoder系列(以及由其扩展的sparse autoencoder, denoise autoencoder, contractive autoencoder, saturating autoencoder等)。
- 只包含decoder部分：比如sparse coding, 和今天要讲的deconvolution network.
- 只包含encoder部分，那就是普通的feed-forward network.
卷积：假设A=B*C 表示的是：B和C的卷积是A，也就是说已知B和C，求A这一过程叫做卷积；
反卷积：如果已知A和B求C或者已知A和C求B，则这个过程就叫做反卷积；【由feature map卷积feature filter，得到input image】
上图表示的是DN(deconvolution network的简称)的第一层，其输入图像是3通道的RGB图，学到的第一层特征有12个，说明每个输入通道图像都学习到了4个特征。而其中的特征图Z是由对应通道图像和特征分别卷积后再求和得到的。
关于卷积网络中涉及的相关计算，列出一道题比较直接：
![]()
Reference
One of the early papers on Deep Q-Learning for Atari games (Mnih et al, 2013) contains this description of its Convolutional Neural Network: "The input to the neural network consists of an 84 × 84 × 4 image.
The first hidden layer convolves 16 8 × 8 filters with stride 4 with the input image and applies a rectifier nonlinearity.
The second hidden layer convolves 32 4 × 4 filters with stride 2, again followed by a rectifier nonlinearity.
The final hidden layer is fully-connected and consists of 256 rectifier units.
The output layer is a fully-connected linear layer with a single output for each valid action.
The number of valid actions varied between 4 and 18 on the games we considered." For each layer in this network, compute the number of weights per neuron in this layer (including bias)？ neurons in this layer？ connections into the neurons in this layer？ independent parameters in this layer？ You should assume there are 18 valid actions (outputs). First Convolutional Layer: J = K = 84, L = 4, M = N = 8, P = 0, s = 4 weights per neuron: 1 + M × N × L = 1 + 8 × 8 × 4 = 257 width and height of layer: 1+(J-M)/s = 1+(84-8)/4 = 20 neurons in layer: 20 × 20 × 16 = 6400 connections: 20 × 20 × 16 × 257 = 1644800 independent parameters: 16 × 257 = 4112
Second Convolutional Layer: J = K = 20, L = 16, M = N = 4, P = 0, s = 2 weights per neuron: 1 + M × N × L = 1 + 4 × 4 × 16 = 257 width and height of layer: 1+(J-M)/s = 1+(20-4)/2 = 9 neurons in layer: 9 × 9 × 32 = 2592 connections: 9 × 9 × 32 × 257 = 666144 independent parameters: 32 × 257 = 8224 Fully Connected Layer: weights per neuron: 1 + 2592 = 2593 neurons in layer: 256 connections: 256 × 2593 = 663808 independent parameters: 663808 Output Layer: weights per neuron: 1 + 256 = 257 neurons in layer: 18 connections: 18 × 257 = 4626 independent parameters: 4626
链接：https://www.zhihu.com/question/43609045/answer/132235276 [1] Zeiler M D, Krishnan D, Taylor G W, etal. Deconvolutional networks[C]. Computer Vision and Pattern Recognition, 2010. [2] Zeiler M D, Taylor G W, Fergus R, etal. Adaptive deconvolutional networks for mid and high level featurelearning[C]. International Conference on Computer Vision, 2011. [3] Zeiler M D, Fergus R. Visualizing andUnderstanding Convolutional Networks[C]. European Conference on ComputerVision, 2013. [4] Long J, Shelhamer E, Darrell T, et al.Fully convolutional networks for semantic segmentation[C]. Computer Vision andPattern Recognition, 2015. [5] Unsupervised Representation Learningwith Deep Convolutional Generative Adversarial Networks [6] Sparse Coding - Ufldl [7] Denoising Autoencoders (dA) [8] Convolution arithmetic tutorial
逆卷积【就是反卷积】的一个很有趣的应用是GAN(Generative Adversarial Network)里用来生成图片：Generative Models