为什么我的 GPU Tensorflow 在尝试运行额外时间时会崩溃？

Question

我是 Tensorflow 的新手，我正在使用 Spyder 在 Anaconda 中安装 GPU。我的程序中反复出现一个问题，在尝试多次运行文件后，它会崩溃并显示“您的 CPU 支持未编译此 TensorFlow 二进制文件以使用的指令：AVX AVX2”。

关闭原始控制台并打开一个新控制台有时可以让我再次运行文件而不会崩溃，但有时不会。再次关闭和打开整个程序也是如此。

通常，在我尝试运行任何 tensorflow 操作之前，程序都可以正常运行，例如，

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    sess.run(predict_op,...)

在此之后程序停止，直到出现此错误：

2019???????? 20:45:10.359441: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2019???????? 20:45:10.612655: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 0 with properties: 
name: GeForce GTX 970 major: 5 minor: 2 memoryClockRate(GHz): 1.253
pciBusID: 0000:01:00.0
totalMemory: 4.00GiB freeMemory: 3.22GiB
2019???????? 20:45:10.684151: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 1 with properties: 
name: GeForce GTX 970 major: 5 minor: 2 memoryClockRate(GHz): 1.253
pciBusID: 0000:02:00.0
totalMemory: 4.00GiB freeMemory: 3.22GiB
2019???????? 20:45:10.687572: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0, 1
2019???????? 20:45:11.509725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:
2019???????? 20:45:11.510626: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0 1 
2019???????? 20:45:11.511179: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0: N Y 
2019???????? 20:45:11.511725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 1: Y N 
2019???????? 20:45:11.512401: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2928 MB memory) ‑> physical GPU (device: 0, name: GeForce GTX 970, pci bus id: 0000:01:00.0, compute capability: 5.2)
2019???????? 20:45:11.901678: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 2928 MB memory) ‑> physical GPU (device: 1, name: GeForce GTX 970, pci bus id: 0000:02:00.0, compute capability: 5.2)
2019???????? 20:45:12.543674: E tensorflow/stream_executor/cuda/cuda_blas.cc:459] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
2019???????? 20:45:13.105336: E tensorflow/stream_executor/cuda/cuda_dnn.cc:352] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR

Answer 1

我可能会崩溃，因为 Tensorflow 在分配 VRAM 时是贪婪的。请看答案here