将 Google Cloud Storage 与 GAE 远程 API 结合使用答案

【问题标题】：Using Google Cloud Storage with GAE remote API将 Google Cloud Storage 与 GAE 远程 API 结合使用
【发布时间】：2013-11-29 17:23:19
【问题描述】：

#!/usr/bin/python
# coding=utf8
import os, sys, hashlib, uuid, json, re

p = '/usr/local/google_appengine'
if p not in sys.path:
  sys.path.append(p)

import dev_appserver
dev_appserver.fix_sys_path()

GAE_USER = 'my email address'
GAE_PASS = '********'

from google.appengine.ext.remote_api import remote_api_stub
from google.appengine.ext import blobstore
import cloudstorage

def gae_login():
  return (GAE_USER, GAE_PASS)
remote_api_stub.ConfigureRemoteApi(None, '/_ah/remote_api', gae_login, 'dannytest.appspot.com')

write_retry_params = cloudstorage.RetryParams(backoff_factor=1.1)

filename = '/dannybucket/test/danny.txt'
gcs_file = cloudstorage.open(filename,'w', content_type='text/plain',options={'x-goog-meta-foo': 'foo','x-goog-meta-bar': 'bar'},retry_params=write_retry_params)
gcs_file.write('abcde\n')
gcs_file.write('f'*1024 + '\n')
gcs_file.close()

我明白了

WARNING:root:suspended generator urlfetch(context.py:1214) raised DownloadError(Unable to fetch URL: http://None/_ah/gcs/dannybucket/test/danny.txt)
WARNING:root:suspended generator do_request_async(rest_api.py:165) raised DownloadError(Unable to fetch URL: http://None/_ah/gcs/dannybucket/test/danny.txt)
WARNING:root:suspended generator do_request_async(storage_api.py:90) raised TimeoutError(('Request to Google Cloud Storage timed out.', DownloadError('Unable to fetch URL: http://None/_ah/gcs/dannybucket/test/danny.txt',)))
Traceback (most recent call last):
  File "./RMQ_Image", line 32, in <module>
    gcs_file = cloudstorage.open(filename,'w', content_type='text/plain',options={'x-goog-meta-foo': 'foo','x-goog-meta-bar': 'bar'},retry_params=write_retry_params)
  File "/usr/local/lib/python2.7/dist-packages/GoogleAppEngineCloudStorageClient-1.8.3.1-py2.7.egg/cloudstorage/cloudstorage_api.py", line 74, in open
    return storage_api.StreamingBuffer(api, filename, content_type, options)
  File "/usr/local/lib/python2.7/dist-packages/GoogleAppEngineCloudStorageClient-1.8.3.1-py2.7.egg/cloudstorage/storage_api.py", line 596, in __init__
    status, resp_headers, _ = self._api.post_object(path, headers=headers)
  File "/usr/local/lib/python2.7/dist-packages/GoogleAppEngineCloudStorageClient-1.8.3.1-py2.7.egg/cloudstorage/rest_api.py", line 41, in sync_wrapper
    return future.get_result()
  File "/usr/local/google_appengine/google/appengine/ext/ndb/tasklets.py", line 325, in get_result
    self.check_success()
  File "/usr/local/google_appengine/google/appengine/ext/ndb/tasklets.py", line 368, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/usr/local/lib/python2.7/dist-packages/GoogleAppEngineCloudStorageClient-1.8.3.1-py2.7.egg/cloudstorage/storage_api.py", line 93, in do_request_async
    'Request to Google Cloud Storage timed out.', e)
cloudstorage.errors.TimeoutError: ('Request to Google Cloud Storage timed out.', DownloadError('Unable to fetch URL: http://None/_ah/gcs/dannybucket/test/danny.txt',))

怎么了？

【问题讨论】：

标签： python google-app-engine python-2.7 google-cloud-storage

【解决方案1】：

http://None 不是有效的 URL。

remote_api_stub.ConfigureRemoteApi(None, '/_ah/remote_api', gae_login, 'dannytest.appspot.com')

需要实际指定应用程序的 APPID，因为您不是在本地运行脚本，而是在远程运行。因此，请将None 替换为您的应用程序 ID。

http://blog.onideas.ws/remote_api.gae

【讨论】：

【解决方案2】：

Google Cloud Storage 客户端不尊重 remote_api_stub 并认为您在本地运行脚本

os.environ['SERVER_SOFTWARE'] = 'Development (remote_api)/1.0'

甚至

os.environ['SERVER_SOFTWARE'] = ''

会有帮助。

函数，从common.py检查你的环境

def local_run():
  """Whether we should hit GCS dev appserver stub."""
  server_software = os.environ.get('SERVER_SOFTWARE')
  if server_software is None:
    return True
  if 'remote_api' in server_software:
    return False
  if server_software.startswith(('Development', 'testutil')):
    return True
  return False

【讨论】：