超出软进程大小限制错误。怎么修？

Question

我的 GAE 应用程序出现以下错误：

2011-06-25 00:15:59.023 /发布 500 30878ms 1796cpu_ms 0kb Picasa/117.430000 (gzip),gzip(gfe)

2011-06-25 00:15:59.010 超软 进程大小限制为 197.977 MB 总共服务了 16 个请求后

代码如下：

def post(self):
    '''Here we receive photos from the GetPostDetailsHandler (/).
    We upload them to vkontakte server one-by-one (vkontakte accepts
    5 files at once, but urlfetch allows only 1 Mb per request), 
    and then save details of photos uploaded in memcache.
    Cookies are also saved there.
    '''
    # list of files for upload (max 5 files can be in the list)
    files = []
    # counts number of uploads to vkontakte server
    posts_sent = 0
    # calculate total number of files received
    # and store field names with files in files_arguments list
    arguments = self.request.arguments()
    files_arguments = []
    for argument in arguments:
        if 'localhost' in argument:
        # if not (argument in ['album', 'albums_list', 'album_custom', 'upload_url', 'album_id', 'user_id', 'body', 'title']): 
            files_arguments.append(argument) 
    logging.info('(POST) ... number of photos received: '+str(len(files_arguments)))

    logging.info('(POST) ... upload process started')
    files_counter = 0 # counts total number of files sent
    for argument in files_arguments:
        files_counter +=1
        file_size = len(self.request.get(argument))/(1024.0*1024.0)
        logging.info('(POST) ... size of file '+str(files_counter)+' is '+str(file_size)+' Mb')
        if file_size <= 1:
            files.append(MultipartParam('file1', self.request.get(argument), 'file'+str(files_counter)+'.jpg', 'application/x-www-form-urlencoded'))
            # sending file
            data, headers = multipart_encode(files)
            # try 3 times to send the file
            for i in range(3):
                try:
                    result = urlfetch.fetch(url=self.request.get('upload_url'), 
                                            payload=''.join(data),
                                            method=urlfetch.POST,
                                            headers=headers,
                                            deadline=10
                                            )              
                    break
                except DownloadError:
                    logging.error('(POST) ... error during file upload, attempt ' + str(i))
                    pass
            if result.status_code == 200:
                # save result in the memcache for 10 minutes
                memcache.add(key=self.request.get('user_id')+'_'+self.request.get('album_id')+'_'+str(files_counter), value=result.content, time=600)
                # save description in the memcache
                memcache.add(key=self.request.get('user_id')+'_'+self.request.get('album_id')+'_'+str(files_counter)+'_desc', value=self.request.get('desc'+str(files_counter)), time=600)
                logging.info('(POST) ... result of photos upload ('+str(files_counter)+'): '+result.content)
                files = []      
    # save cookies
    cookies = self.request.headers.get('Cookie')
    logging.info(cookies)
    memcache.add(key=self.request.get('user_id')+'_'+self.request.get('album_id')+'_'+'cookies', value=cookies, time=600)
    logging.info('(POST) ... upload process finished')
    # return url for Picasa - SavePhotosHandler (/save) - it will be opened in user's default browser
    # so, we have to pass there key of data we saved in memcache
    self.response.out.write('http://picasa2vkontakte.appspot.com/save?'+self.request.get('user_id')+'_'+self.request.get('album_id'))

这里有什么问题？我还发现这样的错误通常意味着应用程序中存在内存泄漏。怎么找到？

Answer 1

这可能不是内存泄漏 - 您只是在使用所有可用内存来处理上传的文件。最好的选择是使用Blobstore 服务来避免自己处理文件。或者，仔细检查您的代码是否存在复制上传文件内容的任何情况（任何操作文件、转换为/从字符串转换等），并尝试将其最小化。

或者，由于您似乎只是立即将文件上传到另一个服务，因此请让您的用户直接在那里上传。

Answer 2

所以问题出现在你的脚本执行之前。所有上传的文件都存储在实例的内存中，并且实例在请求之间共享她的内存。 如果很多用户上传的图片太大，您的实例可能会被“超出内存限制”异常杀死。

服务器端你无法纠正问题，但你可以使用像flash这样的插件在上传之前检查文件的大小。

或者另一种方式是使用服务blobstoreService