在谷歌云存储中创建文件 - IOError：缓冲区已关闭

Question

我创建了一个 python 脚本来从 facebookads API 中提取数据，并使用谷歌应用引擎在谷歌云存储中创建一个文件。

将数据写入谷歌云存储时出现以下错误，但数据在网络浏览器上正常显示：

IOError：缓冲区已关闭。

经过一番研究，我了解到，当无法识别 lin 结尾 ("\n") 时会出现此错误，因此它将整个文件视为单行并引发“缓冲区已关闭”错误。

所以我添加了以下代码，现在在 Web 浏览器上正确显示行，但在写入 gcs 时仍然出错。

data1=data.replace("\n", "<br />")

代码：

class get_region_insights(webapp.RequestHandler):

    _apptitle      = None
    _projectid     = None
    _projectnumber = None


    def get(self):
        #bucket_name = os.environ.get('BUCKET_NAME', app_identity.get_default_gcs_bucket_name())
        cfg=appsettings()
        for i in cfg._templates:

            id=int(i['_id'])

            if id == 7:

                ### Read variables from config file 
                bucket_name = i['_gcsbucket']
                bucket = '/' + bucket_name
                filename = bucket + '/' + i['_filename'] + str(time.strftime("%d_%m_%Y")) + ".csv"
                ad_acct=i['_ad_acct']
                app_id = i['_app_id']
                app_secret = i['_app_secret']
                access_token = i['_access_token']
                needed_keys=ast.literal_eval(i['_needed_keys'])
                self.tmp_filenames_to_clean_up = []

                u = date.today()
                sub_days = 1
                s = u - timedelta(sub_days)
                until = str(u)
                since = str(s)

                params = {
                    'fields': [
                        FBAdsInsights.Field.account_id,
                        FBAdsInsights.Field.campaign_id,
                        FBAdsInsights.Field.campaign_name,
                        FBAdsInsights.Field.adset_id,
                        FBAdsInsights.Field.adset_name,
                        FBAdsInsights.Field.impressions,
                        FBAdsInsights.Field.spend,
                        FBAdsInsights.Field.actions,
                        FBAdsInsights.Field.action_values,
                    ],
                    'filtering': [
                        {
                            'field': 'action_type',
                            'operator': 'IN',
                            'value': ["link_click","comment", "post_reaction", "post", "offsite_conversion.fb_pixel_purchase"] #custom rule filter
                            },        
                        ],    
                    'level': 'adset',
                    'time_range': {
                         'since': since, #user input field
                         'until': until #specify dynamic date range between (today() - (days_entered)) and today()
                        },
                    'time_increment': 1,
                    'breakdowns': ['region'],
                    'action_breakdowns': ['action_type'],
                }

                ### Initializing Google cloud Storage Object 
                write_retry_params = _gcs.RetryParams(backoff_factor=1.1)
                gcs_file=_gcs.open(filename, 'w', content_type='text/plain',retry_params=write_retry_params)


                ### Facebook Initialization 
                session=FacebookSession(app_id,app_secret,access_token)
                api=FacebookAdsApi(session)
                FacebookAdsApi.set_default_api(api)
                ad_account = FBAdAccount(ad_acct)
                stats = ad_account.get_insights(params=params,async=True)
                stats.remote_read()
                while stats[AdReportRun.Field.async_percent_completion] < 100:
                    time.sleep(1)
                    stats.remote_read()
                    time.sleep(1)
                stats1 = stats.get_result()
                x = [x for x in stats1]

                ### Printing the result and writing to Google Cloud Storage

                for i in x:
                    for k in i['actions']:
                        if k['action_type'] == "offsite_conversion.fb_pixel_purchase":
                            Purchase_Facebook_Pixel = k['value']
                        if k['action_type'] == "comment":
                            post_comment= k['value']
                        if k['action_type'] == "link_click":
                            link_click=k['value']
                        if k['action_type'] == "post":
                            post_share=k['value']
                        if k['action_type'] == "post_reaction":
                            post_reaction=k['value']
                    for m in i['action_values']:
                        if m['action_type'] == "offsite_conversion.fb_pixel_purchase" :
                            Purchase_Conversion_Value_Facebook_Pixel=m['value']
                    data=(i['account_id'] + "," + i['adset_id'] + "," + i['campaign_id'] + "," + i['date_start'] + "," + i['date_stop'] + ","+ i['impressions']+ "," + i['region'] + ","+ i['spend']+ "," + link_click + "," + Purchase_Facebook_Pixel + "," + Purchase_Conversion_Value_Facebook_Pixel+"\n")
                    data1=data.replace("\n", "<br />")
                    self.response.write(data.replace("\n", "<br />"))
                    #self.response.write("\n"+i['account_id'] + "," + i['adset_id'] + "," + i['adset_name'] + "," + i['age'] + "," + i['campaign_id'] + "," +i['campaign_name'] + "," + i['date_start'] + "," + i['date_stop'] + ","+i['gender'] + ","+ i['impressions']+","+ i['spend']+ "," + link_click + "," + post_comment + "," + post_share + "," + post_reaction + "," + Purchase_Facebook_Pixel + "," + Purchase_Conversion_Value_Facebook_Pixel+"\n")
                    gcs_file.write(data1.encode('utf-8'))
                    gcs_file.close()
                    self.tmp_filenames_to_clean_up.append(filename)

Answer 1

您在循环外打开云存储文件，但随后在循环内关闭。

            ### Initializing Google cloud Storage Object 
            write_retry_params = _gcs.RetryParams(backoff_factor=1.1)
            gcs_file=_gcs.open(filename, 'w', content_type='text/plain',retry_params=write_retry_params)


            ### Facebook Initialization 
            ...

            ### Printing the result and writing to Google Cloud Storage

            for i in x:
                # do stuff with data
                ...
                gcs_file.write(data1.encode('utf-8'))
                gcs_file.close()   # <-- closing the file buffer
                self.tmp_filenames_to_clean_up.append(filename) 

如果您想为每次循环迭代编写一个文件，请在循环内打开和关闭文件。

如果要将所有数据写入单个文件，请在循环外打开和关闭文件。