Tweepy：抓取实时流媒体推文并保存到 .csv 文件答案

【问题标题】：Tweepy: crawl live streaming tweets and save in to a .csv fileTweepy：抓取实时流媒体推文并保存到 .csv 文件
【发布时间】：2019-09-10 02:25:25
【问题描述】：

阅读streaming with Tweepy 并浏览此example。我尝试编写一个 tweepy 应用程序来使用 tweepy Api 抓取实时流数据并将其保存到 .csv 文件。当我运行我的代码时，它返回空的 csv 文件（'OutputStreaming.csv'），其列名 ['Date'、'Text'、'Location'、'Number_Follower'、'User_Name'、'Friends_count'、'Hash_Tag]、不是流推文。我也尝试以this 的方式也以this one 进行操作，但我的代码得到了相同的输出：-

def on_status(self, status):
with open('OutputStreaming.csv', 'w') as f:
f.write(['Author,Date,Text')
writer = csv.writer(f)
writer.writerow([status.created_at.strftime("%Y-%m-%d \
                %H:%M:%S")status.text.encode,
                status.location,
                status.Number_of_follwers,
                status.author.screen_name,
                status.friends_count])

我卡住了。我不知道代码问题出在哪里，我的代码如下所示：-

import tweepy
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
import json #data
#Variables that contains the user credentials to access Twitter API 
access_token = "***"
access_token_secret = "***"
consumer_key = "***"
consumer_key_secret = "***"
auth = tweepy.OAuthHandler(consumer_key, consumer_key_secret)
auth.set_access_token(access_token, access_token_secret)
#setup api
api = tweepy.API(auth) 
class CustomStreamListener(tweepy.StreamListener):
    def on_data(self,data):
        if data:
           tweet_json = json.loads(data)
        if tweet_json:
           if not tweet_json['text'].strip().startswith('RT '):
              Created = data.created_at.strftime("%Y-%m-%d-%H:%M:%S")`
              Text = data.text.encode('utf8')
              Location = data.location('utf8')
              Follower = data.Number_of_follwers('utf8')
              Name = data.author.screen_name('utf8')
              Friend = data.friends_count('utf8')
              with open('OutputStreaming.csv', 'a') as f:
                   writer = csv.writer(f)
                   writer.writerow([Created, Text ,Loaction\ 
                   ,Follower ,Name ,Friend,status.entities.get('hashtags')])
                   Time.sleep(10)
           return True
    def on_error(self, status_code):
        if status_code == 420:
           return False
        else:
           print >> sys.stderr, 'Encountered error with status code:',\ 
           status_code
    def on_timeout(self):
        print >> sys.stderr, 'Timeout...'
      return True
# Writing csv titles
with open('OutputStreaming.csv', 'a') as f:
writer = csv.writer(f)
writer.writerow(['Date', 'Text', 'Location','Number_Follower', 
'User_Name', 'Friends_count','Hash_Tag'])
if __name__ == '__main__':
l = CustomStreamListener()
streamingAPI = tweepy.streaming.Stream(api.auth, l)
streamingAPI.filter(track=['#Yoga','#Meditation'])

【问题讨论】：

尝试with open('OutputStreaming.csv', 'a') 而不是with open('OutputStreaming.csv', 'w')。您的缩进完全错误，所以这是我最好的猜测，我不会全部阅读。请修正缩进。
@roganjosh 感谢您的建议，我修复了缩进也尝试使用 'a' 但仍然得到与以前相同的结果。请问有什么建议吗？
我建议使用 python linter。您的代码中有太多错误的东西，所有这些变量都没有定义：csv，Loaction，status，Time，sys。并且您打开了两次文件“OutputStreaming.csv”。你应该做一次。

标签： python csv tweepy

【解决方案1】：

这是一个工作代码：

#!/usr/bin/python3
# coding=utf-8

import tweepy

SEP = ';'
csv = open('OutputStreaming.csv','a')
csv.write('Date' + SEP + 'Text' + SEP + 'Location' + SEP + 'Number_Follower' + SEP + 'User_Name' + SEP + 'Friends_count\n')

class MyStreamListener(tweepy.StreamListener):

    def on_status(self, status):
        Created = status.created_at.strftime("%Y-%m-%d-%H:%M:%S")
        Text = status.text.replace('\n', ' ').replace('\r', '').replace(SEP, ' ')
        Location = ''
        if status.coordinates is not None:
            lon = status.coordinates['coordinates'][0]
            lat = status.coordinates['coordinates'][1]
            Location = lat + ',' + lon       
        Follower = str(status.user.followers_count)
        Name = status.user.screen_name
        Friend = str(status.user.friends_count)
        csv.write(Created + SEP + Text + SEP + Location + SEP + Follower + SEP + Name + SEP + Friend + '\n')

    def on_error(self, status_code):
        print(status_code)

consumer_key = '***'
consumer_secret = '***'
access_token = '***'
access_token_secret = '***'

# stream
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
myStream = tweepy.Stream(auth, MyStreamListener())
myStream.filter(track=['#Yoga','#Meditation'])

【讨论】：