列和行之间的Python交互

Question

我有以下数据框：

      topic  student level week
        1      a       1     1
        1      b       2     1
        1      a       3     1
        2      a       1     2
        2      b       2     2
        2      a       3     2
        2      b       4     2
        3      c       1     2
        3      b       2     2
        3      c       3     2
        3      a       4     2
        3      b       5     2

它包含一个列级别，用于指定谁发起了该主题以及谁回复了该主题。如果学生的等级为1，则表示他提出了这个问题。如果学生的等级为2，则表示他回答了提出问题的学生。如果学生的等级为3，则表示他回复了等级为2及以上的学生。

我想提取一个新的数据框，该数据框应该通过每周主题呈现学生之间的交流。它应该包含五列：“学生来源”、“学生目的地”、“周”、“总主题”和“回复数”。

我应该得到类似的东西：

    st_source st_dest  week  total_topics  reply_count
        a        b       1        1             1
        a        b       2        2             1
        a        c       2        1             0
        b        a       1        1             0
        b        a       2        2             0
        b        c       2        1             0
        c        a       2        1             0
        c        b       2        1             1

学生目标是每个学生与之分享主题的学生。

总主题是与其他学生共享的一些主题。我使用以下代码找到了它：

idx_cols = ['topic', 'week']
std_cols = ['student_x', 'student_y']
d1 = df.merge(df, on=idx_cols)
d2 = d1.loc[d1.student_x != d1.student_y, idx_cols + std_cols]

d2.loc[:, std_cols] = np.sort(d2.loc[:, std_cols])

d3 = d2.drop_duplicates().groupby(
    std_cols + ['week']).size().reset_index(name='count')
d3.columns = ['st_source', 'st_dest', 'week', 'total_topics']

我很难找到最后一列“回复数”。

回复计数是学生目的地“直接”回复学生来源的次数。如果一个话题是由学生 A 发起的（通过在级别 1 发送消息），B 回复 A（在级别 2 发送消息），因此 B 直接回复 A。考虑“直接”从 B 回复 A 当且仅当 B 在同一主题的第 k-1 级回复所有级别 k 到 A 的消息。只有 2 级到 1 级的学生回复。

有人有什么建议吗？

如果我应该更好地解释它，请告诉我。

谢谢！

Answer 1

完整答案，经过测试，抱歉之前的版本，有很多错别字......

import pandas as pd
from itertools import permutations

dataframe = {"topic": [1,1,1,2,2,2,2,3,3,3,3,3],
             "student": ["a","b","a","a","b","a","b","c","b","c","a","b"],
             "level": [1,2,3,1,2,3,4,1,2,3,4,5],
             "week": [1,1,1,2,2,2,2,2,2,2,2,2]
             }
dataframe =  pd.DataFrame.from_dict(dataframe)
dataframe = dataframe.reindex_axis(("topic", "student", "level", "week",), axis = 1)


results = {}  # the dictionary where results is going to be stored
source = False  # a simple boolean to make sure message 2 follows message 1
prev_topic = dataframe.get_value(0,'topic')  # boolean to detect topic change
topic_users = set()  # set containing the curent users of the topic
prev_week = None  # variable to check if week is constant in topic.

# print(dataframe)
for row in dataframe.get_values():  # iterate over the dataframe
    # print(prev_topic)

    if prev_topic == row[0]:  # if we are on the same topic
        # print("same_topic")
        # print(row)
        if row[2] == 1:  # if it is an initial message
            # print("first message")
            source = row[1]  # we store users as source
            topic_users.add(source)  # add the user to the topic's set of users
            week = row[3]  # we store the week

        elif row[2] == 2 and source:  # if this is a second message
            # print("scd")
            destination = row[1]  # store user as destination
            topic_users.add(destination)  # add the user to the topic's set of users
            if week != row[3]:  # if the week differs, we print a message
                print("ERROR: Topic " + str(row[0]) + " extends on several weeks")
                # break  # uncomment the line to exit the for loop if error is met

            key = "-".join((source, destination, str(week)))  # construct a key based on source/destination/week
            if key not in results:  # if the key is new to dictionary
                results[key] = [0, 0]  # create the new entry as a list containing topic_counts, reply_counts

            results[key][1] += 1  # add a counter to the reply_counts
            source = False  # reset destination

        else:
            # print("trololo")
            topic_users.add(row[1])  # add the user to the topic's set of users
            if week != row[3]:  # if the week differs, we print a message
                print("ERROR: Topic " + str(row[0]) + " extends on several weeks")
                # break  # uncomment the line to exit the for loop if error is met

            source = False  # reset destination

    else:  # if we enconter a new topic (and not the first one)
        # print('new topic')
        for pair in permutations(topic_users, 2):
            key = "-".join(pair) + "-" + str(week)  # construct a key based on source/destination/week
            if key not in results:   # if the key is new to dictionary
                results[key] = [1, 0]  # create the new entry as a list containing topic_counts, reply_counts
            else:  # otherwise
                results[key][0] += 1  # add a counter to the topic_counts

        topic_users = set()
        if row[2] == 1:  # if it is an initial message
            # print("first message")
            source = row[1]  # we store users as source
            topic_users.add(source)  # add the user to the topic's set of users
            week = row[3]  # we store the week

    prev_topic = row[0]

# redo the topic count feeding for the last topic (for wich we didn't detect a change of topic)
if len(topic_users) > 0:
    for pair in permutations(topic_users, 2):
        key = "-".join(pair) + "-" + str(week)  # construct a key based on source/destination/week
        if key not in results:   # if the key is new to dictionary
            results[key] = [1, 0]  # create the new entry as a list containing topic_counts, reply_counts
        else:  # otherwise
            results[key][0] += 1  # add a counter to the topic_counts

dico = {'source': [], 'destination': [], 'week': [], 'topic': [], 'reply': []}
for k, v in results.items():
    print(k, v)
    s, d, w = k.split('-')
    dico['source'].append(s)
    dico['destination'].append(d)
    dico['week'].append(w)
    dico['topic'].append(v[0])
    dico['reply'].append(v[1])

df = pd.DataFrame.from_dict(dico)
df = df.reindex_axis(("source", "destination", "week", "topic", "reply"), axis = 1)
print(df)

Answer 2

我的建议：

我会使用包含“source-destination-week”作为键和（total_topics，reply_counts）作为值的字典。

遍历第一个数据框，对于每个问题，将发布第一条消息的人存储为目标，将发布第二条消息的人存储为源，将周存储为周，在字典中的键“源-目的地-周”处添加一个计数器.我注意到您不再需要显示没有交互的学生对，因此我将其删除。 例如：

from itertools import permutations

results = {}  # the dictionary where results is going to be stored
source = False  # a simple boolean to make sure message 2 follows message 1
prev_topic = None  # boolean to detect topic change
topic_users = set()  # set containing the curent users of the topic
prev_week = None  # variable to check if week is constant in topic.

for row in dataframe:  # iterate over the dataframe

    if prev_topic = row[0]:  # if we are on the same topic

        if row[2] == 1:  # if it is an initial message
            source = row[1]  # we store users as source
            topic_users.add(source)  # add the user to the topic's set of users
            week = row[3]  # we store the week

        elif row[2] == 2 and source:  # if this is a second message
            destination = row[1]  # store user as destination
            topic_users.add(destination)  # add the user to the topic's set of users
            if week != row[3]:  # if the week differs, we print a message
                print "ERROR: Topic " + str(row[0]) + " extends on several weeks"
                # break  # uncomment the line to exit the for loop if error is met

            key = "-".join((source, destination, week))  # construct a key based on source/destination/week
            if key not in results:  # if the key is new to dictionary
                results[key] = [0, 0]  # create the new entry as a list containing topic_counts, reply_counts

            results[key][1] += 1  # add a counter to the reply_counts
            source = False  # reset destination

        else:
            topic_user.add(row[1])  # add the user to the topic's set of users
            if week != row[3]:  # if the week differs, we print a message
                print "ERROR: Topic " + str(row[0]) + " extends on several weeks"
                # break  # uncomment the line to exit the for loop if error is met

            source = False  # reset destination

    elif prev_topic != None:  # if we enconter a new topic (and not the first one)
        for pair in permutations(topic_users, 2):
            key = "-".join(pair) + "-" + week  # construct a key based on source/destination/week
            if key not in results:   # if the key is new to dictionary
                results[key] = [1, 0]  # create the new entry as a list containing topic_counts, reply_counts
            else:  # otherwise
                results[key][0] += 1  # add a counter to the topic_counts

        topic_users = set()

    prev_topic = row[0]

# redo the topic count feeding for the last topic (for wich we didn't detect a change of topic)
if len(topic_users) > 0: 
    for pair in permutations(topic_users, 2):
        key = "-".join(pair) + "-" + week  # construct a key based on source/destination/week
        if key not in results:   # if the key is new to dictionary
            results[key] = [1, 0]  # create the new entry as a list containing topic_counts, reply_counts
        else:  # otherwise
            results[key][0] += 1  # add a counter to the topic_counts

然后您可以将您的字典转换回数据框。 例如：

dico = {'b-a': [0,1], 'b-c' : [1,1], 'a-b': [2,1]}
df = pd.DataFrame.from_dict(dico, orient='index')
df.rename(index="str", columns={0:'topic', 1:'reply'})

我希望我没有在代码中打错字，还不能测试它......任何问题都可以随时咨询:)