如何清理列移位的 CSV 文件？

Question

我有一个正在尝试清理的电影 CSV 文件。我正在使用 Jupyter 笔记本。

它有 10,000 行和 5 列。以下是一些示例数据：

Movie Name      | Genre  | Date Released  | Length        | Rating      |
The Godfather   | Crime  | March 24, 1972 | 175           | R           |
The Avengers    | Action | May 5, 2012    | 143           | PG-13       |
The Dark Knight | Action | Crime          | July 18, 2008 | 152         | PG-13

请注意，对于“黑暗骑士”，由于有 2 个类型，因此行会向右移动。我想清理数据，使该行变为：

The Dark Knight | Action, Crime | July 18, 2008 | 152 | PG-13

我所做的是（在 Jupyter 笔记本中）

import pandas as pd
path = 'movies.csv'
df = pd.read_csv(path, header=0, names=['Movie Name', 'Genre', 'Date Released','Length','Rating','Extra'])

ctrCheck = 0
months = ["January","February","March","April","May","June","July","August","September","October","November","December"]

while ctrCheck < len(df.index):
    check = str(df['Date Released'][ctrCheck])
    if any(month in check for month in months):
        replaceStr = df.loc[ctrCheck, 'Genre'] + "," + df.loc[ctrCheck, 'Date Released']
        df.loc[ctrCheck, 'Genres'] = replaceStr
        df.loc[ctrCheck, 'Date Released'] = df.loc[ctrCheck, 'Length']
        df.loc[ctrCheck, 'Length'] = df.loc[ctrCheck, 'Rating']
        df.loc[ctrCheck, 'Rating'] = df.loc[ctrCheck, 'Extra']
    ctrCheck = ctrCheck + 1

df.drop(labels='Extra', inplace=True, axis='columns')

除了遍历 10,000 行之外，还有更好的方法吗？

谢谢！

Answer 1

如果我理解正确，您正在寻找一种不包含显式 for 循环的方法，而是使用矢量化 pandas 方法。

我们可以首先注意到需要转换的行是最后一列中值不是Nan的行

因此我可以建议以下代码：

import pandas as pd

# Name the last unnamed column
df = df.rename(columns={'Unnamed: 5': 'Extra'})

# Save the valid lines in a different dataframe
mask = (df['Extra'].isnull())
df_valid = df[mask]

# Fix the invalid lines
# Fix the Genre
df['Genre'] = df['Genre'] + ' ' + df['Date Released']
# Shift left the columns after 'Genre'
cols = df.columns[:-1]
df.drop('Date Released', axis=1, inplace=True)
df.columns = cols

# Restore valid lines
df.loc[mask, :] = df_valid

生成的数据框：

        Movie Name         Genre  Date Released Length Rating
0    The Godfather         Crime  March 24 1972    175      R
1     The Avengers        Action     May 5 2012    143  PG-13
2  The Dark Knight  Action Crime   July 18 2008    152  PG-13

注意此方法仅在每部电影的最大流派数为 2 时才有效，如果我理解正确的话就是这种情况:)