我支持 Romain 的 cmets,您应该限制读取文件的次数,如果可以使用 set 操作,迭代是不好的。
在我看来,您可以在操作中设置行颜色,而无需下拉到 xlwings 来执行此操作。
我将在下面列出一些示例来解释不同的方法:
选项1- 迭代
import numpy as np
import pandas as pd
# Set up the requirements for the row to be coloured
# this will make more sense later
def color(row):
if row["check"] == "matched":
return ['background-color: red'] * len(row)
return [''] * len(row)
# Note this are raw strings to handle the Windows backslash path character
values_to_check = [r'C:\folder\somepath\1234_456_2.pdf', r'C:\folder\somepath\whatever\5932194_123.pdf']
df = pd.read_excel('data.xlsx', sheet_name='My Data')
# Add a blank column as a placeholder
df["check"] = ""
for i in range(len(df)):
# this tests if any of the entries in the file list match the current record
if any(df.loc[i, "value"] in x for x in values_to_check):
df.loc[i, "check"] = "matched"
else:
df.loc[i, "check"] = "not matched"
# now we can apply the colour option
# associate a styler object with the dataframe
styler = df.style
# apply the colour function to select and change the rows
styler.apply(color, axis=1)
# use ExcelWriter rather than using to_Excel directly in order to give access to the append & replace functions
with pd.ExcelWriter("data.xlsx", engine="openpyxl", mode="a", if_sheet_exists="replace") as writer:
styler.to_excel(writer, 'My Data', index=False)
这给出了一个带有附加列的输出,用于标记它是否匹配。
选项 2 - 集合操作(Pandas 合并)
import numpy as np
import pandas as pd
import pathlib
def color_joined(row):
if row["_merge"] == "both":
return ['background-color: red'] * len(row)
return [''] * len(row)
def clean_inputs(input_item:str) -> str:
# Using PureWindowsPath vs Path to handle the backslashes
# stem returns the filename only, no path or extension
# get rid of the underscores to apply int comparisons based on your comments
return int(pathlib.PureWindowsPath(input_item).stem.replace('_',''))
values_to_check = [r'C:\folder\somepath\1234_456_2.pdf', r'C:\folder\somepath\whatever\5932194_123.pdf']
# Let's have only the filenames, and without the underscores, as int
# you may need to fiddle with this a bit to match your real-world data
cleaned_filenames = [ clean_inputs(x) for x in values_to_check ]
# No need to invent a blank check column here
df = pd.read_excel('data.xlsx', sheet_name='My Data')
# Instead, convert the value list into a dataframe too
lookup_list = pd.DataFrame(cleaned_filenames, columns=['value'])
# this uses a left join and leaves a flag
joined_df = df.merge(lookup_list, on='value', how='left', indicator=True)
# the result is a df with all of the records, plus a column called "_merge"
# the values of this column will be either "left_only" for no match or "both" for a match
styler = joined_df.style
styler.apply(color_joined, axis=1)
# Drop the _merge column by writing out only the specified columns
with pd.ExcelWriter("output.xlsx", engine="openpyxl", mode="a", if_sheet_exists="replace") as writer:
styler.to_excel(writer, 'Merged', index=False, columns=['title', 'description', 'value', 'extra_column'])
这给出了与上面相同的输出。从理论上讲,它应该比简单地使用多个循环更优化,但一如既往,您应该测试特定数据的性能。
笔记:
如果您想要仅包含匹配项的列表,请使用选项 2,这可以由 s1 = pd.merge(df, lookup_list, how='inner', on=['value']) 完成。
理论上,您应该能够在写入 Excel 之前使用 styler.hide(subset=['_merge', 'check'], axis="columns") 删除列;但是,我无法在我的测试中使用它。有关详细信息,请参阅styler.hide documentation。
您可以通过指定列的数据类型(例如 int vs dtype)来节省内存(并加快处理速度),因为默认情况下是使用 dtype 对象。