数据清理：从结果中消除 csv 文件中包含的值

Question

我希望我的最终数据不包含我想要清理的初始测试数据的元素。在代码中复制和粘贴数据的过程非常繁琐，并且随着添加的条件越来越多变得复杂。

原始值：

(1, 2, 3), (1, 2, 4),  (1, 2, 5), (1, 3, 4), (1, 3, 5)
(1, 4, 5), (2, 3, 4),  (2, 3, 5), (2, 4, 5), (3, 4, 5)

我想要一个排除 Test.csv

中包含的组合的组合

(1,2,3),   (2,3,4),     (3,4,5),

预期值

(1, 2, 4),
(1, 2, 5),
(1, 3, 4),
(1, 3, 5),
(1, 4, 5),
(2, 3, 5),
(2, 4, 5)

代码尝试 1

a = [1,2,3,4,5]

import csv

with open('Test.csv', newline='') as myFile:  
    reader = csv.reader(myFile)
    list_a = list(reader)

combo_a = [(p,q,r) for p in a for q in a for r in a
                 if q > p and r > q and r > p
                 and (p,q,r) not in list_a]

print (combo_a)

代码尝试 2

 a = [1,2,3,4,5]

import csv

with open('Test.csv', newline='') as myFile:  
    reader = csv.reader(myFile)
    list_a = list(map(tuple, reader))

combo_a = [(p,q,r) for p in a for q in a for r in a
                 if q > p and r > q and r > p
                 and (p,q,r) not in list_a]

print (combo_a)

两个代码都输出不正确的结果

(1, 2, 3),
(1, 2, 4),
(1, 2, 5),
(1, 3, 4),
(1, 3, 5),
(1, 4, 5),
(2, 3, 4),
(2, 3, 5),
(2, 4, 5),
(3, 4, 5),

Answer 1

带有file.csv的内容：

(1,2,3),   (2,3,4),     (3,4,5),

并使用csv 和ast.literal_eval：

a = [1,2,3,4,5]

import csv
from ast import literal_eval
from itertools import combinations

excluded = set()
with open('file.csv', newline='') as myFile:
    reader = csv.reader(myFile, delimiter=' ')
    for row in reader:
        l = list(map(literal_eval, [val for val in row if val]))
        excluded.update(tuple(i[0]) for i in l)

print(',\n'.join(map(str, sorted(set(combinations(a, 3)) - excluded))))

打印：

(1, 2, 4),
(1, 2, 5),
(1, 3, 4),
(1, 3, 5),
(1, 4, 5),
(2, 3, 5),
(2, 4, 5)

Answer 2

看起来您的 list_a 是一个字符串元组，而不是整数。所以如果你的

list_a = [('1', '2', '3'), ('2', '3', '4'), ('3', '4', '5')]

然后使用

将其转换为整数

list_a = [tuple(map(int, i)) for i in list_a]

一旦它是整数元组列表的形式，那么您就可以继续您的combo_a 操作。

Answer 3

所以您正在尝试过滤特定值？

我要做的是保留一个包含您不想要的值的列表。之后只需检查您要过滤的元组是否在列表中。

因此，将 test.csv 中的所有值加载到您的列表中。

dont_want = [some set of tuples you dont want]


combo_a = [(p,q,r) for p in a for q in a for r in a
                 if (p,q,r) not in dont_want]

如果我误解了你的问题，请原谅我，但我想我知道你在问什么。

Answer 4

部分问题是您实际上并没有处理int，而是处理字符串列表，因为元组与条目本身是用逗号分隔的：

from io import StringIO
import csv

c = """(1,2,3),   (2,3,4),     (3,4,5),"""

fh = StringIO(c, newline='')

reader = csv.reader(fh)
next(reader)

# ['(1', '2', '3)', '   (2', '3', '4)', '     (3', '4', '5)', '']

这不是一个元组列表，所以要把它变成一个：

import ast
from io import StringIO # this simulates your file handle

fh = StringIO(c, newline='')

# it's only one line, so call next(fh)
lst = ast.literal_eval(f"[{next(fh)}]")

# [(1, 2, 3), (2, 3, 4), (3, 4, 5)]

ast 会将它们处理成它们的原生数据结构。翻译成你的代码：

import ast

with open('Test.csv', newline='') as fh:
    list_a = ast.literal_eval(f"[{next(fh)}]")

现在list_a 是一个整数元组列表。然后你可以排除列表中的那些：

from itertools import combinations

checked = set()

for c in combinations(list(range(1,6)), 3):
    a = tuple(sorted(c))
    if a not in list_a and a not in checked:
        print(a)
        checked.add(a)