如何有效地比较 2 个列表并从 1 中删除重复项？ [复制]

Question

我需要比较 2 个列表并有效地从第二个数组中删除出现在第一个数组中的元素。我的解决方案如下：

自制算法：

#function accepts 2 arrays. Compares elements from 1st array to 2nd to see if they exist in 2nd, 
#and removes them from 2nd array. 
def compareRemove(self, array1 = [], array2 = []):

    #starting iteration values
    i = 0
    j = 0

    array1_length = len(array1)
    array2_length = len(array2)

    while i < array1_length:
        while j < array2_length:

            #compare element from first array to second array, remove element if same, 
            #and update second array length
            if array1[i] == array2[j]:

                del array2[j]
                array2_length = len(array2)

            #otherwise move j forward
            else:
                j = j + 1

        #when 2nd while loop is done move i forward and reset j to 0
        i = i + 1
        j = 0

    return array2

对 SO 的研究建议：

updated_file_strings = [x for x in array2 if not x in array1]

两者都同样快，但是随着我的阵列长度增加，两者都会变慢。即使数组长度增加，我是否可以使用任何可以很好地执行的方法？顺序无关紧要。

示例：

array1 = [1, 2, 3] array2 = [1, 2, 3, 5, 5, 5]

result_array = [5, 5, 5]

应不删除第二个数组中的重复项。

Answer 1

转换为set，删除元素，然后转换回list。

s1 = set(array1)
s2 = set(array2)
array2 = list(s2.difference(s1))

编辑：要跟踪重复项，您可以使用 collections.Counter 并重建列表。

from collections import Counter

s1 = set(array1)
array2 = [x for x in array2 if x not in s1]
# d2 = Counter(array2)
# array2 = [z for k, v in d2.items() if k not in s1 for z in [k] * v]

EDIT2：我认为使用Counter 会更快，但理解中的二级列表构造似乎抵消了任何收益。最好只创建第一个 set，然后使用它进行存在性检查。

测试：Counter 和双重理解

%%timeit
array1 = [random.randint(0, 10000) for _ in range(200000)]
array2 = [random.randint(0, 20000) for _ in range(200000)]
s1 = set(array1)
d2 = Counter(array2)
[z for k, v in d2.items() if k not in s1 for z in [k]*v]

# returns:
525 ms ± 19.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

测试：带有存在性检查的单一理解

%%timeit
array1 = [random.randint(0, 10000) for _ in range(200000)]
array2 = [random.randint(0, 20000) for _ in range(200000)]
s1 = set(array1)
#d2 = Counter(array2)
[x for x in array1 if x not in s1]

# returns:
510 ms ± 17.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)