合并集合列表

Question

给定一个集合列表（字符串集合，例如setlist = [{'this','is'},{'is','a'},{'test'}]），想法是加入共享字符串的成对联合集合。下面的 sn-p 采用了测试成对重叠、连接和使用内部循环中断重新开始的字面方法。

我知道这是行人的方法，对于可用大小的列表（200K 组，2 到 10 个字符串）确实需要很长时间。

关于如何提高效率的任何建议？谢谢。

j    = 0
while True:
    if j == len(setlist): # both for loops are done
        break # while
    for i in range(0,len(setlist)-1):
        for j in range(i+1,len(setlist)):
            a = setlist[i];
            b = setlist[j];
            if not set(a).isdisjoint(b):     # ... then join them
                newset = set.union( a , b )  # ... new set
                del setlist[j]            # ... drop highest index
                del setlist[i]            # ... drop lowest index
                setlist.insert(0,newset)  # ... introduce consolidated set, which messes up i,j
                break                        # ... back to the top for fresh i,j
        else:
            continue
        break

Answer 1

正如@user2357112 在 cmets 中提到的，这可以被认为是一个图形问题。每个集合都是一个顶点，两个集合之间共享的每个单词都是一条边。然后，您可以迭代顶点并对每个看不见的顶点执行 BFS（或 DFS）以生成 connected component。

其他选项是使用Union-Find。 union find 的优点是不需要构建图，并且当所有集合都具有相同的内容时，不会出现退化的情况。下面是一个实际的例子：

from collections import defaultdict

# Return ancestor of given node
def ancestor(parent, node):
    if parent[node] != node:
        # Do path compression
        parent[node] = ancestor(parent, parent[node])

    return parent[node]

def merge(parent, rank, x, y):
    # Merge sets that x & y belong to
    x = ancestor(parent, x)
    y = ancestor(parent, y)

    if x == y:
        return

    # Union by rank, merge smaller set to larger one
    if rank[y] > rank[x]:
        x, y = y, x

    parent[y] = x
    rank[x] += rank[y]

def merge_union(setlist):
    # For every word in sets list what sets contain it
    words = defaultdict(list)

    for i, s in enumerate(setlist):
        for w in s:
            words[w].append(i)

    # Merge sets that share the word
    parent = list(range(len(setlist)))
    rank = [1] * len(setlist)
    for sets in words.values():
        it = iter(sets)
        merge_to = next(it)
        for x in it:
            merge(parent, rank, merge_to, x)

    # Construct result by union the sets within a component
    result = defaultdict(set)
    for merge_from, merge_to in enumerate(parent):
        result[merge_to] |= setlist[merge_from]

    return list(result.values())

setlist = [
    {'this', 'is'},
    {'is', 'a'},
    {'test'},
    {'foo'},
    {'foobar', 'foo'},
    {'foobar', 'bar'},
    {'alone'}
]

print(merge_union(setlist))

输出：

[{'this', 'is', 'a'}, {'test'}, {'bar', 'foobar', 'foo'}, {'alone'}]