【问题标题】:slow implementation of Union and IntersectionUnion 和 Intersection 执行缓慢
【发布时间】:2009-07-15 13:57:08
【问题描述】:

我正在尝试找到一种更好的方法来实现这些方法,因为对于非常大的集合,它们需要很长时间,有什么想法吗?

import java.util.HashMap;
import java.util.HashSet;

public class Multiset<E> extends HashSet<E> {

    private static final long serialVersionUID = -9013417064272046980L;
    private HashMap<E, Integer> multiplicities = new HashMap<E, Integer>();

    @Override
    public boolean add(E element){
        if(multiplicities.containsKey(element)){
            int x = (int) multiplicities.get(element);
            multiplicities.put(element, ++x);
        }else{
            multiplicities.put(element, 1);
        }
        return super.add(element);    
    }

/**
 * Adds all of the elements of another multiset to this one. 
 * This method allows the preservation of multiplicities
 * which would not occur using the superclass's addAll().
 * @param elements
 * @return true if all elements were successfully added
 */
public boolean addAll(Multiset<E> elements) {
    boolean flag = false;
    for(E element : elements){
        for(int i = 0; i < elements.multiplicity(element); i++)
            flag = add(element);
    }
    return flag;
}

/**
 * The set-view of a multiset is the ordinary set of all 
 * elements with multiplicity >= 1.
 * @return all elements that have multiplicity >= 1
 */
public Multiset<E> setView(){
    Multiset<E> set = new Multiset<E>();
    for(E o : multiplicities.keySet()){
        set.add(o);
    }
    return set;
}

/**
 * provides a union of two multisets whereby the multiplicity of each
 * element is the larger of the two
 * @param second
 * @return
 */
public Multiset<E> union(Multiset<E> second){
    Multiset<E> union = new Multiset<E>();
    Multiset<E> join = new Multiset<E>();
    join.addAll(this);
    join.addAll(second);

    for(E o : join){
        int i = this.multiplicity(o); 
        int j = second.multiplicity(o);
        i = i > j ? i : j;
        for(int c = 0; c < i; c++){
            union.add(o);
        }
    }

    return union;
}

/**
 * provides an intersection of two multisets whereby 
 * the multiplicity of each element is the smaller of the two
 * @param second
 * @return The multiset containing the intersection of two multisets
 */
public Multiset<E> intersect(Multiset<E> second){    

    Multiset<E> intersection = new Multiset<E>();
    for(E o : this.setView()){
        if (second.setView().contains(o)) {
            int i = this.multiplicity(o); 
            int j = second.multiplicity(o);
            i = i < j ? i : j;
            for(int c = 0; c < i; c++){
                intersection.add(o);
            }
        }
    }

    return intersection;        
}

/**
 * The Multiplicity is the number of occurrences of an object 
 * in the multiset
 * @param o
 * @return number of occurrences of o
 */
public int multiplicity(E o){

    return (multiplicities.containsKey(o)) ? multiplicities.get(o) : 0;
}

public int cardinality(){
    int card = 0;
    for(Integer i : multiplicities.values()){
        card += i;
    }

    return card;    
 }

/**
 * Measures the similarity between two multisets
 * @param A
 * @param B
 * @return the cardinality of the difference of A and B 
 */
public int similarityOfMultisets(Multiset<E> second){

    Multiset<E> union, intersection; 
    int difference;

    union = union(second);
    intersection = intersect(second);
    difference = union.cardinality() - intersection.cardinality();

    return difference;

}
}

编辑:

我相信我找到了一种更快的方法来计算similarityOfMultisets 方法:

public int similarityOfMultisets(Multiset<E> second){
    int c = 0;
    for(E elem: this.setView()){
        c += Math.min(this.multiplicity(elem), second.multiplicity(elem));
    }   
    Multiset<E> union = this.union(second);
    return union.cardinality() - c;     
}

【问题讨论】:

  • 什么是 this 类?您自己的 Multiset 实现?
  • yes - 这是 HashSet 的扩展,并在 HashTable 中存储多重性
  • 你看过 Google Collection 的 Multiset 实现吗?
  • 是的,但它没有我需要的功能,它是最终的
  • 在我看来仍然是使用 G-C-Multiset 组合的好人选。

标签: java optimization set abstract-data-type


【解决方案1】:

这是对类的重构。不一定更快 - 除了不在循环内重新运行 setView() - 但在某些方面可能更干净。 FWIW。

import java.util.HashMap;
import java.util.HashSet;

public class Multiset<E> extends HashSet<E> {
    private static final long           serialVersionUID    = -9013417064272046980L;
    private final HashMap<E, Integer>   multiplicities      = new HashMap<E, Integer>();

    public boolean add(E element) {
        return add(element, 1);
    }

    private boolean add(E element, int copies) {
        if (!contains(element))
            multiplicities.put(element, 0);
        int n = multiplicities.get(element);
        multiplicities.put(element, n + copies);
        return super.add(element);
    }

    /**
     * Adds all of the elements of another multiset to this one. This method allows the preservation of multiplicities which would not occur
     * using the superclass's addAll().
     * 
     * @param that
     * @return true if all elements were successfully added
     */
    public boolean addAll(Multiset<E> that) {
        boolean flag = false;
        for (E element : that)
            flag = add(element, that.multiplicity(element));
        return flag;
    }

    /**
     * The set-view of a multiset is the ordinary set of all elements with multiplicity >= 1.
     * 
     * @return all elements that have multiplicity >= 1
     */
    public Multiset<E> setView() {
        Multiset<E> set = new Multiset<E>();
        for (E o : multiplicities.keySet())
            set.add(o);
        return set;
    }

    /**
     * provides a union of two multisets whereby the multiplicity of each element is the larger of the two
     * 
     * @param that
     * @return
     */
    public Multiset<E> union(Multiset<E> that) {
        HashSet<E> both = new HashSet<E>();
        both.addAll(this);
        both.addAll(that);
        Multiset<E> union = new Multiset<E>();
        for (E element : both)
            union.add(element, Math.max(this.multiplicity(element), that.multiplicity(element)));
        return union;
    }

    /**
     * provides an intersection of two multisets whereby the multiplicity of each element is the smaller of the two
     * 
     * @param that
     * @return The multiset containing the intersection of two multisets
     */
    public Multiset<E> intersect(Multiset<E> that) {
        Multiset<E> intersection = new Multiset<E>();
        final Multiset<E> other = that.setView();
        for (E element : this.setView())
            if (other.contains(element))
                intersection.add(element, Math.min(this.multiplicity(element), that.multiplicity(element)));
        return intersection;
    }

    /**
     * The Multiplicity is the number of occurrences of an object in the multiset
     * 
     * @param element
     * @return number of occurrences of o
     */
    public int multiplicity(E element) {
        return contains(element) ? multiplicities.get(element) : 0;
    }

    public int cardinality() {
        int card = 0;
        for (Integer n : multiplicities.values())
            card += n;
        return card;
    }

    /**
     * Measures the similarity between two multisets
     * 
     * @param that
     * @return the cardinality of the difference of A and B
     */
    public int similarityOfMultisets(Multiset<E> that) {
        return union(that).cardinality() - intersect(that).cardinality();
    }
}

【讨论】:

  • 我们能否以某种方式对我们的解决方案进行性能测试?我们需要某个地方的测试数据。
【解决方案2】:

我们算法的第一个变量的性能测试结果:

罗伯特联盟:2263374 我们 罗伯特十字路口:603134 我们 罗伯特相似度:2926389 我们 卡尔联盟:3372 我们 Carl-Intersection: 5097 我们 卡尔相似度:6913 us 大卫联盟:5182 我们 David-Intersection: 2527 我们 大卫相似度:5270 us

卡尔的工会胜过我的工会。

测试代码here。不过我没有验证计算输出的正确性。

测试代码 2 用于各种集合大小和差异 here (JDK 7b59)。结果xslx/ods

【讨论】:

  • 无法解释我工会的缓慢性。也许是因为我使用了一个 beta G-C 库?我试图按照卡尔的方式改变我的工会,但它甚至变得更慢了。
  • +1 用于实际分析 - 这是对性能进行明智评价的唯一方法。
  • 我的新相似性方法比较如何?
  • 我没有对其进行测试,而是将您的更改应用于我链接的代码并运行它。如果它与卡尔的解决方案相同,我会期望相似的值。
  • 对不起,我不只是懒惰——我现在不在工作,也无法访问这台计算机上的 jdk
【解决方案3】:

我不明白 setView 方法的目的...似乎您只是返回自己的副本,但每个键的多重性设置为 1。

对于联合试试这个也许(可能无法编译):

public Multiset<E> union(Multiset<E> second) {
    Multiset<E> union = new Multiset<E>();
    union.addAll(this);
    union.addAll(second);

    for(E o : union){
        int multiplicity = Math.max (this.multiplicity(o), second.multiplicity(o));
        union.multiplicities.put (o, multiplicity);
    }

    return union;
}

【讨论】:

    【解决方案4】:

    我认为问题在于您正在为 this.setView() 中的每个元素调用 second.setView() - 重新创建该集合。试试这个:

    /**
     * provides an intersection of two multisets whereby 
     * the multiplicity of each element is the smaller of the two
     * @param second
     * @return The multiset containing the intersection of two multisets
     */
    public Multiset<E> intersect(Multiset<E> second){    
    
        Multiset<E> intersection = new Multiset<E>();
        Set<E> other = second.setView();
        for(E o : this.setView()){
            if (other.contains(o)) {
                int i = this.multiplicity(o); 
                int j = second.multiplicity(o);
                i = i < j ? i : j;
                for(int c = 0; c < i; c++){
                    intersection.add(o);
                }
            }
        }
    
        return intersection;        
    }
    

    【讨论】:

    • 优化器肯定会在编译时捕捉到它吗?
    • 当然不是。它无法知道第二次通过循环,第二个多重集将产生相同的集。您要求计算机重新生成集合,所以它会这样做。
    【解决方案5】:

    这就是我想出的 G-C:

    import com.google.common.collect.Multiset;
    import com.google.common.collect.Multisets;
    import com.google.common.collect.Multiset.Entry;
    public class MultisetOp {
        public static void main(String[] args) {
            Multiset<Integer> ms1 = Multisets.newHashMultiset(1, 1, 2, 3, 4, 4, 4);
            Multiset<Integer> ms2 = Multisets.newHashMultiset(1, 2, 3, 3,
                4, 5, 5, 5);
            Multiset<Integer> mu = Multisets.newHashMultiset();
            Multiset<Integer> mi = Multisets.newHashMultiset();
            // -------- UNION START -----------
            for (Entry<Integer> e : ms1.entrySet()) {
                int j = ms2.count(e.getElement());
                mu.add(e.getElement(), Math.max(e.getCount(), j));
            }
            for (Entry<Integer> e : ms2.entrySet()) {
                int j = ms1.count(e.getElement());
                if (j == 0) {
                    mu.add(e.getElement(), e.getCount());
                }
            }
            // -------- UNION END -----------
    
            // -------- INTERSECT START -----------
            for (Entry<Integer> e : ms1.entrySet()) {
                int j = ms2.count(e.getElement());
                if (j > 0) {
                    mi.add(e.getElement(), Math.min(e.getCount(), j));
                }
            }
            // -------- INTERSECT END -----------
    
            System.out.printf("Union: %s%n", mu);
            System.out.printf("Intersection: %s%n", mi);
            System.out.printf("Cardinality: %d%n", mu.size() - mi.size());
    
        }
    }
    

    结果:

    [1 x 2, 2, 3 x 2, 4 x 3, 5 x 3]
    [1, 2, 3, 4]

    未进行基准测试。 看来,您的基数可以通过两次遍历而不是三次遍历来计算。

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 2020-04-23
      • 2020-09-11
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2011-08-31
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多