LINQ：枚举列表中的重复项并删除它们

Question

我需要删除重复项，还要记录我要删除的内容。我现在有两种解决方案，一种可以遍历每个重复项，另一种可以删除重复项。我知道在 foreach 中就地删除是危险的，所以我有点坚持如何尽可能高效地做到这一点。

我现在得到了什么

var duplicates = ListOfThings
.GroupBy(x => x.ID)
.Where(g => g.Skip(1).Any())
.SelectMany(g => g);

foreach (var duplicate in duplicates)
{
    Log.Append(Logger.Type.Error, "Conflicts with another", "N/A", duplicate.ID);
}


ListOfThings = ListOfThings.GroupBy(x => x.ID).Select(y => y.First()).ToList();

Answer 1

好吧，ToList() 将实现查询，所以如果您允许副作用（即写入日志），它可能是这样的：

var cleared = ListOfThings
  .GroupBy(x => x.ID)
  .Select(chunk => {
     // Side effect: writing to log while selecting
     if (chunk.Skip(1).Any()) 
       Log.Append(Logger.Type.Error, "Conflicts with another", "N/A", chunk.Key);
     // if there're duplicates by Id take the 1st one
     return chunk.First();
   })
  .ToList();

Answer 2

当可以使用Aggregate 函数来确定报告和结果的重复项时，为什么要分组？

示例

var items = new List<string>() { "Alpha", "Alpha", "Beta", "Gamma", "Alpha"};

var duplicatesDictionary = 
     items.Aggregate (new Dictionary<string, int>(),  
                      (results, itm) => 
                                       {
                                         if (results.ContainsKey(itm))
                                            results[itm]++;
                                         else
                                           results.Add(itm, 1);

                                         return results;
                                  });

这是上面计算和报告每个插入的结果。

现在提取大于 1 的任何计数的重复报告。

duplicatesDictionary.Where (kvp => kvp.Value > 1)
         .Select (kvp => string.Format("{0} had {1} duplicates", kvp.Key, kvp.Value))

现在最终的结果就是提取所有的键。

 duplicatesDictionary.Select (kvp => kvp.Key);

Answer 3

您可以使用哈希集并将其与列表联合以获取唯一项；只需覆盖参考比较。实现IEqualityComparer<T>很灵活；如果只是 ID 使两个对象唯一，那么可以；但如果它更多，你也可以扩展它。

您可以使用 LINQ 获取重复项。

void Main()
{
    //your original class:
    List<Things> originalList = new List<Things> { new Things(5), new Things(3), new Things(5) };
    //i'm doing this in LINQPad; if you're using VS you may need to foreach the object
    Console.WriteLine(originalList);
    //put your duplicates back in a list and log them as you did.
    var duplicateItems = originalList.GroupBy(x => x.ID).Where(x => x.Count() > 1).ToList();//.Select(x => x.GetHashCode());
    Console.WriteLine(duplicateItems);
    //create a custom comparer to compare your list; if you care about more than ID then you can extend this
    var tec = new ThingsEqualityComparer();
    var listThings = new HashSet<Things>(tec);
    listThings.UnionWith(originalList);
    Console.WriteLine(listThings);
}

// Define other methods and classes here
public class Things 
{
    public int ID {get;set;}

    public Things(int id)
    {
        ID = id;
    }
}

public class ThingsEqualityComparer : IEqualityComparer<Things>
{
    public bool Equals(Things thing1, Things thing2)
    {
        if (thing1.ID == thing2.ID)
        {
            return true;
        }
        else
        {
            return false;
        }
    }

    public int GetHashCode(Things thing)
    {
        int hCode = thing.ID;
        return hCode.GetHashCode();
    }
}