如何通过 EF LINQ 展平记录获取但如果存在重复则仅按字段优先级获取一条记录

Question

我准备了一个示例数据模型来说明我的问题，所以不要看模型的含义。

    public class Job
    {
        public int Id { get; set; }
        public ICollection<Task> Tasks { get; set; }
    }

    public class Task
    {
        public int Id { get; set; }
        public Job Job { get; set; }
        public ICollection<Record> Records { get; set; }
    }

    public class Record
    {
        public int Id { get; set; }
        public int RecordID { get; set; }
        public Task Task { get; set; }
        public int? Priority { get; set; }
        //More fields, which are be different for every RecordID
    }

---

Input:JobId

Output: 分配给tasks 的所有records 分配给具有给定ID 的Job。容易吧？但是，对我来说问题很复杂，因为结果输出集合只能包含具有唯一 RecordId 的记录。如果同一个RecordId有重复，则必须选择优先级值最高的Record（如果优先级值为null，则其值最低，如果null记录较多，则顺序可能随机） .

我非常关心应用服务器的性能和 RAM，因此我希望在数据库端（SQL Server）上执行查询，而不是在应用代码中出现奇怪的循环。就我而言，我要查询的records 甚至会达到几百万。

例子：

Input: JobId = 5

Tasks Table:

Id	JobId
1	5
2	5
3	5

Records Table:

Id	TaskId	RecordId	Priority	more fields...
1	1	101	null	AAA
2	2	101	null	BBB
3	1	102	9	CCC
4	2	102	10	DDD
5	2	105	2	EEE
6	3	106	3	FFF

结果：

Id	TaskId	RecordId (unique in results)	Priority	more fields...
1	1	101	null	AAA
4	2	102	10	DDD
5	2	105	2	EEE
6	3	106	3	FFF

---

我开始编写查询，但总是以 C# 中的怪异循环和一个额外的集合来复制结果数据而告终，而在应用程序中，包含一百万条记录的额外集合似乎有点过头了。

var job = await _dbContext.Job.SingleAsync(x=>x.Id = jobId);

return await job.Tasks.SelectMany(s => s.Records). ...//what next?
      or
return = await _dbContext.Records.Where(r => r.Task.Job.Id jobId). ...//what next?

Answer 1

在 LINQ 中有一个简单的模式用于从集合导航属性中选择第一项，如下所示：

var q = from j in db.Jobs
        from t in j.Tasks
        where j.Id == jobId
        select t.Records.OrderByDescending(r => r.Priority).ThenBy(r => r.Id).First();

翻译成：

  SELECT [t1].[Id], [t1].[Priority], [t1].[RecordID], [t1].[TaskId]
  FROM [Jobs] AS [j]
  INNER JOIN [Task] AS [t] ON [j].[Id] = [t].[JobId]
  LEFT JOIN (
      SELECT [t0].[Id], [t0].[Priority], [t0].[RecordID], [t0].[TaskId]
      FROM (
          SELECT [r].[Id], [r].[Priority], [r].[RecordID], [r].[TaskId], ROW_NUMBER() OVER(PARTITION BY [r].[TaskId] ORDER BY [r].[Priority] DESC, [r].[Id]) AS [row]
          FROM [Record] AS [r]
      ) AS [t0]
      WHERE [t0].[row] <= 1
  ) AS [t1] ON [t].[Id] = [t1].[TaskId]
  WHERE [j].[Id] = @__jobId_0

如果您需要为每个任务按优先级选择第一个 RecordId，我无法提供可翻译的 LINQ 表达式。因此，您可以使用商店查询，或在服务器上排序，并在客户端进行过滤。像这样：

var q = from j in db.Jobs
        where j.Id == jobId
        from t in j.Tasks
        from r in t.Records
        orderby t.Id, r.Priority descending, r.RecordID, r.Id
        select r;

var results = new List<Record>();
var lastRecordId = -1;

foreach (var r in q)
{
    if (r.RecordID != lastRecordId)
    {
        results.Add(r);
    }
    lastRecordId = r.RecordID;
}

Answer 2

所以每个Job 有零个或多个Tasks，每个任务有零个或多个Records。使用外键的简单的一对多关系。

您想要属于具有给定 JobId 的 Job 的所有 Record。好吧，不是所有的记录，从 RecordId 重复值的记录中，您想要优先级值最高的记录（null 是最低优先级）。

如果您遵循Entity Framework Conventions，则表中的“一”侧将具有virtual ICollection<...>，而“多”侧将具有指向“一”侧的外键以及虚拟属性到它所属的对象。

类似这样的：

public class Job
{
    public int Id { get; set; }
    ...

    // Every Job has zero or more Tasks (one-to-many)
    public virtual ICollection<Task> Tasks { get; set; }
}

public class Task
{
    public int Id { get; set; }
    ...

    // Every Task is the Task of exactly one Job, using foreign key
    public int JobId {get; set;}
    public virtual Job Job {get; set;}

    // Every Task has zero or more Records (one-to-many)
    public virtual ICollection<Record> Records { get; set; }
}

public class Record
{
    public int Id { get; set; }
    ...

    // Every Record is a Record in exactly one Task, using foreign key:
    public int TaskId {get; set;}
    public virtual Task Task {get; set;}
}

这足以让实体框架检测您的表、表中的列以及表之间的关系。我不确定这对于 EF-core 是否也足够。

在实体框架中，表的列由非虚拟属性表示。虚拟属性表示表之间的关系（一对多、多对多）。

外键是表中的真实列，因此它是非虚拟的。

使用虚拟属性

int jobId = ...
IEnumerable<Record> recordsOfThisJob = dbContext.Records

    // Keep only Records of Jobs with this jobId:
    .Where(record => record.Task.Job.Id == jobId)
    
    // make groups of Records with same value for RecordId
    // from every Group of Records keep the one with the highes value for Priority
    .GroupBy(record => record.RecordId,

    // parameter resultSelector: take every recordId, and all Records with this value
    // for recordId to make one new:
    (recordId, recordsWithThisRecordId) => recordsWithThisRecordId
        .OrderByDescending(record => record.Priority ?? Int32.MinValue)
        // null has lowest priority

    // from this ordered sequence keep only the first one: the one with the highest priority
   .FirstOrDefault());

虽然这可行，但效率不高：在排序期间，它会一遍又一遍地执行record => record.Priority ?? Int32.MinValue 部分。考虑在排序前使用 Select：

.Select(record => new
{
     Priority = record.Priority ?? Int32.MinValue,
     Record = record,
})
.OrderByDescending(record => record.Priority);

排序后，在执行 FirstOrDefault 之前使用第二个 Select 删除 Priority。

实体框架知道您的关系，并且知道在您使用其中一个虚拟属性时使用哪个 (Group-)Join。

自己加入

有些人不喜欢使用虚拟属性，或者他们的实体框架版本不支持自动（Group-）Joining。

当然，您可以自己进行连接。

int jobId = ...
IEnumerable<Record> recordsOfThisJob = dbContext.Records

    // First get the JobId and Prority of each record
    .Select(record => new
    {
        JobId = dbContext.Tasks.Where(task => task.Id == record.Id)
                               .Select(task => task.JobId),
        Priority = record.Priority ?? Int32.MinValue,
        Record = record,
    })

    // then keep only those Record with the correct JobId
    .Where(record => record.JobId == jobId)

    // now do the GroupBy and OrderByDescending
    .GroupBy(record => record.Record.RecordId,
    (recordId, recordsWithThisRecordId) => recordsWithThisRecordId
        .OrderByDescending(record => record.Priority))

    // Get rid of JobId and Prority and Select the FirstOrDefault
    .Select(record => record.Record)
    .FirstOrDefault();

如果你真的想让它变得困难，你可以在外键上做一个 Join：

int jobId = ...
IEnumerable<Record> recordsOfThisJob = dbContext.Records
    .Join(dbContext.Tasks,        // Join Records and Tasks
    record => record.TaskId,      // from every Record take the foreign key to Task
    task => task.Id,              // from every Task take the primary key

    // when they match, take the Record and the Task to make one new
    (record, task) => new
    {
        JobId = task.JobId
        Priority = record.Priority ?? Int32.MinValue,
        Record = record,
    })

等等。其余同上。