【发布时间】:2016-02-24 02:42:22
【问题描述】:
当我尝试处理 270k 条记录时出现此异常。它在 12k 时失败。有人可以向我解释我缺少什么吗?
数据库是 SQL,我使用的是 EF 6。我使用谓词生成器来构建我的 where 子句。 这个想法是 select * from table where ((a = 'v1' and b = 'v2') or (a = 'v11' and b = 'v21') or (a = 'v12' and b = 'v22') ..) 我看不到任何地方仍然持有对代表 EF 类的对象的引用。我正在为要发回查看的结果创建一个 POCO。 有什么想法吗?
此外,我使用 10000 的 CommandTimeout 和失败的点,当我在 sql management studio 中使用相同参数运行查询时,它返回 400 行。
当我运行分析器时,我注意到在出现错误前几秒钟,内存使用量飙升至 1GB+
谢谢
public List<SearchResult> SearchDocuments(List<SearchCriteria> searchCriterias)
{
List<SearchResult> results = new List<SearchResult>();
var fieldSettings = GetData() ;// make a call to database to get this data
using (var context = CreateContext())
{
var theQuery = PredicateBuilder.False<ViewInSqlDatabase>();
int skipCount = 0;
const int recordsToProcessInOneBatch = 100;
while (searchCriterias.Skip(skipCount).Any())
{
var searchCriteriasBatched = searchCriterias.Skip(skipCount).Take(recordsToProcessInOneBatch);
foreach (var searchCriteria in searchCriteriasBatched)
{
var queryBuilder = PredicateBuilder.True<ViewInSqlDatabase>();
// theQuery
if (searchCriteria.State.HasValue)
queryBuilder = queryBuilder.And(a => a.State == searchCriteria.State.Value);
if (!string.IsNullOrWhiteSpace(searchCriteria.StateFullName))
queryBuilder = queryBuilder.And(a => a.StateName.Equals(searchCriteria.StateFullName, StringComparison.CurrentCultureIgnoreCase));
if (searchCriteria.County.HasValue)
queryBuilder = queryBuilder.And(a => a.County == searchCriteria.County.Value);
if (!string.IsNullOrWhiteSpace(searchCriteria.CountyFullName))
queryBuilder = queryBuilder.And(a => a.CountyName.Equals(searchCriteria.CountyFullName, StringComparison.CurrentCultureIgnoreCase));
if (!string.IsNullOrWhiteSpace(searchCriteria.Township))
queryBuilder = queryBuilder.And(a => a.Township == searchCriteria.Township);
// and so on...for another 10 parameters
theQuery = theQuery.Or(queryBuilder.Expand());
}
// this is where I get error after 12k to 15k criterias have been processed
var searchQuery = context.ViewInSqlDatabase.AsExpandable().Where(theQuery).Distinct().ToList();
foreach (var query in searchQuery)
{
var newResultItem = SearchResult.Create(query, fieldSettings); // POCO object with no relation to database
if (!results.Contains(newResultItem))
results.Add(newResultItem);
}
skipCount += recordsToProcessInOneBatch;
}
}
return results.Distinct().OrderBy(a => a.State).ThenBy(a => a.County).ThenBy(a => a.Township).ToList();
}
【问题讨论】:
-
你可以这样做
context.SearchResults.Where(x => ((x.a == 'v1' &&x.b == 'v2') || (x.a = 'v11' &&x.b = 'v21') || (x.a = 'v12' && x.b = 'v22')).Distinct().OrderBy(a => a.State).ThenBy(a => a.County).ThenBy(a => a.Township).ToList(); -
在 for 循环中构建它?同样在任何给定的点上,我最多只能拉出 4000 行。什么参考没有得到清理,我得到这个错误?我猜当我使用上面建议的格式时会出现同样的错误。所有这一切都是用直接参数替换谓词。对吗?
-
替换了 foreach 循环!请在线阅读有关 Linq 和 EF 的更多信息
标签: c# entity-framework-6 predicatebuilder