【问题标题】:LinqToSQL - ToList() appears to be very slowLinqToSQL - ToList() 似乎很慢
【发布时间】:2013-08-26 04:41:47
【问题描述】:

我对 LinqToSQL 很陌生,我正在从事的项目不能更改为其他项目。我正在将一些旧的 SQL 代码翻译成 Linq。对 linq 没有那么热情,我使用 Linqer 为我做翻译。查询运行大约需要 90 秒,所以我认为它一定是 linqToSQL。但是,当我复制 LinqToSQL 生成的查询并在数据上下文上运行 ExecuteQuery 时,它的速度和我预期的一样快。我复制了完整的查询,而不是尝试将其提取出来,但看起来问题出在 LinqToSQL 在幕后所做的事情上。

总结一下,如果我复制 linq 创建的 T-SQL 并运行

var results = DB.ExecuteQuery<InvoiceBalanceCheckDTO.InvoiceBalanceCheck>(@"T-SQL created by Linq - see below").ToList()

它在大约 0.5 秒内完成并获得预期结果。 它直接在 SSMS 中几乎同时运行。但是,如果我使用创建 T-SQL 的 linqToSQL 代码并执行 ToList() 则需要很长时间。结果只有 9 条记录,尽管没有检查余额 0 的约束,大约有 19,000 条记录。就好像它得到了所有 19,000,然后在获得记录后检查 0。 我也将 Linq 改成投影到上面使用的类,而不是匿名类型,但这没什么区别

这是原来的SQL:

SELECT InvoiceNum, Max(AccountCode), Sum(AmountInc) AS Balance
FROM 
    (SELECT InvoiceNum, AccountCode, AmountInc From TourBookAccount WHERE AccDetailTypeE IN(20,30) AND InvoiceNum >= 1000 
    UNION ALL 
    SELECT InvoiceNum, '<no matching invoice>' AS AccountCode, AccountInvoiceDetail.AmountInc 
    FROM AccountInvoiceDetail 
        INNER JOIN AccountInvoice ON AccountInvoiceDetail.InvoiceID=AccountInvoice.InvoiceID 
    WHERE AccDetailTypeE IN(20,30) 
    AND InvoiceNum >= 1000 
    ) as t
GROUP BY InvoiceNum 
HAVING (Sum(t.AmountInc)<>0) 
ORDER BY InvoiceNum

这是 linq

var test =  (from t in
                        (
                            //this gets the TourBookAccount totals
                            from tba in DB.TourBookAccount
                            where
                            detailTypes.Contains(tba.AccDetailTypeE) &&
                            tba.InvoiceNum >= dto.CheckInvoiceNumFrom
                            select new 
                            {
                                InvoiceNum = tba.InvoiceNum,
                                AccountCode = tba.AccountCode,
                                Balance = tba.AmountInc
                            }
                        )
                        .Concat //note that concat, since it's possible that the AccountInvoice record does not actually exist
                        (
                            //this gets the Invoice detail totals.
                            from aid in DB.AccountInvoiceDetail
                            where
                            detailTypes.Contains(aid.AccDetailTypeE) &&
                            aid.AccountInvoice.InvoiceNum >= dto.CheckInvoiceNumFrom &&
                            select new 
                            {
                                InvoiceNum = aid.AccountInvoice.InvoiceNum,
                                AccountCode = "<No Account Records>",
                                Balance = aid.AmountInc
                            }
                        ) 
                group t by t.InvoiceNum into g
                where Convert.ToDecimal(g.Sum(p => p.Balance)) != 0m
                select new 
                {
                    InvoiceNum = g.Key,
                    AccountCode = g.Max(p => p.AccountCode),
                    Balance = g.Sum(p => p.Balance)
                }).ToList();

这是 linq 生成的 T-SQL

  SELECT [t5].[InvoiceNum], [t5].[value2] AS [AccountCode], [t5].[value3] AS [Balance]
FROM (
    SELECT SUM([t4].[AmountInc]) AS [value], MAX([t4].[AccountCode]) AS [value2], SUM([t4].[AmountInc]) AS [value3], [t4].[InvoiceNum]
    FROM (
        SELECT [t3].[InvoiceNum], [t3].[AccountCode], [t3].[AmountInc]
        FROM (
            SELECT [t0].[InvoiceNum], [t0].[AccountCode], [t0].[AmountInc]
            FROM [dbo].[TourBookAccount] AS [t0]
            WHERE ([t0].[AccDetailTypeE] IN (20, 30)) AND ([t0].[InvoiceNum] >= 1000)
            UNION ALL
            SELECT [t2].[InvoiceNum],'<No Account Records>' AS [value], [t1].[AmountInc]
            FROM [dbo].[AccountInvoiceDetail] AS [t1]
            INNER JOIN [dbo].[AccountInvoice] AS [t2] ON [t2].[InvoiceID] = [t1].[InvoiceID]
            WHERE ([t1].[AccDetailTypeE] IN (20, 30)) AND ([t2].[InvoiceNum] >= 1000)
            ) AS [t3]
        ) AS [t4]
    GROUP BY [t4].[InvoiceNum]
    ) AS [t5]
WHERE [t5].[value] <> 0

【问题讨论】:

  • 这个查询是第一个被执行的查询吗?当第一个查询执行时,每个 AppDomain 都需要为 Initization 执行一次成本。
  • 你能在服务器上运行分析器,并验证它是否能一步完成你得到的查询吗?
  • 感谢您的建议。我将在 SQL 框上运行探查器,并仔细检查 VS 告诉我的 SQL 正在运行(这是我得到输出的地方)实际上正在运行。 @Scott - 不,这不是第一个运行的查询
  • 我们正在使用 SQL Server Express,而我目前没有时间安装 SSMS 2012 Sp1,我认为它与分析器一起提供。不过,我在数据上下文中添加了一个日志,这证实了正在发送的 T-SQL 是一次命中,并且与我发布的智能感知显示的相匹配。时间有点短,所以暂时只发送原始 SQL,稍后再回来尝试查找问题。感谢您的帮助。

标签: c# sql-server linq-to-sql


【解决方案1】:

我敢打赌,问题出在这一行:

where Convert.ToDecimal(g.Sum(p => p.Balance)) != 0m

可能发生的情况是,它无法将其转换为 SQL 并默默尝试将所有行从 db 获取到内存,然后在内存对象(LINQ to objects)上进行过滤 也许尝试将其更改为:

where g.Sum(p=>.Balance!=0)

【讨论】:

  • 我也这么认为,但看看生成的 SQL - 它似乎正在翻译它......
  • 通常你会得到一个 NotSupportedException "[blahblah] 方法没有转换为 SQL"
  • 是的,我知道它发生在 EF 上。我不确定 LINQ to SQL。你能在服务器上运行分析器,并验证这实际上是在一个步骤中执行的查询吗?
  • 查看 SQL - 根本没有发生内联“转换为十进制”。所以尝试@Botis 的建议,从.Net 代码中删除转换,看看它是否能提高性能。一旦你验证了这一点,你就可以制定出正确的代码来做你想做的事。
  • 可能源字段已经是正确的数据类型,所以Convert.ToDecimal 被简单地忽略了。
【解决方案2】:

好吧,结果证明不是 LinqToSQL 本身(尽管可能要归咎于它创建查询的方式),而是 SQL 服务器处理查询的方式。当我在数据库上运行查询以检查速度(并在 DB.ExecuteQuery 中运行创建的 T=SQL)时,我对所有变量进行了硬编码。当我将其更改为使用 Linq 生成的确切 sql(即使用被替换的变量)时,它在 SSMS 中的运行速度同样慢。

从两者的执行计划来看,是完全不同的。对 SO 的快速搜索将我带到了这个页面:Why does a parameterized query produces vastly slower query plan vs non-parameterized query 这表明问题是 SQL 服务器的“参数嗅探”。

罪魁祸首竟然是“No Account Records”字符串

为了完整起见,这里是 Linq 创建的生成的 T-SQL。 将@p10 更改为实际的硬编码字符串,它会恢复全速! 最后,我只是从 linq 中删除了该行,然后设置了帐户代码,一切都很好。

感谢@Botis、@Blorgbeard、@ElectricLlama 和@Scott 的建议。

DECLARE @p0 as Int = 20
DECLARE @p1 as Int = 30
DECLARE @p2 as Int = 1000
DECLARE @p3 as Int = 20
DECLARE @p4 as Int = 30
DECLARE @p5 as Int = 1000
DECLARE @p6 as Int = 40
DECLARE @p7 as Int = 10
DECLARE @p8 as Int = 0
DECLARE @p9 as Int = 1
DECLARE @p10 as NVarChar(4000)= '<No Account Records>' /*replace this parameter with the actual text in the SQl and it's way faster.*/
DECLARE @p11 as Decimal(33,4) = 0


SELECT [t5].[InvoiceNum], [t5].[value2] AS [AccountCode], [t5].[value3] AS [Balance]
FROM (
    SELECT SUM([t4].[AmountInc]) AS [value], MAX([t4].[AccountCode]) AS [value2], SUM([t4].[AmountInc]) AS [value3], [t4].[InvoiceNum]
    FROM (
        SELECT [t3].[InvoiceNum], [t3].[AccountCode], [t3].[AmountInc]
        FROM (
            SELECT [t0].[InvoiceNum], [t0].[AccountCode], [t0].[AmountInc]
            FROM [dbo].[TourBookAccount] AS [t0]
            WHERE ([t0].[AccDetailTypeE] IN (@p0, @p1)) AND ([t0].[InvoiceNum] >= @p2)
            UNION ALL
            SELECT [t2].[InvoiceNum], @p10 AS [value], [t1].[AmountInc]
            FROM [dbo].[AccountInvoiceDetail] AS [t1]
            INNER JOIN [dbo].[AccountInvoice] AS [t2] ON [t2].[InvoiceID] = [t1].[InvoiceID]
            WHERE ([t1].[AccDetailTypeE] IN (@p3, @p4)) AND ([t2].[InvoiceNum] >= @p5) AND ([t2].[InvoiceStatusE] <= @p6) AND ([t2].[InvoiceTypeE] = @p7) AND ([t1].[BookNum] <> @p8) AND ([t1].[AccDetailSourceE] = @p9)
            ) AS [t3]
        ) AS [t4]
    GROUP BY [t4].[InvoiceNum]
    ) AS [t5]
WHERE [t5].[value] <> @p11


SELECT [t5].[InvoiceNum], [t5].[value2] AS [AccountCode], [t5].[value3] AS [Balance]
FROM (
    SELECT SUM([t4].[AmountInc]) AS [value], MAX([t4].[AccountCode]) AS [value2], SUM([t4].[AmountInc]) AS [value3], [t4].[InvoiceNum]
    FROM (
        SELECT [t3].[InvoiceNum], [t3].[AccountCode], [t3].[AmountInc]
        FROM (
            SELECT [t0].[InvoiceNum], [t0].[AccountCode], [t0].[AmountInc]
            FROM [dbo].[TourBookAccount] AS [t0]
            WHERE ([t0].[AccDetailTypeE] IN (20, 30)) AND ([t0].[InvoiceNum] >= 1000)
            UNION ALL
            SELECT [t2].[InvoiceNum], '<No Account Records>' AS [value], [t1].[AmountInc]
            FROM [dbo].[AccountInvoiceDetail] AS [t1]
            INNER JOIN [dbo].[AccountInvoice] AS [t2] ON [t2].[InvoiceID] = [t1].[InvoiceID]
            WHERE ([t1].[AccDetailTypeE] IN (20, 30)) AND ([t2].[InvoiceNum] >= 0) AND ([t2].[InvoiceStatusE] <= 40) AND ([t2].[InvoiceTypeE] = 10) AND ([t1].[BookNum] <> 0) AND ([t1].[AccDetailSourceE] = 1)
            ) AS [t3]
        ) AS [t4]
    GROUP BY [t4].[InvoiceNum]
    ) AS [t5]
WHERE [t5].[value] <> 0

【讨论】:

  • 如果您正在创建一个新项目,我建议您使用实体框架,而不是 Linq to SQL。我相信 ef 会自动进行优化。 EF 将继续从 MSFT 获得比 l2s 更好的支持
  • @Aron - 对于我自己的东西,我实际上使用 Lightspeed link,但我们在工作中非常依赖于 LinqToSQL
猜你喜欢
  • 1970-01-01
  • 2021-12-11
  • 1970-01-01
  • 1970-01-01
  • 2015-02-16
  • 1970-01-01
  • 2016-09-08
  • 2018-12-18
  • 1970-01-01
相关资源
最近更新 更多