优化使用多个 CTE 的复杂视图答案

【问题标题】：Optimising a complex view that uses several CTE's优化使用多个 CTE 的复杂视图
【发布时间】：2018-09-17 11:09:22
【问题描述】：

我正在寻找有关我在此视图的性能方面所犯的任何明显错误的一些指导，因为它目前需要超过 1m 46s 来查询视图，如果我添加 WHERE 子句来反对期间和年份，则需要超过 3m .

视图的目的是整理几周的财务信息，并按期间和年份对这些信息进行分组。然后在整个财务系统中引用此视图，以便向其他财务系统报告和生成订单。

查询太长，我无法在此处发布，因此我提供了查询链接，并可应要求提供有关表格的任何信息： https://pastebin.com/x8dV8KLJ

以下只是查询的公用表表达式，因为 StackOverflow 坚持我发布一些代码来附上链接：

ALTER VIEW [dbo].[UnitFinancialFigures] AS

WITH
ctePeriodWeekCount AS (
    SELECT Count(Week.WeekNo) AS WeekCount, PrdNo AS Period, PrdYear AS Year
    FROM Week
    INNER JOIN Period on Week.WeekEndDate BETWEEN Period.StartDate AND Period.EndDate
    GROUP BY PrdNo, PrdYear),

cteLabourFigures AS (
    SELECT Labour.LabUnitId, Labour.LabPeriod, Labour.LabYear,
    SUM(CASE WHEN LabourTypes.LTypeIsPayrollLabour = 1 THEN LabourBreakdown.LBrkAmount ELSE 0 END) AS PayRollLabour,
    SUM(CASE WHEN LabourTypes.LTypeIsPayrollLabour = 0 THEN LabourBreakdown.LBrkAmount ELSE 0 END) AS OtherLabour
    FROM Labour
    INNER JOIN LabourBreakdown on LabourBreakDown.LBrkLabId = Labour.LabId
    INNER JOIN LabourTypes on LabourTypes.LTypeId = LabourBreakdown.LBrkLTypeId
    GROUP BY Labour.LabUnitId, Labour.LabPeriod, Labour.LabYear),

cteMobileRelief AS (
    SELECT McDtUnitId, McHdPeriod, McHdYear, SUM(McDtAmount) AS MobileReliefTotal
    FROM MobileCostHeader
    INNER JOIN MobileCostDetail on MobileCostDetail.McDtHdId = MobileCostHeader.MCHdId
    GROUP BY McDtUnitId, McHdPeriod, McHdYear),

cteInvoices AS (
    SELECT Period.PrdNo, Period.PrdYear, Invoice.AccountID, SUM(Invoice.InvTotalInclVat) AS TotalInvoices,
    SUM(Invoice.InvVatAmount) AS TotalInvoicesVat
    FROM [SOP].Invoice
    INNER JOIN Period on Invoice.InvoiceDate BETWEEN Period.StartDate AND Period.EndDate
    GROUP By Period.PrdNo, Period.PrdYear, Invoice.AccountID),

cteSundryCosts AS (
    SELECT ScDtAccountId, ScDtPeriodNo, ScDtYearNo, SUM(ScdtTotal) AS SundryTotal,
    SUM(ScdtAmount) AS SundryNetTotal,
    SUM(ScdtVatAmount) AS SundryVatAmount
    FROM SundryCostDetail
    WHERE ScdtRepeat = 0
    GROUP BY ScDtAccountId, ScDtPeriodNo, ScDtYearNo),

cteAdditionalProfit AS (
    SELECT ScDtAccountId, ScDtPeriodNo, ScDtYearNo, SUM(ScdtTotal) AS AdditionalProfitTotal,
    SUM(ScdtAmount) AS AdditionalProfitNetTotal,
    SUM(ScdtVatAmount) AS AdditionalProfitVatAmount
    FROM SundryCostDetail
    WHERE ScdtRepeat = 0 AND SCdtExcludeFromGP = 0
    GROUP BY ScDtAccountId, ScDtPeriodNo, ScDtYearNo),

cteCashReceived AS (
    SELECT CtUnitId, CtPeriod, CtYear,
    SUM(CtAmount - CtAgentVatAmount) AS GoodsAmount, SUM(CtVatAmount + CtAgentVatAmount) AS VatTotal, SUM(CtTotal) AS Total,
    SUM(VATFree.CanaAmount - VATFree.CanaAgentVatAmount) AS VATFreeGoodsAmount,
    SUM(Catering.CanaAmount - Catering.CanaAgentVatAmount) AS CateringGoodsAmount,
    SUM(Vending.CanaAmount - Vending.CanaAgentVatAmount) AS VendingGoodsAmount,
    SUM(CansConf.CanaAmount - CansConf.CanaAgentVatAmount) AS CansConfGoodsAmount
    FROM CashTran
    LEFT JOIN CashAnalysis AS VATFree ON CashTran.CtId = VATFree.CanaTranId AND VATFree.CanaCode = 'G201'
    LEFT JOIN CashAnalysis AS Catering ON CashTran.CtId = Catering.CanaTranId AND Catering.CanaCode = 'G205'
    LEFT JOIN CashAnalysis AS Vending ON CashTran.CtId = Vending.CanaTranId AND Vending.CanaCode = 'G202'
    LEFT JOIN CashAnalysis AS CansConf ON CashTran.CtId = CansConf.CanaTranId AND CansConf.CanaCode = 'G203'
    WHERE CtCategoryId = 3 AND CtApproved = 1 ANd CtTranType = 'R'
    GROUP BY CtUnitId, CtPeriod, CtYear),

cteFreeIssues AS (
    SELECT CtUnitId, CtPeriod, CtYear,
    SUM(CtAmount - CtAgentVatAmount) AS GoodsAmount, SUM(CtVatAmount + CtAgentVatAmount) AS VatTotal, SUM(CtTotal) AS Total,
    SUM(Catering.CanaAmount - Catering.CanaAgentVatAmount) AS CateringGoodsAmount,
    SUM(Vending.CanaAmount - Vending.CanaAgentVatAmount) AS VendingGoodsAmount,
    SUM(CansConf.CanaAmount - CansConf.CanaAgentVatAmount) AS CansConfGoodsAmount,
    SUM(Labour.CanaAmount - Labour.CanaAgentVatAmount) AS LabourGoodsAmount
    FROM CashTran
    INNER JOIN [SOP].Accounts ON Accounts.AcUniqueId = CtUnitId
    LEFT JOIN CashAnalysis AS Catering ON CashTran.CtId = Catering.CanaTranId AND Catering.CanaCode = 'G204'
    LEFT JOIN CashAnalysis AS Vending ON CashTran.CtId = Vending.CanaTranId AND Vending.CanaCode = 'G202'
    LEFT JOIN CashAnalysis AS CansConf ON CashTran.CtId = CansConf.CanaTranId AND CansConf.CanaCode = 'G203'
    LEFT JOIN CashAnalysis AS Labour ON CashTran.CtId = Labour.CanaTranId AND Labour.CanaCode = 'H995'
    WHERE CtCategoryId = 2
    GROUP BY CtUnitId, CtPeriod, CtYear),

cteExternalFreeIssues AS (
    SELECT CtUnitId, CtPeriod, CtYear, SUM(CtAmount - CtAgentVatAmount) AS GoodsAmount, SUM(CtVatAmount + CtAgentVatAmount) AS VatTotal, SUM(CtTotal) AS Total
    FROM CashTran
    INNER JOIN [SOP].Accounts on Accounts.AcUniqueId = CtUnitId
    WHERE CtCategoryId = 4
    GROUP BY CtUnitId, CtPeriod, CtYear),

cteAgencyLabour AS (
    SELECT PtUnitId, PtYear, PtPeriod, SUM(CASE WHEN PtTranType = 'I' THEN PanaGoodsAmount - PanaAgentVatAmount ELSE -(PanaGoodsAmount - PanaAgentVatAmount) END) Total
    FROM PurchaseTran
    INNER Join PurchaseAnalysis on PurchaseAnalysis.PanaTranId = PurchaseTran.PtId
    INNER JOIN [SOP].Accounts on Accounts.AcUniqueID = PtUnitId
    INNER JOIN UnitAnalysisConfig on UnitAnalysisConfig.UACfgAnalysisCode = PurchaseAnalysis.PanaPCode AND UnitAnalysisConfig.UACfgType = 1
    WHERE PurchaseAnalysis.PanaPCode = 'H995' AND (PtTranType = 'I' Or PtTranType = 'C') AND PtApproved =1
    GROUP BY PtUnitId, PtPeriod, PtYear),

ctePurchases AS (
    SELECT PurchaseTran.PtUnitId, PurchaseTran.PtPeriod, PurchaseTran.PtYear,
    SUM(CASE WHEN PtTranType = 'I' THEN PtTotal WHEN PtTranType = 'C' THEN - PtTotal ELSE 0 END) AS PurchaseTotal,
    SUM(CASE WHEN PtTranType = 'I' THEN PtAmount - PtAgentVatAmount WHEN PtTranType = 'C' THEN -(PtAmount - PtAgentVatAmount) ELSE 0 END) AS PurchaseNetTotal,
    SUM(CASE WHEN PtTranType = 'I' THEN PtVat + PtAgentVatAmount WHEN PtTranType = 'C' THEN -(PtVat + PtAgentVatAmount) ELSE 0 END) AS VatTotal,
    SUM(CASE WHEN PtSupplierId = CmpCashAccount THEN PtTotal ELSE 0 END) AS TotalCashPaid,
    SUM(CASE WHEN PtTranType = 'I' THEN Catering.PanaGoodsAmount - Catering.PanaAgentVatAmount WHEN PtTranType = 'C' THEN -(Catering.PanaGoodsAmount - Catering.PanaAgentVatAmount) ELSE 0 END) CateringTotal,
    SUM(CASE WHEN PtTranType = 'I' THEN Catering.PanaAgentVatAmount + Catering.PanaVatAmount WHEN PtTranType = 'C' THEN -(Catering.PanaAgentVatAmount + Catering.PanaVatAmount) ELSE 0 END) AS CateringVatAmount,
    SUM(CASE WHEN PtTranType = 'I' THEN Vending.PanaGoodsAmount - Vending.PanaAgentVatAmount WHEN PtTranType = 'C' THEN -(Vending.PanaGoodsAmount - Vending.PanaAgentVatAmount) ELSE 0 END) VendingTotal,
    SUM(CASE WHEN PtTranType = 'I' THEN Vending.PanaAgentVatAmount + Vending.PanaVatAmount WHEN PtTranType = 'C' THEN -(Vending.PanaAgentVatAmount + Vending.PanaVatAmount) ELSE 0 END) AS VendingVatAmount,
    SUM(CASE WHEN PtTranType = 'I' THEN CansConf.PanaGoodsAmount - CansConf.PanaAgentVatAmount WHEN PtTranType = 'C' THEN -(CansConf.PanaGoodsAmount - CansConf.PanaAgentVatAmount) ELSE 0 END) CansConfTotal,
    SUM(CASE WHEN PtTranType = 'I' THEN CansConf.PanaAgentVatAmount + CansConf.PanaVatAmount WHEN PtTranType = 'C' THEN -(CansConf.PanaAgentVatAmount + CansConf.PanaVatAmount) ELSE 0 END) AS CansConfVatAmount
    FROM PurchaseTran
    INNER JOIN CompParam on Compparam.CmpDefault = 1
    INNER JOIN [SOP].Accounts on Accounts.AcUniqueID = PurchaseTran.PtUnitId
    LEFT JOIN PurchaseAnalysis AS Catering on Catering.PanaTranId = PurchaseTran.PtId AND (Catering.PanaPCode = 'H201' OR Catering.PanaPCode = 'H401')
    LEFT JOIN PurchaseAnalysis AS Vending on Vending.PanaTranId = PurchaseTran.PtId AND Vending.PanaPCode = 'H202'
    LEFT JOIN PurchaseAnalysis AS CansConf on CansConf.PanaTranId = PurchaseTran.PtId AND CansConf.PanaPCode = 'H211'
    WHERE (PtTranType = 'I' OR PtTranType = 'C') AND PtApproved = 1
    GROUP BY PtUnitId, PtPeriod, PtYear),

cteSundryPurchases AS (
    SELECT PtUnitId, PtYear, PtPeriod,
    SUM(CASE WHEN PtTranType = 'I' THEN PanaGoodsAmount - PanaAgentVatAmount ELSE -(PanaGoodsAmount - PanaAgentVatAmount) END) Total,
    SUM(CASE WHEN PtTranType = 'I' THEN PanaAgentVatAmount + PanaVatAmount WHEN PtTranType = 'C' THEN -(PanaAgentVatAmount + PanaVatAmount) ELSE 0 END) AS VatAmount
    FROM PurchaseTran
    INNER Join PurchaseAnalysis on PurchaseAnalysis.PanaTranId = PurchaseTran.PtId
    INNER JOIN [SOP].Accounts on Accounts.AcUniqueID = PurchaseTran.PtUnitId
    INNER JOIN UnitAnalysisConfig on UnitAnalysisConfig.UACfgAnalysisCode = PurchaseAnalysis.PanaPCode AND (UnitAnalysisConfig.UACfgType = 1 OR UnitAnalysisConfig.UACfgType = 3)
    WHere UnitAnalysisConfig.UACfgTandOLabel = 'Sundry' AND (PtTranType = 'I' Or PtTranType = 'C') AND PtApproved = 1
    GROUP BY PtUnitId, PtPeriod, PtYear),

cteDiscount AS (
    SELECT PurchaseTran.PtUnitId, SUM(CASE WHEN PtTRanType = 'I' THEN PtDiscountAmount ELSE -PtDiscountAmount END) AS DiscountTotal, PtPeriod, PtYear
    FROM PurchaseTran
    WHERE PurchaseTran.PtApproved = 1 And (PurchaseTran.PtTranType = 'I' OR PurchaseTran.PtTranType = 'C')
    GROUP BY PurchaseTran.PtUnitId, PtPeriod, PtYear),

cteUnitPeriodOpeningStock AS (
    SELECT UStkId, UStkOpening, UnitStock.UStkUnitId, UnitStock.UStkPeriod, UnitStock.UStkYearNo, SUM(OpeningCatering.USAnaAmount) AS OpeningCateringStock, SUM(OpeningBeverage.USAnaAmount) AS OpeningBeverageStock, SUM(OpeningSundry.USAnaAmount) AS OpeningSundryStock, SUM(OpeningCansConf.USAnaAmount) AS OpeningCansConfStock
    FROM UnitStock
    LEFT JOIN UnitStockAnalysis AS OpeningCatering ON UnitStock.UStkId = OpeningCatering.USAnaTranId AND OpeningCatering.USanaType = 'O' AND OpeningCatering.USAnaCode = 'CAT001'
    LEFT JOIN UnitStockAnalysis AS OpeningBeverage ON UnitStock.UStkId = OpeningBeverage.USAnaTranId AND OpeningBeverage.USanaType = 'O' AND OpeningBeverage.USAnaCode = 'BEV001'
    LEFT JOIN UnitStockAnalysis AS OpeningSundry ON UnitStock.UStkId = OpeningSundry.USAnaTranId AND OpeningSundry.USanaType = 'O' AND OpeningSundry.USAnaCode = 'SUN001'
    LEFT JOIN UnitStockAnalysis AS OpeningCansConf ON UnitStock.UStkId = OpeningCansConf.USAnaTranId AND OpeningCansConf.USanaType = 'O' AND OpeningCansConf.USAnaCode = 'CAN001'
    INNER JOIN Week on Week.WeekNo = UnitStock.UStkWeeKNo AND Week.WeekYear = UnitStock.UStkYearNo
    INNER JOIN (
        SELECT UStkUnitId, UStkPeriod, MIN(Week.WeekEndDate) OpeningDate
        FROM UnitStock
        INNER JOIN Week on Week.WeekNo = UnitStock.UStkWeeKNo AND Week.WeekYear = UnitStock.UStkYearNo
        GROUP BY UStkUnitId, UStkPeriod) OpeningData on OpeningData.OpeningDate = Week.WeekEndDate AND OpeningData.UStkUnitId = UnitStock.UStkUnitId
    GROUP BY UStkId, UStkOpening, UnitStock.UStkUnitId, UnitStock.UStkPeriod, UnitStock.UStkYearNo),

cteUnitPeriodClosingStock AS (
    SELECT UStkId, UStkClosing, UnitStock.UStkUnitId, UnitStock.UStkPeriod, UnitStock.UStkYearNo, SUM(ClosingCatering.USAnaAmount) AS ClosingCateringStock, SUM(ClosingBeverage.USAnaAmount) AS ClosingBeverageStock, SUM(ClosingSundry.USAnaAmount) AS ClosingSundryStock, SUM(ClosingCansConf.USAnaAmount) AS ClosingCansConfStock
    FROM UnitStock
    LEFT JOIN UnitStockAnalysis AS ClosingCatering ON UnitStock.UStkId = ClosingCatering.USAnaTranId AND ClosingCatering.USanaType = 'C' AND ClosingCatering.USAnaCode = 'CAT001'
    LEFT JOIN UnitStockAnalysis AS ClosingBeverage ON UnitStock.UStkId = ClosingBeverage.USAnaTranId AND ClosingBeverage.USanaType = 'C' AND ClosingBeverage.USAnaCode = 'BEV001'
    LEFT JOIN UnitStockAnalysis AS ClosingSundry ON UnitStock.UStkId = ClosingSundry.USAnaTranId AND ClosingSundry.USanaType = 'C' AND ClosingSundry.USAnaCode = 'SUN001'
    LEFT JOIN UnitStockAnalysis AS ClosingCansConf ON UnitStock.UStkId = ClosingCansConf.USAnaTranId AND ClosingCansConf.USanaType = 'C' AND ClosingCansConf.USAnaCode = 'CAN001'
    INNER JOIN Week on Week.WeekNo = UnitStock.UStkWeeKNo AND Week.WeekYear = UnitStock.UStkYearNo
    INNER JOIN (
        SELECT UStkUnitId, UStkPeriod, MAX(Week.WeekEndDate) ClosingDate
        FROM UnitStock
        INNER JOIN Week on Week.WeekNo = UnitStock.UStkWeeKNo AND Week.WeekYear = UnitStock.UStkYearNo
        GROUP BY UStkUnitId, UStkPeriod) ClosingData on ClosingData.ClosingDate = Week.WeekEndDate AND ClosingData.UStkUnitId = UnitStock.UStkUnitId
    GROUP BY UStkId, UStkClosing, UnitStock.UStkUnitId, UnitStock.UStkPeriod, UnitStock.UStkYearNo)

回复后编辑 感谢您的反馈。我遵循了这些建议，并分别优化了每个 CTE 查询。我的错误是期望执行计划告诉我在运行视图时是否缺少建议的索引；直到我分解了查询，我才真正得到了任何建议。知道我没有做任何明显损害性能的明显事情也很有帮助。添加各种索引后，我得到了大约 30 秒的视图。

【问题讨论】：

您的查询看起来太大，不可能在这里得到答案，但作为一般性评论，我会说（非物化）视图的性能与基础查询一样好。因此，如果您想加快查看速度，则需要调整底层查询。
考虑索引视图 (docs.microsoft.com/en-us/sql/relational-databases/views/…) ?
很多 CTE 看起来都是多余的，因为它们查询的是相同的基表，但条件略有不同。优化器将尝试尽可能智能地查询它，但它不会主动合并具有不同条件的查询。考虑是否可以将其中一些条件粘贴在列中 (CASE)，然后稍后按它们查询或分组。
另外，您在这里获得的表、连接和条件的绝对数量表明实际上还有另一个数据库正在努力摆脱这个数据库，或者至少是另一种形式的索引。考虑具有列存储索引的（新的或现有的）表或 Analysis Services 数据库，甚至可能是单独维护的仓库数据库，以提前计算和存储这些派生表。虽然这是多余的，但它是一种有用的冗余（结果永远不会被独立修改），而不是邪恶的那种。
另外一个参与者也不喜欢long reads——sql server 的查询优化器。当一个语句那么大（并且整个view 是一个语句）时，它可能无法找到执行它的最佳方式。这个看起来像一个大报告，作为一个有几个@或#的大 SP 可能会更好。一个带有小语句的SP，可以很容易地被sql server的优化器分析，被开发者理解、调试和重构。这再次引出了蒂姆关于“潜在查询”的第一条评论。

标签： sql-server tsql optimization common-table-expression

【解决方案1】：

根据 Tim 的评论，（非物化）视图的性能与底层查询一样好。因此，首先分析查询以进行优化，例如在高度使用的列（在连接中或作为外键）上添加非聚集索引。

除此之外，您还可以查看 Indexed Views 以进一步提高性能。但是索引视图也有一些问题，因为它们也不是提高视图性能的万无一失的方法，例如：

在由大量索引视图或较少但非常复杂的索引视图引用的表上执行 DML 时，这些引用的索引视图也必须更新。因此，DML 查询性能会显着下降，或者在某些情况下，甚至无法生成查询计划。在这种情况下，在生产使用之前测试您的 DML 查询，分析查询计划并调整/简化 DML 语句，例如 UPDATE、DELETE 或 INSERT 操作。
索引视图的成本在于维护聚集索引（以及您可以选择添加的任何非聚集索引）。必须权衡维护索引的成本与索引提供的查询优化的好处。

https://www.red-gate.com/simple-talk/sql/learn-sql-server/sql-server-indexed-views-the-basics/

https://www.codeproject.com/Articles/199058/SQL-Server-Indexed-Views-Speed-Up-Your-Select-Quer

【讨论】：

【解决方案2】：

如果您的所有 CTE 查询都在可接受的水平上执行，但整个语句没有（常见），那么您应该将 CTE 转换为这样的临时：

--WITH
--ctePeriodWeekCount AS (...

--becomes

SELECT Count(Week.WeekNo) AS WeekCount, PrdNo AS Period, PrdYear AS Year
INTO #ctePeriodWeekCount FROM Week
INNER JOIN Period on Week.WeekEndDate BETWEEN Period.StartDate AND Period.EndDate
GROUP BY PrdNo, PrdYear

然后按照 CTE 顺序创建并使用适当的临时表（只需在名称前添加 #）。不要忘记在最后一次使用时删除临时工以尽快释放资源。最后，如果最后使用了过滤器，请确保在临时创建 SELECT 语句时使用它。

即使最终 CTE 的性能接近于单个 CTE，使用 temps 对性能也有好处 - 当未来发生变化时，它不太可能处理性能问题。

【讨论】：