【问题标题】:SQL Replace All Tables with Clustered Columnstore IndexSQL 用聚集列存储索引替换所有表
【发布时间】:2023-03-26 21:26:02
【问题描述】:

我们正在执行一个迁移项目,并希望将大多数行存储索引替换为大型数据仓库的聚集列存储索引。我们正在标识列上添加唯一索引。

是否有人有脚本来更改所有 100 多个表的运行,并将主键聚集索引替换为列存储索引?

测试列存储索引是否有助于我们迁移时的性能调整。

*顺便问一下,Columnstore 中是否推荐使用 Identity 列?需要标识每一行的方法(有时标识列可能会重置,因此放置唯一约束,或者应该使用 Guids?)如果不是标识,请随意使用 Guid 或其他东西删除标识列。

当前:

CREATE TABLE [dbo].[Fact_SalesTransaction] 
(
    [FactSalesTransactionId]        INT              IDENTITY (1, 1) NOT NULL,
    [DimCustomerId]                 INT              NOT NULL,
    [DimSellerId]                   INT              NOT NULL,
    [SalesDatetime]                 DATETIME         NULL,
    [DimSalesDateId]                INT              NULL,
    [SalesAmount]                   DECIMAL (28, 2)  NULL,
    [ETLCreateDate]                 DATETIME         NULL,
    CONSTRAINT [pk_SalesTransactionId] PRIMARY KEY CLUSTERED ([SalesTransactionId] ASC)
);

预期:

CREATE TABLE [dbo].[Fact_SalesTransaction] 
(
    [FactSalesTransactionId]        INT              IDENTITY (1, 1) NOT NULL,
    [DimCustomerId]                 INT              NOT NULL,
    [DimSellerId]                   INT              NOT NULL,
    [SalesDatetime]                 DATETIME         NULL,
    [DimSalesDateId]                INT              NULL,
    [SalesAmount]                   DECIMAL (28, 2)  NULL,
    [ETLCreateDate]                 DATETIME         NULL,
);


CREATE CLUSTERED COLUMNSTORE INDEX ccx_Fact_SalesTransaction ON Fact_SalesTransaction;  

CREATE UNIQUE INDEX unx_FactSalesTransactionId ON dbo.Fact_SalesTransaction (FactSalesTransactionId);  

我们只想使用 T-SQL 在现有数据库上执行此操作。

评论中的有用资源:Generate SQL Create Scripts for existing tables with Query

【问题讨论】:

  • 你尝试过 brent ozar 博客吗?
  • 是否所有行存储表都只有一个主键聚集索引,没有其他需要保留的索引或外键约束?
  • 我去年写了一些类似的东西来归档大量数据。我查询 sys.tables 和 sys.columns 等以获取所有引用,然后编写动态 sql 来生成脚本,这些脚本会生成要复制到的结构。这对你有用吗?您是否尝试过从 sys.tables 获取元数据以进行工作?
  • 我找到了一些可以用作参考的东西来构建脚本,以便从元数据中生成表格代码。看起来它有你需要的一切:stackoverflow.com/questions/706664/…
  • 如果我没记错的话,您需要将“当前”下的主键聚集约束更改为指向 FactSalesTransactionId 而不是“SalesTransactionId”。

标签: sql-server indexing sql-server-2016 data-warehouse columnstore


【解决方案1】:

Devart 回答了this 关于如何为表生成脚本的问题。我调整了他的代码以省略主键部分,并按照他的一般模式将其替换为唯一键和列存储创建脚本。我还必须使用 marc_s 对 this 问题的回答,了解如何确定表的主键是否聚集。我将所有这些都放入一个名为#scriptTable 的存储过程中。这是下面的内容:

create procedure #scriptTable
    @table_name sysname,
    @sql nvarchar(max) output
as

DECLARE 
      @object_name SYSNAME
    , @object_id INT

SELECT 
      @object_name = '[' + s.name + '].[' + o.name + ']'
    , @object_id = o.[object_id]
FROM sys.objects o WITH (NOWAIT)
JOIN sys.schemas s WITH (NOWAIT) ON o.[schema_id] = s.[schema_id]
WHERE s.name + '.' + o.name = @table_name
    AND o.[type] = 'U'
    AND o.is_ms_shipped = 0

;WITH index_column AS 
(
    SELECT 
          ic.[object_id]
        , ic.index_id
        , ic.is_descending_key
        , ic.is_included_column
        , c.name
    FROM sys.index_columns ic WITH (NOWAIT)
    JOIN sys.columns c WITH (NOWAIT) ON ic.[object_id] = c.[object_id] AND ic.column_id = c.column_id
    WHERE ic.[object_id] = @object_id
),
fk_columns AS 
(
     SELECT 
          k.constraint_object_id
        , cname = c.name
        , rcname = rc.name
    FROM sys.foreign_key_columns k WITH (NOWAIT)
    JOIN sys.columns rc WITH (NOWAIT) ON rc.[object_id] = k.referenced_object_id AND rc.column_id = k.referenced_column_id 
    JOIN sys.columns c WITH (NOWAIT) ON c.[object_id] = k.parent_object_id AND c.column_id = k.parent_column_id
    WHERE k.parent_object_id = @object_id
)
SELECT @sql = 'CREATE TABLE ' + @object_name + CHAR(13) + '(' + CHAR(13) + STUFF((
    SELECT CHAR(9) + ', [' + c.name + '] ' + 
        CASE WHEN c.is_computed = 1
            THEN 'AS ' + cc.[definition] 
            ELSE UPPER(tp.name) + 
                CASE WHEN tp.name IN ('varchar', 'char', 'varbinary', 'binary', 'text')
                       THEN '(' + CASE WHEN c.max_length = -1 THEN 'MAX' ELSE CAST(c.max_length AS VARCHAR(5)) END + ')'
                     WHEN tp.name IN ('nvarchar', 'nchar', 'ntext')
                       THEN '(' + CASE WHEN c.max_length = -1 THEN 'MAX' ELSE CAST(c.max_length / 2 AS VARCHAR(5)) END + ')'
                     WHEN tp.name IN ('datetime2', 'time2', 'datetimeoffset') 
                       THEN '(' + CAST(c.scale AS VARCHAR(5)) + ')'
                     WHEN tp.name = 'decimal' 
                       THEN '(' + CAST(c.[precision] AS VARCHAR(5)) + ',' + CAST(c.scale AS VARCHAR(5)) + ')'
                    ELSE ''
                END +
                CASE WHEN c.collation_name IS NOT NULL THEN ' COLLATE ' + c.collation_name ELSE '' END +
                CASE WHEN c.is_nullable = 1 THEN ' NULL' ELSE ' NOT NULL' END +
                CASE WHEN dc.[definition] IS NOT NULL THEN ' DEFAULT' + dc.[definition] ELSE '' END + 
                CASE WHEN ic.is_identity = 1 THEN ' IDENTITY(' + CAST(ISNULL(ic.seed_value, '0') AS CHAR(1)) + ',' + CAST(ISNULL(ic.increment_value, '1') AS CHAR(1)) + ')' ELSE '' END 
        END + CHAR(13)
    FROM sys.columns c WITH (NOWAIT)
    JOIN sys.types tp WITH (NOWAIT) ON c.user_type_id = tp.user_type_id
    LEFT JOIN sys.computed_columns cc WITH (NOWAIT) ON c.[object_id] = cc.[object_id] AND c.column_id = cc.column_id
    LEFT JOIN sys.default_constraints dc WITH (NOWAIT) ON c.default_object_id != 0 AND c.[object_id] = dc.parent_object_id AND c.column_id = dc.parent_column_id
    LEFT JOIN sys.identity_columns ic WITH (NOWAIT) ON c.is_identity = 1 AND c.[object_id] = ic.[object_id] AND c.column_id = ic.column_id
    WHERE c.[object_id] = @object_id
    ORDER BY c.column_id
    FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, CHAR(9) + ' ')
    + ')'  + CHAR(13)
    + ISNULL((SELECT (
        SELECT CHAR(13) +
             'ALTER TABLE ' + @object_name + ' WITH' 
            + CASE WHEN fk.is_not_trusted = 1 
                THEN ' NOCHECK' 
                ELSE ' CHECK' 
              END + 
              ' ADD CONSTRAINT [' + fk.name  + '] FOREIGN KEY(' 
              + STUFF((
                SELECT ', [' + k.cname + ']'
                FROM fk_columns k
                WHERE k.constraint_object_id = fk.[object_id]
                FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, '')
               + ')' +
              ' REFERENCES [' + SCHEMA_NAME(ro.[schema_id]) + '].[' + ro.name + '] ('
              + STUFF((
                SELECT ', [' + k.rcname + ']'
                FROM fk_columns k
                WHERE k.constraint_object_id = fk.[object_id]
                FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, '')
               + ')'
            + CASE 
                WHEN fk.delete_referential_action = 1 THEN ' ON DELETE CASCADE' 
                WHEN fk.delete_referential_action = 2 THEN ' ON DELETE SET NULL'
                WHEN fk.delete_referential_action = 3 THEN ' ON DELETE SET DEFAULT' 
                ELSE '' 
              END
            + CASE 
                WHEN fk.update_referential_action = 1 THEN ' ON UPDATE CASCADE'
                WHEN fk.update_referential_action = 2 THEN ' ON UPDATE SET NULL'
                WHEN fk.update_referential_action = 3 THEN ' ON UPDATE SET DEFAULT'  
                ELSE '' 
              END 
            + CHAR(13) + 'ALTER TABLE ' + @object_name + ' CHECK CONSTRAINT [' + fk.name  + ']' + CHAR(13)
        FROM sys.foreign_keys fk WITH (NOWAIT)
        JOIN sys.objects ro WITH (NOWAIT) ON ro.[object_id] = fk.referenced_object_id
        WHERE fk.parent_object_id = @object_id
        FOR XML PATH(N''), TYPE).value('.', 'NVARCHAR(MAX)')), '')
    + ISNULL(((SELECT
         CHAR(13) + 'CREATE' + CASE WHEN i.is_unique = 1 THEN ' UNIQUE' ELSE '' END 
                + ' NONCLUSTERED INDEX [' + i.name + '] ON ' + @object_name + ' (' +
                STUFF((
                SELECT ', [' + c.name + ']' + CASE WHEN c.is_descending_key = 1 THEN ' DESC' ELSE ' ASC' END
                FROM index_column c
                WHERE c.is_included_column = 0
                    AND c.index_id = i.index_id
                FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, '') + ')'  
                + ISNULL(CHAR(13) + 'INCLUDE (' + 
                    STUFF((
                    SELECT ', [' + c.name + ']'
                    FROM index_column c
                    WHERE c.is_included_column = 1
                        AND c.index_id = i.index_id
                    FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 2, '') + ')', '')  + CHAR(13)
        FROM sys.indexes i WITH (NOWAIT)
        WHERE i.[object_id] = @object_id
            AND i.is_primary_key = 0
            AND i.[type] = 2
        FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)')
    ), '')
    + char(13)

-- Replaced "create primary key" logic in the original with what's below:

    +   ISNULL(
            (
                select  'create clustered columnstore index [ccx_' + @table_name + '] on ' + @object_name + char(13) +
                        'create unique ' + convert(nvarchar(max), i.type_desc) + ' index [' + replace(k.name, 'pk_', 'unx_') + '] ' +
                            'on ' + @object_name + ' (' + 
                                (
                                    SELECT STUFF((
                                        SELECT  ', [' + ic.name + '] ' + CASE WHEN ic.is_descending_key = 1 THEN 'DESC' ELSE 'ASC' END
                                        FROM    index_column ic
                                        WHERE   ic.is_included_column = 0
                                        AND     ic.[object_id] = k.parent_object_id 
                                        AND     ic.index_id = k.unique_index_id     
                                        FOR XML PATH(N''), TYPE
                                    ).value('.', 'NVARCHAR(MAX)'), 1, 2, '')
                                ) + 
                            ')' + CHAR(13)
                FROM sys.key_constraints k WITH (NOWAIT)
                join sys.indexes i with (nowait) on k.unique_index_id = i.index_id and k.parent_object_id = i.object_id
                WHERE k.parent_object_id = @object_id 
                    AND k.[type] = 'PK'
            ), 
            ''
        );

你可以像这样使用#scriptTable:

declare @sql nvarchar(max);
exec #scriptTable 'dbo.Fact_SalesTransaction', @sql output;
print (@sql);

将 'print' 替换为 'exec' 或在准备好时使用 sp_executeSql。

要在所有表上使用它,首先捕获要修改的表:

declare @tables table (
    rowId int identity(1,1),
    name nvarchar(max)
);

insert      @tables
select      schema_name(schema_id) + '.' + name 
from        sys.tables
where       type_desc = 'user_table'

现在您已准备好循环表格并应用#scriptTable:

declare @rowId int = 1;
declare @table nvarchar(max);
declare @sql nvarchar(max);

while @rowId <= (select max(rowId) from @tables) begin
    select @table = name from @tables where rowId = @rowId;
    exec #scriptTable @table, @sql output;
    print (@sql); -- turn 'print' into 'exec' or otherwise use sp_executeSql
    set @rowId += 1;
end

和以前一样,将 'print' 替换为 'exec' 或在准备好时使用 sp_executeSql。

请注意,如果您的表名需要括号括起来,我的部分可能需要进一步修改。

编辑: 稍微更新了代码以简化(稍微)并使用需要括号括起来的表名。

【讨论】:

  • 你去了,devart 的回答又链接到对这类东西感兴趣的人应该阅读的几篇很棒的文章。
【解决方案2】:

要在所有表上创建 CCI,您可以使用以下内容:

    DECLARE @SQLscript nVARCHAR(1000) =
'CREATE CLUSTERED COLUMNSTORE INDEX &&& ON @@@ WITH (DROP_EXISTING = OFF,COMPRESSION_DELAY = 0)'

declare @tables table (
    rowId int identity(1,1),
    name nvarchar(max)
);

insert      @tables
select      schema_name(schema_id) + '.' + name 
from        sys.tables
where       type_desc = 'user_table'

declare @rowId int = 1;
declare @table nvarchar(max);
declare @sql nvarchar(max);

while @rowId <= (select max(rowId) from @tables) begin
    select @table = name from @tables where rowId = @rowId;
    SET @SQLscript = REPLACE(REPLACE(@SQLscript, '@@@',@table) ,'&&&','cci_' + SUBSTRING(REPLACE(@table,'].[','_'),CHARINDEX('.',REPLACE(@table,'].[','_'))+1,LEN(REPLACE(@table,'].[','_'))))
 BEGIN TRY 
  EXEC  sp_executeSql @SQLscript; 
  END TRY 
  BEGIN CATCH 
  PRINT @SQLscript
  END CATCH 

    SET @SQLscript = 'CREATE CLUSTERED COLUMNSTORE INDEX &&& ON @@@ WITH (DROP_EXISTING = OFF,COMPRESSION_DELAY = 0)'
    set @rowId += 1;
end

【讨论】:

    【解决方案3】:

    @pwilcox 的答案是正确的,除了它吐出行存储索引代码的最后一部分。该索引不应聚集,因为上面的行已经创建了聚集列存储索引。

    修复:

    -- Replaced "create primary key" logic in the original with what's below:
    
        +   ISNULL(
                (
                    select  'create clustered columnstore index [ccx_' + @table_name + '] on ' + @object_name + char(13) +
                            'create unique index [' + replace(k.name, 'pk_', 'unx_') + '] ' +
                                'on ' + @object_name + ' (' + 
                                    (
                                        SELECT STUFF((
                                            SELECT  ', [' + ic.name + '] ' + CASE WHEN ic.is_descending_key = 1 THEN 'DESC' ELSE 'ASC' END
                                            FROM    index_column ic
                                            WHERE   ic.is_included_column = 0
                                            AND     ic.[object_id] = k.parent_object_id 
                                            AND     ic.index_id = k.unique_index_id     
                                            FOR XML PATH(N''), TYPE
                                        ).value('.', 'NVARCHAR(MAX)'), 1, 2, '')
                                    ) + 
                                ')' + CHAR(13)
                    FROM sys.key_constraints k WITH (NOWAIT)
                    join sys.indexes i with (nowait) on k.unique_index_id = i.index_id and k.parent_object_id = i.object_id
                    WHERE k.parent_object_id = @object_id 
                        AND k.[type] = 'PK'
                ), 
                ''
            );
    

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 2017-12-02
      • 1970-01-01
      • 1970-01-01
      • 2020-09-15
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2019-01-30
      相关资源
      最近更新 更多