连接 YugabyteDB 中的 8 个表答案

【问题标题】：Joins with 8 tables in YugabyteDB连接 YugabyteDB 中的 8 个表
【发布时间】：2022-10-18 16:37:59
【问题描述】：

我有一个数据集和一个查询，在 10 到 17,000 行之间的表上有 8 个连接，查询返回 4,000 行。我的问题是它非常慢。没有明显的方法可以以更优化的方式重写查询。它只对索引列进行等连接。我试着给它提示，但它们没有被使用。作为比较，我在 PostgreSQL 中加载了相同的集合，它在 < 1 秒内返回。 YB vs Postgres 在这方面的查询计划是完全不同的。我使用了一个单实例 postgres 与 3 个 master 和 3 个 tserver，最新版本，所有 VM 都有 1 cpu 和 3 GB 内存。我意识到 cpu/内存偏低，但没有完全使用，如果这是瓶颈，我会在更大的实例上运行它。你能给一些建议吗？我在哪里寻找优化这个查询？您对进行多次连接的查询有何经验？查询计划如下：

explain (costs off, analyze, verbose) SELECT
    a.ptflio_clstr_mstr_key,
    a.ptflio_clstr_nm,
    a.reference_id,
    pku.kpc_usr_nm,
    a.chng_usr,
    clstrhist.chng_dttm,
    a.ptflio_clstr_desc,
    b.ptflio_bld_blck_grp_mstr_key,
    b.ptflio_bld_blck_grp_nm,
    f.prd_offr_mstr_key,
    f.prd_offr_nm,
    d.atmc_prd_offr_type_ind,
    offrhist.chng_dttm
FROM product_store.pm_ptflio_clstr a
LEFT JOIN product_store.pm_ptflio_clstr_hist clstrhist ON a.ptflio_clstr_mstr_key = clstrhist.ptflio_clstr_mstr_key AND clstrhist.row_seq=1
LEFT JOIN product_store.pm_kpc_usr pku on a.kpc_usr_id = pku.kpc_usr_id
JOIN product_store.pm_ptflio_bld_blck_grp b ON a.ptflio_clstr_mstr_key = b.ptflio_clstr_mstr_key
JOIN product_store.pm_ptflio_bld_blck c ON b.ptflio_bld_blck_grp_mstr_key = c.ptflio_bld_blck_grp_mstr_key
LEFT JOIN product_store.pm_atmc_prd_offr d ON c.ptflio_bld_blck_mstr_key = d.ptflio_bld_blck_mstr_key
LEFT JOIN product_store.pm_prd_offr f ON f.prd_offr_mstr_key = d.atmc_prd_offr_mstr_key
LEFT JOIN product_store.pm_prd_offr_hist offrhist ON f.prd_offr_mstr_key = offrhist.prd_offr_mstr_key AND offrhist.row_seq = 1
WHERE a.ptflio_clstr_mstr_key='b3000fbe-65d5-4c68-9758-0954c7f9a0f1';
Nested Loop Left Join (actual time=12.719..10427.074 rows=3835 loops=1)
  Output: a.ptflio_clstr_mstr_key, a.ptflio_clstr_nm, a.reference_id, pku.kpc_usr_nm, a.chng_usr, clstrhist.chng_dttm, a.ptflio_clstr_desc, b.ptflio_bld_blck_grp_mstr_key, b.ptflio_bld_blck_grp_nm, f.prd_offr_mstr_key, f.prd_offr_nm, d.atmc_prd_offr_type_ind, offrhist.chng_dttm
  Inner Unique: true
  ->  Nested Loop Left Join (actual time=12.338..8896.474 rows=3835 loops=1)
        Output: a.ptflio_clstr_mstr_key, a.ptflio_clstr_nm, a.reference_id, a.chng_usr, a.ptflio_clstr_desc, clstrhist.chng_dttm, pku.kpc_usr_nm, b.ptflio_bld_blck_grp_mstr_key, b.ptflio_bld_blck_grp_nm, d.atmc_prd_offr_type_ind, f.prd_offr_mstr_key, f.prd_offr_nm
        Inner Unique: true
        ->  Nested Loop (actual time=11.903..7197.184 rows=3835 loops=1)
              Output: a.ptflio_clstr_mstr_key, a.ptflio_clstr_nm, a.reference_id, a.chng_usr, a.ptflio_clstr_desc, clstrhist.chng_dttm, pku.kpc_usr_nm, b.ptflio_bld_blck_grp_mstr_key, b.ptflio_bld_blck_grp_nm, d.atmc_prd_offr_type_ind, d.atmc_prd_offr_mstr_key
              ->  Nested Loop Left Join (actual time=1.755..1.759 rows=1 loops=1)
                    Output: a.ptflio_clstr_mstr_key, a.ptflio_clstr_nm, a.reference_id, a.chng_usr, a.ptflio_clstr_desc, clstrhist.chng_dttm, pku.kpc_usr_nm
                    Inner Unique: true
                    ->  Nested Loop Left Join (actual time=1.298..1.301 rows=1 loops=1)
                          Output: a.ptflio_clstr_mstr_key, a.ptflio_clstr_nm, a.reference_id, a.chng_usr, a.ptflio_clstr_desc, a.kpc_usr_id, clstrhist.chng_dttm
                          Inner Unique: true
                          Join Filter: (a.ptflio_clstr_mstr_key = clstrhist.ptflio_clstr_mstr_key)
                          ->  Index Scan using xpkportfolio_cluster on product_store.pm_ptflio_clstr a (actual time=0.764..0.767 rows=1 loops=1)
                                Output: a.ptflio_clstr_mstr_key, a.row_seq, a.ptflio_clstr_nm, a.ptflio_clstr_desc, a.mstr_stat_cd, a.chng_usr, a.chng_dttm, a.chng_rmrk, a.reference_id, a.kpc_usr_id
                                Index Cond: (a.ptflio_clstr_mstr_key = 'b3000fbe-65d5-4c68-9758-0954c7f9a0f1'::uuid)
                          ->  Index Scan using pm_ptflio_clstr_hist_pkey on product_store.pm_ptflio_clstr_hist clstrhist (actual time=0.529..0.529 rows=0 loops=1)
                                Output: clstrhist.ptflio_clstr_mstr_key, clstrhist.row_seq, clstrhist.ptflio_clstr_nm, clstrhist.ptflio_clstr_desc, clstrhist.mstr_stat_cd, clstrhist.chng_usr, clstrhist.chng_dttm, clstrhist.chng_rmrk, clstrhist.reference_id, clstrhist.kpc_usr_id
                                Index Cond: ((clstrhist.ptflio_clstr_mstr_key = 'b3000fbe-65d5-4c68-9758-0954c7f9a0f1'::uuid) AND (clstrhist.row_seq = 1))
                    ->  Index Scan using xpkkpc_user on product_store.pm_kpc_usr pku (actual time=0.453..0.453 rows=0 loops=1)
                          Output: pku.kpc_usr_id, pku.ruisnaam, pku.kpc_usr_nm, pku.kpc_usr_act_ind, pku.chng_usr, pku.chng_dttm
                          Index Cond: (a.kpc_usr_id = pku.kpc_usr_id)
              ->  Nested Loop (actual time=10.131..7193.091 rows=3835 loops=1)
                    Output: b.ptflio_bld_blck_grp_mstr_key, b.ptflio_bld_blck_grp_nm, b.ptflio_clstr_mstr_key, d.atmc_prd_offr_type_ind, d.atmc_prd_offr_mstr_key
                    Inner Unique: true
                    ->  Hash Right Join (actual time=9.673..73.707 rows=16878 loops=1)
                          Output: c.ptflio_bld_blck_grp_mstr_key, d.atmc_prd_offr_type_ind, d.atmc_prd_offr_mstr_key
                          Inner Unique: true
                          Hash Cond: (d.ptflio_bld_blck_mstr_key = c.ptflio_bld_blck_mstr_key)
                          ->  Seq Scan on product_store.pm_atmc_prd_offr d (actual time=8.124..45.709 rows=16865 loops=1)
                                Output: d.atmc_prd_offr_mstr_key, d.row_seq, d.ptflio_bld_blck_mstr_key, d.lcm_phase_cd, d.lcm_phase_start_dttm, d.lcm_phase_end_dttm, d.atmc_prd_offr_type_ind, d.chng_usr, d.chng_dttm, d.lcm_phase_desc, d.lcm_phase_alert_dttm, d.lcm_phase_approved_by, d.lcm_phase_master_key
                          ->  Hash (actual time=1.534..1.534 rows=43 loops=1)
                                Output: c.ptflio_bld_blck_grp_mstr_key, c.ptflio_bld_blck_mstr_key
                                Buckets: 1024  Batches: 1  Memory Usage: 11kB
                                ->  Seq Scan on product_store.pm_ptflio_bld_blck c (actual time=0.531..1.523 rows=43 loops=1)
                                      Output: c.ptflio_bld_blck_grp_mstr_key, c.ptflio_bld_blck_mstr_key
                    ->  Index Scan using xpkportfolio_building_block_gr on product_store.pm_ptflio_bld_blck_grp b (actual time=0.403..0.403 rows=0 loops=16878)
                          Output: b.ptflio_bld_blck_grp_mstr_key, b.row_seq, b.ptflio_bld_blck_grp_nm, b.ptflio_bld_blck_grp_desc, b.ptflio_clstr_mstr_key, b.mstr_stat_cd, b.chng_usr, b.chng_dttm, b.chng_rmrk, b.reference_id, b.kpc_usr_id
                          Index Cond: (b.ptflio_bld_blck_grp_mstr_key = c.ptflio_bld_blck_grp_mstr_key)
                          Filter: (b.ptflio_clstr_mstr_key = 'b3000fbe-65d5-4c68-9758-0954c7f9a0f1'::uuid)
                          Rows Removed by Filter: 1
        ->  Index Scan using xpkproduct_offering on product_store.pm_prd_offr f (actual time=0.419..0.419 rows=1 loops=3835)
              Output: f.prd_offr_mstr_key, f.row_seq, f.prd_offr_nm, f.prop_mod_mstr_key, f.clstr_dsct_prd_offr_grp_cd, f.trgt_ptflio_ind, f.price_brd_nmbr, f.price_brd_stat_cd, f.prd_offr_lnup, f.pm_prd_offr_type_cd, f.comm_prd_id, f.chng_usr, f.chng_dttm, f.reference_id, f.version, f.kpc_usr_id, f.generation
              Index Cond: (f.prd_offr_mstr_key = d.atmc_prd_offr_mstr_key)
  ->  Index Scan using pm_prd_offr_hist_pkey on product_store.pm_prd_offr_hist offrhist (actual time=0.375..0.375 rows=0 loops=3835)
        Output: offrhist.prd_offr_mstr_key, offrhist.row_seq, offrhist.prd_offr_nm, offrhist.prop_mod_mstr_key, offrhist.clstr_dsct_prd_offr_grp_cd, offrhist.trgt_ptflio_ind, offrhist.price_brd_nmbr, offrhist.price_brd_stat_cd, offrhist.prd_offr_lnup, offrhist.pm_prd_offr_type_cd, offrhist.comm_prd_id, offrhist.chng_usr, offrhist.chng_dttm, offrhist.reference_id, offrhist.version, offrhist.kpc_usr_id, offrhist.generation
        Index Cond: ((f.prd_offr_mstr_key = offrhist.prd_offr_mstr_key) AND (offrhist.row_seq = 1))
Planning Time: 0.777 ms
Execution Time: 10655.916 ms

【问题讨论】：

标签： yugabytedb

【解决方案1】：

查询调优的通用方法称为“通过消除一次性的查询调优”http://docplayer.net/20177036-Query-tuning-by-eliminating-throwaway.html，这是丹麦的 Martin Berg 撰写的一篇论文的名称（针对 Oracle 数据库，但通用方法适用于所有数据库）使用带有行源的查询计划）。最重要的是，像 YugabyteDB 这样的分布式数据库有一些细节，其中像嵌套循环这样的行源可能会导致比单体数据库更高的开销，因为在单体中，一切都是本地的。

默认情况下，YugabyteDB 中未启用使提示工作所需的 pg_hint_plan 扩展，但它已安装在 YugabyteDB 服务器软件中。这意味着要启用它，您需要在一个 YugabyteDB 数据库中运行 create extension pg_hint_plan; 来为集群启用它。

在〜 10 秒中，〜 6 秒位于此连接中：

JOIN product_store.pm_ptflio_bld_blck c ON b.ptflio_bld_blck_grp_mstr_key = c.ptflio_bld_blck_grp_mstr_key

使用索引xpkportfolio_building_block_gr。阅读product_store.pm_ptflio_bld_blck_grp b 16878 次，找不到任何行。从它开始可能会更好（我猜b.ptflio_clstr_mstr_key = 'b3000fbe-65d5-4c68-9758-0954c7f9a0f1'::uuid 上的谓词是高度选择性的）： /*+ Leading( (b c) ) */ 可能是一个好的开始，但由于外连接可能无法实现。

所以也许/*+ Leading( (c b) ) HashJoin(c b) */。取决于你的数据。目标是从最有选择性的表开始，然后加入那些减少行数的表，然后是其他表。

【讨论】：