SQLite 索引导致查询性能下降

Question

我正忙着通过测试向查询中的表添加索引的结果来为 1 个大型查询优化 SQLite 数据库。

我注意到的第一件事是索引 beserk 降低了性能，因此我删除了所有索引并缩小了数据库并开始测试一次应用 1 个索引的性能。

这样做我发现了一个索引，它可以将查询速度提高大约 50%。然后，我将此索引应用于另一个 SQLite 数据库（相同的结构只是更多数据），其数据量约为 5 倍（500MB），现在我发现相同的索引会使更大数据库中的性能降低约 35%。

所以我对 PRAGMA cache_size 进行了一些阅读，我了解到，如果您的表很大并且您对其应用索引，那么缓存将很快被索引用完，这会降级表现。所以也许改变这个尺寸会有所帮助......

所以我的问题是，为什么索引在样本数据集较大的数据库中比在数据集较小的数据库中会降低性能？

更多信息：

我的查询只是一个选择查询，数据库只是读取，从不写入（显然第一次填充时除外）。 我应用的索引位于单个整数列上。

查询：

select distinct

u.userName 'User Name',
u.description 'User Description',
ma.abilityName 'Ability',
ifnull(a.applicationName, '') 'Module',
ifnull(rr3.roleName, '') 'Parent Role',
r.responsibilityName 'Responsibility',
r.description 'Responsibility Description',
ff.functionName 'Technical Function Name',
ifnull(ff.userFunctionName, '') 'Function Name',
ff.description 'Function Description',
ff.type 'Function Type',             
ifnull(f.formName, '') 'Form',
ifnull(f.userFormName, '') 'Form Name',             
case ifnull(me.isProcessTab, 0) when 0 then 'N' else 'Y' end as 'Is From Process Tab?',
m.menuName 'Menu Name',
m.menuDisplayName 'Menu Description',
ifnull(sg.securityGroupName, '') 'Data Group Name',
ifnull(rg.requestGroupName, '') 'Request Group Name',
mpath.path as 'Full Menu Path',              
cast(ma.ID as nvarchar) || ':' || cast(u.ID as nvarchar) as 'Ledger Mapping'

from matrix_processes  mp   
inner join matrix_processAbilities  mpa on mpa.processID = mp.ID
inner join matrix_abilities ma on ma.ID = mpa.abilityID
inner join matrix_abilityFunctions maf on maf.abilityID = ma.ID
inner join matrix_functions mf on mf.ID = maf.functionID  --mf.functionName = ff.functionName
inner join formFunctions  ff on ff.functionName = mf.functionName
inner join users_functions uf on uf.functionID = ff.ID
inner join users_responsibilities as ur on ur.userID = uf.userID           
inner join responsibilities_menus_hierarchy_functions  rmhf on rmhf.responsibilityID = ur.responsibilityID and rmhf.functionID = ff.ID         

inner join users as u on u.ID = uf.userID
inner join responsibilities as r on r.ID = ur.responsibilityID

left join requestGroups as rg on rg.ID = r.requestGroupID
left join securityGroups as sg on sg.securityGroupKey = r.dataGroupKey

inner join menus_hierarchy  mh on mh.ID = rmhf.menuHierarchyID
inner join menus  as m on m.id = mh.menuParentID
inner join menuPaths mpath  on mpath.ID = mh.menuPathID
left join menuEntries me on me.menuID = m.ID

left join applications as a on a.ID = r.applicationID
left join forms  as f on f.ID = ff.formID

left join roles_responsibilities rr1 on rr1.responsibilityID = r.ID
left join roles_roles rr2 on rr2.roleID = rr1.roleID
left join roles rr3 on rr3.ID = rr2.roleParentID

where mp.ID = 1033

order by u.userName,
      ma.abilityName,
      r.responsibilityName,
      ff.userFunctionName;

查询计划：

0   0   0   SCAN TABLE matrix_processes AS mp (~100000 rows)
0   1   1   SEARCH TABLE matrix_processAbilities AS mpa USING AUTOMATIC COVERING INDEX (processID=?) (~7 rows)
0   2   2   SEARCH TABLE matrix_abilities AS ma USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   3   3   SEARCH TABLE matrix_abilityFunctions AS maf USING AUTOMATIC COVERING INDEX (abilityID=?) (~7 rows)
0   4   4   SEARCH TABLE matrix_functions AS mf USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   5   5   SEARCH TABLE formFunctions AS ff USING AUTOMATIC COVERING INDEX (functionName=?) (~7 rows)
0   6   6   SEARCH TABLE users_functions AS uf USING INDEX IX_UserAccessDetailsOpt_users_functions2 (functionID=?) (~10 rows)
0   7   7   SEARCH TABLE users_responsibilities AS ur USING AUTOMATIC COVERING INDEX (userID=?) (~7 rows)
0   8   8   SEARCH TABLE responsibilities_menus_hierarchy_functions AS rmhf USING AUTOMATIC COVERING INDEX (responsibilityID=? AND functionID=?) (~7 rows)
0   9   9   SEARCH TABLE users AS u USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   10  10  SEARCH TABLE responsibilities AS r USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   11  11  SEARCH TABLE requestGroups AS rg USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   12  12  SEARCH TABLE securityGroups AS sg USING AUTOMATIC COVERING INDEX (securityGroupKey=?) (~7 rows)
0   13  13  SEARCH TABLE menus_hierarchy AS mh USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   14  14  SEARCH TABLE menus AS m USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   15  15  SEARCH TABLE menuPaths AS mpath USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   16  16  SEARCH TABLE menuEntries AS me USING AUTOMATIC COVERING INDEX (menuID=?) (~7 rows)
0   17  17  SEARCH TABLE applications AS a USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   18  18  SEARCH TABLE forms AS f USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   19  19  SEARCH TABLE roles_responsibilities AS rr1 USING AUTOMATIC COVERING INDEX (responsibilityID=?) (~7 rows)
0   20  20  SEARCH TABLE roles_roles AS rr2 USING AUTOMATIC COVERING INDEX (roleID=?) (~7 rows)
0   21  21  SEARCH TABLE roles AS rr3 USING AUTOMATIC COVERING INDEX (ID=?) (~7 rows)
0   0   0   USE TEMP B-TREE FOR DISTINCT
0   0   0   USE TEMP B-TREE FOR ORDER BY

Answer 1

AUTOMATIC INDEX 表示 SQLite 认为创建临时索引是值得的（即使有时间收集所有表数据并为索引排序）。 写入和排序所有数据所需的时间以 O(n log n) 为单位。

您应该能够通过将所有这些索引永久添加到数据库中来加速此查询：

CREATE INDEX mpa_processID ON matrix_processAbilities(processID);
CREATE INDEX ma_ID ON matrix_abilities(ID);
...

此外，查找where mp.ID = 1033 可以通过此列上的索引来加速。 （优化器认为为此创建一个临时索引是不值得的，因为该表只被搜索一次。）

请注意，任何作为主键且具有整数类型的 ID 列都应在表定义中声明为 INTEGER PRIMARY KEY，这比单独的索引更有效。