【发布时间】:2019-01-01 02:30:08
【问题描述】:
我目前正在尝试为项目的一个相当大的表优化索引,并且在解释结果和实际查询运行时之间遇到了非常反直觉的行为。
服务器正在运行 MariaDB 版本 10.1.26-MariaDB-0+deb9u1,具有以下配置选项:
key_buffer_size = 5G
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
myisam_sort_buffer_size = 512M
read_buffer_size = 2M
read_rnd_buffer_size = 1M
query_cache_type = 0
query_cache_limit = 256K
query_cache_min_res_unit = 2k
query_cache_size = 0M
join_buffer_size = 8M
sort_buffer_size = 8M
tmp_table_size = 64M
max_heap_table_size = 64M
table_open_cache = 4K
performance_schema = ON
innodb_buffer_pool_size = 30G
innodb_log_buffer_size = 4MB
innodb_log_file_size = 1G
innodb_buffer_pool_instances = 10
表格包含大约 680 万行,总计 12.1GB,如下所示:
CREATE TABLE `ad_master_test` (
`ID_AD_MASTER` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
/* Some more attribute fields (mainly integers) ... */
`FK_KAT` BIGINT(20) UNSIGNED NOT NULL,
/* Some more content fields (mainly varchars/integers) ... */
`STAMP_START` DATETIME NULL DEFAULT NULL,
`STAMP_END` DATETIME NULL DEFAULT NULL,
PRIMARY KEY (`ID_AD_MASTER`),
INDEX `TEST1` (`STAMP_START`, `FK_KAT`),
INDEX `TEST2` (`FK_KAT`, `STAMP_START`)
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB
ROW_FORMAT=DYNAMIC
AUTO_INCREMENT=14149037;
我已经尽可能简化了查询以更好地说明问题。我在这里使用 FORCE INDEX 来说明我的问题。
第一个索引使用解释语句进行了优化,看起来很有希望(关于解释输出):
SELECT *
FROM `ad_master_test`
FORCE INDEX (TEST1)
WHERE FK_KAT IN
(94169,94163,94164,94165,94166,94167,94168,94170,94171,94172,
94173,94174,94175,94176,94177,94162,99606,94179,94180,94181,
94182,94183,94184,94185,94186,94187,94188,94189,94190,94191,
94192,94193,94194,94195,94196,94197,94198,94199,94200,94201,
94202,94203,94204,94205,94206,94207,94208,94209,94210,94211,
94212,94213,94214,94215,94216,94217,94218,94219,94220,94221,
94222,94223,94224,94225,94226,94227,94228,94229,94230,94231,
94232,94233,94234,94235,94236,94237,94238,94239,94240,94241,
94178,94161)
ORDER BY STAMP_START DESC
LIMIT 24
这个解释的结果:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ad_master_test index (NULL) TEST1 14 (NULL) 24 Using where
还有这个简介:
Status Duration
starting 0.000180
checking permissions 0.000015
Opening tables 0.000041
After opening tables 0.000013
System lock 0.000011
Table lock 0.000013
init 0.000115
optimizing 0.000044
statistics 0.000050
preparing 0.000039
executing 0.000009
Sorting result 0.000016
Sending data 4.827512
end 0.000023
query end 0.000008
closing tables 0.000004
Unlocking tables 0.000014
freeing items 0.000011
updating status 0.000132
cleaning up 0.000021
第二个索引只是反转的字段(我在这里理解它的方式:https://dev.mysql.com/doc/refman/8.0/en/order-by-optimization.html)看起来非常可怕(关于解释输出):
SELECT *
FROM `ad_master_test`
FORCE INDEX (TEST2)
WHERE FK_KAT IN (94169,94163,94164,94165,94166,94167,94168,94170,94171,94172,94173,94174,94175,94176,94177,94162,99606,94179,94180,94181,94182,94183,94184,94185,94186,94187,94188,94189,94190,94191,94192,94193,94194,94195,94196,94197,94198,94199,94200,94201,94202,94203,94204,94205,94206,94207,94208,94209,94210,94211,94212,94213,94214,94215,94216,94217,94218,94219,94220,94221,94222,94223,94224,94225,94226,94227,94228,94229,94230,94231,94232,94233,94234,94235,94236,94237,94238,94239,94240,94241,94178,94161)
ORDER BY STAMP_START DESC
LIMIT 24
这个解释的结果:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ad_master_test range TEST2 TEST2 8 (NULL) 497.766 Using index condition; Using filesort
还有这个简介:
Status Duration
starting 0.000087
checking permissions 0.000007
Opening tables 0.000021
After opening tables 0.000007
System lock 0.000006
Table lock 0.000005
init 0.000058
optimizing 0.000023
statistics 0.000654
preparing 0.000480
executing 0.000008
Sorting result 0.433607
Sending data 0.001681
end 0.000010
query end 0.000007
closing tables 0.000003
Unlocking tables 0.000011
freeing items 0.000010
updating status 0.000158
cleaning up 0.000021
编辑:当不使用强制索引时解释变化如下:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ad_master_test index TEST2 TEST1 14 (NULL) 345 Using where
配置文件和运行时保持(如预期)与在 TEST1 索引上使用 FORCE INDEX 时相同。
/编辑
老实说,我无法理解这一点。为什么解释和实际查询性能差异如此之大。 5秒“发送数据”期间服务器做了什么?
【问题讨论】:
标签: mysql mariadb query-optimization innodb indices