为什么 MEMBER OF() 或 JSON_CONTAINS() 不使用多值索引？

Question

我有下表，在 JSON 整数数组上设置了多值索引：

CREATE TABLE test (
  id INT UNSIGNED NOT NULL AUTO_INCREMENT,
  catIds JSON NOT NULL,
  PRIMARY KEY (id),
  KEY test_categories ((CAST(catIds AS UNSIGNED ARRAY)))
);

我已插入 200,000 条记录，例如：

INSERT INTO test (catIds) VALUES('[123, 456]');
...

问题是，在有或没有索引的catIds 字段上查询此表不会改变执行速度。我尝试使用MEMBER OF() 和JSON_CONTAINS() 进行查询，有无索引；速度是一样的。

确实，EXPLAIN 表明这些查询不使用索引：

mysql> EXPLAIN SELECT count(*) FROM test WHERE 51 MEMBER OF (catIds);
+----+-------------+-------+------------+------+---------------+------+---------+------+--------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows   | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+------+--------+----------+-------------+
|  1 | SIMPLE      | test  | NULL       | ALL  | NULL          | NULL | NULL    | NULL | 201416 |   100.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+--------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

mysql> SHOW WARNINGS;
+-------+------+----------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message                                                                                                              |
+-------+------+----------------------------------------------------------------------------------------------------------------------+
| Note  | 1003 | /* select#1 */ select count(0) AS `count(*)` from `test`.`test` where <cache>(51) member of (`test`.`test`.`catIds`) |
+-------+------+----------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> EXPLAIN SELECT count(*) FROM test WHERE JSON_CONTAINS(catIds, '51');
+----+-------------+-------+------------+------+---------------+------+---------+------+--------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows   | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+------+--------+----------+-------------+
|  1 | SIMPLE      | test  | NULL       | ALL  | NULL          | NULL | NULL    | NULL | 201416 |   100.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+--------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

mysql> SHOW WARNINGS;
+-------+------+---------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message                                                                                                                   |
+-------+------+---------------------------------------------------------------------------------------------------------------------------+
| Note  | 1003 | /* select#1 */ select count(0) AS `count(*)` from `test`.`test` where json_contains(`test`.`test`.`catIds`,<cache>('51')) |
+-------+------+---------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

为什么catIds 上的索引不用于这些查询？我错过了什么？

Answer 1

您必须为索引定义和查询中的谓词使用 JSON 路径。

https://dev.mysql.com/doc/refman/8.0/en/create-index.html#create-index-multi-valued 说：

多值键部分中唯一允许的表达式类型是 JSON 路径。路径不必指向插入索引列的 JSON 文档中的现有元素，但本身必须在语法上有效。

我对此进行了测试：

mysql> alter table test add key bk1 ((cast(catIds->'$[*]' as unsigned array)));
Query OK, 0 rows affected (0.07 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> explain SELECT count(*) FROM test WHERE 903 MEMBER OF (catIds->'$[*]');
+----+-------------+-------+------------+------+---------------+------+---------+-------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref   | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+-------+------+----------+-------------+
|  1 | SIMPLE      | test  | NULL       | ref  | bk1           | bk1  | 9       | const |    8 |   100.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+-------+------+----------+-------------+

我毫不怀疑使用此功能会增加WTFs per minute during code reviews。

另外请记住，如果优化器认为索引没有帮助，MySQL 将跳过使用索引。就像表只有几行一样，或者您要搜索的值出现在大多数行中。这并不特定于多值索引，多年来它一直是 MySQL 优化器行为的一部分，具有正常索引。

这是一个示例：我的表中有 4096 行，但它们都相同。即使我搜索表中出现的值，MySQL 也会检测到它将匹配大多数行（在本例中为所有行）并避免索引。

mysql> select distinct catIds from test;
+--------------+
| catIds       |
+--------------+
| [258.0, 7.0] |
+--------------+
1 row in set (0.00 sec)

mysql> select count(*) from test;
+----------+
| count(*) |
+----------+
|     4096 |
+----------+
1 row in set (0.01 sec)

mysql> explain SELECT count(*) FROM test WHERE 258 MEMBER OF (catIds);
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | test  | NULL       | ALL  | NULL          | NULL | NULL    | NULL | 4096 |   100.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+

Answer 2

MySQL 中的多值索引如此缓慢的原因有几个： *) 这些总是二级索引，因此与 InnoDB 中的主索引相比会受到惩罚 *) 与常规索引相同，它指向记录，然后包含必须解包才能处理的 JSON。 *) 与常规索引不同，多值索引不能被覆盖，它们总是必须从表中获取行

所有这些都缩小了多值索引有益的条件。他们的最佳条件是： *) 指标选择性高，越高越好 *) 表中有很多行 *) big json docs - 这不会使索引扫描更快，但由于 innodb 中的行 blob 存储不足，普通扫描会很慢，因此可以使索引扫描更加闪亮