Clickhouse 异常：超出内存限制（总计）：28.13 GiB（尝试分配 4543835 字节的块）

Question

我在 clickhouse 数据库中创建了一个带有以下查询的表：

CREATE TABLE events (name String, track_date Date, id UInt64, con_type String, network String) ENGINE = MergeTree() PRIMARY KEY id ORDER BY id;

1. 我正在从 kafka 消费者那里获取数据并解析 JSON 数组以构建查询
2. 尝试每隔 2 秒从 Java 应用程序执行插入查询，如下所示：

   INSERT INTO events(name, track_date , id, con_type , network ) values 
   (ABC,'2021-05-05',111,'android','wxy') , (DEF,'2021-05-05',112,'ios','ewr') , (HJK,'2021-05-05',111,'android','xyz');
   

• 数据大小可能会根据 json 数组长度而变化，可能是更大的插入查询。

在运行 4 天的应用程序后，我得到了 Exception Like：

ru.yandex.clickhouse.except.ClickHouseException: ClickHouse exception, code: 241, host: localhost, port: 8123; Code: 241, e.displayText() = DB::Exception: Memory limit (total) exceeded: would use 28.13 GiB (attempt to allocate chunk of 4543835 bytes), maximum: 28.13 GiB (version 20.8.3.18)

at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:58) at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:28) at ru.yandex.clickhouse.ClickHouseStatementImpl.checkForErrorAndThrow(ClickHouseStatementImpl.java:875) at ru.yandex.clickhouse.ClickHouseStatementImpl.getInputStream(ClickHouseStatementImpl.java:616) at ru.yandex.clickhouse.ClickHouseStatementImpl.executeUpdate(ClickHouseStatementImpl.java:215) at ru.yandex.clickhouse.ClickHousePreparedStatementImpl.executeUpdate(ClickHousePreparedStatementImpl.java:130) at com.way2news.util.DataAppender.pushClickHouse(DataAppender.java:82) at com.way2news.util.KConsumer.run(KConsumer.java:44) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.Throwable: Code: 241, e.displayText() = DB::Exception: Memory limit (total) exceeded: would use 28.13 GiB (attempt to allocate chunk of 4543835 bytes), maximum: 28.13 GiB (version 20.8.3.18)

at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:53) ... 14 more

如何解决此问题。

谢谢，
双马尔。

Answer 1

会不会是 https://github.com/ClickHouse/ClickHouse/issues/15932 的修复没有合并到您使用的 ClickHouse 版本？ 21.1 或更高版本应该有它。