Flink 可查询状态内存不足错误答案

【问题标题】：Flink Queryable State Out Of Memory ErrorFlink 可查询状态内存不足错误
【发布时间】：2021-11-07 06:11:09
【问题描述】：

我正在尝试在 Flink（版本 1.14.0）上使用可查询状态，但不幸的是，我不断收到以下错误：

2021-11-07 11:10:55,795 ERROR org.apache.flink.queryablestate.network.AbstractServerHandler [] - Error while handling request with ID 1
java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Bits.java:175) ~[?:?]
    at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118) ~[?:?]
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317) ~[?:?]
    at org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:632) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:607) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:202) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena.tcacheAllocateSmall(PoolArena.java:172) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena.allocate(PoolArena.java:134) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.shaded.netty4.io.netty.buffer.PoolArena.allocate(PoolArena.java:126) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.shaded.netty4.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:395) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.shaded.netty4.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.shaded.netty4.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.shaded.netty4.io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:139) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.queryablestate.network.NettyBufferPool.ioBuffer(NettyBufferPool.java:95) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.queryablestate.network.messages.MessageSerializer.writePayload(MessageSerializer.java:203) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.queryablestate.network.messages.MessageSerializer.serializeRequest(MessageSerializer.java:96) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.queryablestate.network.ServerConnection$EstablishedConnection.sendRequest(ServerConnection.java:403) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.queryablestate.network.ServerConnection$EstablishedConnection.sendRequest(ServerConnection.java:284) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.queryablestate.network.ServerConnection.sendRequest(ServerConnection.java:91) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.queryablestate.network.Client.sendRequest(Client.java:168) ~[flink-dist_2.12-1.14.0.jar:1.14.0]
    at org.apache.flink.queryablestate.client.proxy.KvStateClientProxyHandler.lambda$getState$2(KvStateClientProxyHandler.java:187) ~[flink-queryable-state-runtime-1.14.0.jar:1.14.0]
    at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072) ~[?:?]
    at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478) [?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
    at java.lang.Thread.run(Thread.java:829) [?:?]

【问题讨论】：

标签： apache-flink flink-streaming

【解决方案1】：

错误本身很清楚。基本上，根据堆栈跟踪中显示的java.lang.OutOfMemoryError，您的内存不足。可能是您没有为作业分配足够的内存，或者超出了该数量。

从广义上讲，可查询状态并非旨在用于任何类型的生产能力（它更适合调试等情况），因此您可能需要考虑它是否最适合您的用例也是。

【讨论】：