Neo4j：从 neo4j-shell 运行密码查询时出现 OutOfMemoryError

Question

我有一个包含大约 130K 节点的 Neo4j 数据库，并且可能介于 17M 关系之间。我运行 Windows 10 的计算机有 16GB 的 RAM，其中 10GB（最大）分配给 Neo4j-shell 堆。

我想在命令提示符下使用 neo4j-shell 运行查询并将结果重定向到 csv 文件。我使用的命令是：

Neo4jShell -v -file query.cql > results.csv

查询的格式为：

MATCH (subject)-[:type1]->(intNode1:label)<-[:type2]-(intNode2:label)<-[:type3]-(object) RETURN subject.property1, object.property1;

问题是，每当我运行此查询时，都会收到 OutOfMemory 错误（请参阅底部的错误消息）。

有人对如何成功运行这样的查询有建议吗？考虑到图形数据库的大小，我觉得 10GB 的 RAM 应该足以完成这样的查询。

我得到的错误信息是：

ERROR (-v for expanded information):
        Error occurred in server thread; nested exception is:
        java.lang.OutOfMemoryError: GC overhead limit exceeded
java.rmi.ServerError: Error occurred in server thread; nested exception is:
        java.lang.OutOfMemoryError: GC overhead limit exceeded
        at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
        at sun.rmi.transport.Transport$2.run(Unknown Source)
        at sun.rmi.transport.Transport$2.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.access$400(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
        at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown Source)
        at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
        at sun.rmi.server.UnicastRef.invoke(Unknown Source)
        at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(Unknown Source)
        at java.rmi.server.RemoteObjectInvocationHandler.invoke(Unknown Source)
        at com.sun.proxy.$Proxy1.interpretLine(Unknown Source)
        at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:149)
        at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:133)
        at org.neo4j.shell.StartClient.executeCommandStream(StartClient.java:393)
        at org.neo4j.shell.StartClient.grabPromptOrJustExecuteCommand(StartClient.java:372)
        at org.neo4j.shell.StartClient.startRemote(StartClient.java:330)
        at org.neo4j.shell.StartClient.start(StartClient.java:196)
        at org.neo4j.shell.StartClient.main(StartClient.java:135)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

Answer 1

当我遇到这个问题时的解决方案是在 neo4j.properties 的末尾添加这一行

# Default values for the low-level graph engine

    cache_type=none
    neostore.nodestore.db.mapped_memory=50M
    neostore.relationshipstore.db.mapped_memory=500M
    neostore.propertystore.db.mapped_memory=100M
    neostore.propertystore.db.strings.mapped_memory=100M
    neostore.propertystore.db.arrays.mapped_memory=0M

也尝试在neo4j-wrapper.conf中增加java这一行

#wrapper.java.initmemory=2048
#wrapper.java.maxmemory=2048

您在查询的节点上有索引吗？

Answer 2

您可以为 Neo4jShell 提供更多堆（使用 JAVA_OPTS=-Xmx4G -Xms4G -Xmn1G 环境变量）。

您是否尝试使用配置文件运行查询？我认为你跨越了数十亿条路径，因为你没有任何限制。

您错过了主题和客体的标签，这导致查询规划器运行全图扫描。

MATCH (subject:label)-[:type1]->(intNode1:label)
               <-[:type2]-(intNode2:label)
               <-[:type3]-(object:label) 
WITH distinct subject, object
RETURN subject.property1, object.property1;

您应该减少中间基数以及输出基数。

MATCH (subject:label)-[:type1]->(intNode1:label)
               <-[:type2]-(intNode2:label)
WITH distinct subject, intNode2
MATCH (intNode2)<-[:type3]-(object:label) 
WITH distinct subject, object
RETURN subject.property1, object.property1;

更好的是：

MATCH (subject:label)-[:type1]->(intNode1:label)
               <-[:type2]-(intNode2:label)
WITH intNode2, collect(distinct subject) as subjects
MATCH (intNode2)<-[:type3]-(object:label) 
WITH distinct object, subjects
UNWIND subjects as subject
RETURN subject.property1, object.property1;