【问题标题】:orientdb load graph csv of nodes and edgesorientdb 加载节点和边的图 csv
【发布时间】:2017-11-22 07:01:39
【问题描述】:

我是 Orientdb 的新手。 我有一个包含节点和边的 csv 文件,我需要从该 csv 文件中创建一个图形。 csv 文件

"p1","p2","score"
"LGG_00001","LGG_01682",282
"LGG_00001",".LGG_01831",183
"LGG_00001","LGG_01491",238

边缘是 IsActingWith,它具有 score 属性

{
"source": {
    "file": {
        "path": "C:/Users/sams/Desktop/OrientDB2/lac2.csv"
    }
},
"extractor": {
    "csv": {}
},
"transformers": [
    {
        "vertex": {
            "class": "lac2"
        }
    },
    {
        "vertex": {
            "class": "lac2"
        }
    },

    {
        "edge":
        {
                "class": "IsActingWith",
                "joinFieldName": "score_p",
                "lookup": "acore",
                "direction": "out"
        }
    }

],
"loader": {
    "orientdb": {
        "dbURL": "plocal:C:/Users/sams/Desktop/OrientDB2/database/proj",
        "dbType": "graph",
        "dbAutoCreate": true,
        "classes": [
            {
                "name": "lac2",
                "extends": "V"
            },
            {
                "name": "lac2",
                "extends": "V"
            },
            {
                "name": "IsActingWith",
                "extends": "E"
            },

        ]

    }
}
}

这是我尝试过的,但对我来说似乎不合逻辑。 我正在寻找的最终结果是有一个由p1->ACTINGWITH-> p2ACTINGWITH 组成的图形具有 score 属性的得分

【问题讨论】:

  • 你用的是什么版本?你得到什么输出?
  • 您好,如果您只需要创建一个边类和一个顶点类,那么您的代码已经正确,否则,您是想在单个顶点类属性之间连接还是有两个不同的顶点类和将它们连接在一起?谢谢
  • @MichelaBonizzi 是的,我需要连接由 p1 和 p2 字段组成的 2 个顶点,并且关系(ACTING WITH)作为属性具有分数
  • 所以 p1 和 p2 必须是同一类的 2 个顶点lac2,对吧?
  • @IvanMainetti 是的

标签: csv graph import etl orientdb


【解决方案1】:

也许有更好的解决方案,但这可行。 我的计划是使用 3 个不同的 etl 脚本:第一个和第二个用于插入顶点,第三个用于边缘。当然,您需要按顺序执行它们。

vertex_import_p1.json

{
    "source": { "file": { "path": "/home/ivan/Cose/OrientDB/issues/stack/44641116/file.csv" } },
    "extractor": { "csv": {
        "separator": ",",
    "columns": ["p1:String","p2:String","s:Integer"] } },
    "transformers": [
        { "command": { "command": "UPDATE lac2 set p='${input.p1}' UPSERT WHERE p='${input.p1}'"} }      
    ],
    "loader": {
        "orientdb": {
            "dbURL": "plocal:/home/ivan/Cose/OrientDB/issues/stack/44641116/db",
            "dbUser": "admin",
            "dbPassword": "admin",
            "dbType": "graph",
            "classes": [
                {"name": "lac2", "extends": "V"},
                {"name": "isActingWith", "extends": "E"}
            ]
        }
    }
}

vertex_import_p2.json

{
    "source": { "file": { "path": "/home/ivan/Cose/OrientDB/issues/stack/44641116/file.csv" } },
    "extractor": { "csv": {
        "separator": ",",
    "columns": ["p1:String","p2:String","s:Integer"] } },
    "transformers": [
        { "command": { "command": "UPDATE lac2 set p='${input.p2}' UPSERT WHERE p='${input.p2}'"} }      
    ],
    "loader": {
        "orientdb": {
            "dbURL": "plocal:/home/ivan/Cose/OrientDB/issues/stack/44641116/db",
            "dbUser": "admin",
            "dbPassword": "admin",
            "dbType": "graph",
            "classes": [
                {"name": "lac2", "extends": "V"},
                {"name": "isActingWith", "extends": "E"}
            ]
        }
    }
}

edge_import_s.json

{
    "source": { "file": { "path": "/home/ivan/Cose/OrientDB/issues/stack/44641116/file.csv" } },
    "extractor": { "csv": {
        "separator": ",",
    "columns": ["p1:String","p2:String","s:Integer"] } },
    "transformers": [
        { "command": { "command": "CREATE EDGE isActingWith FROM (SELECT FROM lac2 WHERE p='${input.p1}') TO (SELECT FROM lac2 WHERE p='${input.p2}') set score=${input.s}"} }
    ],
    "loader": {
        "orientdb": {
            "dbURL": "plocal:/home/ivan/Cose/OrientDB/issues/stack/44641116/db",
            "dbUser": "admin",
            "dbPassword": "admin",
            "dbType": "graph",
            "classes": [
                {"name": "lac2", "extends": "V"},
                {"name": "isActingWith", "extends": "E"}
            ]
        }
    }
}

这是处决后的情况:

orientdb {db=db}> select from lac2

+----+-----+------+---------+-------------------+---------------+
|#   |@RID |@CLASS|p        |out_isActingWith   |in_isActingWith|
+----+-----+------+---------+-------------------+---------------+
|0   |#21:6|lac2  |LGG_00001|[#25:5,#26:1,#27:1]|               |
|1   |#21:7|lac2  |LGG_01682|                   |[#25:5]        |
|2   |#22:3|lac2  |LGG_01831|                   |[#26:1]        |
|3   |#23:1|lac2  |LGG_01491|                   |[#27:1]        |
+----+-----+------+---------+-------------------+---------------+

4 item(s) found. Query executed in 0.003 sec(s).
orientdb {db=db}> select from isActingWith

+----+-----+------------+-----+-----+-----+
|#   |@RID |@CLASS      |score|out  |in   |
+----+-----+------------+-----+-----+-----+
|0   |#25:5|isActingWith|282  |#21:6|#21:7|
|1   |#26:1|isActingWith|183  |#21:6|#22:3|
|2   |#27:1|isActingWith|238  |#21:6|#23:1|
+----+-----+------------+-----+-----+-----+

3 item(s) found. Query executed in 0.004 sec(s).

【讨论】:

  • 所以你说如果我想测量插入的总执行时间,我只需要将 3 的总和对吗?
  • 嗨,是的,我想是的
  • 非常感谢您的帖子。我尝试了 4 个小时来加载一个 csv 文件(将其缩减为非常简单的边列表),非常仔细地阅读了所有 ETL 文档,但仍然遇到一些神秘的错误并且非常沮丧。我发现如果我的数据不适合他们的例子,那我就不走运了。
  • @KrisP 希望它在某种程度上有所帮助。遇到同样的问题
  • @IvanMainetti 我发现加载时间过长,这正常吗?
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多