雪花如何循环对象数组并将数据合并到相关表中而不将数组移动到临时表中？答案

【问题标题】：Snowflake how to loop over an array of objects and merge data into related tables without moving the array into a temporary table?雪花如何循环对象数组并将数据合并到相关表中而不将数组移动到临时表中？
【发布时间】：2021-02-05 11:57:37
【问题描述】：

我有一个具有以下结构的数组：

var array = [
    {
        "dim_label": "...",
        "dim_name": "...",
        "field_label": "...",
        "field_name": "..."
    },
    {
        "dim_label": "...",
        "dim_name": "...",
        "field_label": "...",
        "field_name": "..."
    }
]

我需要在 Snwoflake 过程中循环它并将必要的行合并到它们的相关表中。我当前的程序是从 CSV 暂存文件中读取并像这样进行合并：

var field_label_query = "MERGE INTO SAT_FIELD AS D "+
                "USING (SELECT T.$"+FIELD_LABEL_POSITION+", T.$"+FIELD_ONA_POSITION+" FROM "+FILE_FULL_PATH+"(FILE_FORMAT=>"+FILE_FORMAT_NAME+") T) ST "+
                "ON D.FIELD_NAME_HASH_KEY = md5(ST.$"+FIELD_ONA_POSITION+") "+
                "WHEN NOT MATCHED THEN "+
                "INSERT (SAT_FIELD_HASH_KEY, LOAD_DT, LOAD_END_DT, RECORD_SRC, FIELD_LABEL_NAME, FIELD_NAME_HASH_KEY) "+
                "VALUES(MD5(ST.$"+FIELD_LABEL_POSITION+"), current_timestamp(), NULL, 'ONA', ST.$"+FIELD_LABEL_POSITION+", md5(ST.$"+FIELD_ONA_POSITION+"))";

是否可以在不将数组数据添加到临时表的情况下进行合并？

【问题讨论】：

您是否尝试过仅使用数组的LATERAL FLATTEN？您可以在 USING 选择中使用 CTE。
雪花中的 CTE 是什么？
@MikeWalton 希望您能帮上忙，先生。

标签： snowflake-cloud-data-platform

【解决方案1】：

我使用the answer in this post找到了我的解决方案：

with cte_dim_fields as ( 
    select parse_json('JSON.stringify(dim_field_array)+')::VARIANT as field_var 
), cte_arr_to_table as ( 
    select cs.value:dim_label as dim_label, 
    cs.value:dim_name as dim_name, 
    cs.value:field_label as field_label, 
    cs.value:field_name as field_name 
    from cte_dim_fields, 
    lateral flatten(input=>cte_dim_fields.field_var) cs 
) 
select dim_label, dim_name, field_label, field_name from cte_arr_to_table;

使用 CTE 进行插入：

INSERT INTO my_table with cte_dim_fields as ( 
    select parse_json('JSON.stringify(dim_field_array)+')::VARIANT as field_var 
), cte_arr_to_table as ( 
    select cs.value:dim_label as dim_label, 
    cs.value:dim_name as dim_name, 
    cs.value:field_label as field_label, 
    cs.value:field_name as field_name 
    from cte_dim_fields, 
    lateral flatten(input=>cte_dim_fields.field_var) cs 
) 
select dim_label, dim_name, field_label, field_name from cte_arr_to_table;

【讨论】：