如何在配置单元中为结构字段创建视图

Question

第 1 步： 我编写了一个 UDF，它将形成 2 个或更多 Struct 列，如汽车、自行车、公共汽车。 UDF 还从称为“详细信息”的其他视图中获取一些信息。

cars struct form is: ARRAY<STRUCT<name:string, mfg:string, year:int>>
bikes struct form is: ARRAY<STRUCT<name: string, mfg:string, year: int, price: double>>
buses struct form is: ARRAY<STRUCT<name: string, mfg:string, year: int, price: double>>

我正在使用这个 UDF 创建一个视图“车辆”，如下所示

ADD JAR s3://test/StructFV-0.1.jar;
CREATE TEMPORARY FUNCTION TEST_STRUCT AS "com.test.TestStruct";

CREATE DATABASE IF NOT EXISTS ranjith;
USE ranjith;
DROP VIEW IF EXISTS vehicles;
CREATE VIEW vehicles AS 
SELECT t.cars, t.bikes, t.buses
FROM details d LATERAL VIEW TEST_STRUCT(d.data) t AS
cars, bikes, buses;

第 2 步： 我想将每个结构列分解为另一个视图。 当我尝试以下查询时，我收到类似“AS 子句中提供的别名数与预期的 UDTF 输出的列数不匹配”之类的错误

USE ranjith;
DROP VIEW IF EXISTS cars;
CREATE VIEW cars AS 
SELECT c.name as name, c.mfg as mfg, c.year as year 
FROM vehicles v LATERAL VIEW EXPLODE (v.cars) exploded_table as c;

注意：如果我有只有汽车结构的 UDF，工作正常。仅当 UDF 包含多个 STRUCT 时才会出现问题。

有什么帮助吗？

Answer 1

以下部分不完整的答案，因为您没有包含足够的信息，例如 UDF 名称或您要查询的表的结构/名称，但它应该足以让您继续前进。

您可以创建一个使用 EXPLODE 的视图：

CREATE VIEW productview as 
  SELECT EXPLODE( myudf(.. ).product ) 
      FROM mytable

EXPLODE 将获取一个数组并将其“分解”成一个表。 EXPLODE 实际上是一个 UDTF，即用户定义表生成函数。见https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-explode

Answer 2

USE ranjith;
DROP VIEW IF EXISTS cars;
CREATE VIEW cars AS 
SELECT c.name as name, c.mfg as mfg, c.year, b.name AS bike_name, bb.name AS             bus_name 
FROM vehicles v LATERAL VIEW EXPLODE (v.cars) exploded_table as c LATERAL VIEW     EXPLODE (v.bikes) exploded_table2 AS b LATERAL VIEW EXPLODE (v.buses) exploded_table3 AS bb;

Answer 3

问题在于你的观点

CREATE VIEW vehicles AS 
SELECT t.cars, t.bikes, t.buses
FROM details d LATERAL VIEW TEST_STRUCT(d.data) t AS
cars, bikes, buses;

您的数据类型是结构数组，因此您的横向视图 对于返回的 STRUCT 列，TEST_STRUCT(d.data) 只需要一个别名。

例如

CREATE VIEW vehicles AS 
SELECT columnAlias.cars, columnAlias.bikes, columnAlias.buses
FROM details d LATERAL VIEW TEST_STRUCT(d.data) tableAlias AS columnAlias;