如何将密集向量转换为pyspark中的数据框？

Question

我正在尝试转换下面的密集向量，我通过获取多个线性回归模型的系数得到。我想将其转换为数据框

lr_coefficients = lr_model.coefficients
lr_coefficients.append(lr_coefficients)
lr_coefficients

[DenseVector([-0.0009, -0.2476, 0.5486, 0.396]),
 DenseVector([-0.0016, -1.5333, 0.4269, 0.4363]),
 DenseVector([-0.0492, 0.0, 0.2077, 0.7548]),
 DenseVector([-0.001, -1.2098, 0.545, 0.4148]),
 DenseVector([-0.0001, 0.0, 0.575, 0.3638]),
 DenseVector([-0.001, -1.3361, 0.5402, 0.4113]),
 DenseVector([-0.0049, -1.5534, 0.5747, 0.3934]),
 DenseVector([-0.0049, -1.5534, 0.5747, 0.3934]),
 DenseVector([-0.0049, -1.5534, 0.5747, 0.3934]),
 DenseVector([-0.0049, -1.5534, 0.5747, 0.3934]),
 DenseVector([-0.0049, -1.5534, 0.5747, 0.3934]),
 DenseVector([-0.0049, -1.5534, 0.5747, 0.3934])] 

我想要列中的每个系数。如下表所示

我尝试了以下链接，但对我不起作用。

Convert a Dense Vector to a Dataframe using Pyspark

Answer 1

嗯，你没有提到你是如何尝试的。可能您遇到的问题是您有一个密集向量列表。所以 toArray() 函数必须应用于每个元素

tst_vct = [DenseVector([6603.0, 332.0, 65.8, -0.19]),
           DenseVector([6613.0, 514.0, 60.7, -0.1238]),
           DenseVector([6708.0, 487.0, 60.6, -0.1481]),
           DenseVector([6446.0, 2538.0, 14.0, -0.0178])]
# Convert each vector to array
tst_arr=[x.toArray().tolist() for x in tst_vct]
# create a dataframe from the list
tst_df= sqlContext.createDataFrame(tst_arr)
tst_df.show()
+------+------+------------------+--------------------+
|    _1|    _2|                _3|                  _4|
+------+------+------------------+--------------------+
|6603.0| 332.0| 65.80000000000001| -0.1900000000000067|
|6613.0| 514.0| 60.70000000000002| -0.1238281250000007|
|6708.0| 487.0|60.600000000000016| -0.1481404958677686|
|6446.0|2538.0|              14.0|-0.01775147928994083|
+------+------+------------------+--------------------+

你去吧:-)