来自json文件的Scipy稀疏

Question

我尝试使用 scipy.sparse 从 json 文件创建一个矩阵。

我有这种方式的json文件

{"reviewerID": "A10000012B7CGYKOMPQ4L", "asin": "000100039X", "reviewerName": "Adam", "helpful": [0, 0], "reviewText": "Spiritually and mentally inspiring! A book that allows you to question your morals and will help you discover who you really are!", "overall": 5.0, "summary": "Wonderful!", "unixReviewTime": 1355616000, "reviewTime": "12 16, 2012"} 

这是我的 Json 格式...更多这样的元素（基于亚马逊评论文件）

并且想要对这个矩阵执行 scipy sparse

    count            
object       a   b   c   d
id                   
him       NaN   1 NaN   1
me          1 NaN NaN   1
you         1 NaN   1 NaN

我正在尝试这样做

我

mport numpy as np
import pandas as pd
from scipy.sparse import csr_matrix

df= pd.read_json('C:\\Users\\anto-\\Desktop\\university\\Big Data computing\\Ex. Resource\\test2.json',lines=True)


a= df['reviewerID']
b= df['asin']
data= df.groupby(["reviewerID"]).size()



row = df.reviewerID.astype('category', categories=a).cat.codes
col = df.asin.astype('category', categories=b).cat.codes
sparse_matrix = csr_matrix((data, (row, col)), shape=(len(a), len(b)))

从这个旧示例中读取

Efficiently create sparse pivot tables in pandas?

我的代码中有一些 deprecates 元素的错误，但我不明白如何构建这个矩阵。

这是错误日志：

 FutureWarning: specifying 'categories' or 'ordered' in .astype() is deprecated; pass a CategoricalDtype instead
  from ipykernel import kernelapp as app

我有点困惑。 任何人都可以给我一些建议或类似的例子吗？

Answer 1

生成一个看起来像

的稀疏矩阵

    count            
object       a   b   c   d
id                   
him       NaN   1 NaN   1
me          1 NaN NaN   1
you         1 NaN   1 NaN

您需要生成 3 个数组，例如：

In [215]: from scipy import sparse
In [216]: data = np.array([1,1,1,1,1,1])
In [217]: row = np.array([1,2,0,2,0,1])
In [218]: col = np.array([0,0,1,2,3,3])
In [219]: M = sparse.csr_matrix((data, (row, col)), shape=(3,4))
In [220]: M
Out[220]: 
<3x4 sparse matrix of type '<class 'numpy.int64'>'
    with 6 stored elements in Compressed Sparse Row format>
In [221]: M.A
Out[221]: 
array([[0, 1, 0, 1],
       [1, 0, 0, 1],
       [1, 0, 1, 0]], dtype=int64)

必须将诸如“他”、“我”、“你”之类的类别映射到诸如 0、1、2 之类的唯一索引上。 'a','b','c','d' 也是如此。