【发布时间】:2017-09-09 20:04:21
【问题描述】:
使用循环逐行执行此函数是可行的。使用 pandas.DataFrame.apply 执行相同的函数会返回 ValueError:操作数无法与形状一起广播。 pandas.DataFrame.apply 应该工作吗?如果这是不容易解释的事情之一,关于如何加快处理速度(多处理除外)的任何想法?
#python 3.6
import pandas as pd # version 0.19.2
import numpy as np #
#gensim version 1.0.1
from gensim import models #https://radimrehurek.com/gensim/models/word2vec.html
df=pd.DataFrame({"q1":[['how', 'I', 'from', 'iPhone', 'keep', 'them', 'my', 'but', 'delete', 'iCloud', 'photos', 'in', 'can'],
['use', 'are', 'radio', 'What', 'commercial', 'cognitive', 'technology', 'in'],
['how', 'I', 'razor', 'prevent', 'burns', 'the', 'stomach', 'on', 'can']],
"q2":[['Can', 'remove', 'from', 'I', 'iPhone', 'removing', 'them', 'my', 'storage', 'photos', 'iCloud', 'without'],
['radio', 'from', 'Where', 'do', 'come', 'cognitive', 'distinction'],
['how', 'I', 'razor', 'prevent', 'can', 'burn']]})
#using pretrained model https://code.google.com/archive/p/word2vec/
w2v = models.KeyedVectors.load_word2vec_format('GoogleNews-vectors-negative300.bin', binary=True)
#This works
df['w2v_sim']=np.nan
for i in range(len(df)):
df['w2v_sim'].ix[i]=w2v.n_similarity(df['q1'].ix[i],df['q2'].ix[i])
print(str(df['w2v_sim'].ix[i]))
#this doesn't work
df['w2v_sim']=np.nan
df['w2v_sim']=df.apply(w2v.n_similarity(df['q1'],df['q2']),axis=1)
ValueError: 操作数无法与形状一起广播 (13,300) (8,300)
谢谢
【问题讨论】:
标签: python-3.x pandas gensim word2vec