是否有在 python 中保存和加载 h2o word2vec 模型的首选方法？答案

【问题标题】：Is there a preferred method of saving and loading h2o word2vec models in python?是否有在 python 中保存和加载 h2o word2vec 模型的首选方法？
【发布时间】：2020-09-18 02:52:44
【问题描述】：

我已经在 python h2o 包中训练了一个 word2vec 模型。有没有一种简单的方法可以让我保存该 word2vec 模型并稍后将其加载回来以供使用？

我尝试了 h2o.save_model() 和 h2o.load_model() 函数，但没有成功。使用这种方法时出现错误，例如

ERROR: Unexpected HTTP Status code: 412 Precondition Failed (url = http://localhost:54321/99/Models.bin/)

water.exceptions.H2OIllegalArgumentException
[1] "water.exceptions.H2OIllegalArgumentException: Illegal argument: dir of function: importModel:

我正在使用相同版本的 h2o 来训练并重新加载模型，因此此问题中概述的问题不适用 Can't import binay h2o model with h2o.loadModel() function: 412 Precondition Failed

任何人对如何保存和加载 h2o word2vec 模型有任何见解？

我的示例代码和一些重要的 sn-ps

import h2o
from h2o.estimators import H2OWord2vecEstimator

df['text'] = df['text'].ascharacter()
  
# Break text into sequence of words
words = tokenize(df["text"])
    
# Initializing h2o
print('Initializing h2o.')
h2o.init(ip=h2o_ip, port=h2o_port, min_mem_size=h2o_min_memory) 
   
# Build word2vec model:
w2v_model = H2OWord2vecEstimator(sent_sample_rate = 0.0, epochs = 10)
w2v_model.train(training_frame=words)
    
    
# Calculate a vector for each row
word_vecs = w2v_model.transform(words, aggregate_method = "AVERAGE")

#Save model to path
wv_path = '/models/wordvec/'
model_path = h2o.save_model(model = w2v_model, path= wv_path ,force=True)

# Load model in later script
w2v_model = h2o.load_model(model_path)

【问题讨论】：

你尝试了什么确切的代码来得到那个错误？保存和加载是否都因相同的错误而失败？
@gojomo 刚刚添加了示例代码。只有加载失败并出现该错误。可能是我无法以与机器学习模型（例如梯度提升模型）相同的方式保存 word2vec 模型，也许我只能将词向量配对保存在 .txt 文件中？
什么是完整的错误，完整的堆栈，你得到显示的代码？（因为您显示的错误包括一个本地 URL，http://localhost:54321/99/Models.bin/，这在您的代码中可能没有以其他方式显示。）另外，您确定 save_model() 方法需要您提供的目录路径 '/models/wordvec/'而不是文件路径？

标签： python word2vec h2o

【解决方案1】：

听起来您尝试读取的目录可能存在访问问题。我刚刚在w2v example from docs 之后在 H2O 3.30.0.1 上进行了测试，它运行良好：

job_titles = h2o.import_file(("https://s3.amazonaws.com/h2o-public-test-data/smalldata/craigslistJobTitles.csv"),
                              col_names = ["category", "jobtitle"],
                              col_types = ["string", "string"],
                              header = 1)
STOP_WORDS = ["ax","i","you","edu","s","t","m","subject","can",
              "lines","re","what","there","all","we","one","the",
              "a","an","of","or","in","for","by","on","but","is",
              "in","a","not","with","as","was","if","they","are",
              "this","and","it","have","from","at","my","be","by",
              "not","that","to","from","com","org","like","likes",
              "so"]

# Make the 'tokenize' function:
def tokenize(sentences, stop_word = STOP_WORDS):
    tokenized = sentences.tokenize("\\W+")
    tokenized_lower = tokenized.tolower()
    tokenized_filtered = tokenized_lower[(tokenized_lower.nchar() >= 2) | (tokenized_lower.isna()),:]
    tokenized_words = tokenized_filtered[tokenized_filtered.grep("[0-9]",invert=True,output_logical=True),:]
    tokenized_words = tokenized_words[(tokenized_words.isna()) | (~ tokenized_words.isin(STOP_WORDS)),:]
    return tokenized_words

# Break job titles into a sequence of words:
words = tokenize(job_titles["jobtitle"])

# Build word2vec model:
w2v_model = H2OWord2vecEstimator(sent_sample_rate = 0.0, epochs = 10)

w2v_model.train(training_frame=words)

#Save model
wv_path = 'models/'
model_path = h2o.save_model(model = w2v_model, path= wv_path ,force=True)

#Load Model
w2v_model2 = h2o.load_model(model_path)

【讨论】：