将序列列表的列表转换为一个热点答案

【问题标题】：convert the list of sequence list to one hot将序列列表的列表转换为一个热点
【发布时间】：2020-06-23 03:10:04
【问题描述】：

我有以下示例：我希望将每个序列列表列表转换为一个热编码器。

例如，我有一个包含 2 个句子的列表。我首先将这些句子转换为序列列表。

然后对于序列列表的每个列表，然后我根据每个单词将序列转换为一个热门。

from nltk.tokenize import word_tokenize
from itertools import chain
from keras.preprocessing.sequence import pad_sequences

a = ['hi', 'oh thanks i m fine this is an evening in my timezone']
a_tokens = [word_tokenize(word) for word in a]
tokens_dict = {word:i for i, word in enumerate(set(chain.from_iterable(a_tokens)))}
tokens_sequence = [[tokens_dict[word_t] for word_t in word] for word in a_tokens]

当前输出：

[[4], [2, 5, 3, 1, 8, 7, 9, 0, 12, 10, 11, 6]]

预期输出：

[[[0,0,0,0,0,0,0,0,0,0,0,0,1]],
 [[-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-],
  [-12 0s but 1 for the repsective word-]]]

【问题讨论】：

标签： python scikit-learn one-hot-encoding

【解决方案1】：

您可以使用 keras.utils.np_utils 的 to_categorical 来获取标签的 one_hot_vectors，如下所示：

from nltk.tokenize import word_tokenize
from itertools import chain
from keras.preprocessing.sequence import pad_sequences
import numpy as np
from keras.utils.np_utils import to_categorical

a = ['hi', 'oh thanks i m fine this is an evening in my timezone']
a_tokens = [word_tokenize(word) for word in a]
tokens_dict = {word:i for i, word in enumerate(set(chain.from_iterable(a_tokens)))}
tokens_sequence = [[tokens_dict[word_t] for word_t in word] for word in a_tokens]

labels=np.array(tokens_sequence)

max_label=max([max(l) for l in labels]) # get maximum value in labels= the label of the word with highest label; here is 12

one_hot_labels=[]
for label in labels: 
    label.append(max_label) # add the label of the word with highest label
    one_hot=to_categorical(label,dtype=np.int32)   #get one-hot-labels  
    one_hot_labels.append( one_hot[:-1]) # remove one-hot of the word with highest label and add reaming into the list 
    
one_hot_labels=np.array(one_hot_labels)
print(one_hot_labels)

希望这会有所帮助。

【讨论】：