AttributeError: 'str' 对象在使用带有 PyTorch 的 BertModel 编码张量时没有属性 'shape'（拥抱脸）

Question

AttributeError: 'str' object has no attribute 'shape' 同时使用 BertModel 和 PyTorch（拥抱脸）对张量进行编码。下面是代码

bert_model = BertModel.from_pretrained(r'downloads\bert-pretrained-model')
input_ids

输出是：

tensor([[  101,   156, 13329,  ...,     0,     0,     0],
        [  101,   156, 13329,  ...,     0,     0,     0],
        [  101,  1302,  1251,  ...,     0,     0,     0],
        ...,
        [  101, 25456,  1200,  ...,     0,     0,     0],
        [  101,   143,  9664,  ...,     0,     0,     0],
        [  101,  2586,  7340,  ...,     0,     0,     0]])

下面是代码

last_hidden_state, pooled_output = bert_model(
  input_ids=encoding['input_ids'],
  attention_mask=encoding['attention_mask']
)

下面是代码

last_hidden_state.shape

输出是

AttributeError                            Traceback (most recent call last)
<ipython-input-70-9628339f425d> in <module>
----> 1 last_hidden_state.shape

AttributeError: 'str' object has no attribute 'shape'

完整的代码链接是'https://colab.research.google.com/drive/1FY4WtqCi2CQ9RjHj4slZwtdMhwaWv2-2?usp=sharing'

Answer 1

问题是return type has changed 自 3.xx 版本的转换器。所以，我们明确要求张量的元组。

所以，当我们调用bert_model() 时，我们可以传递一个额外的kwarg return_dict = False 来获得一个对应于last_hidden_state 的实际张量。

last_hidden_state, pooled_output = bert_model(
  input_ids=encoding['input_ids'],
  attention_mask=encoding['attention_mask'],
  return_dict = False   # this is needed to get a tensor as result
)

---

如果你不喜欢以前的方法，那么你可以诉诸：

In [13]: bm = bert_model(
    ...:   encoding_sample['input_ids'],
    ...:   encoding_sample['attention_mask']
    ...: )

In [14]: bm.keys()
Out[14]: odict_keys(['last_hidden_state', 'pooler_output'])

# accessing last_hidden_state 
In [15]: bm['last_hidden_state']

In [16]: bm['last_hidden_state'].shape
Out[16]: torch.Size([1, 17, 768])