如何从 Huggingface（在 sagemaker 中）而不是特征中获取嵌入？答案

【问题标题】：How do I get embeddings from Huggingface(in sagemaker) instead of features?如何从 Huggingface（在 sagemaker 中）而不是特征中获取嵌入？
【发布时间】：2022-10-15 07:31:58
【问题描述】：

我有一个文本分类器模型，它依赖于某个拥抱脸模型的嵌入

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('T-Systems-onsite/cross-en-de-roberta-sentence-transformer')
encodings = model.encode("guckst du bundesliga")

它的形状为 (768,)

tldr：在 sagemaker 上是否有一种干净简单的方法（希望使用它提供的图像）？

语境：查看此huggingface model 的文档，我看到的唯一 sagemaker 选项是特征提取

from sagemaker.huggingface import HuggingFaceModel
import sagemaker

role = sagemaker.get_execution_role()
# Hub Model configuration. https://huggingface.co/models
hub = {
    'HF_MODEL_ID':'T-Systems-onsite/cross-en-de-roberta-sentence-transformer',
    'HF_TASK':'feature-extraction'
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
    transformers_version='4.6.1',
    pytorch_version='1.7.1',
    py_version='py36',
    env=hub,
    role=role, 
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
    initial_instance_count=1, # number of instances
    instance_type='ml.m5.xlarge' # ec2 instance type
)

predictor.predict({
    'inputs': "Today is a sunny day and I'll get some ice cream."
})

这给了我具有形状的特征 (9, 768)

这两个值之间存在联系，从另一个代码示例可以看出

from transformers import AutoTokenizer, AutoModel
import torch


#Mean Pooling - Take attention mask into account for correct averaging
def embeddings(feature_envelope, attention_mask):
    features = feature_envelope[0] #First element of model_output contains all token embeddings
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(features.size()).float()
    sum_embeddings = torch.sum(features * input_mask_expanded, 1)
    sum_mask = torch.clamp(input_mask_expanded.sum(1), min=1e-9)
    return sum_embeddings / sum_mask

#Sentences we want sentence embeddings for
sentences = ['guckst du bundesliga']

#Load AutoModel from huggingface model repository
tokenizer = AutoTokenizer.from_pretrained('T-Systems-onsite/cross-en-de-roberta-sentence-transformer')
model = AutoModel.from_pretrained('T-Systems-onsite/cross-en-de-roberta-sentence-transformer')

#Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, max_length=128, return_tensors='pt')

#Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)
#     print(model_output)

#Perform pooling. In this case, mean pooling
sentence_embeddings = embeddings(model_output, encoded_input['attention_mask'])
sentence_embeddings.shape, sentence_embeddings

但是正如您所看到的，仅给定特征就无法导出嵌入

【问题讨论】：

标签： amazon-sagemaker huggingface-transformers sentence-transformers

【解决方案1】：

您可以通过使用inference.py 文件来定义自己的“用户定义代码”。

https://huggingface.co/docs/sagemaker/inference#user-defined-code-and-modules

【讨论】：

【解决方案2】：

我不是 python 或 ML 人，所以对此持保留态度。我在部署推理端点时遇到了同样的问题。下面提取了我相信您正在寻找的数据。

sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

cos = torch.nn.CosineSimilarity(dim=1, eps=1e-6)

all_sentence_combinations = []
for i in range(len(sentence_embeddings) - 1):
    for j in range(i + 1, len(sentence_embeddings)):
        opt = cos(sentence_embeddings[i].unsqueeze(0), sentence_embeddings[j].unsqueeze(0))
        all_sentence_combinations.append([opt.item(), i, j])

arr = []
for score, i, j in all_sentence_combinations:
    arr.append([sentences[i], sentences[j], score])
    print("{} 	 {} 	 {:.4f}".format(sentences[i], sentences[j], score))

【讨论】：