用 Pegasus 解释整个段落

Question

我一直在尝试使用 Pegasus 来解释事物，到目前为止，它确实可以很好地解释句子，但问题是，它实际上占据了段落的一部分，通常是最后一部分并重新解释它而不是整个给定的“短语” ，这是提供的段落。 我希望它解释整个给定的段落。

我这样做的功能：

from transformers import *


class SomeClass:
    def __init__(self):
        self.model = PegasusForConditionalGeneration.from_pretrained("tuner007/pegasus_paraphrase")
        self.tokenizer = PegasusTokenizerFast.from_pretrained("tuner007/pegasus_paraphrase")


    
    def get_paraphrased_sentences1(self, model, tokenizer, sentence, num_return_sequences=5, num_beams=5):
        # tokenize the text to be form of a list of token IDs
        inputs = tokenizer([sentence], truncation=True, padding="longest", return_tensors="pt")
        # generate the paraphrased sentences
        outputs = model.generate(
            **inputs,
            max_length=8096,
            length_penalty=2.0,
            num_beams=num_beams,
            num_return_sequences=num_return_sequences,
        )
        # decode the generated sentences using the tokenizer to get them back to text
        return tokenizer.batch_decode(outputs, skip_special_tokens=True)

如果我们举个例子： 原始输入：

Randal went to the mall to get some clothes. When he got home he found his wife cheating on him. Randal ended up kicking her out of the house. The kids stay with Randal and not the cheating wife.

接收到的输出序列：

When he got home, he found his wife cheating on him, so he kicked her out of the house and the kids stayed with him.
When he got home, he found his wife cheating on him, so he kicked her out of the house and put the kids with him.
When he returned from the mall, he found his wife cheating on him and he kicked her out of the house and the kids stayed with him.
When he returned from the mall, he found his wife cheating on him and he kicked her out of the house.
When he got home, he found his wife cheating on him, so he kicked her out of the house and the children stayed with him.
When he got home, he found his wife cheating on him, and he kicked her out of the house.
When he got home, he found his wife cheating on him, so he kicked her out of the house and put the kids in his care.
When he got home, he found his wife cheating on him, so he kicked her out of the house.
When he got home, he found his wife cheating on him, so he kicked her out of the house and kept the kids with him.
When he got home, he found his wife cheating on him, so he kicked her out of the house and put his kids with him.

这基本上是我过去几天一直在研究的问题，我查看了Pegasus Documentation 以检查是否有任何参数可以解决这个问题，但没有发现任何有效的问题，我尝试了一些，其中有 max_length 和 min_length 并且没有用。

Answer 1

你应该使用分句器，这篇文章会有所帮助：https://analyticsindiamag.com/how-to-paraphrase-text-using-pegasus-transformer/