spaCy：尝试加载序列化文档时出错

Question

我正在尝试序列化/反序列化 spaCy 文档（设置为 Windows 7、Anaconda）并且出现错误。我一直找不到任何解释。这是一段 sn-p 代码及其生成的错误：

import spacy
nlp = spacy.load('en')
text = 'This is a test.'
doc = nlp(text)
fout = 'test.spacy' # <-- according to the API for Doc.to_disk(), this needs to be a directory (but for me, spaCy writes a file)
doc.to_disk(fout)
doc.from_disk(fout)
Traceback (most recent call last):

  File "<ipython-input-7-aa22bf1b9689>", line 1, in <module>
    doc.from_disk(fout)

  File "doc.pyx", line 763, in spacy.tokens.doc.Doc.from_disk

  File "doc.pyx", line 806, in spacy.tokens.doc.Doc.from_bytes

ValueError: [E033] Cannot load into non-empty Doc of length 5.

我还尝试创建一个新的 Doc 对象并从中加载，如 spaCy docs 中的示例（“示例：保存和加载文档”）所示，这会导致不同的错误：

from spacy.tokens import Doc
from spacy.vocab import Vocab

new_doc = Doc(Vocab()).from_disk(fout)
Traceback (most recent call last):

  File "<ipython-input-16-4d99a1199f43>", line 1, in <module>
    Doc(Vocab()).from_disk(fout)

  File "doc.pyx", line 763, in spacy.tokens.doc.Doc.from_disk

  File "doc.pyx", line 838, in spacy.tokens.doc.Doc.from_bytes

  File "stringsource", line 646, in View.MemoryView.memoryview_cwrapper

  File "stringsource", line 347, in View.MemoryView.memoryview.__cinit__

ValueError: buffer source array is read-only

编辑：

正如回复中指出的，提供的路径应该是一个目录。但是，第一个代码 sn -p 创建了一个文件。将其更改为不存在的目录路径无济于事，因为 spaCy 仍会创建一个文件。尝试写入现有目录也会导致错误：

fout = 'data'

doc.to_disk(fout) Traceback (most recent call last):

  File "<ipython-input-8-6c30638f4750>", line 1, in <module>
    doc.to_disk(fout)

  File "doc.pyx", line 749, in spacy.tokens.doc.Doc.to_disk

  File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 1161, in open
    opener=self._opener)

  File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 1015, in _opener
    return self._accessor.open(self, flags, mode)

  File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)

PermissionError: [Errno 13] Permission denied: 'data'

Python 通过标准文件操作 (open/read/write) 在此位置写入没有问题。

尝试使用 Path 对象会产生相同的结果：

from pathlib import Path

import os

fout = Path(os.path.join(os.getcwd(), 'data'))

doc.to_disk(fout)
Traceback (most recent call last):

  File "<ipython-input-17-6c30638f4750>", line 1, in <module>
    doc.to_disk(fout)

  File "doc.pyx", line 749, in spacy.tokens.doc.Doc.to_disk

  File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 1161, in open
    opener=self._opener)

  File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 1015, in _opener
    return self._accessor.open(self, flags, mode)

  File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)

PermissionError: [Errno 13] Permission denied: 'C:\\Users\\Username\\workspace\\data'

任何想法为什么会发生这种情况？

Answer 1

doc.to_disk(fout)

必须

一个目录的路径，如果它不存在将被创建。 路径可以是字符串或类似路径的对象。

作为 https://spacy.io/api/doc 中 spaCy 状态的文档

尝试将fout 更改为目录，它可能会成功。

编辑： spacy 文档中的示例：

对于doc.to_disk：

doc.to_disk('/path/to/doc')

对于doc.from_disk：

from spacy.tokens import Doc
from spacy.vocab import Vocab
doc = Doc(Vocab()).from_disk('/path/to/doc')