为什么 dask.dataframe.apply 只能处理名为“名称”的列？

Question

我正在尝试将一些 Pandas (Python) 代码移植到 Dask。我正在使用 Pandas 1.1.3 和 Dask 2.30.0。我一直用头撞到我看不见的墙上。也就是说，我无法理解这里发生了什么。我将其归结为以下最小的工作示例：
我的数据是包含以下内容的文件“test.csv”：

age,name
28,Alice

以下 Python 脚本（使用 Pandas）运行良好：

import pandas as pd

df = pd.read_csv("test.csv", dtype={'name': str})
result = df['name'].apply(lambda text: text.upper())
#result = df['age'].apply(lambda num: num + 1)
print(result)

并打印：

0    ALICE
Name: name, dtype: object

在“年龄”列上操作的注释掉的行也可以工作并打印：

0    29
Name: age, dtype: int64

现在，使用 Dask，我的示例变为：

import dask.dataframe as dd

df = dd.read_csv("test.csv", dtype={'name': str})
result = df['name'].apply(lambda text: text.upper(), meta={'name': str})
#result = df['age'].apply(lambda num: num + 1, meta={'age': int})
print(result.compute())

就像 Pandas 示例一样工作正常。但是，如果我尝试在“年龄”列上操作注释掉的行，Python 会抱怨以下错误消息：

Traceback (most recent call last):
  File "test_dask.py", line 7, in <module>
    print(result.compute())
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/base.py", line 167, in compute
    (result,) = compute(self, traverse=False, **kwargs)
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/base.py", line 452, in compute
    results = schedule(dsk, keys, **kwargs)
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/threaded.py", line 76, in get
    results = get_async(
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/local.py", line 486, in get_async
    raise_exception(exc, tb)
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/local.py", line 316, in reraise
    raise exc
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/local.py", line 222, in execute_task
    result = _execute_task(task, data)
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/core.py", line 121, in _execute_task
    return func(*(_execute_task(a, cache) for a in args))
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/optimization.py", line 961, in __call__
    return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/core.py", line 151, in get
    result = _execute_task(task, cache)
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/core.py", line 121, in _execute_task
    return func(*(_execute_task(a, cache) for a in args))
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/utils.py", line 29, in apply
    return func(*args, **kwargs)
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/dask/dataframe/core.py", line 5306, in apply_and_enforce
    c = meta.name
  File "/some/path/miniconda3/envs/testdask/lib/python3.8/site-packages/pandas/core/generic.py", line 5139, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'name'

即使我只是将“名称”列称为其他名称，它也会像这样失败。就好像 Dask 只能处理称为“名称”的 DataFrame 列。这对我来说似乎非常奇怪，我一定是误会了什么。这里到底发生了什么？

Answer 1

The docs 似乎暗示 dict 应该可以工作，所以这很奇怪，但是如果您将 meta 参数替换为元组，您的代码将按预期运行：

df = dd.read_csv("test.csv")
result = df['age'].apply(lambda num: num + 1, meta=('age', 'int64'))
print(result.compute())

变成

0    29
Name: age, dtype: int64