【发布时间】：2022-07-02 08:06:17
【问题描述】：
我有以下dask_cudf.core.DataFrame:-
import pandas as pd
import numpy as np
import dask_cudf
import cudf
data = {"x":range(1,21), "nor":np.random.normal(2, 4, 20), "unif":np.random.uniform(size = 20)}
df = cudf.DataFrame(data)
ddf = dask_cudf.from_cudf(df, npartitions = 2)
ddf.compute()
我想为 nor 和 unif 列创建第 1 到第 5 个滞后值。但是，我通过以下方式创建它们：-
colz = ["nor", "unif"]
ddf[[s + "_" + str(1) for s in colz]] = ddf[colz].shift(1)
ddf[[s + "_" + str(2) for s in colz]] = ddf[colz].shift(2)
我可以创建第一个和第二个滞后值，但不能超过这个值。当我以大于 2 的值运行 shift 时，我收到以下错误::-
/usr/local/lib/python3.7/site-packages/dask/dataframe/utils.py in raise_on_meta_error(funcname, udf)
175 try:
--> 176 yield
177 except Exception as e:
16 frames
cudf/_lib/copying.pyx in cudf._lib.copying.shift()
RuntimeError: parallel_for failed: cudaErrorInvalidConfiguration: invalid configuration argument
The above exception was the direct cause of the following exception:
ValueError Traceback (most recent call last)
/usr/local/lib/python3.7/site-packages/dask/dataframe/utils.py in raise_on_meta_error(funcname, udf)
195 )
196 msg = msg.format(f" in `{funcname}`" if funcname else "", repr(e), tb)
--> 197 raise ValueError(msg) from e
198
199
ValueError: Metadata inference failed in `shift`.
Original error is below:
------------------------
RuntimeError('parallel_for failed: cudaErrorInvalidConfiguration: invalid configuration argument')
Traceback:
---------
File "/usr/local/lib/python3.7/site-packages/dask/dataframe/utils.py", line 176, in raise_on_meta_error
yield
File "/usr/local/lib/python3.7/site-packages/dask/dataframe/core.py", line 5833, in _emulate
return func(*_extract_meta(args, True), **_extract_meta(kwargs, True))
File "/usr/local/lib/python3.7/site-packages/dask/utils.py", line 1021, in __call__
return getattr(__obj, self.method)(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/cudf/core/frame.py", line 1788, in shift
return self._shift(periods)
File "/usr/local/lib/python3.7/site-packages/cudf/core/frame.py", line 1793, in _shift
zip(self._column_names, data_columns), self._index
File "/usr/local/lib/python3.7/site-packages/cudf/core/dataframe.py", line 818, in _from_data
out = super()._from_data(data, index)
File "/usr/local/lib/python3.7/site-packages/cudf/core/frame.py", line 140, in _from_data
Frame.__init__(obj, data, index)
File "/usr/local/lib/python3.7/site-packages/cudf/core/frame.py", line 78, in __init__
self._data = cudf.core.column_accessor.ColumnAccessor(data)
File "/usr/local/lib/python3.7/site-packages/cudf/core/column_accessor.py", line 121, in __init__
data = dict(data)
File "/usr/local/lib/python3.7/site-packages/cudf/core/frame.py", line 1791, in <genexpr>
data_columns = (col.shift(offset, fill_value) for col in self._columns)
File "/usr/local/lib/python3.7/site-packages/cudf/core/column/column.py", line 391, in shift
return libcudf.copying.shift(self, offset, fill_value)
File "cudf/_lib/copying.pyx", line 633, in cudf._lib.copying.shift
我似乎无法理解为什么会这样。
【问题讨论】：