熊猫告诉我非模棱两可的时间是模棱两可的

Question

我有以下测试代码：

import pandas as pd

dt = pd.to_datetime('2021-11-07 01:00:00-0400').tz_convert('America/New_York')
pd.DataFrame({'datetime': dt,
              'value': [3, 4, 5]})

当使用pandas 1.1.5 版本时，它运行成功。但在pandas 1.2.5 或1.3.4 版本下，它会失败并出现以下错误：

Traceback (most recent call last):
  File "test.py", line 5, in <module>
    'value': [3, 4, 5]})
  File "venv/lib/python3.7/site-packages/pandas/core/frame.py", line 614, in __init__
    mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
  File "venv/lib/python3.7/site-packages/pandas/core/internals/construction.py", line 465, in dict_to_mgr
    arrays, data_names, index, columns, dtype=dtype, typ=typ, consolidate=copy
  File "venv/lib/python3.7/site-packages/pandas/core/internals/construction.py", line 124, in arrays_to_mgr
    arrays = _homogenize(arrays, index, dtype)
  File "venv/lib/python3.7/site-packages/pandas/core/internals/construction.py", line 590, in _homogenize
    val, index, dtype=dtype, copy=False, raise_cast_failure=False
  File "venv/lib/python3.7/site-packages/pandas/core/construction.py", line 514, in sanitize_array
    data = construct_1d_arraylike_from_scalar(data, len(index), dtype)
  File "venv/lib/python3.7/site-packages/pandas/core/dtypes/cast.py", line 1907, in construct_1d_arraylike_from_scalar
    subarr = cls._from_sequence([value] * length, dtype=dtype)
  File "venv/lib/python3.7/site-packages/pandas/core/arrays/datetimes.py", line 336, in _from_sequence
    return cls._from_sequence_not_strict(scalars, dtype=dtype, copy=copy)
  File "venv/lib/python3.7/site-packages/pandas/core/arrays/datetimes.py", line 362, in _from_sequence_not_strict
    ambiguous=ambiguous,
  File "venv/lib/python3.7/site-packages/pandas/core/arrays/datetimes.py", line 2098, in sequence_to_dt64ns
    data.view("i8"), tz, ambiguous=ambiguous
  File "pandas/_libs/tslibs/tzconversion.pyx", line 284, in pandas._libs.tslibs.tzconversion.tz_localize_to_utc
pytz.exceptions.AmbiguousTimeError: Cannot infer dst time from 2021-11-07 01:00:00, try using the 'ambiguous' argument

我知道夏令时是在 11 月 7 日。但是这些数据对我来说是明确的，并且是完全本地化的；为什么pandas 忘记了它的时区信息，为什么它拒绝把它放在DataFrame 中？这里有什么解决方法吗？

更新：

我记得几个月前我实际上已经提交了一个关于此的错误，但直到本周我们开始看到实际 DST 转换日期在生产中时，这对我们来说只是有点学术兴趣：@987654321 @

Answer 1

这是不明确的，因为这个特殊时间有 2 个日期：有 DST 和没有 DST：

# Timestamp('2021-11-07 01:00:00-0500', tz='America/New_York')
>>> pd.to_datetime('2021-11-07 01:00:00') \
      .tz_localize('America/New_York', ambiguous=False).dst()
datetime.timedelta(0)


# Timestamp('2021-11-07 01:00:00-0400', tz='America/New_York')
>>> pd.to_datetime('2021-11-07 01:00:00') \
      .tz_localize('America/New_York', ambiguous=True).dst()
datetime.timedelta(3600)

解决方法

dt = pd.to_datetime('2021-11-07 01:00:00-0400')

df = pd.DataFrame({'datetime': dt,
                   'value': [3, 4, 5]})

df['datetime'] = df['datetime'].dt.tz_convert('America/New_York')

Answer 2

我接受了@Corralien 的回答，我还想展示我最终决定采用的解决方法：

# Work around Pandas DST bug, see https://github.com/pandas-dev/pandas/issues/42505 and
# https://stackoverflow.com/questions/69846645/pandas-tells-me-non-ambiguous-time-is-ambiguous
max_len = max(len(x) if self.is_array(x) else 1 for x in data.values())
if max_len > 0 and self.is_scalar(data['datetime']):
    data['datetime'] = [data['datetime']] * max_len
df = pd.DataFrame(data)

is_array() 和is_scalar() 函数检查x 是否是set, list, tuple, np.ndarray, pd.Series, pd.Index 的任何一个实例。

这并不完美，但希望胶带能一直保持到可以在 Pandas 中修复为止。