如何获得n个最长的DataFrame条目？

Question

我正在尝试获取 dask DataFrame 中最长的 n 个条目。我尝试在具有两列的 dask DataFrame 上调用 nlargest，如下所示：

import dask.dataframe as dd

df = dd.read_csv("opendns-random-domains.txt", header=None, names=['domain_name'])
df['domain_length'] = df.domain_name.map(len)
print(df.head())
print(df.dtypes)
top_3 = df.nlargest(3, 'domain_length')
print(top_3.head())

文件 opendns-random-domains.txt 仅包含一长串域名。上面代码的输出是这样的：

                  domain_name  domain_length
0                webmagnat.ro             12
1     nickelfreesolutions.com             23
2  scheepvaarttelefoongids.nl             26
3                  tursan.net             10
4       plannersanonymous.com             21

domain_name       object
domain_length    float64
dtype: object

Traceback (most recent call last):
  File "nlargest_test.py", line 9, in <module>
    print(top_3.head())
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/dask/dataframe/core.py", line 382, in head
    result = result.compute()
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/dask/base.py", line 86, in compute
    return compute(self, **kwargs)[0]
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/dask/base.py", line 179, in compute
    results = get(dsk, keys, **kwargs)
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/dask/threaded.py", line 57, in get
    **kwargs)
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/dask/async.py", line 484, in get_async
    raise(remote_exception(res, tb))
dask.async.TypeError: Cannot use method 'nlargest' with dtype object

Traceback
---------
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/dask/async.py", line 267, in execute_task
    result = _execute_task(task, data)
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/dask/async.py", line 249, in _execute_task
    return func(*args2)
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/dask/dataframe/core.py", line 2040, in <lambda>
    f = lambda df: df.nlargest(n, columns)
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/pandas/core/frame.py", line 3355, in nlargest
    return self._nsorted(columns, n, 'nlargest', keep)
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/pandas/core/frame.py", line 3318, in _nsorted
    ser = getattr(self[columns[0]], method)(n, keep=keep)
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/pandas/util/decorators.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/pandas/core/series.py", line 1898, in nlargest
    return algos.select_n(self, n=n, keep=keep, method='nlargest')
  File "/home/work/Dokumente/ModZero/Commerzbank/DNS_und_Proxylog-Analyse/dask-log-analyzer/venv/lib/python3.5/site-packages/pandas/core/algorithms.py", line 559, in select_n
    raise TypeError("Cannot use method %r with dtype %s" % (method, dtype))

我很困惑，因为我在float64 类型的列上调用nlargest，但仍然收到此错误，说它不能在dtype object 上调用。这在熊猫中也很好用。如何从 DataFrame 中获取最长的 n 个条目？

Answer 1

显式类型转换帮助了我：

df['column'].astype(str).astype(float).nlargest(5)

Answer 2

我试图重现您的问题，但一切正常。我可以建议您生成Minimal Complete Verifiable Example 吗？

熊猫示例

In [1]: import pandas as pd

In [2]: df = pd.DataFrame({'x': ['a', 'bb', 'ccc', 'dddd']})

In [3]: df['y'] = df.x.map(len)

In [4]: df
Out[4]: 
      x  y
0     a  1
1    bb  2
2   ccc  3
3  dddd  4

In [5]: df.nlargest(3, 'y')
Out[5]: 
      x  y
3  dddd  4
2   ccc  3
1    bb  2

Dask 数据框示例

In [1]: import pandas as pd

In [2]: df = pd.DataFrame({'x': ['a', 'bb', 'ccc', 'dddd']})

In [3]: import dask.dataframe as dd

In [4]: ddf = dd.from_pandas(df, npartitions=2)

In [5]: ddf['y'] = ddf.x.map(len)

In [6]: ddf.nlargest(3, 'y').compute()
Out[6]: 
      x  y
3  dddd  4
2   ccc  3
1    bb  2

或者，也许这只是在 git master 版本上工作？

Answer 3

您只需使用.astype() 将相应列的类型更改为int 或float。

例如，在您的情况下：

top_3 = df['domain_length'].astype(float).nlargest(3)

Answer 4

如果您想从字符串类型列中获取出现次数最多的值，您可以使用 value_counts() 和 nlargest(n)，其中 n 是您想要带入的元素数。

df['your_column'].value_counts().nlargest(3)

它将带来该列的前 3 个匹配项。