Cassandra 查询只返回 5000 行

Question

我用这段代码从cassandra查询数据，结果只有5000行，但实际上数据库表有120000行，如何查询所有数据？

from cassandra import ConsistencyLevel
from cassandra.cluster import Cluster
from cassandra.auth import PlainTextAuthProvider
from cassandra.query import SimpleStatement
import pandas as pd

cluster = Cluster(contact_points=['192.168.2.4'],port=9042)
session = cluster.connect()

def testContectRemoteDatabase():
    contact_points = ['192.168.2.4']
    auth_provider = PlainTextAuthProvider(username='XXX', password='XX')
    cluster = Cluster(contact_points=contact_points, auth_provider=auth_provider)
    session = cluster.connect()
    cql_str = 'select * from DB1.mytable ;'
    simple_statement = SimpleStatement(cql_str, consistency_level=ConsistencyLevel.ONE)
    execute_result = session.execute(simple_statement, timeout=None)
    result = execute_result._current_rows
    cluster.shutdown()
    df = pd.DataFrame(result)
    df.to_csv('./my_test.csv', index=False, mode='w', header=True)

if __name__ == '__main__':
    testContectRemoteDatabase()

Answer 1

这是by design：

默认情况下，Session.default_fetch_size 控制每页将获取多少行。这可以通过在Statement 上设置fetch_size 来覆盖每个查询。默认情况下，每个页面最多包含 5000 行。

将此设置为None 将禁用自动分页：

simple_statement = SimpleStatement(
    cql_str, consistency_level=ConsistencyLevel.ONE, fetch_size=None)