通过 sqlalchemy 创建 ReplicatedMergeTree 表

Question

我正在尝试通过 sqlalchemy 在 Clickhouse 集群上创建一个新表。而且我不明白如何为 ReplicatedMergeTree 表的引擎指定所有必需的参数。

我用过https://github.com/xzkostyan/clickhouse-sqlalchemy库

from sqlalchemy import create_engine, Column, MetaData, literal
from clickhouse_sqlalchemy import Table, make_session, get_declarative_base, types, engines

engine = create_engine(URL)
session = make_session(engine)
metadata = MetaData(bind=engine)

# Задаются параметры таблицы ClickHouse
another_table = Table(
      'sqlalchemy_test'
    , metadata
    , Column('id', types.Int32, primary_key=True)
    , Column('value', types.Int32)
    , engines.ReplicatedMergeTree(
          table_path='/clickhouse/schema_name/tables/test_sqlalchemy'
        , replica_name='{replica}-{shard}')
)

another_table.create()

如何将所有必需的参数放入 Table 类中？

DatabaseException: Orig exception: Code: 42, e.displayText() = DB::Exception: Storage ReplicatedMergeTree requires 5 to 6 parameters: 
path in ZooKeeper,
replica name,
name of column with date,
[sampling element of primary key],
primary key expression,
index granularity

MergeTree is a family of storage engines.

MergeTrees are different in two ways:
- they may be replicated and non-replicated;
- they may do different actions on merge: nothing; sign collapse; sum; apply aggregete functions.

So we have 14 combinations:
    MergeTree, CollapsingMergeTree, SummingMergeTree, AggregatingMergeTree, ReplacingMergeTree, GraphiteMergeTree, VersionedCollapsingMergeTree
    ReplicatedMergeTree, ReplicatedCollapsingMergeTree, ReplicatedSummingMergeTree, ReplicatedAggregatingMergeTree, ReplicatedReplacingMergeTree, ReplicatedGraphiteMergeTree, ReplicatedVersionedCollapsingMergeTree


Examples:
   
ReplicatedMergeTree('/clickhouse/tables/{layer}-{shard}/hits', '{replica}', EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID), EventTime), 8192)

Answer 1

运行此代码需要配置的 ClickHouse 集群（详情请参阅CH replication）。

CH 根据集群配置替换所有占位符。

from sqlalchemy import create_engine, Column, MetaData, literal
from clickhouse_sqlalchemy import Table, make_session, get_declarative_base, types, engines

URL = 'clickhouse://default:@localhost/test'
CLUSTER_NAME='test_cluster'

engine = create_engine(URL)
session = make_session(engine)
metadata = MetaData(bind=engine)

another_table = Table(
    'sqlalchemy_test'
    , metadata
    , Column('id', types.Int32)
    , Column('value', types.Int32)
    , engines.ReplicatedMergeTree(
        table_path='/clickhouse/clusters/{cluster}/tables/{database}/{shard}/{table}'
        , replica_name='{replica}'
        , order_by='id'
        , partition_by=None
        , sample_by=None)
    , clickhouse_cluster=CLUSTER_NAME
)

another_table.create()

---

出于测试目的，您可以使用 docker-compose 启动测试 CH 集群（例如，查看 neverlee/clickhouse-cluster-docker-compose）。