doris docker 部署 及监控

evn:  ubuntu20.04  jdk8_172  doris 0.13.12

 

############# 1 启动Fe  BE  ###################

   准备：

sudo mkdir  -p  /opt/docker/doris/ && sudo chmod -R 777 /opt/docker/doris/ 

 

PRIORITY_NETWORKS 可以通过 > ip a 查看

运行(启动fe ):   

 sudo docker run -itd --privileged=true --restart=always --name doris-fe  --net=host  -e FE_ROLE=fe-leader  -e PRIORITY_NETWORKS=192.168.28.151/24 \
-v /opt/docker/doris/fe/log:/data/fe/log  -v /opt/docker/doris/fe/doris-meta:/data/fe/doris-meta  -v /etc/localtime:/etc/localtime:ro  \
   -v /etc/timezone:/etc/timezone    happysea/doris:latest

运行(启动be ): 

sudo docker  run -itd --privileged=true --name doris-be --net=host --restart=always -e FE_ROLE=be -e PRIORITY_NETWORKS=192.168.18.51/24  \
-v /opt/docker/doris/be/log:/data/be/log   -v /opt/docker/doris/be/storage:/data/be/storage  -v /etc/localtime:/etc/localtime:ro   \
 -v /etc/timezone:/etc/timezone   happysea/doris:latest

 

JVM　参数是默认最大4G ,建议改为８Ｇ＞如果需要：

可以修改：

在启动添加参数： -e  JAVA_OPTS="-Xmx4096m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$DATE"

############# 2.连接添加Be  ###################

连接fe
mysql -h host -P 9030 -uroot

修改root 用户密码 (如果需要 ):   >   set password for 用户名@localhost = password('新密码');  

其中 host 为 FE 所在节点 ip；port 为 fe/conf/fe.conf 中的 query_port；默认使用 root 账户，无密码登录。 

 

添加be
ALTER SYSTEM ADD BACKEND “host1:9050”;
ALTER SYSTEM ADD BACKEND “host2:9050”;
ALTER SYSTEM ADD BACKEND “host3:9050”;
(
如果使用多组户功能，则执行以下命令添加BE:
ALTER SYSTEM ADD FREE BACKEND “host:port”;
)

其中 host 为 BE 所在节点 ip；port 为 be/conf/be.conf 中的 heartbeat_service_port。

• BE 进程将启动并进入后台执行。日志默认存放在 be/log/ 目录下。如启动失败，可以通过查看 be/log/be.log 或者 be/log/be.out 查看错误信息。

• 查看BE状态

  使用 mysql-client 连接到 FE，并执行 SHOW PROC '/backends'; 查看 BE 运行情况。如一切正常，isAlive 列应为 true。 

 

显示fe与be状态
show proc "/frontends";
show proc "/backends";

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Dockerfile:

FROM happysea/doris:base
MAINTAINER Sea <lshan523@163.com>
WORKDIR  /opt
COPY docker-entrypoint.sh /opt/
ENV export JAVA_HOME=/soft/softinstall/jdk1.8.0_172
ENV export JRE_HOME=/soft/softinstall/jdk1.8.0_172/jre
EXPOSE 8030 9020 9030 9060 9070 8040 9050 8060 
#ENTRYPOINT["/bin/bash"]
ENTRYPOINT ["/bin/bash","/opt/docker-entrypoint.sh"]

docker-entrypoint.sh

#!/bin/bash
chmod  -R  755 /data/
echo "fe_role:"$FE_ROLE
echo "leader:"$FE_LEADER

source /etc/profile

if [[ $FE_ROLE = "fe-leader" ]]; then
    echo "fe-leader"
    /opt/doris/fe/bin/start_fe.sh
elif [[ $FE_ROLE = "be" ]]; then
      echo "be"
     /opt/doris/be/bin/start_be.sh
elif [[ $FE_ROLE = "fe-follower" ]]; then
    /opt/doris/fe/bin/start_fe.sh --helper $FE_LEADER
else
    #/opt/doris/fs_broker/bin/start_broker.sh
    echo "do nothing "
fi

打包: sudo docker build -t  happysea/doris:latest

 

 

 

docker-compose 参考如下即可:

 

以下可以参考：

使用Docker快速搭建Doris集群： https://blog.csdn.net/jklcl/article/details/112910796

》

制作docker镜像

# centos7:jdk8是我们自己做的基础镜像
FROM centos7:jdk8

RUN mkdir -p /home/doris

ENV JAVA_HOME /usr/lib/jvm/java

COPY ./fe/ /home/doris/fe

COPY ./be/ /home/doris/be

COPY ./apache_hdfs_broker/ /home/doris/fs_broker

EXPOSE 8030 9020 9030 9010 9070 9060 8060 8040 9050 8000

VOLUME ["/home/doris/fe/conf", "/home/doris/fe/log", "/home/doris/fe/doris-meta", "/home/doris/be/conf", "/home/doris/be/log", "/home/doris/be/storage", "/home/doris/fs_brokers/conf"]

COPY entrypoint.sh /

RUN chmod +x entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]

 

entrypoint.sh

#!/bin/sh

echo "fe_role:"$FE_ROLE
echo "leader:"$FE_LEADER

if [[ $FE_ROLE = 'fe-leader' ]]; then
    /home/doris/fe/bin/start_fe.sh
elif [[ $FE_ROLE = 'be' ]]; then
    /home/doris/be/bin/start_be.sh
elif [[ $FE_ROLE = 'fe-follower' ]]; then
    /home/doris/fe/bin/start_fe.sh --helper $FE_LEADER
else
    /home/doris/fs_broker/bin/start_broker.sh
fi

 

创建镜像

docker build -f DockerFile -t doris:0.12.21-release .

docker-compose启动

FE

version: '3.7'
services:
    doris-fe:
        image: doris:0.12.21-release
        restart: always
        network_mode: "host"
        container_name: "doris-fe"
        ports:
            - "8030:8030"
            - "9010:9010"
            - "9020:9020"
            - "9030:9030"
        volumes:
            - "/xxx/doris/fe/log:/home/doris/fe/log"
            - "/xxx/doris/fe/doris-meta:/home/doris/fe/doris-meta"
            - "/xxx/doris/fe/conf:/home/doris/fe/conf"
            - "/etc/localtime:/etc/localtime:ro"
        environment:
            - FE_ROLE=fe-follower
            - FE_LEADER=xxxx:9010
        security_opt:
            - seccomp:unconfined

创建文件夹: mkdir /xxx/doris/fe/conf
启动leader: docker-compose -f docker-compose-doris-fe-leader.yml up -d
启动follower: 略
UI界面: http://s-hadoop-log01:8030/
注：第一次启动时先启动leader，再启动follower，因为follower指向leader所在机器之后的启动，都按照leader起就可以了

BE

version: '3.7'
services:
    doris-be:
        image: doris:0.12.21-release
        restart: always
        network_mode: "host"
        container_name: "doris-be"
        ports:
            - "8040:8040"
            - "8060:8060"
            - "9050:9050"
            - "9060:9060"
            - "9070:9070"
        volumes:
            - "/xxx/doris/be/log:/home/doris/be/log"
            - "/xxx/doris/be/storage:/home/doris/be/storage"
            - "/xxx/doris/be/conf:/home/doris/be/conf/"
            - "/etc/localtime:/etc/localtime:ro"
        environment:
            - FE_ROLE=be

创建文件夹: mkdir /xxx/doris/be/conf
启动be: docker-compose -f docker-compose-doris-be.yml up -d
UI界面: http://s-hadoop-log01:8040/

 

doris-fs-broker:

version: '3.7'
services:
    doris-fs-broker:
        image: doris:0.12.21-release
        restart: always
        network_mode: "host"
        container_name: "doris-fs-broker"
        ports:
            - "8000:8000"
        volumes:
            - "/xxx/doris/fs_broker/conf:/home/doris/fs_broker/conf"
            - "/xxx/doris/fs_broker/log:/home/doris/fs_broker/log"
            - "/etc/localtime:/etc/localtime:ro"
        environment:
            - FE_ROLE=fs

创建文件夹: mkdir /xxx/doris/fs_broker/conf
启动fs: docker-compose -f docker-compose-doris-fs-broker.yml up -d