无法从 Kubernetes 集群内部访问 Cassandra答案

【问题标题】：Unable to access the Cassandra from inside the Kubernetes cluster无法从 Kubernetes 集群内部访问 Cassandra
【发布时间】：2022-08-12 23:55:35
【问题描述】：

我的 Kubernetes 集群中有一个 Cassandra 集群和一个 Spring Boot 应用程序。它们位于相同的（默认）命名空间中。 Spring Boot 应用程序需要连接到 Cassandra，但它无法做到这一点。在连接尝试期间，spring boot 应用程序收到以下异常：

抑制： io.netty.channel.AbstractChannel$AnnotatedConnectException：连接拒绝：cassandra/10.111.117.185:32532 原因： java.net.ConnectException：连接被拒绝 java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) 在 java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777) 在 io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330) 在 io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) 在 io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:710) 在 io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658) 在 io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584) 在 io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) 在 io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) 在 io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) 在 io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) 在 java.base/java.lang.Thread.run(Thread.java:829) 引起： io.netty.channel.StacklessClosedChannelException: null

卡桑德拉 yaml：

apiVersion: v1
kind: Service
metadata:
  labels:
    app: cassandra
  name: cassandra
spec:
  type: NodePort
  ports:
  - port: 9042
    targetPort: 9042
    protocol: TCP
    nodePort: 32532
  selector:
    app: cassandra
---

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: cassandra
  labels:
    app: cassandra
spec:
  serviceName: cassandra
  replicas: 3
  selector:
    matchLabels:
      app: cassandra
  template:
    metadata:
      labels:
        app: cassandra
    spec:
      terminationGracePeriodSeconds: 1800
      containers:
      - name: cassandra
        image: cassandra:latest
        imagePullPolicy: Always
        ports:
        - containerPort: 7000
          name: intra-node
        - containerPort: 7001
          name: tls-intra-node
        - containerPort: 7199
          name: jmx
        - containerPort: 9042
          name: cql
        resources:
          limits:
            cpu: \"500m\"
            memory: 1Gi
          requests:
            cpu: \"500m\"
            memory: 1Gi
        securityContext:
          capabilities:
            add:
              - IPC_LOCK
        lifecycle:
          preStop:
            exec:
              command: 
              - /bin/sh
              - -c
              - nodetool drain
        env:
          - name: MAX_HEAP_SIZE
            value: 512M
          - name: HEAP_NEWSIZE
            value: 100M
          - name: CASSANDRA_SEEDS
            value: \"cassandra-0.cassandra.default.svc.cluster.local\"
          - name: CASSANDRA_CLUSTER_NAME
            value: \"K8SCassandra\"
          - name: CASSANDRA_DC
            value: \"DC1-K8SCassandra\"
          - name: CASSANDRA_RACK
            value: \"Rack1-K8SCassandra\"
          - name: POD_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
        
        volumeMounts:
        - name: cassandra-data
          mountPath: /cassandra_data
  volumeClaimTemplates:
  - metadata:
      name: cassandra-data
    spec:
      accessModes: [ \"ReadWriteOnce\" ]
      storageClassName: standard
      resources:
        requests:
          storage: 1Gi

Spring Boot 应用程序 yaml：

apiVersion: v1
kind: Service
metadata:
  name: service-cassandraapp
  labels:
     app: cassandraapp
spec:
  selector:
    app: cassandraapp
  type: LoadBalancer
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
      nodePort: 32588
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-cassandraapp
  labels:
    app: cassandraapp
spec:
  replicas: 1
  strategy:
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1
  selector:
    matchLabels:
      app: cassandraapp
  template:
    metadata:
      labels:
        app: cassandraapp
    spec:
      containers:
      - name: cassandraapp
        image: ek/cassandraapp:latest
        ports:
          - containerPort: 8080
        resources:
          limits:
            memory: \"1Gi\"
            cpu: \"1000m\"
          requests:
            memory: \"256Mi\"
            cpu: \"500m\"
        env:
          - name: CONFIG_CASSANDRA_HOST
            value: \"cassandra\"
          - name: CONFIG_CASSANDRA_PORT
            value: \"32532\"

春季启动应用程序.properties：

spring.data.cassandra.local-datacenter=datacenter1
spring.data.cassandra.keyspace-name=testkeyspace
spring.data.cassandra.port=${CONFIG_CASSANDRA_PORT}
spring.data.cassandra.contact-points=${CONFIG_CASSANDRA_HOST}
spring.data.cassandra.username=cassandra
spring.data.cassandra.password=cassandra
spring.data.cassandra.schema-action=CREATE_IF_NOT_EXISTS

当我检查 Cassandra pod 时，它们处于运行状态。但是spring boot应用程序被拒绝了。任何帮助将不胜感激。

您是否尝试过在 spring boot 应用程序中使用 cassandra 服务的目标端口 9042 而不是节点端口 32532？
@TAM 是的，我也尝试过目标端口 9042。但它也没有工作。我收到了同样的“拒绝连接”错误。
尝试使用无头服务而不是 NodePort 类型的服务。

标签： java spring-boot kubernetes cassandra

【解决方案1】：

我解决了这个问题。我假设 spring.data.cassandra.schema-action=CREATE_IF_NOT_EXISTS 会创建密钥空间，但它不会。所以我手动创建了它。

我更新的 Spring Boot 应用程序 yml：

apiVersion: v1
kind: ConfigMap
metadata:
  name: configmap-cassandra-cassandraapp
data:
  cassandra-host: "cassandra.default.svc.cluster.local"
  cassandra-port: "9042"
  cassandra-keyspace: "testkeyspace"
  cassandra-datacenter: "datacenter1"
---
apiVersion: v1
kind: Service
metadata:
  name: service-cassandraapp
  labels:
     app: cassandraapp
spec:
  selector:
    app: cassandraapp
  type: LoadBalancer
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-cassandraapp
  labels:
    app: cassandraapp
spec:
  replicas: 1
  strategy:
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1
  selector:
    matchLabels:
      app: cassandraapp
  template:
    metadata:
      labels:
        app: cassandraapp
    spec:
      containers:
      - name: cassandraapp
        image: ek/cassandraapp:latest
        ports:
          - containerPort: 8080
        resources:
          limits:
            memory: "1Gi"
            cpu: "1000m"
          requests:
            memory: "256Mi"
            cpu: "500m"
        env:
          - name: CONFIG_CASSANDRA_HOST
            valueFrom:
              configMapKeyRef:
                name: configmap-cassandra-cassandraapp
                key: cassandra-host
          - name: CONFIG_CASSANDRA_PORT
            valueFrom:
              configMapKeyRef:
                name: configmap-cassandra-cassandraapp
                key: cassandra-port
          - name: CONFIG_CASSANDRA_KEYSPACE
            valueFrom:
              configMapKeyRef:
                name: configmap-cassandra-cassandraapp
                key: cassandra-keyspace
          - name: CONFIG_CASSANDRA_DATACENTER
            valueFrom:
              configMapKeyRef:
                name: configmap-cassandra-cassandraapp
                key: cassandra-datacenter

Spring Boot 应用程序.properties

spring.data.cassandra.local-datacenter=${CONFIG_CASSANDRA_DATACENTER}
spring.data.cassandra.keyspace-name=${CONFIG_CASSANDRA_KEYSPACE}
spring.data.cassandra.port=${CONFIG_CASSANDRA_PORT}
spring.data.cassandra.contact-points=${CONFIG_CASSANDRA_HOST}

最后我还尝试了 Bitnami Helm Chart。我也推荐它。 [https://bitnami.com/stack/cassandra/helm][1]

【讨论】：