点燃集群和 Kubernetes 集成

Question

我是 Ignite 的新手。

Test1：（没有 Kubernetes）

步骤 1：我在 2 个虚拟机（ubuntu）中安装了 Ignite 2.6.0，在一个虚拟机中启动了节点。使用以下命令。 bin/ignite.sh 示例/config/example-ignite.xml

step2：我所有的配置都在example-default.xml中

step3：在其他VM中执行client.jar，其中包含datagrid逻辑（这个VM既是客户端也是节点）。

第4步：能够将数据存储在缓存中。

Test2：（Kubernetes 安装在云端）

第1步：按照url中提到的步骤，https://apacheignite.readme.io/docs/stateful-deployment。

1. 按原样运行“RBAC 授权”和“Ignite Servie”超链接 yamls 的所有命令。
2. 接下来在“数据库和 WAL 文件的相同存储”部分中，运行“ignite-stateful-set.yaml”
3. ignite-stateful-set.yaml 中的 CONFIG_URI 指向“Kubernetes IP Finder”部分的“example-default.xml（重命名）”。
4. 在 TcpDiscoveryKubernetesIpFinder bean 的 example-default.xml 'addresses' 部分中添加了 2 个 VM 的 IP。

第 2 步：将“example-default.xml”复制到 2 个虚拟机的 /opt/apache-ignite-fabric-2.6.0-bv-in/examples/config（安装了 ignite 的位置）中。

第 3 步：现在我启动了相同的 client.jar，它给出了异常（附加） 附加了运行客户端的命令 （Kubernetes 和 Ignite 安装在同一个虚拟机（ubuntu）中）

请让我知道 Test2 中的任何错误。提前致谢。

example-default.xml

 <bean abstract="true" id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <!-- Set to true to enable distributed class loading for examples, default is false. -->
        <property name="peerClassLoadingEnabled" value="true"/>
        <property name="clientMode" value="false"/>

        <!-- Enable task execution events for examples. -->
        <property name="includeEventTypes">
            <list>

                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_TIMEDOUT"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_SESSION_ATTR_SET"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_REDUCED"/>


                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_PUT"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_READ"/>
                <util:constant static-field="org.apache.ignite.events.EventType.EVT_CACHE_OBJECT_REMOVED"/>
            </list>
        </property>


    <property name="publicThreadPoolSize" value="40"/>
    <property name="systemThreadPoolSize" value="30"/>

     <property name="dataStorageConfiguration">
                 <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
                         <property name="dataRegionConfigurations">
                                 <list>
                                         <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                                         <!-- Custom region name. -->
                                         <property name="name" value="6GB_Region"/>

                                         <!-- 100 MB initial size. -->
                                         <property name="initialSize" value="#{100L * 1024 * 1024}"/>

                                         <!-- 6 GB maximum size. -->
                                         <property name="maxSize" value="#{6L * 1024 * 1024 * 1024}"/>

                                         <!-- Enabling persistence for the region. -->
                                         <property name="persistenceEnabled" value="true"/>
                                         <property name="metricsEnabled" value="true"/>
                                        </bean>
                                 </list>
                         </property>
                 </bean>
    </property>

    <property name="cacheConfiguration">
        <list>
         <bean id="myCacheTemplate" class="org.apache.ignite.configuration.CacheConfiguration">
                 <property name="dataRegionName" value="6GB_Region"/>
                 <property name="name" value="myCache"/>
                <property name="atomicityMode" value="ATOMIC"/>
                 <property name="cacheMode" value="REPLICATED"/>
                 <property name="backups" value="0"/>
                 <property name="copyOnRead" value="false"/>
                                 <property name="rebalanceBatchSize" value="#{1024 * 1024}"/>
                                 <property name="rebalanceThrottle" value="0"/>
                                 <property name="rebalanceThreadPoolSize" value="4"/>
                                <!-- <property name="swapEnabled" value="false"/> 
                                 <property name="startSize" value="#{100 * 1024 * 1024}"/> 
                <property name="evictionPolicy">

                         <bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">

                         <property name="maxSize" value="1000000"/>
                         </bean>
                </property>
                <property name="offHeapMaxMemory" value="0"/>-->
         </bean>
         </list>
    </property>


        <!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    <!--
                        Ignite provides several options for automatic discovery that can be used
                        instead os static IP based discovery. For information on all options refer
                        to our documentation: http://apacheignite.readme.io/docs/cluster-config
                    -->
                    <!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
           <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">-->
                   <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
                    <property name="namespace" value="ignite"/>
                    <property name="serviceName" value="ignite"/>
                    <!-- <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">-->
                        <property name="addresses">
                            <list>
                                <!-- In distributed environment, replace with actual host IP address. -->
                               <value>xxx.xxx.xxx.xxx:47500</value>
                                <value>xxx.xxx.xxx.xxx:47500</value>                                
                            </list>
                        </property>
                    </bean>
                    <!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
                <property name="namespace" value="ignite"/>
                    </bean>-->
                </property>
            </bean>
        </property>
    </bean>
</beans>

代码片段：

    try (Ignite ignite = Ignition.start("/opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-ignite.xml")) {
    //try (Ignite ignite = Ignition.start("D:\\ApacheIgnite\\apache-ignite-fabric-2.6.0-bin\\examples\\config\\example-ignite.xml")) {


        ignite.cluster().active(true);
        System.out.println(">>> Cache asynchronous API started.");



        // Auto-close cache at the end of the example.
        try (IgniteCache<String, String> cache = ignite.getOrCreateCache("myCache")) {

           Collection<IgniteFuture<?>> futs = new ArrayList<>();

            // Execute several puts asynchronously.
           String key = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
           String value = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";

           System.out.println("CAME HERE 11"); 

           long put_start_time = System.currentTimeMillis();
           // Execute several puts asynchronously.
           for(int i=0;i<messageCount;i++) {

             futs.add(cache.putAsync(key+String.valueOf(i), value));

           }        
           long put_end_time = System.currentTimeMillis();

           System.out.println("Time taken to put="+(put_end_time-put_start_time)+" Milli Seconds ");

          key = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
          long get_start_time = System.currentTimeMillis();

          // Wait for completion of all futures.
          for (IgniteFuture<?> fut : futs) fut.get();



          // Execute get operation asynchronously and wait for result.
          for(int i=0;i<messageCount;i++) {
              cache.getAsync(key+String.valueOf(i)).listen(new IgniteInClosure<IgniteFuture<String>>() {

                  /**
                 * 
                 */
                private static final long serialVersionUID = -2066693655303313807L;

                @Override
                  public void apply(IgniteFuture<String> fut) {
                      //System.out.println("Get operation completed [value=" + fut.get() + ']');
                  }
              });
          }

          long get_end_time = System.currentTimeMillis();
          System.out.println("Time taken to get="+(get_end_time-get_start_time)+" Milli Seconds ");


          // Get the metrics of all the data regions configured on a node.
          Collection<DataRegionMetrics> regionsMetrics = ignite.dataRegionMetrics();
          System.out.println("----------------------------------Data Region Metrics---------------------------------- ");         
          // Print out some of the metrics.
          for (DataRegionMetrics metrics : regionsMetrics) {

              System.out.println(">>> Memory Region Name: " + metrics.getName());
              System.out.println(">>> Allocation Rate: " + metrics.getAllocationRate());
              System.out.println(">>> Fill Factor: " + metrics.getPagesFillFactor());
              System.out.println(">>> Allocated Size: " + metrics.getTotalAllocatedSize());
              System.out.println(">>> Physical Memory Size: " + metrics.getPhysicalMemorySize());
              System.out.println(">>> Total allocated Size: " + metrics.getTotalAllocatedSize());                 
              System.out.println(">>> Eviction Rate: " + metrics.getEvictionRate());                  
              System.out.println(">>> Off Heap Size: " + metrics.getOffHeapSize());
              System.out.println(">>> Off Heap used Size: " + metrics.getOffheapUsedSize());

          }


          System.out.println("Cached size=="+cache.size(CachePeekMode.ALL));
          ignite.compute().broadcast(()->System.out.println(cache.get("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa0")));
          //cache.removeAll();
          cache.clear();
          //cache.removeAllAsync();
          System.out.println("Cached size1=="+cache.size(CachePeekMode.ALL));

        }
        finally {
            // Distributed cache could be removed from cluster only by #destroyCache() call.
            //ignite.destroyCache("myCache");
        }
    }

ignite-stateful-set.yaml

apiVersion: apps/v1beta2
kind: StatefulSet
metadata:
  name: ignite
  namespace: ignite
spec:
  selector:
    matchLabels:
      app: ignite
  serviceName: ignite
  replicas: 2
  template:
    metadata:
      labels:
        app: ignite
    spec:
      serviceAccountName: ignite
      containers:
      - name: ignite
        image: apacheignite/ignite:2.6.0
        env:
        - name: OPTION_LIBS
          value: ignite-kubernetes,ignite-rest-http
        - name: selfLink
          value: file:///opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-default.xml
        - name: IGNITE_QUIET
          value: "false"
        - name: JVM_OPTS
          value: "-Djava.net.preferIPv4Stack=true"
        ports:
        - containerPort: 11211 # JDBC port number.
        - containerPort: 47100 # communication SPI port number.
        - containerPort: 47500 # discovery SPI port number.
        - containerPort: 49112 # JMX port number.
        - containerPort: 10800 # SQL port number.
        - containerPort: 8080 # REST port number.
        - containerPort: 10900 #Thin clients port number.
        volumeMounts:
        - mountPath: "/data/ignite"
          name: ignite-storage
  volumeClaimTemplates:
  - metadata:
      name: ignite-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

启动客户端后的观察。

例外：

Exception in thread "main" class org.apache.ignite.IgniteException: Failed to instantiate Spring XML application context (make sure all classes used in Spring configuration are present at CLASSPATH) [springUrl=file:/opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-ignite.xml]
        at org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:990)
        at org.apache.ignite.Ignition.start(Ignition.java:355)

Caused by: class org.apache.ignite.IgniteCheckedException: Failed to instantiate Spring XML application context (make sure all classes used in Spring configuration are present at CLASSPATH) [springUrl=file:/opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-ignite.xml]

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ignite.cfg$child#0' defined in URL [file:/opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-ignite.xml]: Cannot create inner bean 'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#c038203' of type [org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi] while setting bean property 'discoverySpi'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#c038203' defined in URL 

[file:/opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-default.xml]; nested exception is java.lang.ClassNotFoundException: org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313)

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi#c038203' defined in URL [file:/opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-default.xml]: Cannot create inner bean

[file:/opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-default.xml]; nested exception is java.lang.ClassNotFoundException: org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder

问题的其他形式...

一旦 Kubernetes 按照上述步骤启动并使用 2 个 pod，如何以及在何处部署我的 ignite 数据网格逻辑 client.jar 文件以在其中执行 java 文件。

@clarification

** 下面的文件...

• 名称：selfLink 值：file:///opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-default.xml

应仅包含以下内容**

  <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
    <property name="namespace" value="ignite"/>
  </bean>

**和文件

try (Ignite ignite = Ignition.start("/opt/apache-ignite-fabric-2.6.0-bin/examples/config/example-ignite.xml")) 包含我的自定义配置如下

       <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                <property name="namespace" value="ignite"/>
                <property name="serviceName" value="ignite"/>
                <bean 
                    <property name="addresses">
                        <list>
                            <value>xxx.xxx.xxx.xxx:47500</value>
                            <value>xxx.xxx.xxx.xxx:47500</value>                                
                        </list>
                    </property>
                </bean>

以下标签将帮助 Kubernetes Cluster 识别 Ignite 集群节点。我可以开始运行我的客户端了。

属性名称=“命名空间”值=“点燃” 属性名称=“服务名称”值=“点燃”

如果这不是正确的方法，请更正

@客户端运行

java -Xms5g -Xmx15g -XX:+UseG1GC -XX:+DisableExplicitGC -cp "/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-shmem-1.0.0.jar:/opt /apache-ignite-fabric-2.6.0-bin/libs/cache-api-1.0.0.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/annotations-13.0.jar :/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-core-2.6.0.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-indexing/lucene -queryparser-5.5.2.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-indexing/lucene-core-5.5.2.jar:/opt/apache-ignite-fabric-2.6 .0-bin/libs/ignite-indexing/lucene-analyzers-common-5.5.2.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-indexing/ignite-indexing-2.6。 0.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-indexing/h2-1.4.195.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/ ignite-indexing/commons-codec-1.11.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-spring/spring-tx-4.3.16.RELEASE.jar:/opt/apache- ignite-fabric-2.6.0-bin/libs/ignite-spring/spring-jdbc-4.3.16.RELEASE.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-spring/spring -expression-4.3.16.RELEASE.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-spring/spring-core-4.3.16.RELEASE.jar:/opt/apache-ignite -fabric-2.6.0-bin/libs/ignite-spring/spring-context-4.3.16.RELEASE.jar:/opt/apa che-ignite-fabric-2.6.0-bin/libs/ignite-spring/spring-beans-4.3.16.RELEASE.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-spring /spring-aop-4.3.16.RELEASE.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/ignite-spring/ignite-spring-2.6.0.jar:/opt/apache-ignite -fabric-2.6.0-bin/libs/ignite-spring/commons-logging-1.1.1.jar:/opt/apache-ignite-fabric-2.6.0-bin/libs/optional/ignite-kubernetes.jar: /opt/apache-ignite-fabric-2.6.0-bin/libs/apache-ignite.jar" com.ecoenergy.cortix.ignite.IgniteFutureAsync 1000000

Answer 1

您不能将addresses 用于TcpDiscoveryKubernetesIpFinder。因此，系统无法从 XML 创建它。删除 addresses 部分。