【问题标题】:How to integrate ElasticSearch 7.0 version with Spring Boot?如何将 ElasticSearch 7.0 版本与 Spring Boot 集成?
【发布时间】:2019-09-09 16:23:42
【问题描述】:

我正在尝试使用 maven 存储库中已经提供的最新版本的 Elastic Search 库。

<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>7.0.0</version>
</dependency>

但不确定如何将第 7 版与导入 6.5 的 Spring Boot 一起使用。 我的 Maven 依赖项:

  <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
  </dependency>

【问题讨论】:

  • 在您的 pom.xml 文件中添加 elasticsearch (版本 7.0.0)版本而不是 spring-boot-starter-data-elasticsearch 依赖项,您就可以使用它了。
  • This answer 可能会有所帮助。根据您的需要替换版本号。
  • 对于我的情况,我切换到通过弹性而不是弹簧数据弹性搜索使用 RestHighLevelClient
  • @SarvarNishonboev 这里是 GitHub 上拉取请求的链接。我猜我们必须等待...github.com/spring-projects/spring-data-elasticsearch/pull/284
  • 据我所知,它应该在Spring Data Elasticsearch 4.x 版本中得到支持。有人知道该版本的预计发布日期吗?

标签: spring spring-boot elasticsearch


【解决方案1】:

更新

Spring Boot 2.3 正在集成 spring-data-elasticsearch 4,因此它将支持 ElasticSearch 7.x 开箱即用。它很快就会发布,但你已经可以尝试了:

plugins {
  id 'org.springframework.boot' version '2.3.0.RC1'
  id 'io.spring.dependency-management' version '1.0.9.RELEASE'
}

我已经对它进行了积极的测试,并且我的所有测试场景都通过了,所以我肯定会推荐这种方式。对于由于某些原因无法升级到 2.3 的人,我将在下面保留答案。

OLD WORKAROUND(以前版本的原始答案)

由于我们真的不知道 Spring Data Elastic Search 4.x 何时发布,所以我发布了我集成当前 Spring 的方式数据弹性搜索 4.x 和稳定的 Spring Boot 2.1.7。如果您想使用 Spring Repositories 和最新的 Elastic Search,它可能会作为您的临时解决方法。

1) 在您的依赖项中强制使用最新的弹性搜索客户端(在我的例子中:build.gradle

dependencies {
    //Force spring-data to use the newest elastic-search client
    //this should removed as soon as spring-data-elasticsearch:4.0.0 is released!
    implementation('org.springframework.data:spring-data-elasticsearch:4.0.0.BUILD-SNAPSHOT') {
        exclude group: 'org.elasticsearch'
        exclude group: 'org.elasticsearch.plugin'
        exclude group: 'org.elasticsearch.client'
    }

    implementation('org.elasticsearch:elasticsearch:7.3.0') { force = true }
    implementation('org.elasticsearch.client:elasticsearch-rest-high-level-client:7.3.0') { force = true }
    implementation('org.elasticsearch.client:elasticsearch-rest-client:7.3.0') { force = true }
}

2) 禁用 Elastic Search 自动配置和健康检查组件,因为它们变得不兼容(您以后可能想要实施自己的健康检查)。

@SpringBootApplication(exclude = {ElasticsearchAutoConfiguration.class, ElasticSearchRestHealthIndicatorAutoConfiguration.class})
@EnableElasticsearchRepositories
public class SpringBootApp {

    public static void main(String[] args) {
        SpringApplication.run(SpringBootApp.class, args);
    }

}

3) 当我们禁用自动配置时,我们需要自己初始化ElasticsearchRestTemplate。我们还需要提供自定义MappingElasticsearchConverter 以避免类不兼容。

/**
 * Manual configuration to support the newest ElasticSearch that is currently not supported by {@link org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchAutoConfiguration}.
 *
 * @author aleksanderlech
 */
@Configuration
@EnableConfigurationProperties(ElasticsearchProperties.class)
public class ElasticSearchConfiguration {

    @Primary
    @Bean
    public ElasticsearchRestTemplate elasticsearchTemplate(ElasticsearchProperties configuration) {
        var nodes =  Stream.of(configuration.getClusterNodes().split(",")).map(HttpHost::create).toArray(HttpHost[]::new);
        var client = new RestHighLevelClient(RestClient.builder(nodes));
        var converter = new CustomElasticSearchConverter(new SimpleElasticsearchMappingContext(), createConversionService());
        return new ElasticsearchRestTemplate(client, converter, new DefaultResultMapper(converter));
    }

    private DefaultConversionService createConversionService() {
        var conversionService = new DefaultConversionService();
        conversionService.addConverter(new StringToLocalDateConverter());
        return conversionService;
    }
}

CustomElasticSearchConverter:

/**
 * Custom version of {@link MappingElasticsearchConverter} to support newest Spring Data Elasticsearch integration that supports ElasticSearch 7. Remove when Spring Data Elasticsearch 4.x is released.
 */
class CustomElasticSearchConverter extends MappingElasticsearchConverter {

    private CustomConversions conversions = new ElasticsearchCustomConversions(Collections.emptyList());

    CustomElasticSearchConverter(MappingContext<? extends ElasticsearchPersistentEntity<?>, ElasticsearchPersistentProperty> mappingContext) {
        super(mappingContext);
        setConversions(conversions);
    }

    CustomElasticSearchConverter(MappingContext<? extends ElasticsearchPersistentEntity<?>, ElasticsearchPersistentProperty> mappingContext, GenericConversionService conversionService) {
        super(mappingContext, conversionService);
        setConversions(conversions);
    }

    @Override
    protected <R> R readValue(@Nullable Object source, ElasticsearchPersistentProperty property,
                              TypeInformation<R> targetType) {

        if (source == null) {
            return null;
        }

        if (source instanceof List) {
            return readCollectionValue((List) source, property, targetType);
        }

        return super.readValue(source, property, targetType);
    }

    private Object readSimpleValue(@Nullable Object value, TypeInformation<?> targetType) {

        Class<?> target = targetType.getType();

        if (value == null || target == null || ClassUtils.isAssignableValue(target, value)) {
            return value;
        }

        if (conversions.hasCustomReadTarget(value.getClass(), target)) {
            return getConversionService().convert(value, target);
        }

        if (Enum.class.isAssignableFrom(target)) {
            return Enum.valueOf((Class<Enum>) target, value.toString());
        }

        return getConversionService().convert(value, target);
    }


    private <R> R readCollectionValue(@Nullable List<?> source, ElasticsearchPersistentProperty property,
                                      TypeInformation<R> targetType) {

        if (source == null) {
            return null;
        }

        Collection<Object> target = createCollectionForValue(targetType, source.size());

        for (Object value : source) {

            if (isSimpleType(value)) {
                target.add(
                        readSimpleValue(value, targetType.getComponentType() != null ? targetType.getComponentType() : targetType));
            } else {

                if (value instanceof List) {
                    target.add(readValue(value, property, property.getTypeInformation().getActualType()));
                } else {
                    target.add(readEntity(computeGenericValueTypeForRead(property, value), (Map) value));
                }
            }
        }

        return (R) target;
    }

    private Collection<Object> createCollectionForValue(TypeInformation<?> collectionTypeInformation, int size) {

        Class<?> collectionType = collectionTypeInformation.isCollectionLike()//
                ? collectionTypeInformation.getType() //
                : List.class;

        TypeInformation<?> componentType = collectionTypeInformation.getComponentType() != null //
                ? collectionTypeInformation.getComponentType() //
                : ClassTypeInformation.OBJECT;

        return collectionTypeInformation.getType().isArray() //
                ? new ArrayList<>(size) //
                : CollectionFactory.createCollection(collectionType, componentType.getType(), size);
    }

    private ElasticsearchPersistentEntity<?> computeGenericValueTypeForRead(ElasticsearchPersistentProperty property,
                                                                            Object value) {

        return ClassTypeInformation.OBJECT.equals(property.getTypeInformation().getActualType())
                ? getMappingContext().getRequiredPersistentEntity(value.getClass())
                : getMappingContext().getRequiredPersistentEntity(property.getTypeInformation().getActualType());
    }

    private boolean isSimpleType(Object value) {
        return isSimpleType(value.getClass());
    }

    private boolean isSimpleType(Class<?> type) {
        return conversions.isSimpleType(type);
    }

}

【讨论】:

  • 嗨,Aleksander,谢谢,刚刚测试了你的方法,对我来说效果很好。存储库 findBy 方法的先前问题已经消失。不过,有一个问题:您是否只是忽略以下警告?:[299 Elasticsearch-7.3.0-de777fa "[types removal] Specifying types in document index requests is deprecated 或者您有解决方案吗?
  • 这对我不起作用。抛出:“ClassNotFoundException:org.springframework.data.mapping.model.EntityInstantiators”
  • 可能你使用了从未使用过的快照,无论如何我都会更新我的答案,因为现在它是一个更好的解决方案
  • 不幸的是,它不适用于 org.springframework.data:spring-data-elasticsearch:4.0.0.RELEASE
【解决方案2】:

如果有人使用 Spring Boot 2.1.2Kotlin,以下代码可能会对您有所帮助。我刚刚从@Alexander Lech 的回答中翻译了它,做了一些小改动:

第一次更改 Alexanders 答案:

@SpringBootApplication(exclude = [ElasticsearchAutoConfiguration::class, 
ElasticsearchDataAutoConfiguration::class])

我必须排除 ElasticsearchDataAutoConfiguration 才能使其正常工作。

第二:由于我们使用 Kotlin,并且自定义转换器是很多代码,也许这种对 Kotlin 的翻译会对某人有所帮助:

class CustomElasticSearchConverter(mappingContext: MappingContext<out ElasticsearchPersistentEntity<*>, ElasticsearchPersistentProperty>, customConversionService: GenericConversionService?) : MappingElasticsearchConverter(mappingContext, customConversionService) {

    private val conversionsNew = ElasticsearchCustomConversions(emptyList<Any>())

    init {
        setConversions(conversionsNew)
    }

    override fun <R : Any?> readValue(source: Any?, property: ElasticsearchPersistentProperty, targetType: TypeInformation<R>): R? {
        if (source == null) {
            return null
        }

        if (source is Collection<*>) {
            return readCollectionValue(source, property, targetType) as R?;
        }

        return super.readValue(source, property, targetType);
    }

    private fun readCollectionValue(source: Collection<*>?, property: ElasticsearchPersistentProperty, targetType: TypeInformation<*>): Any? {

        if (source == null) {
            return null
        }

        val target = createCollectionForValue(targetType, source.size)

        for (value in source) {
            require(value != null) { "value must not be null" }

            if (isSimpleType(value)) {
                target.add(readSimpleValue(value, if (targetType.componentType != null) targetType.componentType!! else targetType))
            } else {
                if (value is MutableCollection<*>) {
                    target.add(readValue(value, property, property.typeInformation.actualType as TypeInformation<out Any>))
                } else {
                    @Suppress("UNCHECKED_CAST")
                    target.add(readEntity(computeGenericValueTypeForRead(property, value), value as MutableMap<String, Any>?))
                }
            }
        }

        return target
    }

    private fun readSimpleValue(value: Any?, targetType: TypeInformation<*>): Any? {

        val target = targetType.type

        @Suppress("SENSELESS_COMPARISON")
        if (value == null || target == null || ClassUtils.isAssignableValue(target, value)) {
            return value
        }

        if (conversionsNew.hasCustomReadTarget(value.javaClass, target)) {
            return conversionService.convert(value, target)
        }

        @Suppress("UNCHECKED_CAST")
        return when {
            Enum::class.java.isAssignableFrom(target) -> enumByName(target as Class<Enum<*>>, value.toString())
            else -> conversionService.convert(value, target)
        }
    }

    private fun enumByName(target: Class<Enum<*>>, name: String): Enum<*> {
        val enumValue = target.enumConstants.find { it.name == name }
        require(enumValue != null) { "no enum value found for name $name and targetClass $target" }
        return enumValue
    }

    private fun createCollectionForValue(collectionTypeInformation: TypeInformation<*>, size: Int): MutableCollection<Any?> {

        val collectionType = when {
            collectionTypeInformation.isCollectionLike -> collectionTypeInformation.type
            else -> MutableList::class.java
        }

        val componentType = when {
            collectionTypeInformation.componentType != null -> collectionTypeInformation.componentType
            else -> ClassTypeInformation.OBJECT
        }

        return when {
            collectionTypeInformation.type.isArray -> ArrayList(size)
            else -> CollectionFactory.createCollection(collectionType, componentType!!.type, size)
        }
    }

    private fun computeGenericValueTypeForRead(property: ElasticsearchPersistentProperty, value: Any): ElasticsearchPersistentEntity<*> {

        return when {
            ClassTypeInformation.OBJECT == property.typeInformation.actualType -> mappingContext.getRequiredPersistentEntity(value.javaClass)
            else -> mappingContext.getRequiredPersistentEntity(property.typeInformation.actualType!!)
        }
    }

    private fun isSimpleType(value: Any): Boolean {
        return isSimpleType(value.javaClass)
    }

    private fun isSimpleType(type: Class<*>): Boolean {
        return conversionsNew.isSimpleType(type)
    }

}

在此之后,解决了一些存储库查询的问题。还请注意不要使用spring-boot-starter-data-elasticsearch,而是使用spring-data-elasticsearch:4.0.0.BUILD-SNAPSHOT。 (这花了我一些时间)。

是的,代码很丑,但是spring-data-elasticsearch:4.0.0发布后,你可以扔掉了。

【讨论】:

  • 嗨,我正在尝试使用上面提到的代码,但我收到以下错误2019-11-26 14:33:45,287 ERROR [restartedMain] org.springframework.data.elasticsearch.repository.support.AbstractElasticsearchRepository : failed to load elasticsearch nodes : org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=mapper_parsing_exception, reason=No type specified for field [properties]]
  • 好的,首先我注意到,使用 4.0.0-SNAPHOST 有点不安全,我建议使用“4.0.0.DATAES-690-SNAPSHOT”,因为 4.0.0 -SNAPSHOT 变化太频繁。其次,对于您的问题:它表明您在@Document 中使用了一个未知类型,您需要使用@Field(type = FieldType.&lt;AnyfieldType&gt;) 对其进行注释。也许这可以帮助您绘制地图。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2019-02-12
  • 2018-01-03
  • 2020-01-20
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多