【问题标题】:Elasticsearch Java API: Object mapping for [eventDefinitions] tried to parse field [null] as object, but found a concrete value?Elasticsearch Java API:[eventDefinitions] 的对象映射尝试将字段 [null] 解析为对象,但找到了具体值?
【发布时间】:2020-02-26 00:15:58
【问题描述】:

我正在尝试为以下文档进行映射:

{
  "eventDatabase": "abc",
  "usageLibraryEventType": "ABC",
  "name": "Prionti",
  "namespace": "Prionti's namespace",
  "latestBuildTimestamp": 1581348323634,
  "flattenedEventProperties": [
    "User Id"
  ],
  "eventDefinitions": [
    {
      "buildInfo": {
        "baseVersion": "1",
        "branch": "master",
        "buildName": "something.com",
        "orgName": "Prionti's org",
        "repoName": "myrepo",
        "buildTimestamp": 1581348323634,
        "packageName": "myrepo",
        "packagePath": "",
        "resolvedVersion": "1.2920",
        "rootModuleName": "repo",
        "rootPackagePath": ""
      },
      "eventKey": "myEvent",
      "eventDefinition": {
        "name": "myName",
        "namespace": "myNamespace",
        "meta": {
          "description": "No description available",
          "database": "myDatabase",
          "owner": null,
          "codeOwners": [
            "Prionti Nasir"
          ],
          "imgSrc": null,
          "isPublic": null,
          "yamlSrc": {
            "packageName": "my-package",
            "packageVersion": "static-1.2920",
            "relativePath": "something.yaml"
          }
        },
        "properties": {
          "userId": {
            "type": "number",
            "options": null,
            "isOptional": false,
            "description": null
          }
        },
        "class": "interaction"
      }
    }
  ]
}

我将排除 buildInfo 和其他一些字段,因此我相应地创建了一个映射:


{
  "settings": {
    "index": {
      "number_of_replicas": "2",
      "number_of_shards": "25",
      "analysis": {
        "analyzer": {
          "autocomplete": {
            "filter": [
              "lowercase",
              "autocomplete_filter"
            ],
            "tokenizer": "standard",
            "type": "custom"
          },
          "prefixMatch": {
            "filter": [
              "lowercase"
            ],
            "tokenizer": "standard",
            "type": "custom"
          }
        },
        "filter": {
          "autocomplete_filter": {
            "min_gram": "3",
            "max_gram": "10",
            "type": "edge_ngram"
          }
        }
      }
    }
  },
  "mappings": {
    "usageLibraryEventType": {
      "dynamic": false,
      "properties": {
        "eventDatabase": {
          "properties": {
            "name": {
              "type": "string"
            }
          },
          "enabled": "false"
        },
        "eventType": {
          "type": "string",
          "analyzer": "autocomplete"
        },
        "name": {
          "type": "string",
          "analyzer": "autocomplete"
        },
        "namespace": {
          "type": "string",
          "analyzer": "autocomplete"
        },
        "latestBuildTimestamp": {
          "type": "long"
        },
        "flattenedEventProperties": {
          "type": "string"
        },
        "eventDefinitions": {
          "properties": {
            "eventKey": {
              "type": "string",
              "index": "not_analyzed"
            },
            "eventDefinition": {
              "properties": {
                "name": {
                  "type": "string",
                  "index": "no"
                },
                "namespace": {
                  "type": "string",
                  "index": "no"
                },
                "meta": {
                  "properties": {
                    "description": {
                      "type": "string",
                      "analyzer": "prefixMatch"
                    },
                    "owner": {
                      "type": "string",
                      "index": "not_analyzed",
                      "null_value" : "N/A"
                    },
                    "codeOwners": {
                      "type": "string",
                      "index": "not_analyzed",
                      "null_value" : "N/A"
                    }
                  }
                },
                "class": {
                  "type": "string",
                  "index": "not_analyzed"
                }
              }
            }
          }
        }
      }
    }
  }
}

这给了我一个 MapperParsingException。 eventDefinitions 应该是一个 json 对象列表,每个对象将包含 buildInfoeventKeyeventDefinition。如您所见,eventDefinition 还包含 json 对象。

@POST
  public UsageLibraryEventType indexEventType(UsageLibraryEventType usageLibraryEventType) {
    elasticsearchIndexerClient.addDocumentRequest(
        new IndexRequest(config.getUsageLibrarySourceTopic(), TYPE)
            .source(
                "eventDatabase", usageLibraryEventType.getEventDatabase(),
                "eventType", usageLibraryEventType.getUsageLibraryEventType(),
                "name", usageLibraryEventType.getName(),
                "namespace", usageLibraryEventType.getNamespace(),
                // fix these field names
                "latestBuildTimestamp", usageLibraryEventType.getLatestBuildTimestamp(),
                "flattenedEventProperties", usageLibraryEventType.getFlattenedEventProperties(),
                "eventDefinitions", usageLibraryEventType.getEventDefinitions()),
        config.getUsageLibrarySourceTopic())
        .join();
    return usageLibraryEventType;
  }

(eventDefinitionsEventDefinitionWithBuildInfo的列表,每个EventDefinitionWithBuildInfo包含buildInfoeventKeyeventDefinitionEventDefinition还包含一些字段和一个名为meta的对象。虽然我已经在映射中映射了所有这些,我没有明确地将每个字段的值移交给树中的最后一个分支。我无法在每个 eventDefinitionWithBuildInfo 中输入每个字段,然后在 @ 987654338@ 当然是分开的,所以我必须给它一个列表,因此它不会一直映射到最后一个单元。我该怎么办?我应该定义名为 EventDefinitionWithBuildInfo 的新类型和EventDefinition ?

【问题讨论】:

  • 你用的是哪个版本的es?
  • 如果您发布可以重现此错误的 Java 代码的小示例,我们应该能够帮助您!
  • @IanGabes 已发布!
  • 您的 java 对象不能像这样序列化为 JSON。您正在传递 Elasticsearch 客户端内容,例如“EventDefinition 列表”。 ES 不知道如何将其转换为 JSON。您可以使用 Jackson 之类的库来做到这一点。尝试使用 Jackson 将您的 UsageLibraryEventType 转换为 JSON,然后给 ES 那个字符串。
  • @IanGabes 你是英雄

标签: java elasticsearch elasticsearch-mapping


【解决方案1】:

@IanGabes 是救世主。我整个周末都在哭泣并试图让它发挥作用。但是我的对象是如此嵌套,如此分层,我希望 ES 能够自行检测内部字段。我向这么多人展示了这一点,他们一无所知。然后,当我绝望地看着 Rick 和 Morty 从嘴里掉下薯条时,@IanGabes 像天使一样来了,要我简单地使用 Jackson 将我的对象转换为 Json,并且它不会像这样反序列化到最后一个单元我正在做。谢谢你。你摇滚,我烂。

【讨论】:

    猜你喜欢
    • 2020-10-02
    • 1970-01-01
    • 2023-03-27
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多