org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroRuntimeException: 未知数据类型答案

【问题标题】：org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroRuntimeException: Unknown datum typeorg.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroRuntimeException: 未知数据类型
【发布时间】：2021-08-06 18:12:44
【问题描述】：

我正在使用 avro 版本 1.7.7，scala 版本 2.11.8

我的 avro 架构称为 EntityAuth，其中的枚举如下所示

            {
              "name": "reason",
              "type": [
                "null", 
                {
                  "type": "enum",
                  "name": "BlockedSiteReasonType",
                  "symbols": [
                    "REASON_1",
                    "REASON_2",
                    "REASON_3",
                    "REASON_4",
                    "ADMIN_SITE_CATEGORY_FILTER",
                    "REASON_6",
                    "REASON_7",
                    "REASON_8",
                    "REASON_9"
                  ]
                }
              ],
              "doc": "Agent blocked the entity",
              "default": null
            }

我编写 avro 文件的代码如下所示。 EntityAuth 是 avro 架构类

class EntitySerializationStream(codec: CodecFactory = CodecFactory.snappyCodec()) {
  private val schema = EntityAuth.getClassSchema
  private val datumWriter = new GenericDatumWriter[EntityAuth](schema)
  private val fileName = "some-file-name"
  private val tempFile = File.createTempFile(fileName, "avro.tmp")
  tempFile.deleteOnExit()

  private val dataFileWriter = new DataFileWriter[EntityAuth](datumWriter)
  dataFileWriter.setCodec(codec)
  dataFileWriter.create(schema, tempFile)

  def append(e: EntityAuth) = {
    dataFileWriter.append(e)
  }

当我尝试将记录附加到通用数据文件写入器时遇到的异常是

org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroRuntimeException: Unknown datum type com.goguardian.events.auth.BlockedSiteReasonType: ADMIN_SITE_CATEGORY_FILTER
        at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:296)
Caused by: org.apache.avro.AvroRuntimeException: Unknown datum type com.goguardian.events.auth.BlockedSiteReasonType: ADMIN_SITE_CATEGORY_FILTER
        at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:636)
        at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:601)
        at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:151)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
        at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
        at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
        at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:138)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:68)
        at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
        at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
        at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
        at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:290)
        ... 22 more

这仅在我编写 avro 文件时发生。当我阅读 avro 文件时，我可以很好地解析它。

【问题讨论】：

为什么异常中枚举类型的类名是BlockedSiteReasonType，和schema json中的ReasonType不一样？
@tianzhipeng 好点。他们是一样的。编辑了架构。
请说明如何定义EntityAuth。您是使用avro4s 或类似工具生成的吗？

标签： scala enums avro

【解决方案1】：

如果您已使用此模式生成 Java 类。请更换

private val schema = EntityAuth.getClassSchema
private val datumWriter = new GenericDatumWriter[EntityAuth](schema)

到

private val datumWriter = new SpecificDatumWriter[EntityAuth](classOf[EntityAuth])

SpecificDatumWriter 用于生成的 Java 类。
GenericDatumWriter 用于通用 Java 对象，无需代码生成。

【讨论】：