当被调用者成员发生序列化异常时，Hazelcast 调用者成员抛出超时异常

Question

我在有 2 个成员的场景中使用 Hazelcast（JAVA，版本 3.7.5）。第一个成员通过IExecutorService 将任务委派给第二个成员。经过一些处理后，第二个成员尝试发回一个不可序列化的响应。

由于无法发回响应，第二个成员打印与HazelcastSerializationException相关的堆栈跟踪。

当没有响应到达时，第一个成员会在达到 operation-heartbeat-timeout 时抛出 OperationTimeoutException。

目前，当 IExecutorService 无法解析 Callable 结果时，它会打印一个堆栈跟踪（在被调用方）。 假设我有一个简单的调用者：

private Future<Object> startFlow() {
    //This throws an OperationTimeoutException
    return hazelcastInstance.getExecutorService("myExecutor").submit(myRunnable);
}

调用一个简单的被调用者：

@Override
public Object call() throws Exception {
    //The object returned is not serializable, therefore an HazelcastSerializationException is thrown
    return service.execute();
}

被调用者在解析响应失败后打印一个堆栈跟踪（见帖子末尾）。

就我而言，不可能知道service 可能返回什么样的对象，也不可能相信service 会发回可序列化的对象。

我想知道调用方超时的原因。

经过一番搜索，我发现当 IExecutorService 无法序列化响应时，没有可用的配置/API 来拦截 IExecutorService 引发的异常。

所以我尝试看看是否可以检查一个对象是否可以被 Hazelcast 解析，但还是没有成功。

有什么想法吗？

谢谢

---

被调用者打印的堆栈跟踪将如下所示：

Exception in thread "hz._hzInstance_1_dev.cached.thread-1" com.hazelcast.nio.serialization.HazelcastSerializationException: Failed to serialize 'com.hazelcast.spi.impl.operationservice.impl.responses.NormalResponse'
    at com.hazelcast.internal.serialization.impl.SerializationUtil.handleSerializeException(SerializationUtil.java:73)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:143)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:124)
    at com.hazelcast.spi.impl.operationservice.impl.OperationServiceImpl.send(OperationServiceImpl.java:427)
    at com.hazelcast.spi.impl.operationservice.impl.RemoteInvocationResponseHandler.sendResponse(RemoteInvocationResponseHandler.java:51)
    at com.hazelcast.spi.Operation.sendResponse(Operation.java:291)
    at com.hazelcast.executor.impl.DistributedExecutorService$CallableProcessor.sendResponse(DistributedExecutorService.java:269)
    at com.hazelcast.executor.impl.DistributedExecutorService$CallableProcessor.run(DistributedExecutorService.java:253)
    at com.hazelcast.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:212)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at com.hazelcast.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
    at com.hazelcast.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:92)
Caused by: com.hazelcast.nio.serialization.HazelcastSerializationException: Failed to serialize 'com.myomain.UnserialiableObject'
    at com.hazelcast.internal.serialization.impl.SerializationUtil.handleSerializeException(SerializationUtil.java:73)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.writeObject(AbstractSerializationService.java:236)
    at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:371)
    at com.hazelcast.spi.impl.operationservice.impl.responses.NormalResponse.writeData(NormalResponse.java:91)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:189)
    at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.write(DataSerializableSerializer.java:54)
    at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.write(StreamSerializerAdapter.java:43)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toBytes(AbstractSerializationService.java:140)
    ... 12 more
Caused by: com.hazelcast.nio.serialization.HazelcastSerializationException: There is no suitable serializer for class com.myomain.UnserialiableObject
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.serializerFor(AbstractSerializationService.java:469)
    at com.hazelcast.internal.serialization.impl.AbstractSerializationService.writeObject(AbstractSerializationService.java:232)
    ... 18 more

编辑（解决方案）

所以我最终注册了一个全局序列化程序，只要调用它就会简单地发送一个异常。像这样的：

public class GlobalSerializerException  implements StreamSerializer<Object> {

    @Override
    public void write(ObjectDataOutput out, Object object) throws IOException {
        String objectInfo;
        if(object == null){
            objectInfo = "Object was null.";
        }else{
            objectInfo = String.format("Object of class %s and printed as String gives %s", object.getClass().getCanonicalName(), object.toString());
        }
        objectInfo = "Hazelcast was unable to serialize an object. " + objectInfo;
        out.writeUTF(objectInfo);
    }

    @Override
    public Object read(ObjectDataInput in) throws IOException {
        String message = in.readUTF();
        HazelcastSerializationException hazelcastSerializationException = new HazelcastSerializationException(message);
        return hazelcastSerializationException;
    }

    @Override
    public int getTypeId() {
        return 63426;
    }

    @Override
    public void destroy() {

    }
} 

Answer 1

调用者因超时而失败，因为它没有得到目标的响应。目标节点未能序列化响应，原因是未能发送响应。这是当前的行为，但我认为也可以发送一个特殊的异常来表示响应失败。

默认情况下，Hazelcast 能够序列化实现 java.io.Serializable、java.io.Externalizable 和一些 Hazelcast 特定接口的类，例如 DataSerializable、Portable。也可以定义自定义序列化程序或委托给另一个序列化库。有关更多信息，请参阅Hazelcast Reference Manual - Serialization 部分。

在分布式系统中，节点之间交换的消息必须可序列化为二进制形式才能通过网络传输。因此，参与分布式系统的实体/服务必须确保其消息以某种形式可序列化。

如果您不知道消息的类型，那么您可以注册 Hazelcast 一个global serializer，它首先尝试使用已知格式（Serializable、Externalizable 等）进行序列化，如果类型未知则它改为写入自定义错误消息。 或者，您可以使用自定义的可序列化包装器对象来包装服务的执行结果。在序列化过程中，如果原始包装结果序列化失败，则再次编写自定义错误消息。

例如；

class NonSerializableResponseException extends Exception {}

class ServiceResponseWrapper implements DataSerializable {

    private Object response;

    @Override
    public void writeData(ObjectDataOutput out) throws IOException {
        try {
            out.writeObject(response);
        } catch (HazelcastSerializationException e) {
            out.writeObject(new NonSerializableResponseException());
        }
    }

    @Override
    public void readData(ObjectDataInput in) throws IOException {
        response = in.readObject();
    }
}