Spring Boot IMAP 通道适配器在处理大量入站电子邮件时丢失电子邮件

Question

我有一个正在工作的 Spring Boot 应用程序，它正在处理来自专用邮箱的电子邮件。当电子邮件数量有限时，一切都在测试环境中运行良好。在生产环境中，这些电子邮件是由计划的作业生成的，有时一批中有超过 10000 封电子邮件。每封电子邮件都有一些大附件，应用程序正在解析和存储这些附件。 这是应用程序严重减速的地方，在最坏的情况下也会丢失电子邮件。看起来像线程饥饿的情况，因为该过程是资源密集型的。 有没有办法优化这个流程？

我的电子邮件配置如下。 Service Activator sn-p 来自另一个服务来处理电子邮件。

    @Bean
    public QueueChannel emailReceiveChannel() {
        return new QueueChannel(100);
    }

    @Bean(name = "emailPoller")
    public PollerMetadata poller() {
        PollerMetadata poller = new PollerMetadata();
        poller.setReceiveTimeout(0);
        poller.setTrigger(new PeriodicTrigger(2000));
        poller.setMaxMessagesPerPoll(100);
        poller.setTaskExecutor(taskExecutor());
        
        return poller;
    }
    
    @Bean(name = "emailTaskExecutor")
    public AsyncTaskExecutor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(10);
        executor.setMaxPoolSize(1000);
        executor.setQueueCapacity(100);
        
        return executor;
    }

    @Bean
    public ImapIdleChannelAdapter mailAdapter() {
        
        ImapIdleChannelAdapter imapAdapter = new ImapIdleChannelAdapter(imapMailReceiver());
        imapAdapter.setAutoStartup(true);
        imapAdapter.setOutputChannel(emailReceiveChannel());
        
        return imapAdapter;
    }

    @Bean
    public ImapMailReceiver imapMailReceiver() {

        StringBuilder impaURI = new StringBuilder();
        impaURI.append(MAIL_PROTOCOL).append("://").append(MAIL_USERNAME).append(":").append(MAIL_PASSWORD)
        .append("@").append(MAIL_HOST).append(":").append(MAIL_PORT).append("/").append(MAIL_FOLDER);
        
        logger.info("IMAP URL = " + impaURI.toString());
        
        ImapMailReceiver mailReceiver = new ImapMailReceiver(impaURI.toString());
        mailReceiver.setShouldDeleteMessages(true);
        mailReceiver.setShouldMarkMessagesAsRead(true);
        mailReceiver.setJavaMailProperties(javaMailProperties());
        mailReceiver.setAutoCloseFolder(false);

        return mailReceiver;
    }
     
    @ServiceActivator(inputChannel = "emailReceiveChannel", poller = @Poller(value = "emailPoller"))
    public void handleMessage(final Message message) {
        
        logger.info("Email Message received on " + LocalDate.now());
        if (message != null && message instanceof MimeMessage) {
            /*
             * Delegate the message handling to another thread and release this one.
             */
            processEmailMessage(message);
        }
    }

    /*
     * Global Task scheduler
     */
    @Bean(name = "taskScheduler")
    public TaskScheduler taskScheduler() {
        ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler();
        scheduler.setPoolSize(100);
        return scheduler;
    }

Answer 1

你可能需要考虑增加TaskScheduler线程池：https://docs.spring.io/spring-integration/docs/current/reference/html/configuration.html#namespace-taskscheduler。

您也可以考虑不要将 emailReceiveChannel 用作队列。 ImapIdleChannelAdapter 本质上是异步的，并且已经在调度线程中执行其逻辑。