处理 HardDeadlineExceededError 的任务策略答案

【问题标题】：task strategies for handling HardDeadlineExceededError处理 HardDeadlineExceededError 的任务策略
【发布时间】：2011-02-25 18:58:05
【问题描述】：

我有许多任务/servlet 遇到了 HardDeadlineExceededError，这使所有内容都处于“仍在执行”状态。

正在完成的工作很容易超过 29 秒的阈值。

我尝试捕获 DeadlineExceededException 和基本异常以保存退出状态，但这些异常处理程序都没有被捕获...

有没有办法确定哪些任务在队列中或当前正在执行？

还有其他应对这种情况的策略吗？

我正在处理的情况记录在 "The Request Timer" 标题下。

// task handler for retrieving information from external web services
protected void doPost(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {

    String taskRetryCountParam = req.getParameter( "X-AppEngine-TaskRetryCount" );
    int taskRetryCount = (taskRetryCountParam==null) ? 0 : Integer.parseInt(taskRetryCountParam);
            // look up the persistent 'task' and mark it as 'running'

    logger.info( this.getClass().getName() + ".doPost( "+ taskId + " ) retryCount=" + taskRestryCount );


    // Do lots of heavy lifting here 
    // like calling external web services using URL fetch service
            // and saving the contents into our database.

            // look up the persistent 'task' and mark it as 'completed'

    } catch ( DeadlineExceededException deadline ) {
        // got this deadline exception
                    // look up the persistent 'task' and mark it as 'errored - try again'
        logger.warning( "DeadlineExceeded Exception while loading content " + deadline.getMessage() );
        resp.setStatus( HttpServletResponse.SC_REQUEST_TIMEOUT );

        }
    } catch ( Exception unknown ) {
        // got some unknown exception
                    // look up the persistent 'task' and mark it as 'errored - cancelled'
        logger.severe( "General Exception while loading content  exception:" + unknown.getMessage() );
        resp.setStatus( HttpServletResponse.SC_OK );

    }
}

这是我遇到这种情况时的日志文件条目...看来我的数据库事务处理时间太长了。

  W 05-30 12:42PM 09.535
  Error for /loadstatus
  com.google.apphosting.runtime.HardDeadlineExceededError: This request (083793d1091c2ca3) started at 2010/05/30 19:41:39.814 UTC and was still executing at 2010/05/30 19:42:09.529 UTC.
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:443)
    at java.util.concurrent.TimeUnit.timedWait(Unknown Source)
    at com.google.apphosting.runtime.AsyncFuture.get(AsyncFuture.java:60)
    at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:326)
    at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:217)
    at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:131)
    at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:43)
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:104)
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:102)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:102)
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:43)
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98)
    at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:58)
    at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:42)
    at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:56)
    at com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java:66)
    at org.datanucleus.store.appengine.DatastoreTransaction.commit(DatastoreTransaction.java:61)
    at org.datanucleus.store.appengine.DatastoreXAResource.commit(DatastoreXAResource.java:88)
    at org.datanucleus.transaction.Transaction.commit(Transaction.java:149)
    at org.datanucleus.transaction.TransactionManager.commit(TransactionManager.java:95)
    at org.datanucleus.TransactionImpl.internalCommit(TransactionImpl.java:390)
    at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:258)
    at org.datanucleus.jdo.JDOTransaction.commit(JDOTransaction.java:83)
    at org.datanucleus.store.appengine.jdo.DatastoreJDOTransaction.commit(DatastoreJDOTransaction.java:56)
    <snip>
  C 05-30 12:42PM 09.629
  Uncaught exception from servlet
  com.google.apphosting.runtime.HardDeadlineExceededError: This request (083793d1091c2ca3) started at 2010/05/30 19:41:39.814 UTC and was still executing at 2010/05/30 19:42:09.529 UTC.
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:443)
    at java.util.concurrent.TimeUnit.timedWait(Unknown Source)
    at com.google.apphosting.runtime.AsyncFuture.get(AsyncFuture.java:60)
    at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:326)
    at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:217)
    at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:131)
    at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:43)
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:104)
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:102)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:102)
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:43)
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98)
    at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:58)
    at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:42)
    at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:56)
    at com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java:66)
    at org.datanucleus.store.appengine.DatastoreTransaction.commit(DatastoreTransaction.java:61)
    at org.datanucleus.store.appengine.DatastoreXAResource.commit(DatastoreXAResource.java:88)
    at org.datanucleus.transaction.Transaction.commit(Transaction.java:149)
    at org.datanucleus.transaction.TransactionManager.commit(TransactionManager.java:95)
    at org.datanucleus.TransactionImpl.internalCommit(TransactionImpl.java:390)
    at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:258)
    at org.datanucleus.jdo.JDOTransaction.commit(JDOTransaction.java:83)
    at org.datanucleus.store.appengine.jdo.DatastoreJDOTransaction.commit(DatastoreJDOTransaction.java:56)
    <snip>
  W 05-30 12:42PM 09.644
  A serious problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may be throwing exceptions during the initialization of your application. (Error code 104)

【问题讨论】：

你应该可以捕捉到DeadlineExceededException。你能发布一些你的代码吗？
@Matthew：我不做 GAE，但HardDeadlineExceededError 听起来不像是Exception。
@Balus，我没说是。 DeadlineExceededException 和 HardDeadlineExceededError 是两个不同的东西。前者可以抓。后者不能，如果您处理前者的时间超过一秒钟，则会被抛出。
@Matthew：我把它解释为：他们不能被抓住，因为实际上一个HardDeadlineExceededError被抛出了。
我没有看到任何引发/捕获的 DeadlineExceededExceptions

标签： java google-app-engine servlets scheduled-tasks

【解决方案1】：

http://groups.google.com/group/google-appengine-java/msg/e3fd2b621bb96013

如果发生在您自己的代码中，则可以在没有 DEE 的情况下抛出 HDEE。
通常最耗时的事情是等待 API 调用
return 所以这里的超时将导致 API 调用以 a
停止迪。因此，如果您不经常调用 API，您可以直接点击 HDEE。

我也有长时间运行的任务，通过数据处理进行迭代
并存储结果。我使用了一个停止返回结果的迭代器
20 秒后保存最后处理的对象，然后踢
关闭一个新任务继续处理。

我最初的解决方案捕获了 DEE，然后进行了清理，但这
停止可靠工作。

【讨论】：

感谢马克的输入。您是在捕捉错误并从中推断 HDEE 还是其他什么？
您无法捕获 HDEE 错误。不是因为你不能添加这样的捕获（你可以），而是你的代码被停止了。为此，有 DEE，它应该在 HDEE 之前大约半秒被抛出，但如果你在自己的代码中，这不起作用。