适用于 .Net 的 Google API 客户端：请求失败时实现重试

Question

如果作为批处理请求的一部分的请求在与谷歌的 API 交互时失败，我该如何实现重试。在他们的documentation 中，他们建议添加“指数退避”算法。我在他们的文档中使用了以下codesn-p：

UserCredential credential;
using (var stream = new FileStream("client_secrets.json", FileMode.Open, FileAccess.Read))
{
    credential = await GoogleWebAuthorizationBroker.AuthorizeAsync(
        GoogleClientSecrets.Load(stream).Secrets,
        new[] { CalendarService.Scope.Calendar },
        "user", CancellationToken.None, new FileDataStore("Calendar.Sample.Store"));
}

// Create the service.
var service = new CalendarService(new BaseClientService.Initializer()
    {
        HttpClientInitializer = credential,
        ApplicationName = "Google Calendar API Sample",
    });

// Create a batch request.
var request = new BatchRequest(service);
request.Queue<CalendarList>(service.CalendarList.List(),
     (content, error, i, message) =>
     {
         // Put your callback code here.
     });
request.Queue<Event>(service.Events.Insert(
     new Event
     {
         Summary = "Learn how to execute a batch request",
         Start = new EventDateTime() { DateTime = new DateTime(2014, 1, 1, 10, 0, 0) },
         End = new EventDateTime() { DateTime = new DateTime(2014, 1, 1, 12, 0, 0) }
     }, "YOUR_CALENDAR_ID_HERE"),
     (content, error, i, message) =>
     {
         // Put your callback code here.
     });
// You can add more Queue calls here.

// Execute the batch request, which includes the 2 requests above.
await request.ExecuteAsync();

Answer 1

这是一个简单的帮助器类，可以轻松地为 Google 在其 API 错误页面上讨论的许多情况实现指数退避：https://developers.google.com/calendar/v3/errors

如何使用：

• 编辑下面的类以包含您在https://console.developers.google.com 上设置的客户端密码和应用程​​序名称
• 在你的应用程序启动时（或当你要求用户授权时），调用GCalAPIHelper.Instance.Auth();
• 您可以在任何地方调用 Google 日历 API（例如 Get、Insert、Delete 等），而是通过以下方式使用此类：GCalAPIHelper.Instance.CreateEvent(event, calendarId);（您可能需要根据需要将此类扩展到其他 API 端点）

using Google;
using Google.Apis.Auth.OAuth2;
using Google.Apis.Calendar.v3;
using Google.Apis.Calendar.v3.Data;
using Google.Apis.Services;
using Google.Apis.Util.Store;
using System;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Threading;
using static Google.Apis.Calendar.v3.CalendarListResource.ListRequest;

/*======================================================================================
 * This file is to implement Google Calendar .NET API endpoints WITH exponential backoff.
 * 
 * How to use:
 *    - Install the Google Calendar .NET API (nuget.org/packages/Google.Apis.Calendar.v3)
 *    - Edit the class below to include your client secret and application name as you 
 *      set up on https://console.developers.google.com
 *    - In the startup of your application (or when you ask the user to authorize), call
 *      GCalAPIHelper.Instance.Auth();
 *    - Anywhere you would call the Google Calendar API (eg Get, Insert, Delete, etc),
 *      instead use this class by doing: 
 *      GCalAPIHelper.Instance.CreateEvent(event, calendarId); (you may need to expand
 *      this class to other API endpoints as your needs require) 
 *======================================================================================
 */

namespace APIHelper
{
    public class GCalAPIHelper
    {
        #region Singleton
        private static GCalAPIHelper instance;

        public static GCalAPIHelper Instance
        {
            get
            {
                if (instance == null)
                    instance = new GCalAPIHelper();

                return instance;
            }
        }
        #endregion Singleton

        #region Private Properties
        private CalendarService service { get; set; }
        private string[] scopes = { CalendarService.Scope.Calendar };
        private const string CLIENTSECRETSTRING = "YOUR_SECRET"; //Paste in your JSON client secret here. Don't forget to escape special characters!
        private const string APPNAME = "YOUR_APPLICATION_NAME"; //Paste in your Application name here
        #endregion Private Properties

        #region Constructor and Initializations
        public GCalAPIHelper()
        {

        }

        public void Auth(string credentialsPath)
        {
            if (service != null)
                return;

            UserCredential credential;
            byte[] byteArray = Encoding.ASCII.GetBytes(CLIENTSECRETSTRING);

            using (var stream = new MemoryStream(byteArray))
            {
                credential = GoogleWebAuthorizationBroker.AuthorizeAsync(
                    GoogleClientSecrets.Load(stream).Secrets,
                    scopes,
                    Environment.UserName,
                    CancellationToken.None,
                    new FileDataStore(credentialsPath, true)).Result;
            }

            service = new CalendarService(new BaseClientService.Initializer()
            {
                HttpClientInitializer = credential,
                ApplicationName = APPNAME
            });
        }
        #endregion Constructor and Initializations

        #region Private Methods
        private TResponse DoActionWithExponentialBackoff<TResponse>(CalendarBaseServiceRequest<TResponse> request)
        {
            return DoActionWithExponentialBackoff(request, new HttpStatusCode[0]);
        }

        private TResponse DoActionWithExponentialBackoff<TResponse>(CalendarBaseServiceRequest<TResponse> request, HttpStatusCode[] otherBackoffCodes)
        {
            int delay = 100;
            while (delay < 1000) //If the delay gets above 1 second, give up
            {
                try
                {
                    return request.Execute();
                }
                catch (GoogleApiException ex)
                {
                    if (ex.HttpStatusCode == HttpStatusCode.Forbidden || //Rate limit exceeded
                        ex.HttpStatusCode == HttpStatusCode.ServiceUnavailable || //Backend error
                        ex.HttpStatusCode == HttpStatusCode.NotFound ||
                        ex.Message.Contains("That’s an error") || //Handles the Google error pages like https://i.imgur.com/lFDKFro.png
                        otherBackoffCodes.Contains(ex.HttpStatusCode))
                    {
                        Common.Log($"Request failed. Waiting {delay} ms before trying again");
                        Thread.Sleep(delay);
                        delay += 100;
                    }
                    else
                        throw;
                }
            }

            throw new Exception("Retry attempts failed");
        }
        #endregion Private Methods

        #region Public Properties
        public bool IsAuthorized
        {
            get { return service != null; }
        }
        #endregion Public Properties

        #region Public Methods
        public Event CreateEvent(Event eventToCreate, string calendarId)
        {
            EventsResource.InsertRequest eventCreateRequest = service.Events.Insert(eventToCreate, calendarId);
            return DoActionWithExponentialBackoff(eventCreateRequest);
        }

        public Event InsertEvent(Event eventToInsert, string calendarId)
        {
            EventsResource.InsertRequest eventCopyRequest = service.Events.Insert(eventToInsert, calendarId);
            return DoActionWithExponentialBackoff(eventCopyRequest);
        }

        public Event UpdateEvent(Event eventToUpdate, string calendarId, bool sendNotifications = false)
        {
            EventsResource.UpdateRequest eventUpdateRequest = service.Events.Update(eventToUpdate, calendarId, eventToUpdate.Id);
            eventUpdateRequest.SendNotifications = sendNotifications;
            return DoActionWithExponentialBackoff(eventUpdateRequest);
        }

        public Event GetEvent(Event eventToGet, string calendarId)
        {
            return GetEvent(eventToGet.Id, calendarId);
        }

        public Event GetEvent(string eventIdToGet, string calendarId)
        {
            EventsResource.GetRequest eventGetRequest = service.Events.Get(calendarId, eventIdToGet);
            return DoActionWithExponentialBackoff(eventGetRequest);
        }

        public CalendarListEntry GetCalendar(string calendarId)
        {
            CalendarListResource.GetRequest calendarGetRequest = service.CalendarList.Get(calendarId);
            return DoActionWithExponentialBackoff(calendarGetRequest);
        }

        public Events ListEvents(string calendarId, DateTime? startDate = null, DateTime? endDate = null, string q = null, int maxResults = 250)
        {
            EventsResource.ListRequest eventListRequest = service.Events.List(calendarId);
            eventListRequest.ShowDeleted = false;
            eventListRequest.SingleEvents = true;
            eventListRequest.OrderBy = EventsResource.ListRequest.OrderByEnum.StartTime;

            if (startDate != null)
                eventListRequest.TimeMin = startDate;

            if (endDate != null)
                eventListRequest.TimeMax = endDate;

            if (!string.IsNullOrEmpty(q))
                eventListRequest.Q = q;

            eventListRequest.MaxResults = maxResults;

            return DoActionWithExponentialBackoff(eventListRequest);
        }

        public CalendarList ListCalendars(string accessRole)
        {
            CalendarListResource.ListRequest calendarListRequest = service.CalendarList.List();
            calendarListRequest.MinAccessRole = (MinAccessRoleEnum)Enum.Parse(typeof(MinAccessRoleEnum), accessRole);
            return DoActionWithExponentialBackoff(calendarListRequest);
        }

        public void DeleteEvent(Event eventToDelete, string calendarId, bool sendNotifications = false)
        {
            DeleteEvent(eventToDelete.Id, calendarId, sendNotifications);
        }

        public void DeleteEvent(string eventIdToDelete, string calendarId, bool sendNotifications = false)
        {
            EventsResource.DeleteRequest eventDeleteRequest = service.Events.Delete(calendarId, eventIdToDelete);
            eventDeleteRequest.SendNotifications = sendNotifications;
            DoActionWithExponentialBackoff(eventDeleteRequest, new HttpStatusCode[] { HttpStatusCode.Gone });
        }
        #endregion Public Methods
    }
}

Answer 2

derekantrican 有一个我基于我的答案。有两件事，如果资源“未找到”，等待它不会有任何好处。那是他们在没有找到对象的情况下响应请求，因此无需后退。我不确定是否还有其他代码需要处理，但我会仔细研究。根据谷歌：https://cloud.google.com/iot/docs/how-tos/exponential-backoff 应该重试所有 5xx 和 429。

此外，Google 希望这是指数级的回退；不是线性的。所以下面的代码以指数方式处理它。他们还希望您在重试超时中添加随机数量的 MS。我不这样做，但这很容易做到。我只是认为这并不重要。

我还需要异步请求，因此我将工作方法更新为这种类型。请参阅 derekantrican 的示例，了解如何调用这些方法；这些只是工人方法。除了在 notFound 上返回“默认值”，您还可以重新抛出异常并在上游处理它。

    private async Task<TResponse> DoActionWithExponentialBackoff<TResponse>(DirectoryBaseServiceRequest<TResponse> request)
    {
        return await DoActionWithExponentialBackoff(request, new HttpStatusCode[0]);
    }

    private async Task<TResponse> DoActionWithExponentialBackoff<TResponse>(DirectoryBaseServiceRequest<TResponse> request, HttpStatusCode[] otherBackoffCodes)
    {
        int timeDelay = 100;
        int retries = 1;
        int backoff = 1;

        while (retries <= 5) 
        {
            try
            {
                return await request.ExecuteAsync();
            }
            catch (GoogleApiException ex)
            {
                if (ex.HttpStatusCode == HttpStatusCode.NotFound)
                    return default;
                else if (ex.HttpStatusCode == HttpStatusCode.Forbidden || //Rate limit exceeded
                    ex.HttpStatusCode == HttpStatusCode.ServiceUnavailable || //Backend error
                    ex.Message.Contains("That’s an error") || //Handles the Google error pages like https://i.imgur.com/lFDKFro.png
                    otherBackoffCodes.Contains(ex.HttpStatusCode))
                {
                    //Common.Log($"Request failed. Waiting {delay} ms before trying again");
                    Thread.Sleep(timeDelay);
                    timeDelay += 100 * backoff;
                    backoff = backoff * (retries++ + 1);
                }
                else
                    throw ex;  // rethrow exception
            }
        }

        throw new Exception("Retry attempts failed");
    }