在 Laravel 中使用 API 时的处理速率限制

Question

在我的 Laravel 应用程序中，我广泛使用 HubSpot API 来执行各种操作。我在文档中读到，您可以每 10 秒发出 150 个请求。

要监控此 HubSpot，请在进行任何 API 调用时提供以下标头。

"X-HubSpot-RateLimit-Daily" => array:1 [▶]
"X-HubSpot-RateLimit-Daily-Remaining" => array:1 [▶]
"X-HubSpot-RateLimit-Interval-Milliseconds" => array:1 [▶]
"X-HubSpot-RateLimit-Max" => array:1 [▶]
"X-HubSpot-RateLimit-Remaining" => array:1 [▶]
"X-HubSpot-RateLimit-Secondly" => array:1 [▶]
"X-HubSpot-RateLimit-Secondly-Remaining" => array:1 [▶]

在我的应用程序中，我使用了 Laravel 的 Http Client，它基本上只是 Guzzle 的包装器。

为了遵守速率限制，我真的只需要在每个请求周围都包含一个 if 语句吗？

这是一个例子：

$endpoint = 'https://api.hubapi.com/crm/v3/owners/';

$response = Http::get($endpoint, [
    'limit' => 100,
    'hapikey' => config('hubspot.api_key'),
]);

在这种情况下，$response 将包含标头，但是否有办法有效地使用它们，因为我肯定只有在进行 API 调用后才能知道费率是多少？

我问，因为我必须删除 1,000 多个交易，然后更新一些记录，但这肯定会超过 API 限制。作为参考，这是我写的命令。

<?php

namespace App\Console\Commands;

use App\Events\DealImportedFromHubspot;
use App\Hubspot\PipelineHubspot;
use App\Models\Deal;
use App\Models\DealStage;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\Http;

class ImportHubspotDeals extends Command
{
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'import:hubspot-deals
        {--force : Whether we should force the command}
    ';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'Import Deal objects from the HubSpot API in bulk.';

    /**
     * An array to store imported Deals
     *
     * @var array
     */
    private $importedDeals = [];

    /**
     * Create a new command instance.
     *
     * @return void
     */
    public function __construct()
    {
        parent::__construct();
    }

    /**
     * Execute the console command.
     *
     * @return int
     */
    public function handle()
    {
        $this->line('Importing Pipelines & Deal Stages from HubSpot API...');

        PipelineHubspot::import();

        $this->line('Importing Deals from HubSpot API...');

        $this->getDealsFromHubspot();

        $this->line('Found ' . count($this->importedDeals) . ' Deals to import');

        if ($this->option('force')) {
            $this->doImport();
        } else {
            if ($this->confirm('Do you want to import these deals? (yes|no)', false)) {
                $this->doImport();
            } else {
                $this->line('Process aborted');
            }
        }
    }

    /**
     * Grab Deals from Hubspot by calling the Deals API and looping through the paginated data
     *
     * @param int    $limit: the number of deals per page
     * @param string $next:  the link to the next page of results
     */
    private function getDealsFromHubspot(?int $limit = 100, string $next = null)
    {
        $endpoint = 'https://api.hubapi.com/crm/v3/objects/deals';

        $properties = [
            'limit' => $limit,
            'properties' => implode(',', Deal::HUBSPOT_DEAL_PROPERTIES),
            'hapikey' => config('hubspot.api_key'),
            'associations' => 'engagements',
        ];

        // If there's another page, append the after parameter.
        if ($next) {
            $properties['after'] = $next;
        }

        $response = Http::get($endpoint, $properties);

        if ($response->successful()) {
            $data = $response->json();

            // If there are results, get them.
            if (isset($data['results'])) {
                foreach ($data['results'] as $hubspotDeal) {
                    $this->importedDeals[] = $hubspotDeal['properties'];
                }
            }

            // If there's paginate we need to call the function on itself
            if (isset($data['paging']['next']['link'])) {
                $this->getDealsFromHubspot(null, $data['paging']['next']['after']);
            }
        }

        $response->json();
    }

    /**
     * Pull the Deal data in order to create a Deal model.
     *
     * @param array $data
     */
    private function syncDeal(array $data)
    {
        $excludedDealStages = DealStage::excludeFromDealImport()->pluck('hubspot_id');

        if ($excludedDealStages->contains($data['dealstage'])) {
            return false;
        }

        $deal = Deal::updateOrCreate([
            'hubspot_id' => $data['hs_object_id'],
        ], [
            'name' => $data['dealname'],
            'deal_stage_id' => $data['dealstage'],
            'hubspot_owner_id' => $data['hubspot_owner_id'] ?? null,
        ]);

        event(new DealImportedFromHubspot($deal));

        return $deal;
    }

    /**
     * Create and increment a nice progress bar as we import deals.
     */
    private function doImport()
    {
        $bar = $this->output->createProgressBar(count($this->importedDeals));

        $bar->start();

        foreach ($this->importedDeals as $deal) {
            $this->syncDeal($deal);

            $bar->advance();
        }

        $bar->finish();

        $this->newLine(2);

        $this->line('Successfully imported ' . count($this->importedDeals) . ' Deals from HubSpot.');
    }
}

在此 event(new DealImportedFromHubspot($deal)); 的基础上，还会对 HubSpot 进行 API 回调，以添加刚刚被拉入的门户的 URL。

在这种情况下，我想我要么需要将交易导入视为自己的工作，要么添加某种速率限制器。

只使用sleep(10) 来绕过速率限制会是不好的做法吗？

Answer 1

听起来像是Queue 的工作。

您可以在Queue 上定义自己的rate limiter，但正确的解决方案可能是扩展ShouldQueue 并在收到表示您的请求已被限制的响应时运行$this->fail()。