【问题标题】:Azure Functions / Events Hub and Blob ouput binding. How to save messages efficiently (Python)Azure 函数/事件中心和 Blob 输出绑定。如何有效地保存消息(Python)
【发布时间】:2021-03-11 13:42:11
【问题描述】:

我被连接到事件中心的 Azure Functions 卡住了,我发现文档与此无关。

如果我以小 Python 示例为现有事件创建一个循环,以便手动将它们保存到 blob 存储:

from typing import List
import logging
import azure.functions as func


def main(events: List[func.EventHubEvent], outputBlob: func.Out[func.InputStream]):
    for event in events:
        logging.info('Python EventHub trigger processed an event: %s', event.get_body().decode('utf-8'))
        outputBlob.set(event.get_body().decode('utf-8')) # Save to storage.

对于 function.json :

"bindings": [ ..., 
{
  "type": "blob",
  "direction": "out",
  "name": "outputBlob",
  "path": "outcontainer/{rand-guid}",
  "connection": "storage_STORAGE"
}

它可以工作......但不正确:如果事件中有 3 条消息,则保存第一条和第三条消息......但不会保存第二条。有什么方法可以改善这一点,并根据消息的内容使用名称保存每个事件,或者通过根据事件的某些值提供文件名并通过示例在循环中管理来保存其他事件?

谢谢,

【问题讨论】:

  • 检查日志有没有错误?

标签: python azure azure-functions azure-blob-storage


【解决方案1】:

使用storage output binding确实会出现这样的问题,您可以使用storage SDK作为替代解决方案,请参考以下代码:

代码:

from typing import List
import logging, time
import azure.functions as func
import os, uuid
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient, __version__
import tempfile


def main(events: List[func.EventHubEvent]):
    connect_str = '<your-storage-connection-string>'
    blob_service_client = BlobServiceClient.from_connection_string(connect_str)
    container_name = '<your-container-name>'
    container_client = blob_service_client.get_container_client(container_name)
    for event in events:
        logging.info('Python EventHub trigger processed an event: %s', event.get_body().decode('utf-8'))

        temp_path = tempfile.gettempdir()

        # Create a file in the local data directory to upload and download
        local_file_name = str(uuid.uuid4()) + ".txt"
        upload_file_path = os.path.join(temp_path, local_file_name)

        # Write text to the file
        file = open(upload_file_path, 'w')
        file.write(event.get_body().decode('utf-8'))
        file.close()

        # Create a blob client using the local file name as the name for the blob
        blob_client = blob_service_client.get_blob_client(container=container_name, blob=local_file_name)

        print("\nUploading to Azure Storage as blob:\n\t" + local_file_name)

        # Upload the created file
        with open(upload_file_path, "rb") as data:
            blob_client.upload_blob(data)

函数.json:

{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "type": "eventHubTrigger",
      "name": "events",
      "direction": "in",
      "eventHubName": "frankhub",
      "connection": "frankeventhub_RootManageSharedAccessKey_EVENTHUB",
      "cardinality": "many",
      "consumerGroup": "$default",
      "dataType": "binary"
    }
  ]
}

您可以参考以下教程:

https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-python

【讨论】:

  • 事实上,这是我终于想到的事情,即使有一个输出的直接解决方案可能更直接(看到在我的搜索过程中更新了文档的同一页面,更多可能是微软在这个主题的未来)。感谢您的建议,将测试。
  • @user15367534。您好,如果答案有帮助,请accept作为答案(单击我的答案旁边的复选标记,将其从灰色切换为已填充)?谢谢。它可能会帮助其他人。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2021-02-09
  • 2020-07-19
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多