带芹菜的 Django 大型 CSV 导出（异步）答案

【问题标题】：Django large CSV export with celery (async)带芹菜的 Django 大型 CSV 导出（异步）
【发布时间】：2023-02-13 22:11:11
【问题描述】：

我正在尝试导出需要一段时间才能生成的 CSV 文件。我已经编写了成功运行的导出 CSV 视图，但是我需要对其进行调整以使用 celery 以使其在具有大数据量的生产中工作。我写了下面的代码（主要是https://thoslin.github.io/async-download-with-celery/）：

看法：

from tasks import export_report
@staff_member_required()
def export_status_report(request):
    task = export_report.delay()
    return render(request, "admin/poll_for_download.html", {"task_id": task.task_id })

# Check if task is finished
def poll_for_download(request):
    task_id = request.GET.get("task_id")
    filename = request.GET.get("filename")

    if request.is_ajax():
        result = AsyncResult(task_id)
        if result.ready():
            return HttpResponse(json.dumps({"filename": result.get()}))
        return HttpResponse(json.dumps({"filename": None}))

    try:
        f = open("/path/to/export/"+filename)
    except:
        return HttpResponseForbidden()
    else:
        response = HttpResponse(file, mimetype='text/csv')
        response['Content-Disposition'] = 'attachment; filename=%s' % filename
    return response

任务：

@app.task
def export_report():
    date = datetime.now()
    date = date.strftime("%m/%d/%Y")
    filename = "reoport"+date+".csv"
    response = HttpResponse(content_type='text/csv')
    response['Content-Disposition'] = 'attachment; filename= "{}"'.format(filename)
    payouts = Payout.objects.all()

    writer = csv.writer(response) #instantiate writer

    # write headers
    writer.writerow(['Field1', 'Field2', 'Field3'])

    for payout in payouts:
        list = [payout.field1, payout.field2, payout.field3]
        writer.writerow(list)

    return filename

HTML

<script>
$(function(){
    $.ajaxSetup({ cache: false, timeout: 360000 });
    var url = "admin/poll-for-download/";
    var i = 0;
    (function worker() {
        $.getJSON(url+"?task_id=", function(data){
            if(data.filename) {
                var file_url = url+"?filename="+data.filename;
                $("#content").html("If your download doesn't start automatically, please click <a href='"+file_url+"'>here</a>.");
                window.location.href = file_url;
            } else {
                setTimeout(worker, 5000);
            }
        });
    })();
    setInterval(function() {
        i = ++i % 4;
        $("#loading").html("loading"+Array(i+1).join("."));
    }, 1000);
});

但是，这是行不通的。我收到一个错误：

with open("%s%s" % ("/path/to/export/", filename), "w+") as f:
2022-11-10T14:03:49.564942+00:00 app[worker.1]: FileNotFoundError: [Errno 2] No such file or directory: '/path/to/export/31e39843-421d-49f5-8a59-46394c22a3ce.csv'

我该如何解决这个问题，以便脚本在准备就绪时自动下载 CSV 文件并向用户显示一个链接，以防下载没有自动开始。我也不希望报告公开可用，因此不应将它们上传到我们的 S3/Cloudfront 静态文件夹。

【问题讨论】：

标签： django asynchronous celery export-to-csv django-celery

【解决方案1】：

替换file

任务response = HttpResponse(file, mimetype='text/csv')

和

response = HttpResponse(filename, mimetype='text/csv') 或f文件对象

【讨论】：