如何访问标准输出作为 create_subprocess_shell 协程的一部分？

Question

我正在编写一个 python 脚本，它为数百个子文件夹中的每一个调用 3 个 shell 脚本。这些脚本对 AWS S3 进行 ls 调用，从服务器接收响应很慢，所以我决定异步编写代码。

我已经为这个问题建模了一个更简单的版本，我为三个参与者中的每一个调用三个 shell 脚本。为了理解如何异步调用 shell 脚本，我在下面使用的 shell 脚本不会 ping AWS，它们只是打印随机数。我只是想捕获 randomList.sh 的输出并将其作为列表（或稍后我将决定的其他数据类型）返回。

我的问题：如何访问脚本调用的输出？

#randomList.sh

x=$(( ( RANDOM % 10 )  + 1 ))
echo "sleeping for $x for $1"
sleep $x
x=$(( ( RANDOM % 10 )  + 1 ))
echo "sleeping for $x for $1"
sleep $x
x=$(( ( RANDOM % 10 )  + 1 ))
echo "sleeping for $x for $1"
sleep $x

import asyncio
import subprocess

async def callScript(script, participant):
    call = "./" + script + " " + participant
    proc = asyncio.create_subprocess_shell(call, stdout=subprocess.PIPE)
    result = await proc
    #HELP!
    #this is where i'm not sure how to access stdout..
    output = result.communicate()
    print(output)
    return output


async def callParticipant(participant):
    script1 = 'randomList.sh'
    script2 = 'randomList1.sh'
    script3 = 'randomList2.sh'
    scriptCalls = []
    scriptCalls.append(asyncio.create_task(callScript(script1, participant)))
    scriptCalls.append(asyncio.create_task(callScript(script2, participant)))
    scriptCalls.append(asyncio.create_task(callScript(script3, participant)))
    results = await asyncio.gather(*scriptCalls)
    return results

async def getRows():
    participants = ["1","2","3"]
    tasks = []
    for participant in participants:
        tasks.append(asyncio.create_task(callParticipant(participant)))
    rows = await asyncio.gather(*tasks)
    return rows

rows = asyncio.run(getRows())

---

编辑，自己解决了 请参阅 callScript 函数。请注意，我还将rows = asyncio.run(getRows()) 放入main() 函数中，但这与解决方案无关。

async def callScript(script, participant):
    call = "./" + script + " " + participant
    #I CHANGED THE FOLLOWING LINES
    proc = await asyncio.create_subprocess_shell(call, stdout=asyncio.subprocess.PIPE)
    stdout = await proc.communicate()
    output = stdout[0]
    output = output.decode('ascii')
    output = output.split()
    return output


async def callParticipant(participant):
    script1 = 'randomList.sh'
    script2 = 'randomList1.sh'
    script3 = 'randomList2.sh'
    scriptCalls = []
    scriptCalls.append(asyncio.create_task(callScript(script1, participant)))
    scriptCalls.append(asyncio.create_task(callScript(script2, participant)))
    scriptCalls.append(asyncio.create_task(callScript(script3, participant)))
    results = await asyncio.gather(*scriptCalls)
    return results

async def getRows():
    participants = ["1","2","3"]
    tasks = []
    for participant in participants:
        tasks.append(asyncio.create_task(callParticipant(participant)))
    rows = await asyncio.gather(*tasks)
    return rows

def main ():
    rows = asyncio.run(getRows())
    print(rows)

if __name__ == "__main__":
    main()

Answer 1

你需要使用await函数

-- your code until --
rows = asyncio.run(getRows())
result = await rows

您可以阅读更多关于它的实际工作原理： https://www.aeracode.org/2018/02/19/python-async-simplified/#:~:text=When%20you%20have%20an%20asynchronous,result%20when%20you%20await%20me%22.

这个例子运行良好：

async def get_chat_id(name):
    await asyncio.sleep(3)
    return "chat-%s" % name

async def main():
    id_coroutine = get_chat_id("django")
    result = await id_coroutine