python asyncio - as_completed identifying coroutie objects

Question

I'm using asyncio to await set of coroutines in following way:

# let's assume we have fn defined and that it can throw an exception

coros_objects = []
for x in range(10):
    coros_objects.append(fn(x))


for c in asyncio.as_completed(coros_objects):
    try:
       y = await c
    exception:
       # something
       # if possible print(x)

Question is how can I know which coroutine failed and for which argument? I could append "x" to the outputs but this would give me info about successful executions only.

I can know that form order because it's different from the order of coros_objects

Can I somehow identify what coro just yielded result?

Answer 1

Question is how can I know which coroutine failed and for which argument?

You can't with the current as_completed. Once this PR is merged, it will be possible by attaching the information to the future (because as_completed will then yield the original futures). At the moment there are two workarounds:

• wrap the coroutine execution in a wrapper that catches exceptions and stores them, and also stores the original arguments that you need, or
• not use as_completed at all, but write your own loop using tools like asyncio.wait.

The second option is easier than most people expect, so here it is (untested):

# create a list of tasks and attach the needed information to each
tasks = []
for x in range(10):
    t = asyncio.create_task(fn(x))
    t.my_task_arg = x
    tasks.append(t)

# emulate as_completed with asyncio.wait()
while tasks:
    done, tasks = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
    for t in done:
        try:
            y = await t
        except Exception as e:
            print(f'{e} happened while processing {t.my_task_arg}')