Support coroutine tasks

[nicktimko]

Some of the libraries I use are built around a bunch of coroutines and they make liberal use of async functions. In order to call them, I can either wrap my async function in a dirty-but-works way (specifying the args three times):

@task
def mytask(c, someopt=False):
    loop = asyncio.get_event_loop()
    return loop.run_until_complete(_mytask(c, someopt))

async def _mytask(c, someopt=False):
    pass

Or something even hackier that intercepts the inspect.getargspec call and gives the original argspec so Invoke is happy that it has a context object and knows about the other arguments.

import inspect

class ArgSpecPatch:
    def __init__(self, original):
        self.registry = {}
        self.original = original

    def register(self, func, wrapped):
        self.registry[wrapped] = self.original(func)

    def __call__(self, *args, **kwargs):
        if not (len(args) == 1 and len(kwargs) == 0):
            return self.original(*args, **kwargs)
        func = args[0]
        if func not in self.registry:
            return self.original(*args, **kwargs)
        return self.registry[func]

inspect.getargspec = ArgSpecPatch(inspect.getargspec)

def atask(func):
    @functools.wraps(func)
    def _wrap(*args, **kwargs):
        loop = asyncio.get_event_loop()
        return loop.run_until_complete(func(*args, **kwargs))
    inspect.getargspec.register(func, _wrap)
    return task(_wrap)

@atask
async def mytask(c, option=False):
    pass

Is there an easier place to insert a check if the task passed to @task is a coroutine function and if so, exhaust the task in an event loop? Not asking for any task concurrency/parallelism (like #63); they could just be run serially.

[bitprophet]

Haven't used asyncio myself yet. The tl;dr answer is "you need to look at the (should be thoroughly docstring'd and relatively cleanly organized) guts of Executor and to a lesser extent Task" - I suspect it shouldn't be too large a diff to get this working.

Longer answer: just goofed around with asyncio and Invoke's guts in a REPL and found the following:

• Creating tasks with async functions as their body doesn't appear to explode, and the resulting Task instances appear to understand the non-context args (same as with a regular function).
  • So, not sure why you implied Invoke was having issues with a lack of context arg - an example of actual explosions might be useful there since I'm probably missing something.
• As anticipated, the main problem seems to be that Executor executing the task just does a normal __call__, which for async functions gets us the coroutine and doesn't actually do any work (thus your need for the loop method).
• Offhand, what's probably necessary is Executor (or a subclass, perhaps) understanding that a given task wraps a coroutine, so it can deal with the appropriate loop method(s).
  • Likely requires lightweight addition to Task (or again, a subclass) so it can use inspect.iscoroutinefunction at creation time. Could just do it in Executor but having Task understand what it's wrapping seems better.
• Executor.execute "wants" to store/return the return value of executed tasks, and my coroutine-fu is weak enough I don't remember exactly how that works (I assume a queue or yield from...somewhere...) but it's something that should be accounted for and documented even if it's not totally "fixed".
  • I'd personally be fine with a handwavy "coroutine-type tasks have no defined return value from Executor's perspective" if there's no obvious quick clean way to do this.

[bitprophet]

2.0.0 finally uses inspect.signature, and the inner method(s) related to this also tend to now just pass around the Signature object in question, so this may want to get another look from somebody sometime. (Likely not me directly but I would certainly take a look at something that passes tests and has new tests of its own 🤔 )

[Lordfirespeed]

I'd be up for looking at this, perhaps behind a feature-flag, if the current maintainers still find this feature desirable :)