[Feature Request] Adding support for passing references as inputs/outputs in orchestrations

[olitomlinson]

Is your feature request related to a problem? Please describe.

Not a problem per-se, just an optimisation.

Describe the solution you'd like

Given an Orchestration where multiple Activities are called
And each Activity returns a large Object graph to the orchestration
When all Activities are complete, references to the Object graphs (re-use the large message store in TaskHub maybe?) are returned to the orchestration (rather than the actual Object Graphs themselves)
Then the list of references are passed as an input to a subsequent Activity
And in the Activity, the binding automatically loads the reference/s and deserializes into a List of the Objects, ready for processing.

Describe alternatives you've considered

I could manually use blob storage to store the object graphs and pass around the references, but this is just unsightly boilerplate. It would be very convenient if the extension could do this for me.

Additional context

The purpose of this is stop the Orchestration having to load and unload many large objects (from the result of an activity) into memory, just for them to passed to subsequent Activities.

The reason for raising this is that I'm using Activities to load pages of data from CosmosDb. Once all pages are loaded, I need to process them all together in an final Activity. It seems very wasteful that I have to build up a list of the pages in the Orchestration, and isn't a pattern that scales. Equally, using blob storage to manually store the pages seems like a step-backwards considering Durable Functions is so good at hiding the underlying transient data as part of a workflow.

[cgillum]

Hey @olitomlinson, can I think of this feature request as being similar to the Claim-Check pattern? Basically, you want the ability to pass around references to your large object graphs without actually pulling them into and out of the orchestration?

If so, I agree this is something we should consider.

[olitomlinson]

@cgillum I never knew there was a name for it it, but yes, that claim-check pattern sounds right!

If the result of an Activity returns a large object and I know I’m not going to make use of the large object directly in the orchestration, other than to forward that object to a subsequent activity, why bother loading it? Just pass the reference 🙂

[ConnorMcMahon]

I think whatever approach we end up taking will have to be explicitly opted into., The reason for this is that we can't just lazily load activity results once the orchestration has started, as we can't perform IO at that point. Therefore, we have to know what data we need to fully load before starting the orchestration, and I think the easiest way to achieve that is via some opt-in flag and/or datatype.

[olitomlinson]

@ConnorMcMahon

I agree. Opt-in should be the way, and not the default experience.

I don't know if this is viable, but I had in my mind something like Reference<T> class which could instruct the extension to automatically store the blob into storage, and then also be used to tell the extension not to load T in the orchestration.

[FunctionName("Orchestration")]
public async Task Orchestration([OrchestrationTrigger] IDurableOrchestrationContext context)
{
    List<Reference<Foo>> fooRefs = new List<Reference<Foo>>();        
    foreach(var i in Enumerable.Range(0,100)
    {
        fooRefs.Add(await context.CallActivityAsync<Reference<Foo>>("GetFoo", i ));
    }

    await context.CallActivityAsync("ProcessFoos", fooRefs, LazyLoadReferences : true | false );
}

[FunctionName("GetFoo")]
public async Task<Reference<Foo>> GetFoo([ActivityTrigger] int id)
{
    var foo = await getLargeFooObject();
    return new Reference<Foo>(foo);
}
        
[FunctionName("ProcessFoo")]
public async Task ProcessFoo([ActivityTrigger] List<Reference<Foo>> foos )
{
    foreach(var @ref in foos ) 
    {
         var loadedFoo = await @ref.GetDataAsync();
    }
}

[ConnorMcMahon]

Yeah, I agree a data type like Reference<T> would be a great way to support this in C#, though we also want a solution for languages like Node and Python. I suppose we could define similar data types in those languages, we would just need to make sure that we can safely mark those as "references" back in the extension without accidentally doing so for any existing types in those languages.

[ConnorMcMahon]

Starting on the design for implementing this. Hopefully it will make it into our v2.5.0 release.