callHttp does no polling

[itpropro]

Describe the bug
I have two durable function runtimes running locally on different ports. I try to make a durable http call with yield context.df.callHttp.
Regarding to the docs, the orchestrator function should poll based on the return as long as 202 and a location header is returned.
In my case, callhttp does no polling at all and returns the plain result of the request containing status code 202 and a location header:

[2021-08-29T20:17:24.596Z] {
[2021-08-29T20:17:24.597Z]   statusCode: 202,
[2021-08-29T20:17:24.597Z]   headers: {
[2021-08-29T20:17:24.598Z]     Date: 'Sun, 29 Aug 2021 20:17:24 GMT',
[2021-08-29T20:17:24.598Z]     Server: 'Kestrel',
[2021-08-29T20:17:24.599Z]     'Transfer-Encoding': 'chunked',
[2021-08-29T20:17:24.599Z] Executed 'Functions.UpdateFunction' (Succeeded, Id=85f4cbae-34da-4ae8-9f27-469041954a81, Duration=6ms)
[2021-08-29T20:17:24.599Z]     Location: 'http://localhost:7072/runtime/webhooks/durabletask/instances/53fccd02ea564e088b5b03c06520aaf3?taskHub=kametlicensingapihub&connection=DfStorageConnectionString&code=jD7lnp5vqS34wvyYTmSKUpKCWxZy5BXmuCLMPAYCsqGXgAYNUvdhDQ==',
[2021-08-29T20:17:24.600Z]     'Retry-After': '10'
[2021-08-29T20:17:24.601Z]   },
[2021-08-29T20:17:24.601Z]   content: '{\r\n' +
[2021-08-29T20:17:24.602Z]     '  "id": "53fccd02ea564e088b5b03c06520aaf3",\r\n' +
[2021-08-29T20:17:24.603Z]     '  "statusQueryGetUri": "http://localhost:7072/runtime/webhooks/durabletask/instances/53fccd02ea564e088b5b03c06520aaf3?taskHub=kametlicensingapihub&connection=DfStorageConnectionString&code=jD7lnp5vqS34wvyYTmSKUpKCWxZy5BXmuCLMPAYCsqGXgAYNUvdhDQ==",\r\n' +
[2021-08-29T20:17:24.603Z]     '  "sendEventPostUri": "http://localhost:7072/runtime/webhooks/durabletask/instances/53fccd02ea564e088b5b03c06520aaf3/raiseEvent/{eventName}?taskHub=kametlicensingapihub&connection=DfStorageConnectionString&code=jD7lnp5vqS34wvyYTmSKUpKCWxZy5BXmuCLMPAYCsqGXgAYNUvdhDQ==",\r\n' +
[2021-08-29T20:17:24.604Z]     '  "terminatePostUri": "http://localhost:7072/runtime/webhooks/durabletask/instances/53fccd02ea564e088b5b03c06520aaf3/terminate?reason={text}&taskHub=kametlicensingapihub&connection=DfStorageConnectionString&code=jD7lnp5vqS34wvyYTmSKUpKCWxZy5BXmuCLMPAYCsqGXgAYNUvdhDQ==",\r\n' +
[2021-08-29T20:17:24.604Z]     '  "rewindPostUri": "http://localhost:7072/runtime/webhooks/durabletask/instances/53fccd02ea564e088b5b03c06520aaf3/rewind?reason={text}&taskHub=kametlicensingapihub&connection=DfStorageConnectionString&code=jD7lnp5vqS34wvyYTmSKUpKCWxZy5BXmuCLMPAYCsqGXgAYNUvdhDQ==",\r\n' +
[2021-08-29T20:17:24.605Z]     '  "purgeHistoryDeleteUri": "http://localhost:7072/runtime/webhooks/durabletask/instances/53fccd02ea564e088b5b03c06520aaf3?taskHub=kametlicensingapihub&connection=DfStorageConnectionString&code=jD7lnp5vqS34wvyYTmSKUpKCWxZy5BXmuCLMPAYCsqGXgAYNUvdhDQ==",\r\n' +
[2021-08-29T20:17:24.605Z]     '  "restartPostUri": "http://localhost:7072/runtime/webhooks/durabletask/instances/53fccd02ea564e088b5b03c06520aaf3/restart?taskHub=kametlicensingapihub&connection=DfStorageConnectionString&code=jD7lnp5vqS34wvyYTmSKUpKCWxZy5BXmuCLMPAYCsqGXgAYNUvdhDQ=="\r\n' +
[2021-08-29T20:17:24.606Z]     '}'
[2021-08-29T20:17:24.607Z] }

I tried calling it with POST and with GET, which made no difference at all:

  const result = yield context.df.callHttp(
    "POST",
    `http://localhost:7072/api/OtherFunctionOrchestrator`,
    { a, b, c}
  );

  const result = yield context.df.callHttp(
    "GET",
    `http://localhost:7072/api/OtherFunctionOrchestrator?parametera=${a}&parameterb=${b}&parameterc=${c}`
  );

Investigative information

• Durable Functions extension version: ExtensionBundle "[2.*, 3.0.0)"
• durable-functions npm module version: 1.5.2
• Language (JavaScript/TypeScript) and version: TypeScript 4.4.2
• Node.js version: 14.17.4

Expected behavior
callHttp should poll the endpoint until the return code is 200 and then return the actual result.

Actual behavior
callHttp does one single call, does no polling despite the location header and status 202 being present and returns the first reply package containing the statusUris.

I am using Windows 10 19043.

[itpropro]

I thought it may be connected to the extension bundle, but even after removing the bundle and installing all extensions manually with func extensions install, callHttp still doesn't poll and returns the initial reply from the other orchestrator.

[cgillum]

Pinging @bachuv and @davidmrdavid.

[davidmrdavid]

Hi @itpropro,

Thank you for reaching out! Just to make sure we're looking at the same problem, could you please provide a minimal reproducer for this issue? An attached .zip or a link to a repro would suffice, and it would help accelerate our investigation; thanks!

[itpropro]

Sorry I couldn't get back earlier @davidmrdavid. I just created a minimal reproduction mono repo with two Azure Functions. They both have local.settings.template.json files that contain my configuration (just adjusted the default port). I did the following to reproduce the error:

1. Started azurite
2. Executed yarn build && func start in both folders
3. Executed Invoke-RestMethod -Method Get -Uri http://localhost:7071/api/Orchestrator1 with pwsh

The two functions apps have each two functions:

• FunctionApp 1
  • Activity1 -> Activity Function that sleeps for 5 seconds and returns
  • Orchestrator 1 -> Orchestrator Function that executes Activity1 then executes callHttp to call Orchestrator2 then returns the result of the callHttp call.
• FunctionApp 2
  • Activity2 -> Activity Function that sleeps for 5 seconds and returns
  • Orchestrator2 => Orchestrator Function that calls Activity2 and returns

The same behavior can be seen here. the callHttp from Orchestrator1 doesn't poll or anything. It just returns the response package including the location header and the status code 202. Regarding to the docs, it should do this: "If a called API returns an HTTP 202 response with a Location header, the orchestrator function automatically polls the Location resource until receiving a response other than 202."

https://github.com/itpropro/durable-functions-call-http

[itpropro]

Any updates @cgillum and @davidmrdavid? Could you reproduce the issue with the repo or do you need anything else from me?

[davidmrdavid]

Hi @itpropro, I'll be looking into this today. I'll look to provide an update in a few hours.

[davidmrdavid]

Hi @itpropro! I looked into this today. I can reproduce the issue and I'm currently investigating why this isn't working out of the box. I have some theories already, but nothing I'm ready to share just yet. I'll keep this thread updated, thanks!

[davidmrdavid]

Good news! I found the root cause, @itpropro.

There are two different blockers preventing this behavior at the moment.

The first is that the underlying durable extension (written in C#) requires that durable-http-requests that support polling be marked with a particular boolean flag that indicates the client supports the polling behavior. The JS/TS SDK currently does not send that flag, but it could.

The second is more complicated. Even after setting the right flag in our durable-http-request, the polling behavior introduces a variable number of history events in our History array, which we use for orchestration replay and fault tolerance. You can imagine that, in a non-polling scenario, the JS/TS SDK assumes that it can retrieve the requests' result from the next http-result event in the History array, but with a polling scenario we make several requests so now there's extra resolution logic to be made on the SDK's side to find the event containing the actual result. This extra task resolution logic is exactly what we're missing on the SDK's side to enable this feature.

So that's what we need to do to resolve this issue :)

However, we're currently days away from starting an effort to revamp the JS/TS SDK with the changes described in this post, so I would like to wait until those changes are completed before tackling this feature. These changes will also make it easier to correctly implement this feature's logic, so I think they're a necessary precondition :)

So I'm going to tag this issue as a top-priority after our revamping of the SDK, and update our documentation to indicate this temporary limitation. Is that ok with you?