Record and play back requests

[GeoffreyBooth]

This would solve...

For tests, it's useful to be able to mock the responses to outbound network requests so that tests are not reliant on external resources. This is already possible via MockAgent, but all the examples at https://github.com/nodejs/undici/blob/main/docs/docs/api/MockAgent.md involve writing code to explicitly define each desired mock response. For an app that makes many outbound network requests, it becomes tedious if not impractical to define every response by hand. I would like to somehow record the outbound network requests and their responses, so that later I can attach them to MockAgent to be played back as mocks.

The implementation should look like...

I think this should be achievable using existing APIs, though it isn’t obvious how to do so. I found an implementation by @aduh95 in this comment from several years ago, and a version for fetch is in Corepack. At minimum I think there should be an example in the Undici docs of how to record requests and then use those recordings with MockAgent, assuming this is all possible using existing APIs; and if the example has excessive boilerplate, perhaps a new API such as RecordingAgent could be considered.

I have also considered...

The tool nock has APIs for recording mocks, and the latest version of nock supports Node global fetch, but as far as I can tell nock doesn’t support Undici request. I would like to use request for its increased performance, and because I am using RetryAgent and cacheable-lookup with request.

Additional context

I’ve been searching online for examples of intercepting requests in Undici and besides the mentioned snippets above, the article https://blog.platformatic.dev/http-fundamentals-understanding-undici-and-its-working-mechanism has some interesting examples.

[metcoder95]

Really like this idea; currently the recently introduced the MockCallHistory can be of help here.

I imagine an interceptor (or even an Agent) that actively records the requests and returns a MockCallHistory instance that can be used to return and reply the recorded requests.

Happy to support this feature

[blephy]

Hi,

If i understand well, you want to be able to run your tests suite with real external endpoint and register all the response according to a request configuration.
So then subsequence test suite run, you can mock the registered response for each request.

As @metcoder95 mentioned, you now have the ability to access every request configration like this :

for (const log of mockAgent.getCallHistory()) {
  // the toString is like an hash, it's a string computed with every part of the request configuration
  // @see https://undici.nodejs.org/#/docs/api/MockCallHistoryLog?id=tostring
  console.log(log.toString())
}

With that in mind, you can implements your own solution to automatically intercept request if any DB / FileSystem exist.
Get all mock response with all .toString() value appaired. Parse toSring values to get all request configuration then iterate on it to setup every interceptor.

I think this can be achieved easly but putting undici responsability in mind (no code for registering in DB...)

NB: there will be one problem with MockCallHistory : it register every HTTP call regardless there are actually intercepted or not.

[guyellis]

As an FYI another library that does something similar to what the OP mentioned with nock is PollyJS. (I think that PollyJS is now abandoned.)

Their Adapters & Persisters feature allow the registration of a persister which will write the network interaction to the persister if the artifact does not already exist (configurable) and then use that for playback.