Understand task tester performance

[nikku]

What should we do?

Task tester is a new component that we roll out to our users. Some reported that it may be slow (which in turn may hinder adoption). But is it really?

Beyond basic usage metrics I'd like us to collect performance metrics, to understand

(1) how fast or slow task tester is in real-world scenarios
(2) what parts of it is actually slow

Why should we do it?

Understand if improvements are required, and where the time is spent.

[sbuettner]

If we could pass information to the engine (via the existing termination instruction, a new instruction or based on the dev tag) that we are running a test when starting the instance, we could for example immediately flush the export batch after reaching the end of the process instance record.

This could lead to faster secondary storage exports and to a faster experience in general.

[barmac]

One thing which is slow, and probably will remain slow, is the incident case. The engine uses retries to handle intermittent failures but this means that in the task testing case the user needs to wait up to a minute until the incident is reported. Not sure how much of it is due to secondary storage delay.

[philippfromme]

We do not track execution times yet. We emit events for start and end that can be handled by the host application. We use these for tracking general usage. Measuring the time from start to end should probably within task testing and simply be provided as part of the end event (regardless of whether the test was successful or not). Example implementation: https://github.com/camunda/task-testing/pull/64

[Balazs-PM]

The engine uses retries to handle intermittent failures but this means that in the task testing case the user needs to wait up to a minute until the incident is reported.

• Can we feed back the retries to the UI when running Task Tester?
• Turning the output to an Activity log could also help our case, and the users get a better perception as well.