Embed Nats server in gitea

[TheFox0x7]

Feature Description

Since gitea and nats are both written in go, gitea can embed nats service and have it always available regardless of environment.

Why is it beneficial?

• ability to have a single provider for a feature set regardless of environment (standalone binary, container, cluster all have it available with the same feature set)
• Simplification of queues - nats can handle them replacing need for multiple providers in the first place.
• PubSub channel is already there if needed in the future
• Various services and system might benefit from having a request/reply channel
• JetStream features can be enabled if needed providing an even wider feature set out of the box
• Common system between clustered gitea and standalone

Potential issues I've identified so far:

• Migration from current user setups
• Added memory usage is unknown
• New binary size is unknown pulling just the client (but not doing much with it - only connecting) brings less than 800kB. Server adds 5.5MB.
• Redis is also used for cache so this won't be removing a dependency (aside from queue only ones but they aren't as big)
• Nats does not have counters for future rate limiting cases (they are planned to be in alpha for 2.12 release)

---

It's a very low priority idea but I figured I should write it down so there's a place for discussions on this. I've seen it mentioned in relation to preparing gitea for clusters before.

Screenshots

No response

[TheFox0x7]

Small update - I have working queue that's based on NATS Jetstream with WorkQueuePolicy retention.
It's more or less the same design as redis based queue but can be embedded in gitea directly but for queue it's a no go in my opinion.

TLDR the approach to have unique queue would require subject per item (for example per repository)

It's not possible to my current knowledge to check if queue has a particular item in it. Doing so would require consumer iterating through messages, which I'm not exactly sure if it would break something or not. Deduplication could work here but the window looks back into acked messages too so that's a dead end.

The only way to make properly unique queue is to use approach from https://nats.io/blog/new-per-subject-discard-policy/ (discardPerSubject policy, discard new items, single msg per subject limit, unique key as part of the subject) but there's higher memory cost of this setup according to the blog.
I did try it with a synthetic test on 500k subjects (in one burst so it's really the worst case scenario) and it's added around 220MB of memory, then dropping to 90MB.
So it's a trade off - either memory consumption grows, or it has to rely on deduplication window which takes into account potentially processed items.

The idea might find other uses - event notifications, audit logs etc. I think there's a large benefit in ability to have nats by default in binary.