Instance.GetFunction Cache

[martindevans]

Added a cache into the Instance.GetFunction and Instance.GetMemory methods. This reduces a benchmark fetching a function 1,000,000 times from 300ms to 30ms.

[peterhuene]

Hi @martindevans, sorry for the delay in reviewing this!

These changes look good, only I'm having a hard time thinking about the real world use case that would necessitate the caching in the API, as I think the typical use case would, at worst, simply store the returns of GetFunction where needed.

I would imagine that it's fairly rare to call GetFunction more than a few times for an instance (at most once per export?) in typical usage and I'm less concerned with implementing a fix to improve a micro benchmark.

Would you mind sharing a usage pattern that you think would directly benefit from having an internal synchronized cache?

[martindevans]

The various performance improvement PRs I've been submitting recently are all driven by a game I'm working on at the moment, which allows users to write code (in WASM) to run in game units. There may be around 100 different WASM instances all running in the game, all of which require various calls every frame. The game runs a "sim" as quickly as possible and then saves the replay, so speed is very important!

Initially I was not saving the results of GetFunction, it's easier to simply pass in an Instance to a game system and have it get whatever functions it needs. However profiling revealed that was taking a lot of time (100 ticks/second * 20 minute replay * 100 units * 10 calls per unit == ~30 seconds) over one run of a sim! Obviously I've solved that internally by wrapping the Instance in another class and caching functions, but I thought it'd be better to push that upstream so everyone can get the benefits :)

[Joshhua5]

It's good to know these can be cached, but I can see the hesitation since the optimisation mustn't negatively affect other use cases, such as a single module with a large number of exports and infrequent usage from taking up more memory when idle.

[martindevans]

These objects are already cached in the Store (but with an inconvenient key), so this new cache is adding fairly minimal extra memory pressure (mainly just the name string).

If there's interest in merging this, a hard cap could be added (e.g. stop adding to the cache once it has 1000 items in it) to handle truly extreme cases. The same logic should probably be applied to other caches in that case though.