Add input flavors (raw, json-to-messagepack, and json)

[makrisoft]

#gsd:39011

• Drop the log buffer altogether and only warn when the log length will be greater than the current limit:

• Added an Codec to the runner so that input can be passed as:
  • json (default for today)
  • binary untouched passthrough (helps us test argo, messagepack etc)
  • messagepack json -> messagepack transcoding (adds rmp-serde as a new dependency)

This pull request introduces several changes to the src/engine.rs, src/function_run_result.rs, src/logs.rs, src/main.rs, tests/fixtures/log_truncation_function/src/main.rs and Cargo.toml files. The most significant changes include the addition of a new library, the restructuring of function parameters into a struct, changes to error handling, and updates to test cases.

Dependency Changes:

• Cargo.toml: Added rmp-serde = "1.1" to the list of dependencies. This library provides serialization and deserialization between Rust and the MessagePack format.

Function Parameter Restructuring:

• src/engine.rs: The parameters of the run function were moved into a new struct called FunctionRunParams. This change simplifies the function's signature and makes it easier to add, remove, or modify parameters in the future.

Error Handling and Logging:

• src/engine.rs: Removed the .expect("Couldn't append error logs"); call when appending error logs, which simplifies error handling.
• src/function_run_result.rs: Added a check for the length of logs and a warning message if the logs exceed a certain limit.
• src/logs.rs: Removed the capacity limit for LogStream and the associated truncation logic. This change simplifies the log handling process. [1] [2] [3] [4]

Test Case Updates:

• src/engine.rs and tests/fixtures/log_truncation_function/src/main.rs: Updated test cases to reflect the changes in function parameters and log handling. [1] [2] [3]

Input Handling:

• src/main.rs: Added support for different input codecs, including JSON, raw input, and JSON converted to MessagePack. This change adds flexibility in how input data can be provided. [1] [2] [3]

[jbourassa]

How did you pick l?

I've been thinking we could call this codec.

[makrisoft]

the next letter after f in flavor :P
codec is better! thanks!

[jbourassa]

We could probably drop the max altogether, and emit a warning if the log would've beeb truncated in production? Not sure when log truncation is ever desirable here.

[jbourassa]

LGTM outside the 20kb -> 1kb log limit change.

[jbourassa]

Logs are at most 1kB (1,000 bytes), from the docs:

Function logs to STDERR are truncated after 1 kB. Some Wasm toolchains might crash if STDERR fails to write full logs.

[makrisoft]

oof! I confused it with output :)
I'll fix, thanks!

[jbourassa]

It'd be nice to extend this to include output flavor, but we can do that later.