Embed binary files as base64 encoded strings

[jandubois]

They need to be broken into smaller lines; yqlib doesn't do this, and yamlfmt fails with a buffer overflow when it tries to keep existing line breaks.

Fixes #3403

[jandubois]

I'm going to switch this to draft because I just thought of 2 changes I want to make:

• Add an INFO level log message when a file is base64 encoded
• Check max line length for scripts / text data files, and encode them as well if they are too long

[jandubois]

I'm going to switch this to draft because I just thought of 2 changes I want to make:

• Add an INFO level log message when a file is base64 encoded
• Check max line length for scripts / text data files, and encode them as well if they are too long

Implemented:

❯ cat tmpl.yaml
base: template://alpine
provision:
- mode: data
  file: sample.data
  path: /tmp/data
❯ perl -E 'say "A" x 66000' > sample.data
❯ limactl tmpl copy --embed tmpl.yaml /dev/null
WARN[0000] `base` is experimental
WARN[0000] `provision[*].file` and `probes[*].file` are experimental
INFO[0000] File "/Users/jan/suse/lima/sample.data" is being base64 encoded: line 1 (offset 0) is longer than 65000 characters
❯ cp /bin/cat sample.data
❯ limactl tmpl copy --embed tmpl.yaml /dev/null
WARN[0000] `base` is experimental
WARN[0000] `provision[*].file` and `probes[*].file` are experimental
INFO[0000] File "/Users/jan/suse/lima/sample.data" is being base64 encoded: unprintable character '\x00' at offset 4

[jandubois]

Full example:

❯ cat tmpl.yaml
base: template://alpine
provision:
- mode: system
  file: my.script

❯ echo -e 'echo "Hello world\a\n"' > my.script
❯ hexdump -C my.script
00000000  65 63 68 6f 20 22 48 65  6c 6c 6f 20 77 6f 72 6c  |echo "Hello worl|
00000010  64 07 0a 22 0a                                    |d..".|
00000015

❯ limactl start -y tmpl.yaml
WARN[0000] `base` is experimental
WARN[0000] `provision[*].file` and `probes[*].file` are experimental
INFO[0000] File "/Users/jan/suse/lima/my.script" is being base64 encoded: unprintable character '\a' at offset 17
…

❯ grep -A2 provision ~/.lima/tmpl/lima.yaml
provision:
- mode: system
  script: !!binary ZWNobyAiSGVsbG8gd29ybGQHCiIK

❯ limactl shell tmpl sudo grep Hello /var/log/cloud-init-output.log
Hello world

[jandubois]

Sorry, did one more force-push to move emcodeScriptReason just behind the definition of maxLineLength, and before binaryScript. Feels more logical to me this way.

[jandubois]

Not that it really matters, but I've decided that maxLineLength should include the newline at the end, so I've updated the PR once more.

[alexandear]

I'm not sure if it's a good idea to declare maxLineLength as var instead of const.
For unit tests, we can use strings.Repeat to construct a very long string.

Suggested change

	var maxLineLength = 65000
	const maxLineLength = 65000

[jandubois]

I was using strings.Repeat before, but changed it in response to a request from @AkihiroSuda to add a test for `maxLineLength'.

It is not an exported variable, and the comment right above says it is only mutable so unit tests can mock it. Unit tests should use constant data for "expected" results. Otherwise a thinko in the implementation of the function that is being tested can carry over in the test function constructing the test data and cancel out the error in the implementation. Not that it matters in a trivial case like this.

I think it is fine as is.

[AkihiroSuda]

Thanks