Memory usage optimization when booting

[niusmallnan]

RancherOS Version: (ros os version)
RancherOS v1.4.0
Where are you running RancherOS? (docker-machine, AWS, GCE, baremetal, etc.)
Any

It should be able to start on 1G memory.

[niusmallnan]

In the boot process, there are two places where the memory requirements are very large.

1. Decompress the initrd
2. Loading the system-docker images

If there is not enough memory in these two processes, then the kernel panic will appear.

As long as we control the file size of initrd and system-docker image, there will be no memory shortage.

So possible options include the following:

1. Streamline rootfs. It’s now lean enough, and the space for continued streamlining is small.
2. Streamline system-docker image. The latest version currently has built-in 18.03 by default. If we switch back to the earlier docker version, it may save space.

I might provide a document that lets users know how to reduce memory dependencies. But the mainline version may still maintain the status quo.

[niusmallnan]

When building the ISO, I tried to change the default user-docker to 17.03.2, and finally the ISO can be booted in 1GB memory.

[albertdb]

Do not replace xz with gzip, there are much better alternatives nowadays such as zstd, in speed (which will make RancherOS boot faster), compression level and memory usage. Please take a look: http://facebook.github.io/zstd/

[niusmallnan]

@albertdb We are using kernel 4.14.x, the zstd kernel compression seems not to be supported.
Consider now and referring to some other distributions, gzip may not be the best choice, but a universal choice.

Ubuntu will use lz4, we will continue to pay attention.
https://news.softpedia.com/news/ubuntu-18-10-will-boot-faster-thanks-to-lz4-initramfs-compression-520317.shtml

[albertdb]

@niusmallnan I'm talking about docker images compressed TARs, not kernel initramfs. As long as the zstd binary is available, recent versions of tar support zstd: https://www.phoronix.com/scan.php?page=news_item&px=Tar-Zstd-Compression

[niusmallnan]

@albertdb Our system-docker is based on docker 17.06(has made some changes). This version should not support images with zstd formatting, I can see that it should only support an archive (.tar, .tar.gz, .tgz, .bzip, .tar.xz, or .txz.).

Maybe your idea is to first extract it with zstd and then import that file into docker. I may need to try if it works, how much will improve. If you have already done this part, I would like to know the result.

But what you said is a good suggestion, this part is really worth optimizing. Thanks！

[albertdb]

@niusmallnan Yes, if TAR version is old, thats what I'm talking about. You can even pipe decompression to docker load, and it would be as fast as docker load is (zstd is almost real time on decompression).

This is what we do when generating our custom RancherOS images (in which we docker load some images) with Packer. Not only is faster than gzip, it also result in smaller source docker images tars.