Is 128Gb RAM enough to plot using SSD for tmp1 and Ramdisk for tmp2 disks?

[ninjaproger]

• Is 128Gb RAM enough if using NVME SSD as tmp1 and 110Gib Ramdisk as tmp2?
• What maximum number of threads I can use in that setup?

[Kiggles]

Yes - but only up to ~16 threads, and may want to run OS headless? I'm not sure how tight the memory constraints would be. You will have more room to play with less than 16 threads.

[marramgg]

@Kiggles could you explain why only up to ~16 threads?

I'm able to expand my system to 128GB and currently have a 3950X (16 cores / 32 threads). Any reason why I wouldn't be able to use all 32 threads here? Thanks!

[Kiggles]

Each thread requires a certain amount of memory to do it's own job. 1GB at most, assuming default bucket count. Increasing bucket count could help - but then you pay a RAM thrashing tax. Worth it? I have no clue.

110GiB ram drive will be 119GB. 16 threads will generally use a bit over 6GB at any single time. Could be instances it is a lot more and you pay a swap file tax, but only periodically, if ever. That leaves about 1-3 GB for operating system and anything else. 16 is "safe", but isn't perfect. 17, or even 18 might perform just as well.

You can go higher. Maybe even out of the box. Worst case you get more swap file hits. But breaking it down in a performance/cost benefit analysis is beyond my expertise.

[casa10]

How do you configure ramdisk on Linux and windows please?

[farchord]

@F1DNA I'm sorry, I do not know. I know how to dabble in linux and learn as I go, shm is one thing I never heard of before.

[F1DNA]

Ah, by default (ubuntu anyway), /dev/shm is set = to half of your total RAM. It is not reserved, however. Meaning it doesn't steal half of RAM away right from the get-go. Applications can then use it as needed. For example. On my plex server, I have memory used for transcoding go to /dev/shm. On my PiHole server, logs go to /dev/shm. Now, you edit /etc/fstab and override the default value to whatever you want. In this case, I tossed it 112GB.

https://askubuntu.com/questions/108135/how-do-i-set-the-default-size-of-dev-shm

Having looked this over, I think that link answered my question. /dev/shm is just a standard tmpfs

[marramgg]

@F1DNA thanks for sharing that info. May I assume you have 128GB total RAM? How does the system behave if you need more than the difference between what's assigned to /dev/shm?

How many threads are you able to assign the plotter with this setup?

[F1DNA]

I currently have 122GB of RAM allocated to this VM and 20 vCPU. I give the plotter 16 threads but it hammers all 20 on its own. As far as RAM usage goes, I gave /dev/shm more than what madmax says it needs because if it doesn't take it all, then other resources can use it. With the 16 threads running and the RAM disk, I have not run out of memory with 122GB in total thus far. Unfortunately I am not currently running anything to datalog stuff to give you a better idea. All I know is I am not running out. If I get around to setting up prometheus and grafana soon, I can let you know but suffice to say, 128GB is enough.

[marramgg]

Thanks for taking the time to reply. I need to read a bit more about /dev/shm and check if I commit to adding an extra 64GB RAM after confirming there's enough clearance with the cooler.

My understanding of the /dev/shm is that you may allocate and resize an arbitrary amount (mount with resize option for example) and if that's not used, the system is able to use the shared memory. I'm not clear on this. Just read something very quickly.