[FEATURE]Finetune LLama-2-70B using 16 * A10 GPUs.

[babytdream]

Hello!Can I use LLM Studio to Finetune LLama-2-70B using 16 * A10 (16 * 24G)GPUs ?
Can you give me a command if possible？Thanks!

[psinger]

We currently do not directly support multi-node. I assume this is a multi-node setup, right?

[babytdream]

@psinger Yes, it is. Can I finetune it in FSDP + 16-bit LoRA , FSDP + 8-bit qLoRA or FSDP or 4-bit qLoRA.

[psinger]

FSDP deepspeed support is currently ongoing work: #288

And relevant open issues:
#98
#239

Multi-node is currently not on the roadmap.

[pascal-pfeiffer]

Actually, multi node should be possible with CLI (as using native torchrun), but I never tried myself.