[PD] Remove the requirement of config file for mooncake backend

[ShangmingCai]

This PR completely removes the Mooncake backend's dependency on configuration files, allowing us to say goodbye to the tedious configuration preparation process. From now on, there's no need to prepend the MOONCAKE_CONFIG_PATH=./mooncake.json environment variable before commands as well.

Note: Need to update the mooncake package version with pip3 install mooncake-transfer-engine==0.3.0b5.

The PR includes the following key changes:

• Removed Mooncake Metadata Server Dependency:

  • Replaced the metadata server component in the bootstrap server with a peer-to-peer (P2P) connection approach for handshakes. Provides better usability, flexibility, and eliminates potential single points of failure.

  • Special thanks to @doujiang24 from the Mooncake community and Ant Group for his contribution of the "P2PHANDSHAKE" method in PR #240 and [PD] transfer engine remove the metadata server. #5394.

• Automatic RDMA Device Discovery:

  • Implemented automatic detection of underlying RDMA devices via PR #244. Eliminates the need for manual IB device specification in most cases. Thanks to @alogfans from the Mooncake team.
  • Users can still explicitly specify devices using --disaggregation-ib-device if needed.

• Automatic local IP address detection.

• Temporarily restricted protocol to "rdma" as the current implementation only supports RDMA-enabled environments.

Conclusion: All previously required configurations in mooncake.json can now be automatically detected, making the configuration file safely removable.

Next Steps:

• We've implemented GPU and RDMA device affinity detection, but integration will be delayed due to potential CUDA dependency requirements in pip packaging. This optimization will be included in future updates.

CC List: @doujiang24 @alogfans @zhyncs @ByronHsu @hnyls2002 @whybeyoung

Motivation

Modifications

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[whybeyoung]

LGTM, i will test it

[whybeyoung]

Crashed! See the log .. kvcache-ai/Mooncake#244 (comment)

[ShangmingCai]

Crashed! See the log .. kvcache-ai/Mooncake#244 (comment)

@whybeyoung Thank you for testing this. We will submit a PR to fix this in the mooncake repo and release a new pip wheel tomorrow.

[ShangmingCai]

For anyone interested in this PR, if you encounter an error with the new Automatic RDMA Device Discovery feature, you can always use --disaggregation-ib-device to specify the correct devices that used to be in the config JSON file.

[whybeyoung]

The current bottleneck of this PR lies in the need to explicitly specify the IB device using --disaggregation-ib-device. Based on our testing, the automatic IB device detection in the underlying C++ TransferEngine still requires some adaptation work in certain cloud vendor environments.

That said, this is already a significant improvement compared to the previous requirement of manually specifying a JSON file, and it also removes the dependency on the MetadataServer.

[whybeyoung]

@hnyls2002 i think we can merge this first .