@@ -623,7 +623,6 @@ MPI Functionality and Features
623
623
- portals4
624
624
625
625
(2) The ob1 PML and the following BTLs support MPI_THREAD_MULTIPLE:
626
- - openib (see exception below)
627
626
- self
628
627
- sm
629
628
- smcuda
@@ -632,10 +631,6 @@ MPI Functionality and Features
632
631
- usnic
633
632
- vader (shared memory)
634
633
635
- The openib BTL's RDMACM based connection setup mechanism is also not
636
- thread safe. The default UDCM method should be used for
637
- applications requiring MPI_THREAD_MULTIPLE support.
638
-
639
634
Currently, MPI File operations are not thread safe even if MPI is
640
635
initialized for MPI_THREAD_MULTIPLE support.
641
636
@@ -794,7 +789,8 @@ Network Support
794
789
- In prior versions of Open MPI, InfiniBand and RoCE support was
795
790
provided through the openib BTL and ob1 PML plugins. Starting with
796
791
Open MPI 4.0.0, InfiniBand support through the openib plugin is both
797
- deprecated and superseded by the ucx PML component.
792
+ deprecated and superseded by the ucx PML component. The openib BTL
793
+ was removed in Open MPI v5.0.0.
798
794
799
795
While the openib BTL depended on libibverbs, the UCX PML depends on
800
796
the UCX library.
@@ -809,15 +805,6 @@ Network Support
809
805
for OpenSHMEM support, and "--mca osc ucx" for MPI RMA (one-sided)
810
806
operations.
811
807
812
- - Although the ob1 PML+openib BTL is still the default for iWARP and
813
- RoCE devices, it will reject InfiniBand defaults (by default) so
814
- that they will use the ucx PML. If using the openib BTL is still
815
- desired, set the following MCA parameters:
816
-
817
- # Note that "vader" is Open MPI's shared memory BTL
818
- $ mpirun --mca pml ob1 --mca btl openib,vader,self \
819
- --mca btl_openib_allow_ib 1 ...
820
-
821
808
- The usnic BTL is support for Cisco's usNIC device ("userspace NIC")
822
809
on Cisco UCS servers with the Virtualized Interface Card (VIC).
823
810
Although the usNIC is accessed via the OpenFabrics Libfabric API
@@ -850,8 +837,8 @@ Network Support
850
837
851
838
http://lwn.net/Articles/343351/
852
839
853
- - The use of fork() with OpenFabrics-based networks (i.e., the openib
854
- BTL ) is only partially supported, and only on Linux kernels >=
840
+ - The use of fork() with OpenFabrics-based networks (i.e., the UCX
841
+ PML ) is only partially supported, and only on Linux kernels >=
855
842
v2.6.15 with libibverbs v1.1 or later (first released as part of
856
843
OFED v1.2), per restrictions imposed by the OFED network stack.
857
844
@@ -1206,51 +1193,6 @@ NETWORKING SUPPORT / OPTIONS
1206
1193
--with-usnic
1207
1194
Abort configure if Cisco usNIC support cannot be built.
1208
1195
1209
- --with-verbs=<directory>
1210
- Specify the directory where the verbs (also known as OpenFabrics
1211
- verbs, or Linux verbs, and previously known as OpenIB) libraries and
1212
- header files are located. This option is generally only necessary
1213
- if the verbs headers and libraries are not in default
1214
- compiler/linker search paths.
1215
-
1216
- The Verbs library usually implies operating system bypass networks,
1217
- such as InfiniBand, usNIC, iWARP, and RoCE (aka "IBoIP").
1218
-
1219
- --with-verbs-libdir=<directory>
1220
- Look in directory for the verbs libraries. By default, Open MPI
1221
- will look in <verbs_directory>/lib and <verbs_ directory>/lib64,
1222
- which covers most cases. This option is only needed for special
1223
- configurations.
1224
-
1225
- --with-verbs-usnic
1226
- Note that this option is no longer necessary in recent Linux distro
1227
- versions. If your Linux distro uses the "rdma-core" package (instead
1228
- of a standalone "libibverbs" package), not only do you not need this
1229
- option, you shouldn't use it, either. More below.
1230
-
1231
- This option will activate support in Open MPI for disabling a
1232
- dire-sounding warning message from libibverbs that Cisco usNIC
1233
- devices are not supported (because Cisco usNIC devices are supported
1234
- through libfabric, not libibverbs). This libibverbs warning can
1235
- also be suppressed by installing the "no op" libusnic_verbs plugin
1236
- for libibverbs (see https://github.com/cisco/libusnic_verbs, or
1237
- download binaries from cisco.com).
1238
-
1239
- This option is disabled by default for two reasons:
1240
-
1241
- 1. It causes libopen-pal.so to depend on libibverbs.so, which is
1242
- undesirable to many downstream packagers.
1243
- 2. As mentioned above, recent versions of the libibverbs library
1244
- (included in the "rdma-core" package) do not have the bug that
1245
- will emit dire-sounding warnings about usnic devices. Indeed,
1246
- the --with-verbs-usnic option will enable code in Open MPI that
1247
- is actually incompatible with rdma-core (i.e., cause Open MPI to
1248
- fail to compile).
1249
-
1250
- If you enable --with-verbs-usnic and your system uses the rdma-core
1251
- package, configure will safely abort with a helpful message telling
1252
- you that you should not use --with-verbs-usnic.
1253
-
1254
1196
1255
1197
RUN-TIME SYSTEM SUPPORT
1256
1198
0 commit comments