TECHPUBS-4735: Updated Soft-RoCE and Lustre information (#138)

* TECHPUBS-4735: updated soft-roce and lustre instructions

* TECHPUBS-4735: review feedback

* TECHPUBS-4735: updated prereq wording

* TECHPUBS-4735: updated title and intro text

* TECHPUBS-4735: fixed errors in values

* TECHPUBS-4735: updated language around lustre

Author: nrockershousen

.spelling

@@ -80,6 +80,7 @@ chroot
 CLI
 cli
 client-server
+ClusterStor
 cm-cli
 COCN
 conditionalization
@@ -387,6 +388,7 @@ NUMA
 Nvidia
 NVIDIA
 ntp
+o2ib
 OData
 ogopogod
 onloaded
@@ -604,6 +606,7 @@ S-9009
 S-9010
 S-9011
 S-9012
+S-9100
 S-9929
 
 

docs/portal/developer-portal/install/lustre_network_driver_lnd_ko2iblnd_configuration.md

@@ -1,6 +1,15 @@
-# Lustre Network Driver (LND) ko2iblnd configuration
+# Configure ko2iblnd Lustre Network Driver (LND) for Soft-RoCE performance
 
-The ko2iblnd.ko changes are needed for better Soft-RoCE performance on LNDs.
+The ko2iblnd.ko module requires modifications to optimize Soft-RoCE performance.
+If your setup does not involve Soft-RoCE connections, this section does not apply.
+
+## Prerequisites
+
+Ensure that Lustre is installed with the ko2iblnd module built for the Soft-RoCE driver (RXE) by specifying `--with-o2ib=yes` for `/.configure` or `rpmbuild`.
+If this option is not specified, the build process will attempt to automatically detect external OFED installations or internal o2ib support.
+If neither is detected, the ko2iblnd module will not be built.
+
+For detailed instructions, see the _Cray ClusterStor Lustre Client Build Configuration Guide S-9100_.
 
 ## Compute Node tuning for Soft-RoCE
 
@@ -49,7 +58,7 @@ Tuning on compute node can be achieved in two ways. Follow the steps that work b
       /sys/module/ko2iblnd/parameters/cksum:0
       /sys/module/ko2iblnd/parameters/concurrent_sends:84
       /sys/module/ko2iblnd/parameters/conns_per_peer:4
-      /sys/module/ko2iblnd/parameters/credits:84
+      /sys/module/ko2iblnd/parameters/credits:1024
       /sys/module/ko2iblnd/parameters/dev_failover:1
       /sys/module/ko2iblnd/parameters/fmr_cache:1
       /sys/module/ko2iblnd/parameters/fmr_flush_trigger:384
@@ -112,7 +121,7 @@ NOTE: These changes will not persist on file system upgrade and should be reappl
    ```console
    [root@hpelus1n01 ~]# vim /mnt/nfsdata/images/$(nodeattr -UV ver)/appliance.x86_64/etc/modprobe.d/ko2iblnd.conf
 
-   options ko2iblnd conns_per_peer=4 ntx=2048 peer_credits=42 peer_credits_hiw=64 concurrent_sends=256 credits=1024 map_on_demand=1
+   options ko2iblnd conns_per_peer=4 ntx=2048 peer_credits=42 peer_credits_hiw=64 concurrent_sends=84 credits=1024 map_on_demand=1
    ```
 
 7. Recreate the SquashFS image after updating the `ko2iblnd.conf` file in the image.
@@ -163,7 +172,7 @@ NOTE: These changes will not persist on file system upgrade and should be reappl
     /sys/module/ko2iblnd/parameters/cksum:0
     /sys/module/ko2iblnd/parameters/concurrent_sends:84
     /sys/module/ko2iblnd/parameters/conns_per_peer:4
-    /sys/module/ko2iblnd/parameters/credits:84
+    /sys/module/ko2iblnd/parameters/credits:1024
     /sys/module/ko2iblnd/parameters/dev_failover:1
     /sys/module/ko2iblnd/parameters/fmr_cache:1
     /sys/module/ko2iblnd/parameters/fmr_flush_trigger:384

docs/portal/developer-portal/install/softroce_on_HPE_Slingshot_200Gbps.md

@@ -5,6 +5,10 @@
 1. `cray-cxi-driver` RPM package must be installed.
 2. `cray-rxe-driver` RPM package must be installed.
 3. HPE Slingshot 200Gbps NIC Ethernet must be configured and active.
+4. Lustre is being used.
+
+   **NOTE:** This procedure contains configuration settings specific to Lustre.
+   If you are using a different filesystem, you will need to determine and apply the appropriate configuration settings for that filesystem. Proceed with caution
 
 ## Configuration
 
@@ -18,9 +22,8 @@ Follow the relevant procedures to achieve the needed configuration. Contact a sy
    cxi-eth.large_pkts_buf_count=10000
    ```
 
-2. If Lustre is being used, modify the client and server parameters.
+2. Modify the client and server parameters.
 
-   Skip this step if Lustre is not being used.
    See the [Lustre configuration](lustre_network_driver_lnd_ko2iblnd_configuration.md#lustre-network-driver-lnd-ko2iblnd-configuration) procedure for more details.
 
    Update the parameters on the client.
@@ -40,6 +43,10 @@ Follow the relevant procedures to achieve the needed configuration. Contact a sy
 
 4. Create an RXE (Soft-RoCE) device by running `rxe_init.sh [devices list]` as root.
 
+   The `rxe_init.sh` script is provided in the DKMS package.
+   It is in the installed source directory's `scripts` subdirectory.
+   If not done already, copy the `rxe_init.sh` script to the binary RPM's install location of `/usr/bin` or run the script from the `/usr/src/cray-rxe-driver-<version>/scripts` directory of the DKMS package.
+
    NOTE: At this time, HPE Slingshot 200Gbps NICs do not automatically create RXE devices, so it must be done manually.
 
    `[devices list]` is a list of interfaces to create RXE devices for.