RoCE Configuration

Easy RoCE

Easy RoCE View

Table 1 Easy RoCE View

Command	Purpose
show qos roce [all\|summary\|RoCE_profile_name]	Display “Easy RoCE” configuration.
show counters qos roce interfaceethernet interface_name queue queue-id	Display RoCE statistics of the interface.
clear counters qos roce	Clear RoCE statistics of all interfaces.

show qos roce [all|summary|RoCE_profile_name]

[Command]

show qos roce [all|summary|RoCE_profile_name]

[Purpose]

Display RoCE configuration.

[View]

Global Configuration View

[Notes]

CX732Q-N-V2 devices currently do not support.

[Use Cases]

sonic# show qos roce
Notice: Displaying configuraions of in-use RoCE profiles
==> RoCE Profile: roce_lossless_5m_low_latency | RoCE Policy Map: roce_lossless_5m_low_latency_100g <==
+--------------------+-----------------+-----------------------------------------------------+
|                    | Operational     | Description                                         |
+====================+=================+=====================================================+
| Mode               | Lossless        | QoS RoCE mode                                       |
+--------------------+-----------------+-----------------------------------------------------
| Status             | Bind: 0/0-0/252 | QoS RoCE binding status                             |
+--------------------+-----------------+-----------------------------------------------------+
| Cable Length       | 5m              | Cable length in meters for QoS RoCE lossless config |
+--------------------+-----------------+-----------------------------------------------------+
| Congestion-Control | -               | -                                                   |
|  - Congestion Mode | ECN             | Congestion control mode                             |
|  - Enabled TC      | 3,4             | Congestion control config enabled traffic class     |
|  - Max Threshold   | 1025024         | Congestion control config max threshold             |
|  - Min Threshold   | 15360           | Congestion control config max threshold             |
+--------------------+-----------------+-----------------------------------------------------+
| PFC                | -               | -                                                   |
|  - PFC Priority    | 3,4             | PFC enabled switch priority                         |
|  - TX Status       | Enabled         | PFC RX status                                       |
|  - RX Status       | Enabled         | PFC TX status                                       |
+--------------------+-----------------+-----------------------------------------------------+
| Trust              | -               | -                                                   |
|  - Trust Mode      | DSCP            | Trust setting for packet classification             |
+--------------------+-----------------+-----------------------------------------------------+

====> RoCE DSCP->SP Mapping Configurations <====
+-------------------------+-------------------+
| DSCP                    | Switch Priority   |
+=========================+===================+
| 0,1,2,3,4,5,6,7         | 0                 |
| 8,9,10,11,12,13,14,15   | 1                 |
| 16,17,18,19,20,21,22,23 | 2                 |
| 24,25,26,27,28,29,30,31 | 3                 |
| 32,33,34,35,36,37,38,39 | 4                 |
| 40,41,42,43,44,45,46,47 | 5                 |
| 48,49,50,51,52,53,54,55 | 6                 |
| 56,57,58,59,60,61,62,63 | 7                 |
+-------------------------+-------------------+

====> RoCE SP->TC Mapping & ETS Configurations <====
+-------------------+--------+----------+
| Switch Priority   | Mode   | Weight   |
+===================+========+==========+
| 6                 | SP     | -        |
| 7                 | SP     | -        |
+-------------------+--------+----------+

====> PFC Profile Configurations <====
+-------------------------------------------+-------------------+
| Profile Name                              | Switch Priority   |
+===========================================+===================+
| egress_lossless_profile                   | 3,4               |
| egress_lossy_profile                      | 0,1,2,5,6,7       |
| ingress_lossy_profile                     | 0,1,2,5,6,7       |
| pg_lossless_10000_40m_profile             | 3,4               |
| roce_lossless_5m_low_latency_100g_profile | 3,4               |
+-------------------------------------------+-------------------+

sonic# show qos roce summary
+------------------------------+----------+----------+
| Profile                      | Mode     | Status   |
+==============================+==========+==========+
| roce_lossless_5m_low_latency | lossless | Bind     |
+------------------------------+----------+----------+

show counters qos roce interface ethernet interface_name queue queue-id

[Command]

show counters qos roce interface ethernet interface_name queue queue-id

[Purpose]

Display RoCE statistics of the interface.

[Parameters]

Parameter	Description
interface_name	Interface name (eg.0/1)
queue-id	Queue ID, the range is [0,7]

[View]

Global Configuration View

[Notes]

CX732Q-N-V2 devices currently do not support.

[Use Cases]

sonic# show counters qos roce interface ethernet 0/32 queue 3
                               operational
-----------------------------  -----------------
roce states                    Ethernet32.3
pfc-stats
- pfc_rx_stats                 0
- pfc_tx_stats                 402
- pg-stats
-   total_packet               11,380,786,999
-   total_bytes                1,456,740,735,872
-   drop_packet                0
-   curr_occupancy             0
ecn-stats
- ecn_stats                    0
- ecn_buffer
-   shared_use_watermark_byte  0
-   total_use_watermark_byte   0
-   total_use_count_byte       0
queue-stats
- Counter_pkts                 0
- Counter_bytes                0
- Drop_pkts                    0
- Drop_bytes                   0
- CounterRate_pkts             0.0
- CounterRate_bytes            0.0
- DropRate_pkts                0.0
- DropRate_bytes               0.0
- Occupancy_bytes              0
- SharedOccupancy_bytes        0

clear counters qos roce

[Command]

clear counters qos roce

[Purpose]

Clear RoCE statistics of all interfaces.

[View]

Global Configuration View

[Notes]

CX732Q-N-V2 devices currently do not support.

[Use Cases]

sonic# show counters qos roce interface 0/32 queue 3

Easy RoCE Config

Table 2 Easy RoCE Config

Command	Purpose
qos roce {lossless\|lossy} [cable-length length] [incast-level level] [traffic-model model]	Enable “Easy RoCE”, and create a configuration template.
qos service-policy {roce_lossless\|roce_lossy\|RoCE_profile_name}	Apply “Easy RoCE” configuration to all interfaces.
qos roce RoCE_profile_name	Enter the RoCE template configuration view.
bind interface {all\|ethernet interface_name\|range interface_name_list}	Apply the RoCE configuration template to the specified interfaces.

qos roce {lossless|lossy} [cable-length length] [incast-level level] [traffic-model model]

[Command]

qos roce {lossless|lossy} [cable-length length] [incast-level level] [traffic-model model]

[Purpose]

Enable “Easy RoCE”, and create a configuration template.

[Parameters]

Parameter	Description
length	Specify the cable length, choose from 5m/40m/100m/300m.
level	Specify the traffic incast model, choose from low/medium/high
model	Specify the traffic model, choose from throughput/latency/balance.

[View]

Global Configuration View

[Notes]

To enhance the user-friendliness of AsterNOS in RoCE scenarios, we support “Easy RoCE” configuration and management. There are two modes in RoCE scenario: “Lossless Ethernet” and “Lossy Ethernet”. In lossless mode, the system will automatically generate lossless configuration templates, with default DSCP mapping, enabling PFC and ECN for queue 3 and 4, and setting strict priority scheduling for queue 6 and 7. At the same time, it is supported for you to enter the corresponding configuration view to modify the parameters. Run command no qos roce {lossless|lossy} to remove “Easy RoCE” configuration. CX732Q-N-V2 devices currently do not support.

[Use Cases]

sonic(config)# qos roce lossless
or
sonic(config)# qos roce lossless cable-length 5m incast-level low traffic-model latency
Notice: Creating new RoCE profile roce_lossless_5m_low_latency

qos service-policy {roce_lossless|roce_lossy|RoCE_profile_name}

[Command]

qos service-policy {roce_lossless|roce_lossy|RoCE_profile_name}

[Purpose]

Apply “Easy RoCE” configuration to all interfaces.

[Parameters]

Parameter	Description
RoCE_profile_name	Specify the name of RoCE profile.

[View]

Global Configuration View

[Notes]

Run command no qos service-policy {roce_lossless|roce_lossy|RoCE_profile_name} to unbind all interfaces. CX732Q-N-V2 devices currently do not support.

[Use Cases]

sonic(config)# qos service-policy roce_lossless
Notice: interface 0/0,0/4 has already bind other policy, pleace manually bind roce_lossless to the interfaces
or
sonic(config)# qos service-policy roce_lossless_5m_low_latency
Notice: RoCE is not supported on interface 0/256-0/257, skipping
Notice: Binding RoCE profile roce_lossless_5m_low_latency to interface 0/0-0/252

qos roce RoCE_profile_name

[Command]

qos roce RoCE_profile_name

[Purpose]

Enter the RoCE template configuration view.

[Parameters]

Parameter	Description
RoCE_profile_name	Specify the name of RoCE profile.

[View]

Global Configuration View

[Notes]

Run command no qos roce RoCE_profile_name to delete the template you created. CX732Q-N-V2 devices currently do not support.

[Use Cases]

sonic(config)# qos roce roce_lossless
or
sonic(config)# qos roce roce_lossless_5m_low_latency

bind interface {all|ethernet interface_name|range interface_name_list}

[Command]

bind interface {all|ethernet interface_name |range interface_name_list}

[Purpose]

Apply the RoCE configuration template to the specified interfaces.

[Parameters]

Parameter	Description
all	Bind to all interfaces.
interface_name	Specify the certain interface name id (eg. 0/1)
interface_name_list	Specify the name list (eg. 0/0-0/64,0/128)

[View]

RoCE Template Configuration View

[Notes]

Run command no bind interface {all|ethernet interface_name|range interface_name_list} to unbind. CX732Q-N-V2 devices currently do not support.

[Use Cases]

sonic(config)# qos roce roce_lossless_5m_low_latency
sonic(config-roce_lossless_5m_low_latency)# bind interface range 0/0-0/12
Notice: Binding RoCE profile roce_lossless_5m_low_latency to interface 0/0-0/12

Load Balance

Features below are only supported on CX864E-N.

Packet Spray

Table 3 Packet Spray

Command	Purpose
hash algorithm {random\|round_robin}	Configure the hash algorithm of LAG based on packets.
hash algorithm ecmp {random\|round_robin}	Configure the hash algorithm of ECMP based on packets.

hash algorithm {random|round_robin}

[Command]

hash algorithm {random|round_robin}

[Purpose]

Configure the hash algorithm of LAG based on packets.

[Parameters]

Parameter	Description
random	Random hash allocation of packets.
round_robin	Sequential round-robin allocation of packets.

[View]

LAGIF Configuration View

[Notes]

Run commit to take effect. Defaults to hash based on five-tuple. Only CX864-N supports this feature. Run command no hash algorithm to restore to the default configuration.

[Use Cases]

sonic# configure terminal
sonic(config)# interface link-aggregation 100
sonic(config-lagif-100)# hash algorithm random
Please enter 'commit' to make hash algorithm command take effect
sonic(config-lagif-100)# commit

hash algorithm ecmp {random|round_robin}

[Command]

hash algorithm ecmp {random|round_robin}

[Purpose]

Configure the hash algorithm of ECMP based on packets.

[Parameters]

Parameter	Description
random	Random hash allocation of packets.
round_robin	Sequential round-robin allocation of packets.

[View]

Global Configuration View

[Notes]

Defaults to hash based on five-tuple. Only CX864-N supports this feature. The ECMP hash algorithm and the LAG hash algorithm are independently controlled. Please configure the hash algorithm before route distribution. If the hash algorithm is modified after routes have been distributed, the routes need to be regenerated. Run command no hash algorithm ecmp to restore to the default configuration.

[Use Cases]

onic# configure terminal
sonic(config)# hash algorithm ecmp random

ARS

ARS View

Table 4 ARS View

Command	Purpose
show ars instance	Display ARS instance configuration
show ars interface	Display the ARS status of the interface.
show ars profile	Display ARS profile configuration.

show ars instance

[Command]

show ars instance

[Purpose]

Display ARS instance configuration.

[View]

Privileged User View

[Use Cases]

sonic# show ars instance
Instance Name    Assign Mode            Idle Time    Max Flows  Binding Status
---------------  -------------------  -----------  -----------  ----------------
test             per_flowlet_quality          256          512  N/A

show ars interface

[Command]

show ars instance

[Purpose]

Display the ARS status of the interface.

[View]

Privileged User View

[Use Cases]

sonic# show ars interface
Interface    Status       Scaling Factor
-----------  -----------  ----------------
0/0          Enabled      10000
0/1          Enabled      10000
0/2          Enabled      10000
0/3          Enabled      10000
0/4          Enabled      10000
0/5          Enabled      10000
0/6          Enabled      10000
0/7          Enabled      10000
0/8          Enabled      10000
……

show ars profile

[Command]

show ars profile

[Purpose]

Display ARS profile configuration.

[View]

Privileged User View

[Use Cases]

sonic# show ars profile
Algorithm    IPv4 Enable    IPv6 Enable      Load Past Min    Load Past Max    Overall Max Flows    Remaining Max Flows
-----------  -------------  -------------  ---------------  ---------------  -------------------  ---------------------
EWMA         True           True                         0             4096                 8128                   7616

ARS Config

Table 5 ARS Config

Command	Purpose
ars profile	Create ARS profile.
ars instance instance-name	Create ARS instance.
ars nexthop-group {A.B.C.D/M\|A::B/M} instance instance-name	Bind the ARS instance to an ECMP group.
idle-time time_value	Set the flow splitting interval
max-flows flow-value	Set the maximum number of flowlets that ECMP can track simultaneously in macro-flow.
ars disable	Disable ARS at interface level.

ars profile

[Command]

ars profile

[Purpose]

Create ARS profile.

[View]

Global Configuration View.

[Notes]

Enable ARS globally and create a profile to automatically bind to the switch. This will take effect globally. Currently, the device supports only one profile. Run command no ars profile to delete the ARS profile.

[Use Cases]

sonic# configure terminal
sonic(config)# ars profile
sonic(config-arsprofile)# show this
!
ars profile

ars instance instance-name

[Command]

ars instance instance-name

[Purpose]

Create ARS instance.

[Parameters]

Parameter	Description
instance-name	Name of ARS instance, in string form

[View]

Global Configuration View.

[Notes]

Run command no ars instance instance-name to delete the ARS instance .

[Use Cases]

sonic# configure terminal
sonic(config)# ars instance test
sonic(config-ars-instance-test)# show this
!
ars instance test

ars nexthop-group {A.B.C.D/M|A::B/M} instance instance-name

[Command]

ars nexthop-group {A.B.C.D/M|A::B/M} instance instance-name

[Purpose]

Bind the ARS instance to an ECMP group.

[Parameters]

Parameter	Description
A.B.C.D/M	IPv4 destination address, bound to the ECMP group where the next hop of the destination address belongs
A::B/M	IPv6 destination address, bound to the ECMP group where the next hop of the destination address belongs
instance-name	Name of ARS instance, in string form

[View]

Global Configuration View.

[Notes]

ECMP groups and ARS instances can only be bound one-to-one. Run command no ars nexthop-group {A.B.C.D/M |A::B/M} instance instance-name to delete the bond with ECMP group.

[Use Cases]

sonic# configure terminal
sonic(config)# ars instance test
sonic(config-ars-instance-test)# exit
sonic(config)# ars nexthop-group 10.0.0.0/24 instance test

idle-time time_value

[Command]

idle-time time_value

[Purpose]

Set the flow splitting interval.

[Parameters]

Parameter	Description
time_value	Idle duration in microseconds. The range is 1-1048, and the default value is 256.

[View]

Flow splitting is triggered if the frame interval exceeds this time; ARS instance Configuration View.

[Notes]

Run command no idle-time to restore the idle duration value to the default value.

[Use Cases]

sonic# configure terminal
sonic(config)# ars instance test
sonic(config-ars-instance-test)# idle-time 200
sonic(config-ars-instance-test)# show this
!
ars instance test
 idle-time 200

max-flows flow-value

[Command]

max-flows flow-value

[Purpose]

Set the maximum number of flowlet that ECMP group can track simultaneously in macro-flow table.

[Parameters]

Parameter	Description
flow_value	The number of flowlet, optional parameters 32/64/ 128/256/512/1k/2k, and the default value is 512

[View]

ARS instance Configuration View.

[Notes]

It is recommended to use the default value. Run command no max-flows to restore the number of flowlet to the default value.

[Use Cases]

sonic# configure terminal
sonic(config)# ars instance test
sonic(config-ars-instance-test)# max-flows 1k
!
ars instance test
 max-flows 1k

ars disable

[Command]

ars disable

[Purpose]

Disable ARS at interface-level.

[View]

Physical Interface Configuration View.

[Notes]

Run command no ars disable to enable ARS.

[Use Cases]

sonic# configure terminal
sonic(config)# interface ethernet 0/400
sonic(config-if-0/400)# ars disable
sonic(config-if-0/400)# show this
!
interface ethernet 0/400
 ars disable
 fec rs
 mtu 9216
 speed 800000