qperf measures bandwidth and latency between two nodes. It can work over TCP/IP as well as the RDMA transports.

How to install qperf?

On CentOS/RHEL8, qperf is available from the EPEL repository. Install it like this:

# enable the EPEL repo
$ sudo yum install epel-release

# install the package itself
$ sudo yum install qperf

The Ubuntu/Debian repositories do not have qperf, so you’ll need to build it from source.

# get the tools needed to build qperf
$ sudo apt-get install -y make gcc libc-dev

# download the qperf source tarball
$ wget https://www.openfabrics.org/downloads/qperf/qperf-0.4.9.tar.gz

# unpack it
$ tar xvf qperf-0.4.9.tar.gz

# build it
$ cd qperf-0.4.9
$ ./configure
$ make

# src/qperf --version

Open the firewall on the Server

# firewall-cmd --permanent --add-port=19765-19766/tcp
# firewall-cmd --reload

On Server

# Start Server to listen
$ qperf

On Client

Connect to qperf Server with qperf Client and measure bandwidth

$ qperf -ip 19766 -t 60 qperf_server_ip_address tcp_bw
tcp_bw:
    bw  =  2.52 GB/sec

Connect to qperf Server with qperf Client and measure latency

$ qperf -vvs qperf_server_ip_address tcp_lat
tcp_lat:
latency = 20.7 us
msg_rate = 48.1 K/sec
loc_send_bytes = 48.2 KB
loc_recv_bytes = 48.2 KB
loc_send_msgs = 48,196
loc_recv_msgs = 48,196
rem_send_bytes = 48.2 KB
rem_recv_bytes = 48.2 KB
rem_send_msgs = 48,197
rem_recv_msgs = 48,197

UDP bandwidth and latency tests 

$ qperf -v 10.99.0.1 udp_bw udp_lat
udp_bw:
    send_bw         =   248 MB/sec
    recv_bw         =   238 MB/sec
    msg_rate        =  7.25 K/sec
    send_cost       =   302 ms/GB
    recv_cost       =   547 ms/GB
    send_cpus_used  =   7.5 % cpus
    recv_cpus_used  =    13 % cpus
udp_lat:
    latency        =  46.7 us
    msg_rate       =  21.4 K/sec
    loc_cpus_used  =  11.5 % cpus
    rem_cpus_used  =    10 % cpus

TCP bandwidth and latency tests 

$ qperf -t 60 -v 10.99.0.1 tcp_bw tcp_lat
tcp_bw:
    bw              =   239 MB/sec
    msg_rate        =  3.65 K/sec
    time            =    60 sec
    send_cost       =   254 ms/GB
    recv_cost       =  1.15 sec/GB
    send_cpus_used  =  6.07 % cpus
    recv_cpus_used  =  27.6 % cpus
tcp_lat:
    latency        =  47.4 us
    msg_rate       =  21.1 K/sec
    time           =    60 sec
    loc_cpus_used  =    11 % cpus
    rem_cpus_used  =    10 % cpus

References

https://www.opsdash.com/blog/network-performance-linux.html
https://thelinuxcluster.com/2021/02/15/using-qperf-to-measure-network-bandwidth-and-latency/