It’s really funny to degrade your network performance by yourself after adding a higher
delay and packet loss on the link. Even 2 servers are directly connected with a cable
within a 2-meter distance but still, you can add the delay or packet loss like they are
2000km apart from each other.
To edit the network or link properties such as the Delay and jitter and the link loss.in
Linux there is a tool called NetEM.
NetEm (Network Emulation) is Linux kernel functionality that allows the emulation of link
properties for controlling delay, jitter, packet loss, and more.
NetEM is the most appropriate way to play with the delay and drop packets in Linux.This
can be applied to any network port, physical or virtual, whether a physical interface or
a bridge.
tc is a command which we will use for manipulating the delay and packet loss. and tc is a part
of IP route package and that is installed by default so we don’t need to install any additional
package for tc.
1-Modifying the Delay.
Before modifying the delay, Lets ping google.com and see the actual latency .as you can see it below
the latency is now 38ms.
[root@localhost ~]# ping -c 5 google.com PING google.com (172.217.31.206) 56(84) bytes of data. 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=1 ttl=121 time=38.8 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=2 ttl=121 time=38.7 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=3 ttl=121 time=38.2 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=4 ttl=121 time=38.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=5 ttl=121 time=38.2 ms --- google.com ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4006ms rtt min/avg/max/mdev = 38.272/38.501/38.809/0.323 ms [root@localhost ~]#
Now let’s see the default link properties, As you can see below, There is everything default and nothing has been configured on the link. Here, My active interface is eno2.
[root@localhost ~]# tc -s qdisc qdisc noqueue 0: dev lo root refcnt 2 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc mq 0: dev eno2 root Sent 104897762954 bytes 96124760 pkt (dropped 0, overlimits 0 requeues 4147) backlog 0b 0p requeues 4147
Now let’s add the 10ms delay on our interface which is eno2.
[root@localhost ~]# [root@localhost ~]# tc qdisc add dev eno2 root netem delay 10ms [root@localhost ~]#
Now let’s verify the link properties whether the delay is added successfully or not.
[root@localhost ~]# tc -s qdisc qdisc noqueue 0: dev lo root refcnt 2 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc netem 8001: dev eno2 root refcnt 9 limit 1000 delay 10.0ms Sent 4347999 bytes 3511 pkt (dropped 0, overlimits 0 requeues 3) backlog 150b 1p requeues 3
Now it’s time to verify, Earlier we were getting 38ms to google. com, now we have added
another 10ms delay and the total delay should be above 48ms. now let’s ping and check.
As you can see below we are getting above 48ms to google.com.
[root@localhost ~]# ping -c 5 google.com PING google.com (172.217.31.206) 56(84) bytes of data. 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=1 ttl=121 time=48.6 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=2 ttl=121 time=48.5 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=3 ttl=121 time=50.0 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=4 ttl=121 time=48.2 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=5 ttl=121 time=48.2 ms --- google.com ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4005ms rtt min/avg/max/mdev = 48.225/48.759/50.085/0.698 ms [root@localhost ~]#
Now let’s delete the configuration on the link .here is the command given below to remove
the configuration.
[root@localhost ~]# [root@localhost ~]# tc qdisc del dev eno2 root netem [root@localhost ~]#
As you can see below, once we deleted the delay configuration we get 38ms as it was before.
[root@localhost ~]# ping -c 5 google.com PING google.com (142.250.182.78) 56(84) bytes of data. 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=1 ttl=121 time=38.1 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=2 ttl=121 time=38.0 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=3 ttl=121 time=38.0 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=4 ttl=121 time=38.9 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=5 ttl=121 time=38.2 ms --- google.com ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4004ms rtt min/avg/max/mdev = 38.963/39.278/40.112/0.460 ms [root@localhost ~]#
2-Modifying The Packet Loss.
Link loss or packet loss is a dropping mechanism on the link, if you want to drop some
incoming and outgoing packets on the link, you can configure this mechanism.
Before modifying the link loss, let’s ping and verify the current link loss. as you can see
below there is nothing configured and everything is the default.
[root@localhost ~]# tc -s qdisc qdisc noqueue 0: dev lo root refcnt 2 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc mq 0: dev eno2 root Sent 5084 bytes 43 pkt (dropped 0, overlimits 0 requeues 0)
Now let’s add the link loss of 20% and the command given below.
[root@localhost ~]# tc qdisc add dev eno2 root netem loss 20% [root@localhost ~]#
Now let’s verify the currently configured link loss, you can see below it has been configured successfully
, and it’s showing a 20% link loss.
[root@localhost ~]# tc -s qdisc qdisc noqueue 0: dev lo root refcnt 2 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc netem 8009: dev eno2 root refcnt 9 limit 1000 loss 20% Sent 5361 bytes 21 pkt (dropped 8, overlimits 0 requeues 0) backlog 0b 0p requeues 0
Now its time to verify, The server I am testing is a live server and most of the live
traffic is riding on the link so I will not get the exact result of 20% loss because the
server is already dropping some other packet as well so I will collect a statistics of the
link loss after pinging multiple times. lets ping 3 times and in each ping, we will send
20 ping packet and then you can compare 3 pings given below and link loss are almost 20%.
[root@localhost ~]# ping -c 20 google.com PING google.com (142.250.182.78) 56(84) bytes of data. 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=1 ttl=121 time=39.8 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=2 ttl=121 time=41.0 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=4 ttl=121 time=40.8 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=5 ttl=121 time=39.5 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=7 ttl=121 time=40.7 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=8 ttl=121 time=40.6 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=9 ttl=121 time=39.1 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=10 ttl=121 time=41.3 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=11 ttl=121 time=39.0 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=12 ttl=121 time=41.3 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=13 ttl=121 time=39.1 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=14 ttl=121 time=41.0 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=15 ttl=121 time=41.5 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=16 ttl=121 time=39.9 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=17 ttl=121 time=39.0 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=18 ttl=121 time=39.5 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=19 ttl=121 time=39.0 ms 64 bytes from maa05s20-in-f14.1e100.net (142.250.182.78): icmp_seq=20 ttl=121 time=41.5 ms --- google.com ping statistics --- 20 packets transmitted, 18 received, 10% packet loss, time 19023ms rtt min/avg/max/mdev = 39.053/40.245/41.528/0.946 ms [root@localhost ~]# [root@localhost ~]# ping -c 20 google.com PING google.com (172.217.31.206) 56(84) bytes of data. 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=2 ttl=121 time=38.9 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=3 ttl=121 time=38.5 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=4 ttl=121 time=38.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=5 ttl=121 time=38.5 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=6 ttl=121 time=38.8 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=7 ttl=121 time=38.4 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=9 ttl=121 time=38.7 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=10 ttl=121 time=38.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=11 ttl=121 time=38.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=12 ttl=121 time=38.4 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=13 ttl=121 time=39.0 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=14 ttl=121 time=38.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=15 ttl=121 time=38.2 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=16 ttl=121 time=38.0 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=18 ttl=121 time=1035 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=19 ttl=121 time=38.1 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=20 ttl=121 time=38.1 ms --- google.com ping statistics --- 20 packets transmitted, 17 received, 15% packet loss, time 23098ms rtt min/avg/max/mdev = 38.064/97.100/1035.062/234.490 ms, pipe 2 [root@localhost ~]# ping -c 20 google.com PING google.com (172.217.31.206) 56(84) bytes of data. 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=1 ttl=121 time=38.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=2 ttl=121 time=38.1 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=3 ttl=121 time=38.1 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=5 ttl=121 time=38.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=8 ttl=121 time=38.1 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=9 ttl=121 time=38.1 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=10 ttl=121 time=38.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=11 ttl=121 time=38.2 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=12 ttl=121 time=38.2 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=13 ttl=121 time=38.2 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=14 ttl=121 time=38.2 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=15 ttl=121 time=38.1 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=16 ttl=121 time=38.2 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=17 ttl=121 time=38.1 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=19 ttl=121 time=38.1 ms --- google.com ping statistics --- 20 packets transmitted, 15 received, 25% packet loss, time 19018ms rtt min/avg/max/mdev = 38.120/38.230/38.358/0.160 ms [root@localhost ~]#
Now let’s delete the link loss configuration.
[root@localhost ~]# tc qdisc del dev eno2 root netem loss 20% [root@localhost ~]#
After deleting the link loss, you can see no loss detected on the ping .output given below.
[root@localhost ~]# ping -c 20 google.com PING google.com (172.217.31.206) 56(84) bytes of data. 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=1 ttl=121 time=39.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=2 ttl=121 time=38.5 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=3 ttl=121 time=39.8 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=4 ttl=121 time=40.4 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=5 ttl=121 time=40.4 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=6 ttl=121 time=38.5 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=7 ttl=121 time=40.6 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=8 ttl=121 time=38.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=9 ttl=121 time=40.4 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=10 ttl=121 time=39.4 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=11 ttl=121 time=40.5 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=12 ttl=121 time=40.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=13 ttl=121 time=40.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=14 ttl=121 time=38.1 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=15 ttl=121 time=38.4 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=16 ttl=121 time=40.4 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=17 ttl=121 time=40.4 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=18 ttl=121 time=38.3 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=19 ttl=121 time=40.1 ms 64 bytes from maa03s28-in-f14.1e100.net (172.217.31.206): icmp_seq=20 ttl=121 time=38.6 ms --- google.com ping statistics --- 20 packets transmitted, 20 received, 0% packet loss, time 19011ms rtt min/avg/max/mdev = 38.199/39.612/40.649/0.935 ms [root@localhost ~]#