Performance testing with VyOS

Far too long ago, I started tinkering with VyOS, a linux router distribution.  I say “too long ago” because that was back in May, and I haven’t found (or made) the time to do much more with it.

I still haven’t done any of the real testing I want to do with it, largely because I have some external dependencies that I need to resolve, but I did log in to try a couple of things this evening, and I discovered a drastic change in the initial numbers I’d posted.

This is the virtual environment I’d built within my Proxmox hypervisor:

VyOS Test Environment
VyOS Test Environment

TestHost1 is an Ubuntu instance on my main network.  It’s actually one of my DNS/DHCP servers.  TestHost2 is another Ubuntu instance contained within the VyOS test environment.  All of this, mind you, is contained within the same physical box, all running on the same hypervisor.

Theoretically, in an environment like this, we’re able to test the actual throughput of a VM running VyOS without being bound by physical interface adapters, or having to hit an actual “wire.”  My assumption is that so long as the VM itself performs well enough, my only limitations in real world use cases would be the hardware characteristics of a production hypervisor (which would include network interfaces), and the physical switching fabric within an actual data center.

A couple of months back, I ran some iperf numbers from the external host to the internal host, and this is what I got:

Client connecting to, TCP port 5001
TCP window size: 65.0 KByte (default)
[  3] local port 13836 connected with port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 5.0 sec  1.10 GBytes  1.90 Gbits/sec
[  3]  5.0-10.0 sec   998 MBytes  1.67 Gbits/sec
[  3]  0.0-10.0 sec  2.08 GBytes  1.79 Gbits/sec

Not too shabby, approaching 2Gbps…

Over the course of the past couple of months, I’d logged in a few times, and had discovered some minor issues, but nothing serious – in fact, I don’t even remember what they were.  I’d just fixed them and moved on.

They were trivial enough that I’d never re-run the iperf tests.  Until now, and WOW

From the outside in:

Client connecting to, TCP port 5001
TCP window size: 85.0 KByte (default)
[  3] local port 53578 connected with port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  19.8 GBytes  17.0 Gbits/sec
[  3] 10.0-20.0 sec  19.6 GBytes  16.8 Gbits/sec
[  3] 20.0-30.0 sec  19.4 GBytes  16.6 Gbits/sec
[  3] 30.0-40.0 sec  19.4 GBytes  16.7 Gbits/sec
[  3] 40.0-50.0 sec  19.3 GBytes  16.5 Gbits/sec
[  3] 50.0-60.0 sec  19.4 GBytes  16.7 Gbits/sec
[  3]  0.0-60.0 sec   117 GBytes  16.7 Gbits/sec

That’s through two sequential VyOS instances, VyOS1 & VyOS2.  And remember, these are minimally provisioned VMs, with a single core, and just 512MB of RAM.

I will be spinning some of these up on some big hypervisors at work shortly to start testing the real stuff I want to see – setting VyOS up as a spoke in a DMVPN network with Cisco Hubs.

Share this post:
war Written by:

One Comment

Leave a Reply

Your email address will not be published. Required fields are marked *