DMVPN Interoperability – Part 1

If you search for DMVPN between Cisco and VyOS, there’s not a lot out there – at least, not much that I found, in terms of some ready to go configuration examples.  The good news: That’s what I’m about to give you.  Last night, I spun up a Cisco CSR 1000v on my home hypervisor so that I could do some mixed environment interoperability testing.

The focus so far is simply on getting things to work, rather than throughput.  At this point, I don’t have any licensing applied to the CSR, so I’m expecting there to be performance limitations.  The VyOS developers, who’ve taken an interest in what I’ve been up to, are going to try to get me some trial CSR licenses so that I’ll be able to do some performance testing similar to what I’ve previously posted.

Here’s the current topology:

DMVPN Interoperability - Cisco Hub
DMVPN Interoperability – Cisco Hub

Essentially identical to what I’d set up for the homogenous VyOS DMVPN testing.  The only differences are that the DMVPN hub is now a Cisco CSR1000v, and the ip addressing of the hub is changed (My VyOS hub is still out there) so as to avoid any conflicts.

Hub Config (Cisco)

crypto keyring DMVPN 
  pre-shared-key address 0.0.0.0 0.0.0.0 key SECRET
!         
crypto isakmp policy 10
encr aes 256
authentication pre-share
group 2
!
crypto isakmp keepalive 60
!
crypto isakmp profile DMVPN
   keyring DMVPN
   match identity address 0.0.0.0          
!         
crypto ipsec transform-set DMVPN-AES256 esp-aes 256 esp-sha-hmac
mode transport
!         
crypto ipsec profile DMVPN
set security-association idle-time 720
set transform-set DMVPN-AES256
!
interface GigabitEthernet1
ip address 10.0.255.10 255.255.255.0
negotiation auto
!         
interface Tunnel0
ip address 10.0.0.10 255.255.255.0
no ip redirects
ip nhrp map multicast dynamic
ip nhrp network-id 1
ip nhrp holdtime 600
ip nhrp server-only
ip nhrp registration timeout 75
tunnel source GigabitEthernet1
tunnel mode gre multipoint
tunnel key 1
tunnel path-mtu-discovery
tunnel protection ipsec profile DMVPN
!         
router bgp 65000
bgp router-id 10.0.0.10
bgp log-neighbor-changes
neighbor DMVPN peer-group
neighbor DMVPN remote-as 65000
neighbor DMVPN timers 5 15
neighbor DMVPN route-reflector-client
neighbor 10.0.0.11 peer-group DMVPN
neighbor 10.0.0.12 peer-group DMVPN
neighbor 10.0.0.13 peer-group DMVPN

Spoke Config (VyOS)

set interfaces ethernet eth0 address 10.0.255.11/24
set interfaces ethernet eth0 description Outside
!
set interfaces ethernet eth1 address 10.0.1.1/24
set interfaces ethernet eth1 description Inside
!
set interfaces tunnel tun0 address 10.0.0.11/24
set interfaces tunnel tun0 encapsulation gre
set interfaces tunnel tun0 local-ip 10.0.255.11
set interfaces tunnel tun0 multicast enable
set interfaces tunnel tun0 parameters ip key 1
!
set protocols bgp 65000 neighbor 10.0.0.10 remote-as 65000
set protocols bgp 65000 neighbor 10.0.0.10 update-source 10.0.0.11
set protocols bgp 65000 network 10.0.1.0/24
set protocols bgp 65000 parameters router-id 10.0.0.11
set protocols bgp 65000 timers holdtime 15
set protocols bgp 65000 timers keepalive 5
!
set protocols nhrp tunnel tun0 cisco-authentication CISCO
set protocols nhrp tunnel tun0 map 10.0.0.10/24 cisco
set protocols nhrp tunnel tun0 map 10.0.0.10/24 nbma-address 10.0.255.10
set protocols nhrp tunnel tun0 map 10.0.0.10/24 register
set protocols nhrp tunnel tun0 multicast nhs
set protocols nhrp tunnel tun0 redirect
set protocols nhrp tunnel tun0 shortcut
!
set vpn ipsec esp-group ESP-SPOKE compression disable
set vpn ipsec esp-group ESP-SPOKE lifetime 1800
set vpn ipsec esp-group ESP-SPOKE mode tunnel
set vpn ipsec esp-group ESP-SPOKE pfs dh-group2
set vpn ipsec esp-group ESP-SPOKE proposal 1 encryption aes256
set vpn ipsec esp-group ESP-SPOKE proposal 1 hash sha1
!
set vpn ipsec ike-group IKE-SPOKE ikev2-reauth no
set vpn ipsec ike-group IKE-SPOKE key-exchange ikev1
set vpn ipsec ike-group IKE-SPOKE lifetime 3600
set vpn ipsec ike-group IKE-SPOKE proposal 1 encryption aes256
set vpn ipsec ike-group IKE-SPOKE proposal 1 hash sha1
!
set vpn ipsec ipsec-interfaces interface eth0
!
set vpn ipsec profile DMVPN authentication mode pre-shared-secret
set vpn ipsec profile DMVPN authentication pre-shared-secret SECRET
set vpn ipsec profile DMVPN bind tunnel tun0
set vpn ipsec profile DMVPN esp-group ESP-SPOKE
set vpn ipsec profile DMVPN ike-group IKE-SPOKE
!
set vpn ipsec site-to-site

Spokes 2 & 3 are identical, with the exception of ip addressing.

Here’s what the world looks like from the Hub’s perspective:

csr1000v#sh ip nhrp brief
 ****************************************************************************
     NOTE: Link-Local, No-socket and Incomplete entries are not displayed
 ****************************************************************************
 Legend: Type --> S - Static, D - Dynamic
         Flags --> u - unique, r - registered, e - temporary, c - claimed
         a - authoritative, t - route
 ============================================================================
 Intf     NextHop Address                                    NBMA Address
          Target Network                              T/Flag
 -------- ------------------------------------------- ------ ----------------
 Tu0      10.0.0.11                                          10.0.255.11
          10.0.0.11/32                                D/ur
 Tu0      10.0.0.12                                          10.0.255.12
          10.0.0.12/32                                D/ur
 Tu0      10.0.0.13                                          10.0.255.13
          10.0.0.13/32                                D/ur

csr1000v#sh crypto ipsec sa
 interface: Tunnel0
     Crypto map tag: Tunnel0-head-0, local addr 10.0.255.10
 
    protected vrf: (none)
    local  ident (addr/mask/prot/port): (10.0.255.10/255.255.255.255/47/0)
    remote ident (addr/mask/prot/port): (10.0.255.11/255.255.255.255/47/0)
    current_peer 10.0.255.11 port 500
      PERMIT, flags={origin_is_acl,}
     #pkts encaps: 616, #pkts encrypt: 616, #pkts digest: 616
     #pkts decaps: 633, #pkts decrypt: 633, #pkts verify: 633
     #pkts compressed: 0, #pkts decompressed: 0
     #pkts not compressed: 0, #pkts compr. failed: 0
     #pkts not decompressed: 0, #pkts decompress failed: 0
     #send errors 0, #recv errors 0
      local crypto endpt.: 10.0.255.10, remote crypto endpt.: 10.0.255.11
      plaintext mtu 1426, path mtu 1472, ip mtu 1472, ip mtu idb Tunnel0
      current outbound spi: 0xC86A4F26(3362410278)
      PFS (Y/N): Y, DH group: group2
      inbound esp sas:
       spi: 0x87F5D021(2281033761)
         transform: esp-256-aes esp-sha-hmac ,
         in use settings ={Transport, }
         conn id: 2165, flow_id: CSR:165, sibling_flags FFFFFFFF80000008, crypto map: Tunnel0-head-0
         sa timing: remaining key lifetime (k/sec): (4607933/2018)
         IV size: 16 bytes
         replay detection support: Y
         Status: ACTIVE(ACTIVE)
      inbound ah sas:
      inbound pcp sas:
      outbound esp sas:
       spi: 0xC86A4F26(3362410278)
         transform: esp-256-aes esp-sha-hmac ,
         in use settings ={Transport, }
         conn id: 2166, flow_id: CSR:166, sibling_flags FFFFFFFF80000008, crypto map: Tunnel0-head-0
         sa timing: remaining key lifetime (k/sec): (4607964/2018)
         IV size: 16 bytes
         replay detection support: Y
         Status: ACTIVE(ACTIVE)
      outbound ah sas:
      outbound pcp sas:
           
    protected vrf: (none)
    local  ident (addr/mask/prot/port): (10.0.255.10/255.255.255.255/47/0)
    remote ident (addr/mask/prot/port): (10.0.255.13/255.255.255.255/47/0)
    current_peer 10.0.255.13 port 500
      PERMIT, flags={origin_is_acl,}
     #pkts encaps: 908, #pkts encrypt: 908, #pkts digest: 908
     #pkts decaps: 920, #pkts decrypt: 920, #pkts verify: 920
     #pkts compressed: 0, #pkts decompressed: 0
     #pkts not compressed: 0, #pkts compr. failed: 0
     #pkts not decompressed: 0, #pkts decompress failed: 0
     #send errors 0, #recv errors 0
      local crypto endpt.: 10.0.255.10, remote crypto endpt.: 10.0.255.13
      plaintext mtu 1426, path mtu 1472, ip mtu 1472, ip mtu idb Tunnel0
      current outbound spi: 0xC6D242EC(3335668460)
      PFS (Y/N): Y, DH group: group2
      inbound esp sas:
       spi: 0xA78E6474(2811126900)
         transform: esp-256-aes esp-sha-hmac ,
         in use settings ={Transport, }
         conn id: 2151, flow_id: CSR:151, sibling_flags FFFFFFFF80000008, crypto map: Tunnel0-head-0
         sa timing: remaining key lifetime (k/sec): (4607995/1272)
         IV size: 16 bytes
         replay detection support: Y
         Status: ACTIVE(ACTIVE)
       spi: 0x67988C26(1738050598)
         transform: esp-256-aes esp-sha-hmac ,
         in use settings ={Transport, }
         conn id: 2159, flow_id: CSR:159, sibling_flags FFFFFFFF80000008, crypto map: Tunnel0-head-0
         sa timing: remaining key lifetime (k/sec): (4607914/1586)
         IV size: 16 bytes
         replay detection support: Y
         Status: ACTIVE(ACTIVE)
      inbound ah sas:
      inbound pcp sas:
      outbound esp sas:
       spi: 0xCF57FB8D(3478649741)
         transform: esp-256-aes esp-sha-hmac ,
         in use settings ={Transport, }
         conn id: 2152, flow_id: CSR:152, sibling_flags FFFFFFFF80000008, crypto map: Tunnel0-head-0
         sa timing: remaining key lifetime (k/sec): (4607997/1272)
         IV size: 16 bytes
         replay detection support: Y
         Status: ACTIVE(ACTIVE)
       spi: 0xC6D242EC(3335668460)
         transform: esp-256-aes esp-sha-hmac ,
         in use settings ={Transport, }
         conn id: 2160, flow_id: CSR:160, sibling_flags FFFFFFFF80000008, crypto map: Tunnel0-head-0
         sa timing: remaining key lifetime (k/sec): (4607955/1586)
         IV size: 16 bytes
         replay detection support: Y
         Status: ACTIVE(ACTIVE)
      outbound ah sas:
      outbound pcp sas:
 
    protected vrf: (none)
    local  ident (addr/mask/prot/port): (10.0.255.10/255.255.255.255/47/0)
    remote ident (addr/mask/prot/port): (10.0.255.12/255.255.255.255/47/0)
    current_peer 10.0.255.12 port 500
      PERMIT, flags={origin_is_acl,}
     #pkts encaps: 746, #pkts encrypt: 746, #pkts digest: 746
     #pkts decaps: 737, #pkts decrypt: 737, #pkts verify: 737
     #pkts compressed: 0, #pkts decompressed: 0
     #pkts not compressed: 0, #pkts compr. failed: 0
     #pkts not decompressed: 0, #pkts decompress failed: 0
     #send errors 0, #recv errors 0
      local crypto endpt.: 10.0.255.10, remote crypto endpt.: 10.0.255.12
      plaintext mtu 1426, path mtu 1472, ip mtu 1472, ip mtu idb Tunnel0
      current outbound spi: 0xC520AF45(3307253573)
      PFS (Y/N): Y, DH group: group2
      inbound esp sas:
       spi: 0x8EC131FD(2395025917)
         transform: esp-256-aes esp-sha-hmac ,
         in use settings ={Transport, }
         conn id: 2153, flow_id: CSR:153, sibling_flags FFFFFFFF80000008, crypto map: Tunnel0-head-0
         sa timing: remaining key lifetime (k/sec): (4607995/1272)
         IV size: 16 bytes
         replay detection support: Y
         Status: ACTIVE(ACTIVE)
       spi: 0x9A2CEF62(2586636130)
         transform: esp-256-aes esp-sha-hmac ,
         in use settings ={Transport, }
         conn id: 2167, flow_id: CSR:167, sibling_flags FFFFFFFF80000008, crypto map: Tunnel0-head-0
         sa timing: remaining key lifetime (k/sec): (4607937/2126)
         IV size: 16 bytes
         replay detection support: Y
         Status: ACTIVE(ACTIVE)
      inbound ah sas:
      inbound pcp sas:
      outbound esp sas:
       spi: 0xC90E7A74(3373169268)
         transform: esp-256-aes esp-sha-hmac ,
         in use settings ={Transport, }
         conn id: 2154, flow_id: CSR:154, sibling_flags FFFFFFFF80000008, crypto map: Tunnel0-head-0
         sa timing: remaining key lifetime (k/sec): (4607997/1272)
         IV size: 16 bytes
         replay detection support: Y
         Status: ACTIVE(ACTIVE)
       spi: 0xC520AF45(3307253573)
         transform: esp-256-aes esp-sha-hmac ,
         in use settings ={Transport, }
         conn id: 2168, flow_id: CSR:168, sibling_flags FFFFFFFF80000008, crypto map: Tunnel0-head-0
         sa timing: remaining key lifetime (k/sec): (4607966/2126)
         IV size: 16 bytes
         replay detection support: Y
         Status: ACTIVE(ACTIVE)
      outbound ah sas:
      outbound pcp sas:

csr1000v#sh ip bgp sum          
  BGP router identifier 10.0.0.10, local AS number 65000
  BGP table version is 14, main routing table version 14
  3 network entries using 744 bytes of memory
  3 path entries using 360 bytes of memory
  1/1 BGP path/bestpath attribute entries using 248 bytes of memory
  0 BGP route-map cache entries using 0 bytes of memory
  0 BGP filter-list cache entries using 0 bytes of memory
  BGP using 1352 total bytes of memory
  BGP activity 8/5 prefixes, 8/5 paths, scan interval 60 secs
  Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
  10.0.0.11       4        65000     326     321       14    0    0 00:26:53        1
  10.0.0.12       4        65000     306     302       14    0    0 00:25:14        1
  10.0.0.13       4        65000     415     406       14    0    0 00:34:15        1

csr1000v#sh ip bgp
  BGP table version is 14, local router ID is 10.0.0.10
  Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
                r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
                x best-external, a additional-path, c RIB-compressed,
  Origin codes: i - IGP, e - EGP, ? - incomplete
  RPKI validation codes: V valid, I invalid, N Not found
       Network          Next Hop            Metric LocPrf Weight Path
  *>i 10.0.1.0/24      10.0.0.11                1    100      0 i
  *>i 10.0.2.0/24      10.0.0.12                1    100      0 i
  *>i 10.0.3.0/24      10.0.0.13                1    100      0 i

And here it is from the Spoke’s perspective:

VyOS-Spoke1:~$ sh nhrp tunnel
  Status: ok
  
  Interface: tun0
  Type: local
  Protocol-Address: 10.0.0.255/32
  Alias-Address: 10.0.0.11
  Flags: up
  
  Interface: tun0
  Type: local
  Protocol-Address: 10.0.0.11/32
  Flags: up
  
  Interface: tun0
  Type: cached
  Protocol-Address: 10.0.0.13/32
  NBMA-Address: 10.0.255.13
  Flags: used up
  Expires-In: 119:54
  
  Interface: tun0
  Type: cached
  Protocol-Address: 10.0.0.12/32
  NBMA-Address: 10.0.255.12
  Flags: used up
  Expires-In: 119:51
  
  Interface: tun0
  Type: static
  Protocol-Address: 10.0.0.10/24
  NBMA-Address: 10.0.255.10
  Flags: used up

VyOS-Spoke1:~$ show vpn ipsec sa
  Peer ID / IP                            Local ID / IP
  ------------                            -------------
  10.0.255.12                             10.0.255.11                           
    Tunnel  State  Bytes Out/In   Encrypt  Hash  NAT-T  A-Time  L-Time  Proto
    ------  -----  -------------  -------  ----  -----  ------  ------  -----
    tun0    up     92.0/92.0      aes256   sha1  no     283     3600    gre
  
  Peer ID / IP                            Local ID / IP
  ------------                            -------------
  10.0.255.10                             10.0.255.11                           
      Tunnel  State  Bytes Out/In   Encrypt  Hash  NAT-T  A-Time  L-Time  Proto
      ------  -----  -------------  -------  ----  -----  ------  ------  -----
      tun0    up     49.2K/48.3K    aes256   sha1  no     2866    3600    gre
  
  Peer ID / IP                            Local ID / IP
  ------------                            -------------
  0.0.0.0                                 10.0.255.11                           
      Tunnel  State  Bytes Out/In   Encrypt  Hash  NAT-T  A-Time  L-Time  Proto
      ------  -----  -------------  -------  ----  -----  ------  ------  -----
      tun0    down   n/a            n/a      n/a   no     0       1800    gre
  
  Peer ID / IP                            Local ID / IP
  ------------                            -------------
  10.0.255.13                             10.0.255.11                           
      Tunnel  State  Bytes Out/In   Encrypt  Hash  NAT-T  A-Time  L-Time  Proto
      ------  -----  -------------  -------  ----  -----  ------  ------  -----
      tun0    up     92.0/92.0      aes256   sha1  no     280     3600    gre

VyOS-Spoke1:~$ sh ip bgp sum
  BGP router identifier 10.0.0.11, local AS number 65000
  IPv4 Unicast - max multipaths: ebgp 1 ibgp 1
  RIB entries 5, using 480 bytes of memory
  Peers 1, using 4560 bytes of memory
  Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
  10.0.0.10       4 65000     425     433        0    0    0 00:35:49        2
  Total number of neighbors 1

VyOS-Spoke1:~$ sh ip bgp
  BGP table version is 0, local router ID is 10.0.0.11
  Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
                r RIB-failure, S Stale, R Removed
  Origin codes: i - IGP, e - EGP, ? - incomplete
     Network          Next Hop            Metric LocPrf Weight Path
  *> 10.0.1.0/24      0.0.0.0                  1         32768 i
  *>i10.0.2.0/24      10.0.0.12                1    100      0 i
  *>i10.0.3.0/24      10.0.0.13                1    100      0 i
  Total number of prefixes 3
Share this post:
Facebooktwittergoogle_plusredditpinterestlinkedintumblrmail
war Written by:

3 Comments

  1. March 11, 2017
    Reply

    Interested in discussing your VyOS experience….

    • March 11, 2017
      Reply

      Well stand by… I’m about to post the second half of DMVPN interoperability. This time, Cisco Spoke, and a VyOS Hub.

Leave a Reply

Your email address will not be published. Required fields are marked *