DMVPN Phase 3 – Multicast with source and receiver on spokes

Assumption :

  • PIM open standard (rp-candidate + bsr-candidate), PIM SM
  • RP is connected behind the HUB. (10.15.15.15)
  • The source is connected to the Spoke 1.
  • The receiver is connected to the spoke 2

HUB configuration :

ip multicast-routing
!
interface Tunnel0
 bandwidth 1000
 ip address 10.0.1.1 255.255.255.248
 no ip redirects
 ip mtu 1400
 ip pim nbma-mode
 ip pim sparse-mode
 ip nhrp authentication key
 ip nhrp map multicast dynamic
 ip nhrp network-id 12345
 ip nhrp holdtime 300
 ip nhrp redirect
 ip tcp adjust-mss 1360
 delay 1000
 tunnel source Ethernet0/0
 tunnel mode gre multipoint
 tunnel protection ipsec profile PROFILE

Spoke 1 configuration :

ip multicast-routing
!
ip pim spt-threshold infinity
!
interface Tunnel0
 bandwidth 1000
 ip address 10.0.1.2 255.255.255.248
 no ip redirects
 ip mtu 1400
 ip pim sparse-mode
 ip nhrp authentication key
 ip nhrp map multicast 172.16.0.1
 ip nhrp map 10.0.1.1 172.16.0.1
 ip nhrp network-id 12345
 ip nhrp holdtime 300
 ip nhrp nhs 10.0.1.1 
 ip nhrp shortcut
 ip tcp adjust-mss 1360
 delay 1000
 tunnel source Serial1/0
 tunnel mode gre multipoint
 tunnel protection ipsec profile PROFILE

Spoke 2 configuration :

ip multicast-routing
!
ip pim spt-threshold infinity
!
interface Tunnel0
 bandwidth 1000
 ip address 10.0.1.3 255.255.255.248
 no ip redirects
 ip mtu 1400
 ip pim sparse-mode
 ip nhrp authentication key
 ip nhrp map multicast 172.16.0.1
 ip nhrp map 10.0.1.1 172.16.0.1
 ip nhrp network-id 12345
 ip nhrp holdtime 300
 ip nhrp nhs 10.0.1.1
 ip nhrp shortcut
 ip tcp adjust-mss 1380
 delay 1000
 tunnel source Serial1/0
 tunnel mode gre multipoint
 tunnel protection ipsec profile PROFILE

Tshoot

IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
 L - Local, P - Pruned, R - RP-bit set, F - Register flag,
 T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
 X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
 U - URD, I - Received Source Specific Host Report,
 Z - Multicast Tunnel, z - MDT-data group sender,
 Y - Joined MDT-data group, y - Sending to MDT-data group,
 G - Received BGP C-Mroute, g - Sent BGP C-Mroute,
 Q - Received BGP S-A Route, q - Sent BGP S-A Route,
 V - RD & Vector, v - Vector
Outgoing interface flags: H - Hardware switched, A - Assert winner
 Timers: Uptime/Expires
 Interface state: Interface, Next-Hop or VCD, State/Mode

RP on HUB and Spokes routers

#sh ip pim rp
Group: 232.1.1.1, RP: 10.15.15.15, v2, uptime 00:13:52, expires 00:01:46
! Source multicast with real IP address 10.1.18.1

SPOKE1#sh ip mroute 232.1.1.1 

(*, 232.1.1.1), 00:02:40/stopped, RP 10.15.15.15, flags: SPF
 Incoming interface: Tunnel0, RPF nbr 10.0.1.1
 Outgoing interface list: Null

(10.1.18.1, 232.1.1.1), 00:02:40/00:03:23, flags: FT
 Incoming interface: Ethernet0/0, RPF nbr 0.0.0.0, Registering
 Outgoing interface list:
 Tunnel0, Forward/Sparse, 00:02:40/00:02:47, A
! Receiver multicast for the group 232.1.1.1

interface Ethernet0/0
 ip address 10.1.19.1 255.255.255.0
 ip pim sparse-mode
 ip igmp join-group 232.1.1.1

SPOKE2#sh ip mroute 232.1.1.1

(*, 232.1.1.1), 00:09:21/00:02:42, RP 10.15.15.15, flags: SCL
 Incoming interface: Tunnel0, RPF nbr 10.0.1.1
 Outgoing interface list:
 Ethernet0/0, Forward/Sparse, 00:09:19/00:02:42
HUB#sh ip mroute 232.1.1.1


(*, 232.1.1.1), 00:09:13/00:03:08, RP 10.15.15.15, flags: S
 Incoming interface: Ethernet0/1, RPF nbr 10.20.1.17
 Outgoing interface list:
 Tunnel0, 10.0.1.3, Forward/Sparse, 00:09:13/00:03:08

(10.1.18.1, 232.1.1.1), 00:06:17/00:01:35, flags: T
 Incoming interface: Tunnel0, RPF nbr 10.0.1.3
 Outgoing interface list:
 Tunnel0, 10.0.1.3, Forward/Sparse, 00:06:17/00:03:08
SPOKE2#mtrace 10.1.19.1 232.1.1.1
Type escape sequence to abort.
Mtrace from 10.1.19.1 to 10.1.19.1 via group 232.1.1.1
From source (?) to destination (?)
Querying full reverse path...
 0  10.1.19.1
-1  10.1.19.1 ==> 10.0.1.3 PIM  [using shared tree]       << Interface Tu0
-2 10.0.1.1 ==> 10.20.1.18 PIM  [using shared tree]       << Interface Tu0 Hub to next router 
-3 10.20.1.17 ==> 10.20.1.2 PIM  [using shared tree]      << Next router to RP router 
-4 10.20.1.1 ==> 0.0.0.0 PIM_MT Reached RP/Core [using shared tree] << RP router

DMVPN with IPSEC

Case 1 without VRF

crypto isakmp policy 10
 encr aes
 authentication pre-share
 group 2
crypto isakmp key MyKey address 0.0.0.0
!
!
crypto ipsec transform-set TRANS esp-aes
 mode transport
!
crypto ipsec profile PROF_DMVPN
 set transform-set TRANS

interface Tunnel0
 <...>
 tunnel source e0/0
 tunnel mode gre multipoint
 tunnel protection ipsec profile PROF_DMVPN

Case 2 with VRF

crypto keyring CCIE vrf VRF1
 pre-shared-key address 0.0.0.0 0.0.0.0 key MyKey
!
!
!
!
crypto isakmp policy 10
 encr aes
 authentication pre-share
 group 2
!
!
crypto ipsec transform-set TRANS esp-aes
 mode transport
!
crypto ipsec profile PROF_DMVPN
 set transform-set TRANS

interface Tunnel0
 <..>
 tunnel source e0/0
 tunnel mode gre multipoint
 tunnel vrf VRF1
 tunnel protection ipsec profile PROF_DMVPN

 

DMVPN Phase 3 with OSPF

Diagram

Hub (R1)

HUB#
interface Ethernet0/0
ip address 10.0.1.2 255.255.255.0
!
ip route 0.0.0.0 0.0.0.0 10.0.1.1
!
interface Tunnel0
ip address 100.0.0.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp map multicast dynamic
ip nhrp network-id 1234
ip nhrp holdtime 360
ip nhrp redirect
ip tcp adjust-mss 1360
ip ospf network point-to-multipoint
ip ospf 1 area 0
tunnel source Ethernet0/0
tunnel mode gre multipoint

Spoke (R2, R3)

SPOKE1#
 interface Ethernet0/0
 ip address 10.0.2.2 255.255.255.0
 !
 ip route 0.0.0.0 0.0.0.0 10.0.2.1
 !
 interface Tunnel0
 ip address 100.0.0.2 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp map 100.0.0.1 10.0.1.2
 ip nhrp map multicast 10.0.1.2
 ip nhrp network-id 1234
 ip nhrp holdtime 360
 ip nhrp nhs 100.0.0.1
 ip nhrp shortcut
 ip tcp adjust-mss 1360
 ip ospf network point-to-multipoint
 ip ospf 1 area 0
 tunnel source Ethernet0/0
 tunnel mode gre multipoint

SPOKE2#
 interface Ethernet0/0
 ip address 10.0.3.2 255.255.255.0
 !
 ip route 0.0.0.0 0.0.0.0 10.0.3.1

!
 interface Tunnel0
 ip address 100.0.0.3 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip nhrp map 100.0.0.1 10.0.1.2
 ip nhrp map multicast 10.0.1.2
 ip nhrp network-id 1234
 ip nhrp holdtime 360
 ip nhrp nhs 100.0.0.1
 ip nhrp shortcut
 ip tcp adjust-mss 1360
 ip ospf network point-to-multipoint
 ip ospf 1 area 0
 tunnel source Ethernet0/0
 tunnel mode gre multipoint

Troubleshooting

HUB#sh dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
 N - NATed, L - Local, X - No Socket
 # Ent --> Number of NHRP entries with same NBMA peer
 NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
 UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel0, IPv4 NHRP Details
Type:Hub, NHRP Peers:2,

# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
 1 10.0.2.2 100.0.0.2 UP 01:15:52 D
 1 10.0.3.2 100.0.0.3 UP 01:15:09 D

Before flow between spokes:

SPOKE1#sh dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
 N - NATed, L - Local, X - No Socket
 # Ent --> Number of NHRP entries with same NBMA peer
 NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
 UpDn Time --> Up or Down Time for a Tunnel
==========================================================================

Interface: Tunnel0, IPv4 NHRP Details
Type:Spoke, NHRP Peers:1,

# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
 ----- --------------- --------------- ----- -------- -----
 1 10.0.1.2 100.0.0.1 UP 01:16:18 S
 

 

SPOKE1#ping 100.0.0.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 100.0.0.3, timeout is 2 seconds:
!!!!!

After a dynamic tunnel to the other spoke is dynamically enabled

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
SPOKE1#sh dmvpn
Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete
N - NATed, L - Local, X - No Socket
# Ent --> Number of NHRP entries with same NBMA peer
NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting
UpDn Time --> Up or Down Time for a Tunnel
==========================================================================
Interface: Tunnel0, IPv4 NHRP Details
Type:Spoke, NHRP Peers:2,

# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb
----- --------------- --------------- ----- -------- -----
1 10.0.1.2 100.0.0.1 UP 01:16:31 S
1 10.0.3.2 100.0.0.3 UP 00:00:01 D

Cisco Nexus 9000 – POAP

In this article, we will discuss about POAP to provision multiple switches.

We need a DHCP, TFTP and SCP server. We can also use an HTTP server to deliver the software and the configuration.

POAP Infrastructure:

http://www.cisco.com/c/dam/en/us/td/i/300001-400000/330001-340000/331001-332000/331649.eps/_jcr_content/renditions/331649.jpg

POAP Process:

http://www.cisco.com/c/dam/en/us/td/i/300001-400000/330001-340000/332001-333000/332315.eps/_jcr_content/renditions/332315.jpg

Software used:

  • ISC-DHCP-SERVER – version 4.3.1
  • ATFTPD – version 0.7
  • OPENSSH-server 6.7p1

DHCP configuration example :

Subnet used : 192.168.255.0/24

In the following block, I reserve a baud for the Client XXXXXXXX. XXXX is the serial number of the switch.

In the option dhcp-client-identifier you need to add « \000 » before the serial number.

We have to assign the following parameter:

  • IP address
  • Default Gateway
  • IP address TFTP Server
  • Filename
  • DNS server

In the file: /etc/dhcp/dhcpd.conf

option domain-name-servers 192.168.255.254;

subnet 192.168.255.0 netmask 255.255.255.0 {

host switch1 {
 option dhcp-client-identifier "\000XXXXXXXXXX";
 fixed-address 192.168.255.1;
 option routers 192.168.255.254;
 option bootfile-name "/nxos/poap.py";
 option tftp-server-name "192.168.255.200";
 }

}

TFPT server:

I kept the default configuration, in the file /etc/default/atftpd

USE_INETD=true
 OPTIONS="--tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr 239.239.239.0-255 --mcast-ttl 1 --maxthread 100 --verbose=5 /srv/tftp"

In the directory /srv/tftp, I downloaded the poap.py file on github. (https://github.com/datacenter/nexus9000/blob/master/nx-os/poap/poap.py)

This script is provided by Cisco. In this file, you need to customize one part. In the following part you enter the information for:

  • The target image
  • Directory for the image and configuration
  • Method to download the image and configuration here scp
  • The credential of the SCP Server
  • The name of the configuration file (here based on the serial number)
# system and kickstart images, configuration: location on server (src) and target (dst)
 n9k_image_version       = "7.0.3.I5.2" # this must match your code version
 image_dir_src           = "/srv/tftp/nxos"  # Sample - /Users/bob/poap
 ftp_image_dir_src_root  = image_dir_src
 tftp_image_dir_src_root = image_dir_src
 n9k_system_image_src    = "nxos.%s.bin" % n9k_image_version
 config_file_src         = "/srv/tftp/nxos/conf" # Sample - /Users/bob/poap/conf
 image_dir_dst           = "bootflash:" # directory where n9k image will be stored
 system_image_dst        = n9k_system_image_src
 config_file_dst         = "volatile:poap.cfg"
 md5sum_ext_src          = "md5"
 # Required space on /bootflash (for config and system images)
 required_space          = 800000

# copy protocol to download images and config
 # options are: scp/http/tftp/ftp/sftp
 protocol                = "scp" # protocol to use to download images/config

# Host name and user credentials
 username                = "root" # server account
 ftp_username            = "anonymous" # server account
 password                = "password" # password
 hostname                = "192.168.255.200" # ip address of ftp/scp/http/sftp server
 config_file_type        = "serial_number"

After you need to generate a md5 of the poap.py script. The following line will replace the second line with the MD5. If the MD5 is not valided the POAP process will fail and restart.

f=poap.py ; cat $f | sed '/^#md5sum/d' > $f.md5 ; sed -i "s/^#md5sum=.*/#md5sum=\"$(md5sum $f.md5 | sed 's/ .*//')\"/" $f
#!/bin/env python
#md5sum="3b614973cbde2742388b5997228678cd"
# Still needs to be implemented.
# Return Values:

You also need to generate an md5 for the image:

md5sum nxos.7.0.3.I5.2.bin > nxos.7.0.3.I5.2.bin.md5

Don’t forget your configuration file name « conf.XXXXXXX » where XXXX is the serial number and to configure the credential in this file.