Cisco Nexus 9000 – POAP

In this article, we will discuss about POAP to provision multiple switches.

We need a DHCP, TFTP and SCP server. We can also use an HTTP server to deliver the software and the configuration.

POAP Infrastructure:

POAP Process:

Software used:

  • ISC-DHCP-SERVER – version 4.3.1
  • ATFTPD – version 0.7
  • OPENSSH-server 6.7p1

DHCP configuration example :

Subnet used :

In the following block, I reserve a baud for the Client XXXXXXXX. XXXX is the serial number of the switch.

In the option dhcp-client-identifier you need to add « \000 » before the serial number.

We have to assign the following parameter:

  • IP address
  • Default Gateway
  • IP address TFTP Server
  • Filename
  • DNS server

In the file: /etc/dhcp/dhcpd.conf

option domain-name-servers;

subnet netmask {

host switch1 {
 option dhcp-client-identifier "\000XXXXXXXXXX";
 option routers;
 option bootfile-name "/nxos/";
 option tftp-server-name "";


TFPT server:

I kept the default configuration, in the file /etc/default/atftpd

 OPTIONS="--tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr --mcast-ttl 1 --maxthread 100 --verbose=5 /srv/tftp"

In the directory /srv/tftp, I downloaded the file on github. (

This script is provided by Cisco. In this file, you need to customize one part. In the following part you enter the information for:

  • The target image
  • Directory for the image and configuration
  • Method to download the image and configuration here scp
  • The credential of the SCP Server
  • The name of the configuration file (here based on the serial number)
# system and kickstart images, configuration: location on server (src) and target (dst)
 n9k_image_version       = "7.0.3.I5.2" # this must match your code version
 image_dir_src           = "/srv/tftp/nxos"  # Sample - /Users/bob/poap
 ftp_image_dir_src_root  = image_dir_src
 tftp_image_dir_src_root = image_dir_src
 n9k_system_image_src    = "nxos.%s.bin" % n9k_image_version
 config_file_src         = "/srv/tftp/nxos/conf" # Sample - /Users/bob/poap/conf
 image_dir_dst           = "bootflash:" # directory where n9k image will be stored
 system_image_dst        = n9k_system_image_src
 config_file_dst         = "volatile:poap.cfg"
 md5sum_ext_src          = "md5"
 # Required space on /bootflash (for config and system images)
 required_space          = 800000

# copy protocol to download images and config
 # options are: scp/http/tftp/ftp/sftp
 protocol                = "scp" # protocol to use to download images/config

# Host name and user credentials
 username                = "root" # server account
 ftp_username            = "anonymous" # server account
 password                = "password" # password
 hostname                = "" # ip address of ftp/scp/http/sftp server
 config_file_type        = "serial_number"

After you need to generate a md5 of the script. The following line will replace the second line with the MD5. If the MD5 is not valided the POAP process will fail and restart. ; cat $f | sed '/^#md5sum/d' > $f.md5 ; sed -i "s/^#md5sum=.*/#md5sum=\"$(md5sum $f.md5 | sed 's/ .*//')\"/" $f
#!/bin/env python
# Still needs to be implemented.
# Return Values:

You also need to generate an md5 for the image:

md5sum nxos.7.0.3.I5.2.bin > nxos.7.0.3.I5.2.bin.md5

Don’t forget your configuration file name « conf.XXXXXXX » where XXXX is the serial number and to configure the credential in this file.



Cisco Nexus 9000 – Erase your configuration

To erase the startup-configuration, you need to enter the following command:

Switch# write erase 
Warning: This command will erase the startup-configuration.
Do you wish to proceed anyway? (y/n)  [n] y
Leaf3# reload 
This command will reboot the system. (y/n)?  [n] y

Cisco Nexus 9K – Recovery password

To recover the password on Cisco Nexus 9000, you need to restart the switch.

During the boot process, you need to escape with Ctrl+C

Version 2.16.1240. Copyright (C) 2013 American Megatrends, Inc. 
Board type 2
IOFPGA @ 0xc8000000
SLOT_ID @ 0xf
Aborting config file read and autoboot 
No autoboot or failed autoboot. falling to loader 

 Loader Version 7.34

loader > help 
? Print the command list
boot Boot image
bootmode Display/Change current boot mode
dir List file contents on a device
eobc Booting image from active sup via EOBC channel
help Print the command list or the specific command usage
ip Setting IP address or gateway address
reboot Reboot the system
serial Serial console setting
set Set network configuration
show Show loader configuration

Enter in recovery mode with the following command cmdline recoverymode=1 and boot the image.

loader > cmdline recoverymode=1 

loader > dir 



loader > boot nxos.7.0.3.I5.2.bin 
Booting nxos.7.0.3.I5.2.bin 
Trying diskboot 
 Filesystem type is ext2fs, partition type 0x83
Image valid

Image Signature verification was Successful.

Boot Time: 4/10/2017 8:33:0
Installing klm_card_index
INIT: version 2.88 booting
Installing ata_piix module ... done.
Unsquashing rootfs ...
Installing isan procfs ... done.
Installing SSE module with card index 21025 ... done.
Creating SSE device node 248 ... done.
Loading I2C driver ... done.
Installing CCTRL driver for card_type 19 without NEED_GEM ... done.
Loading IGB driver ... done.

In configuration mode, change the admin password and load the image.

A copy of each such license is available at and and and
switch(boot)# config ter
Enter configuration commands, one per line. End with CNTL/Z.
switch(boot)(config)# admin-password ?
 <WORD> Password for user admin (Max Size - 64)
switch(boot)(config)# admin-password C1sco1234
WARNING! Enabling local authentication for login through console due to password recovery
switch(boot)(config)# exit

switch(boot)# load-nxos 
Unsquashing rootfs ...
Creating /dev/mcelog
Starting mcelog daemon
Overwriting dme stub lib
INIT: Switching to runlevel: 3
INIT: Sending processes the TERM signal
File /etc/shared/localtime exists.
INIT: (boot)# 
Running S93thirdparty-script...

Populating conf files for hybrid sysmgr ...
Starting hybrid sysmgr ...

Your device is UP with the new password.

Cisco DHCP with client-identifier 27 bytes

How configure the good value for a DHCP reservation with a client-identifier 27 bytes?

R1 will be the dhcp server with a DHCP POOL SERVER3. The client MAC Address will be : aacf.a2e3.aaff

Configuration ont the client :

 interface Ethernet0/0
 mac-address aacf.a2e3.aaff
 ip address dhcp

The problem here is to find the good value for the client identifier with 27 bytes (vendor-xxxx.xxxx.xxxx-Interface)

The first possibility is to find on the Internet a convertor Hex to ASCII.

The other one is to use the debug information on the client to find the good value with debug dhcp detail command.

Now we will shutting down the interface and no shut to generate a DHCP negotiation.

Now we see the good value here :

Retry count: 1 Client-ID: cisco-aacf.a2e3.aaff-Et0/0
 Client-ID hex dump: 636973636F2D616163662E613265332E

The request is the following in ASCII : Client-ID: cisco-aacf.a2e3.aaff-Et0/0

In Hexadecimal : 636973636F2D616163662E613265332E616166662D4574302F30

Now you just need to configure the DHCP pool on the server and add « 00 » to the Hexadecimal value like this :

ip dhcp pool SERVER3
 client-identifier 00636973636F2D616163662E613265332E616166662D4574302F30

Now the client can receive the IP address :

*Jul 23 17:44:36.638: DHCP: SRequest attempt # 1 for entry:
*Jul 23 17:44:36.638: Temp IP addr: for peer on Interface: Ethernet0/0
*Jul 23 17:44:36.638: Temp sub net mask:
*Jul 23 17:44:36.638: DHCP Lease server:, state: 4 Requesting
*Jul 23 17:44:36.638: DHCP transaction id: B43
*Jul 23 17:44:36.638: Lease: 86400 secs, Renewal: 0 secs, Rebind: 0 secs
*Jul 23 17:44:36.638: Next timer fires after: 00:00:03
*Jul 23 17:44:36.638: Retry count: 1 Client-ID: cisco-aacf.a2e3.aaff-Et0/0
*Jul 23 17:44:36.638: Client-ID hex dump: 636973636F2D616163662E613265332E
*Jul 23 17:44:36.639: 616166662D4574302F30
*Jul 23 17:44:39.657: DHCP: Releasing ipl options:
*Jul 23 17:44:39.657: DHCP: Applying DHCP options:
*Jul 23 17:44:39.657: DHCP: Sending notification of ASSIGNMENT:
*Jul 23 17:44:39.657: Address mask
*Jul 23 17:44:39.657: DHCP Client Pooling: ***Allocated IP address:
*Jul 23 17:44:39.730: Allocated IP address =
Client(config-if)#do sh ip int brief
 Interface IP-Address OK? Method Status Protocol
 Ethernet0/0 YES DHCP up up

Convert LWAPP to Autonomous AP


Password: <= Cisco

AP4403.xxxx.xxxx#sh ver

Cisco IOS Software, C2600 Software (AP3G2-RCVK9W8-M), Version 15.2(2)JA, RELEASE SOFTWARE (fc1)Technical Support: (c) 1986-2012 by Cisco Systems, Inc.Compiled Thu 23-Aug-12 02:43 by prod_rel_team
ROM: Bootstrap program is C2600 boot loaderBOOTLDR: C2600 Boot Loader (AP3G2-BOOT-M)
LoaderVersion 12.4(25e)JA1, RELEASE SOFTWARE (fc1)
AP4403.a7a0.db3e uptime is 4 minutesSystem returned to ROM by power-onSystem image file is « flash:/ap3g2-rcvk9w8-mx/ap3g2-rcvk9w8-xx »

P4403.xxxx.xxxx#debug capwap console cli
This command is meant only for debugging/troubleshooting
Any configuration change may result in different
behavior from centralized configuration.

CAPWAP console CLI allow/disallow debugging is on

AP4403.a7a0.db3e(config)#ip defa
AP4403.a7a0.db3e(config)#ip default-g
AP4403.a7a0.db3e(config)#ip default-gateway
AP4403.a7a0.db3e(config)#int gi0
Not in Bound state.
*Mar 1 00:06:53.019: %CAPWAP-3-DHCP_RENEW: Could not discover WLC using DHCP IP. Renewing DHCP IP.nt gi0
AP4403.a7a0.db3e(config-if)#int gi0
AP4403.a7a0.db3e(config-if)#int gi0
*Mar 1 00:06:56.023: %CAPWAP-3-ERRORLOG: Invalid event 38 & state 2 combination.
*Mar 1 00:06:56.091: %DHCP-6-ADDRESS_ASSIGN: Interface BVI1 assigned DHCP address, mask, hostname AP4403.a7a0.db3e

AP4403.a7a0.db3e(config-if)#ip add
AP4403.a7a0.db3e(config-if)#ip address 1
Translating « »…domain server (
*Mar 1 00:07:04.019: %CAPWAP-3-ERRORLOG: Did not get log server settings from DHCP.
AP4403.a7a0.db3e(config-if)#ip address 10.0.10
*Mar 1 00:07:07.019: %CAPWAP-3-ERRORLOG: Could Not resolve
AP4403.a7a0.db3e(config-if)#ip address
% overlaps with BVI1
AP4403.a7a0.db3e(config-if)#no sh
*Mar 1 00:07:28.527: %SYS-5-CONFIG_I: Configured from console by console10.0.100.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to, timeout is 2 seconds:
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

AP4403.a7a0.db3e#archive download-sw /force-reload /overwrite tftp://
examining image…
Loading ap3g2-k9w7-tar.default from (via BVI1): !
extracting info (279 bytes)
Image info:
Version Suffix: k9w7-.153-3.JC
Image Name: ap3g2-k9w7-mx.153-3.JC
Version Directory: ap3g2-k9w7-mx.153-3.JC
Ios Image Size: 10322432
Total Image Size: 13384192
Image Feature: WIRELESS LAN
Image Family: AP3G2
Wireless Switch Management Version:
Extracting files…
ap3g2-k9w7-mx.153-3.JC/ (directory) 0 (bytes)
extracting ap3g2-k9w7-mx.153-3.JC/ap3g2-k9w7-mx.153-3.JC (215867 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/ap3g2-k9w7-tx.153-3.JC (73 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/ap3g2-bl-2600 (190140 bytes)!
extracting ap3g2-k9w7-mx.153-3.JC/ap3g2-bl-3600 (189183 bytes)!
ap3g2-k9w7-mx.153-3.JC/html/ (directory) 0 (bytes)
ap3g2-k9w7-mx.153-3.JC/html/level/ (directory) 0 (bytes)
ap3g2-k9w7-mx.153-3.JC/html/level/1/ (directory) 0 (bytes)

extracting ap3g2-k9w7-mx.153-3.JC/html/level/1/appsui.js (563 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/html/level/1/back.shtml (512 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/html/level/1/cookies.js (5032 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/html/level/1/forms.js (20442 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/HA5.bin (2049 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/B2.bin (10512 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/B5.bin (1995 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/Y2.bin (7008 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/Y5.bin (1555 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/8006.img (568619 bytes)!!!
extracting ap3g2-k9w7-mx.153-3.JC/triggerfish.jed (0 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/uart_firmware_upgrade.bin (18239 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/MCU.bin (8799 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/info (279 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/file_hashes (36832 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/final_hash (141 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/final_hash.sig (513 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/img_sign_rel.cert (1375 bytes)
extracting ap3g2-k9w7-mx.153-3.JC/img_sign_rel_sha2.cert (1371 bytes)
extracting info.ver (279 bytes)
[OK – 13434880 bytes]

Deleting current version: flash:/ap3g2-rcvk9w8-mx…done.
New software image installed in flash:/ap3g2-k9w7-mx.153-3.JC
Writing out the event log to flash:/event.log …
guring system to use new image…done.
Requested system reload in progress…
archive download: takes 220 seconds

Write of event.log done

*Mar 1 00:13:17.647: %SYS-5-RELOAD: Reload requested by Exec. Reload Reason: Reason unspecified.
*Mar 1 00:13:17.647: %LWAPP-5-CHANGED: CAPWAP changed state to DOWN
IOS Bootloader – Starting system.
flash is writable
FLASH CHIP: Numonyx Mirrorbit (0089)
Xmodem file system is available.
flashfs[0]: 237 files, 8 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 31997952
flashfs[0]: Bytes used: 13329408
flashfs[0]: Bytes available: 18668544
flashfs[0]: flashfs fsck took 16 seconds.
Reading cookie from SEEPROM
Base Ethernet MAC address: 44:03:a7:a0:db:3e
Ethernet speed is 1000 Mb – FULL Duplex
Loading « flash:/ap3g2-k9w7-mx.153-3.JC/ap3g2-k9w7-mx.153-3.JC »…#########################

File « flash:/ap3g2-k9w7-mx.153-3.JC/ap3g2-k9w7-mx.153-3.JC » uncompressed and installed, entry point: 0x2003000

Secondary Bootloader – Starting system.
Tide MB – 32MB of flash
Xmodem file system is available.
flashfs[0]: 237 files, 8 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 31997952
flashfs[0]: Bytes used: 13329408
flashfs[0]: Bytes available: 18668544
flashfs[0]: flashfs fsck took 8 seconds.
flashfs[1]: 0 files, 1 directories
flashfs[1]: 0 orphaned files, 0 orphaned directories
flashfs[1]: Total bytes: 12257280
flashfs[1]: Bytes used: 1024
flashfs[1]: Bytes available: 12256256
flashfs[1]: flashfs fsck took 1 seconds.
Base Ethernet MAC address: 44:03:a7:a0:db:3e

From TFTP Server :
May 5 19:53:52 srv1 in.tftpd[5529]: connect from (
May 5 19:53:52 srv1 atftpd[5529]: Advanced Trivial FTP server started (0.7)
May 5 19:53:52 srv1 atftpd[5529]: Serving ap3g2-k9w7-tar.default to
May 5 19:53:52 srv1 atftpd[5529]: Serving ap3g2-k9w7-tar.default to
May 5 19:54:11 srv1 atftpd[5529]: timeout: retrying…
May 5 19:55:08 srv1 atftpd[5529]: timeout: retrying…