Cisco 6500 Cosmetic bugs

Ever had this error before on a Cisco 6500 catalyst?

6500#  sh module
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  1    5  Supervisor Engine 720 10GE (Active)    VS-S720-10G        SAL-------
  2   48  48-port 10/100/1000 RJ45 EtherModule   WS-X6148A-GE-TX    SAL---------
  3   48  CEF720 48 port 1000mb SFP              WS-X6748-SFP       SAL----------

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  1  001d.45e1.ed48 to 001d.45e1.ed4f   2.0   8.5(2)       12.2(33)SXH1 Ok
  2  001f.9ec6.7d70 to 001f.9ec6.7d9f   1.6   8.4(1)       8.7(0.22)BUB Ok
  3  001b.d4ec.ab60 to 001b.d4ec.ab8f   1.12  12.2(14r)S5  12.2(33)SXH1 Ok

Mod  Sub-Module                  Model              Serial       Hw     Status
---- --------------------------- ------------------ ----------- ------- -------
  1  Policy Feature Card 3       VS-F6K-PFC3C       SAL----------  1.0    Ok
  1  MSFC3 Daughterboard         VS-F6K-MSFC3       SAL----------  1.0    Ok
  3  Centralized Forwarding Card WS-F6700-CFC        SAL----------  3.1    Ok

Mod  Online Diag Status
---- -------------------
  1  Minor Error
  2  Pass
  3  Pass

Continue reading “Cisco 6500 Cosmetic bugs”


Cisco Nexus User Roles

IOS relies on privilege levels.  Privilege levels (0-15) defines locally what level of access a user has when logged into an IOS device, i.e. what commands are permitted. This only applies in the absence of AAA being configured. There are 3 default privilege levels on IOS, but really only two that are relevant:

  • Privilege Level 1 — Normal level on Telnet; includes all user-level commands at the router> prompt.
  • Privilege Level 15 — Includes all enable-level commands at the router# prompt.

NX-OS uses a different concept for the same purpose, known as User Roles. User Roles contain rules that define the operations allowed for a particular user assigned to a role. There are default User Roles:

  • Network-Admin—Complete read-and-write access to the entire NX-OS device (only available in the default VDC).
  • Network-Operator—Complete read access to the entire NX-OS device (Default User Role).
  • VDC-Admin—Read-and-write access limited to a VDC (VDCs are not yet available on Nexus 5000).
  • VDC-Operator—Read access limited to a VDC (Default User Role).

A VDC (Virtual Device Context) is a logical separation of control plane hardware resources into virtualized layer3 switches. Don’t worry to much about what a VDC is for now, it is not really relevant to the purpose of this post.

When a NX-OS device is setup for the first time, during the first login, a Network-Admin account must be specified and subsequently be used to login. Arguably a bit more secure that IOS. Any additional users created locally after that will by default receive the User Role “Network-Operator“, unless specified differently:

User Roles are local to a switch and only relevant in the absence of AAA Authorization being configured. To see the permissions of a particular User Role use:

N5K-2# sh role name network-operator
Role: network-operator
  Description: Predefined network operator role has access to all read
  commands on the switch
  Rule    Perm    Type        Scope               Entity
  1       permit  read

Continue reading “Cisco Nexus User Roles”

Nexus’ improved CLI

The Cisco Nexus Series platform has some good things going. Having spent much of my time recently using them, I have come to appreciate some very neat improvements NX-OS is offering over standard IOS. For the most part driving NX-OS is very similar to IOS, but it’s been greatly improved.

One such example is the output from the most used IOS command “show ip int brief”, which on NX-OS only shows ‘IP’ (being layer 3) interfaces. To see the brief state of all types of interfaces use “sh int brief” instead.

N5K-2(config)# sh ip int brief
IP Interface Status for VRF "default"(1)
Interface            IP Address      Interface Status
Vlan19            protocol-up/link-up/admin-up
Vlan22            protocol-up/link-up/admin-up

N5K-2(config)# sh int brief
Ethernet      VLAN   Type Mode   Status  Reason                   Speed     Port
Interface                                                                   Ch #
Eth1/1        1      eth  trunk  up      none                       1000(D) 51
Eth1/2        22     eth  access up      none                        10G(D) -
Eth1/3        1      eth  trunk  down    SFP not inserted            10G(D) 50
Eth1/4        1      eth  trunk  down    SFP not inserted            10G(D) 50
Eth1/5        1      eth  trunk  down    SFP not inserted            10G(D) -
Eth1/6        19     eth  access down    SFP not inserted            10G(D) -
Eth1/7        1      eth  trunk  down    Link not connected          10G(D) 5
Eth1/8        1      eth  trunk  down    Link not connected          10G(D) 5
Eth1/9        1      eth  fabric down    Administratively down       10G(D) 9
Eth1/10       1      eth  fabric down    FEX identity mismatch       10G(D) 7
Eth1/11       1      eth  fabric down    vpc peerlink is down        10G(D) 34
Eth1/12       1      eth  fabric down    SFP not inserted            10G(D) 12
Eth1/13       1      eth  fabric up      none                        10G(D) 15
Eth1/14       1      eth  fabric down    Administratively down       10G(D) 9

Continue reading “Nexus’ improved CLI”

Jumbo MTU on Nexus 5000

Setting a per interface MTU (maximum transmission unit) is not supported on the Nexus 5000/2000 series switches.
If a Jumbo packet is required to traverse a Nexus 5000 series switch , the jumbo MTU must be set in a policy-map and applied to the ‘Sytem QOS’.


Configuration, PRE NX-OS 4.1:
policy-map JUMBO
 class class-default
  mtu 9216
system qos
 service-policy JUMBO

Configuration with POST NX-OS 4.1:
policy-map type network-qos JUMBO
 class type network-qos class-default
  mtu 9216
system qos
 service-policy type network-qos JUMBO

Continue reading “Jumbo MTU on Nexus 5000”


Really sad when you have to reboot a production switch that’s been up for this long. Suppose another question is why was the switch never upgraded? Until now not needed.  :)

bry-asw1>show version
Cisco Internetwork Operating System Software
IOS (tm) C2950 Software (C2950-I6Q4L2-M), Version 12.1(9)EA1, RELEASE SOFTWARE (fc1)
Copyright (c) 1986-2002 by cisco Systems, Inc.
Compiled Wed 24-Apr-02 06:57 by antonino
Image text-base: 0x80010000, data-base: 0x804E8000
ROM: Bootstrap program is CALHOUN boot loader
bry-asw1 uptime is 7 years, 48 weeks, 6 days, 6 hours, 19 minutes
System returned to ROM by power-on
System restarted at 12:00:24 SAST Thu Feb 13 2003
System image file is "flash:/c2950-i6q4l2-mz.121-9.EA1.bin"
cisco WS-C2950G-24-EI (RC32300) processor (revision D0) with 20815K bytes of memory.
Processor board ID FOC0633Y2T5

What is the longest your production devices have been up for?

Nexus defaults to PAP authentication

Ever configured a Nexus switch to use AAA to query a Tacacs+ server? Had some troubles applying standard IOS config to NX-OS?

Possibly if your Tacacs+ server is configured to only allow PAM (Password Authentication Manager) authentication for the users. See when a NX-OS switch sends a AAA authentication packet, by default it is encapsulated using PAP encoding. This is in contrast to normal IOS devices, that use PAM encoding by default.

To illustrate I used the following config:

ip tacacs source-interface mgmt0
tacacs-server host key password
aaa group server tacacs+ TAC
use-vrf management
source-interface mgmt0
aaa authentication login default group TAC
aaa authorization config-commands default group TAC
aaa authorization commands default group TAC
aaa accounting default group TAC

Continue reading “Nexus defaults to PAP authentication”

Troubleshooting random Nexus reboots

November last year, a pair of Cisco Nexus 5010 switches, suddenly started rebooting randomly without user intervention.  Since these boxes were a front to a VM environment, stability were of urgent concern. But in order to stabilize the environment, the root cause of the reboots had to be isolated, and quickly.

The Cisco Nexus platform might not be as mature as many would like, but it is quickly becoming a very needed switch in Next-Generation datacenters. Of the things I like most about the Nexus boxes are the readily available local reporting and intuitive system checks.  Obviously there are many other features which is making the platform so popular. I’ll cover some of these in time.

Coming back to the rebooting issue. Unlike IOS devices that looses all local logging info, unless a crash dump was saved to NVRAM, the Nexus writes most of its log information to disk. Thus even after the reboot, you have all the information.
Continue reading “Troubleshooting random Nexus reboots”