My CCIE Booklist

A quick post. I’ve had many requests from guys asking details surrounding my studies and preparation. As always I am more than happy to help and aid other candidates where I can. After all I did not get this far on my own.

So first off I have create a new page called CCIE BOOKLIST (on the right) of books I bought and used for both the R&S and SP. I have added a small review of most of them.

In the next week or two, I will post the methods I used to get through the theory, labs, my approach and lab strategies etc.


Terminal Server in Dynamips

I find using a terminal server to connect to routers while labbing very efficient. I personally don’t like having 10 windows open when configuring devices. I tried it back when I started studying for my R&S but found I made more errors than worth. Since then I have gotten used to jumping between terminal sessions on one screen.

Like most I used Dynamips when I studied for the SP. I built a quad-core PC at home with Ubuntu. My laptop at the time was running Windows XP, but during my 4 months trial I got a Mac Book Pro. Obviously I had to study whenever I had time regardless of the platform. So I configured the same setup across all three platforms.

Configuring a terminal server in Dynamips requires a real interface to be bridged to a virtual router interface. This is done by using a loopback interface. This is done very differently on the three  platforms:

  • Windows XP (32-bit)
  • Ubuntu 9.10 (64-bit)
  • Snow Leopard 10.6 (32/64-bit)

The .NET file I used for the Internetwork Expert SP labs are at the bottom of the article.

Continue reading “Terminal Server in Dynamips”

VRF-lite route leaking

The purpose of VRF-lite is to extend the logical separation of two different networks from a MPLS network down to a single CE router, connected to both these networks. It’s called VRF-lite because it is done without running MPLS (LDP/TDP) or MP-BGP between the PE and CE. Traffic is mapped to the VRF assigned to the ingress interface on the CE router.

But VRF-lite could be used without connecting to a MPLS network entirely! Consider what a VRF is?

A VRF is a mechanism used to provide logical separation between routing tables on the same router. It is locally significant to the router. Each interface on a router can only be assigned to one VRF, but a VRF can have multiple interfaces.

So VRF-lite could be used to separate multiple networks using the same equipment. (Not exactly something you should ever plan in a design, but it could be useful to know)

Once you have the separation you needed, you might need a way to selectively bridge that separation to allow communication between the VRF’s.

Assume the following scenario:
Continue reading “VRF-lite route leaking”

Output101- sh run vrf

Now that the hard work is behind me, the awesome holiday has past, I can finally get back to all the outstanding fun stuff. That said I have some good half completed posts are on the way :)

I came across the following command browsing the DOC-CD a couple months back, and I have used it ever since.

sh run vrf [vrf-name]

The show running vrf feature provides the option to display a subset of the running configuration on a router that is linked to a VRF instance. It can be used to display the configuration of a specific VRF or of all VRFs configured on a router. The command is unfortunately only available on the more recent IOS versions, but if available makes life easy.

Continue reading “Output101- sh run vrf”

Cisco CCIE SP – 1st Time PASS

I got my results this morning. I passed the CCIE Service-Provider lab exam I took in RTP, first attempt on the dreaded Friday the 13th of August! (I’m not superstitious, that stuff is for the special kind).

I’ve had overwhelming support from family and friends. HUGE THANKS!!

The last 4 months was without a doubt the most difficult time I have ever experienced. Initially the time frame looked good, and my schedule was nicely packed with some study leave. But then life happens. Changing jobs in July (no more study leave), a very demanding 3rd baby, my boys getting sick a couple times, and all this while my wife was working and running the house.

My wife has been unbelievable, juggling work and house stuff  in my absence. Without her support this would not have been possible. Okay I’m not getting soppy. There are many guys in similar positions that needs encouragement.

Continue reading “Cisco CCIE SP – 1st Time PASS”

Memory problems

From a question on groupstudy; the following output was posted of a Cisco 2600 that did not boot up even after the Flash was upgraded to 32MB??????

System Bootstrap, Version 12.1(3r)T2, RELEASE SOFTWARE (fc1)
 Copyright (c) 2000 by cisco Systems, Inc.
 C2600 platform with 65536 Kbytes of main memory

 program load complete, entry point: 0x80008000, size: 0x1c8a8e4

 Error : memory requirements exceed available memory
 Memory required     : 0x04746F90

 *** System received a Software forced crash ***
 signal= 0x17, code= 0x4, context= 0x80080630
 PC = 0x0, Vector = 0x0, SP = 0x0

 System Bootstrap, Version 12.1(3r)T2, RELEASE SOFTWARE (fc1)
 Copyright (c) 2000 by cisco Systems, Inc.
 C2600 platform with 65536 Kbytes of main memory

 program load complete, entry point: 0x80008000, size: 0x1c8a8e4

 Error : memory requirements exceed available memory
 Memory required     : 0x04746F90

 rommon 1 > dir flash:
 File size           Checksum   File name
 29928068 bytes (0x1c8aa84)  0x6ee9    c2600-adventerprisek9-mz.124-23.bin
 rommon 2 >

What could the problem be?

Continue reading “Memory problems”

Export RTs?

(This article assumes some understanding of MPLS VPNs)

The different methods to attach MPLS VPN RTs when routes are exported from a VRF table can be confusing.
This could be done in two ways with additional options.

  1. The default ‘all’ export RT could be used.
  2. Or the RTs could be attached using an export-map.

The first method is the most common and the easiest to understand. The command below will attach the configured RT to any routes exported from the router’s VRF RIB table into the MPBGP table for advertising.

route-target export {asn:xx}

But what if another RT should be attached in place of, or additionally but for only one prefix?

This is where the second method becomes necessary. By using an export-map one can selectively attach RTs to individual prefixes, separate or in conjunction with the default export if it is configured. This usually raises the question of when the ‘additive’ keyword is needed.

Allow me to explain by using the following diagram :

Continue reading “Export RTs?”

CRC Errors on an ATM Trunk

How does one localise the errors on the ATM trunk to a specific VC?

Assume for a second that the following interface ATM1/0 is terminating multiple VCs (Virtual Circuits), and when you issue the following command you see CRC errors. How would you know which one of VCs are the problem child?

#show interfaces atm 1/0
ATM1/0 is up, line protocol is up
  Hardware is ENHANCED ATM PA Plus
  Description: bob's ATM
  MTU 4470 bytes, sub MTU 4470, BW 149760 Kbit, DLY 80 usec,
     reliability 255/255, txload 7/255, rxload 5/255
  Encapsulation ATM, loopback not set
  Encapsulation(s): AAL5
  8191 maximum active VCs, 16 current VCCs
  VC Auto Creation Disabled.
  VC idle disconnect time: 300 seconds
  Signalling vc = 1, vpi = 0, vci = 5
         UNI Version = 4.0, Link Side = user
  0 carrier transitions
  Last input 00:00:01, output 00:00:00, output hang never
  Last clearing of "show interface" counters 00:23:50
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1115
  Queueing strategy: Per VC Queueing
  30 second input rate 1966000 bits/sec, 1032 packets/sec
  30 second output rate 3226000 bits/sec, 1025 packets/sec
     885563 packets input, 129820445 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     350 input errors, 350 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort            '<----Not cool'
     1373823 packets output, 456299872 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 output buffer failures, 0 output buffers swapped out

Continue reading “CRC Errors on an ATM Trunk”

Understanding and Configuring a HWIC-3G-GSM

Apologies for the long absence from posting. I find myself without any hours left in a day before I got to everything I wanted to do.  And before you know it, more than a month has gone past.

In my previous post I presented a quick solution to an Out-of-Band network and I talked about some options. I’ve had mails asking how to show some of the configurations. I’ll cover those and do other posts I have been promising in the next couple days.

This post will focus on the current Cisco 3G WAN card, the HWIC-3G-GSM. This card is supported by Cisco’s 1841, 1861, 2800-series and 3800-series ISR routers. This card only supports High-Speed Downlink Packet Access (HSDPA) “up to” 3.6 Mb/s downlink, 384 kb/s uplink (presumably HSDPA Category 5/6, but not sure)

“3G” is a broad category of standards and services around “broadband” mobile wireless voice and data. Universal Mobile Telecommunications System (UMTS) is part of this family. High Speed Packet Access (HSPA) is a collection of mobile telephony protocols that extend and improve the performance of existing UMTS protocols. Two standards, HSDPA and HSUPA have been established and is fairly well known.

Continue reading “Understanding and Configuring a HWIC-3G-GSM”

Out-of-Band network

An Out-of-Band management network plays an integral part in supporting any network. Without it when core devices go down, unnecessary time is spend driving to the downed site to fix and correct the problem if remote connectivity in unavailable.

For those that don’t know, an Out-of-Band (OOB) management network is a small support network that usually runs alongside the production network at key locations, with the sole purpose to provide console level access to core devices remotely. This access can be vital to assure downtime is minimized.

The usual OOB requirements are:

  1. Low implementation cost since it is used only for support.
  2. Low monthly cost for the same reason.
  3. OOB should not depend on any existing infrastructure.
  4. Should be easily accessible from remote locations.
  5. Must be secure, since it connects to the core devices.

ISDN and dialup technologies are most commonly used, due to the low monthly line costs. But ISDN and Dialup has the inherit cost problem if the line is connected for extended periods (days), either due engineer negligence or configuration troubles. I have also seen 64k Diginet links used, which is really not the best option cost wise, when the OOB network spans different geographical regions.

I was recently task to fix a OOB design that were using Diginet links. I looked at the design, and I cancelled all the serial links days later due to insanely high monthly costs.

Instead, to address all the required points above, I proposed a new design similar to the diagram below. (This diagram only depicts one site though)

Continue reading “Out-of-Band network”

Upgrading a Cisco 6500

Upgrading a 6500 is pretty straight forward, provided the necessary is done in the right order. I’ve listed the steps I would typically take to fully upgrade a single Cisco-6509-E (single Route-Processor) with a IPSEC VPN SPA blade.

Please lab this if possible BEFORE trying it in a production network. I have illustrated the steps to be taken if some of the known funnies occur during an upgrade. Feel free to use this as a guideline.

Firstly download the IOS and image versions, you need. Obviously do a little homework and check the specific IOS for known bugs using the Bug Toolkit. Don’t just pick any IOS. Make sure all the required features are relatively bug free.

Copy the downloaded files to the following locations:

  • ROMMON firmware to sup-bootflash
  • BOOTLDR to bootflash
  • IOS to flash disk

I always use FTP if possible, due to the higher transfer rates. is connected to the switch and is running a FTP server, expecting a username:password of cisco:pass.

copy ftp://cisco:pass@ sup-bootflash:
copy ftp://cisco:pass@ bootflash:
copy ftp://cisco:pass@ disk0:
dir sup-bootflash:
dir bootflash:
dir disk0:

Continue reading “Upgrading a Cisco 6500”

Troubleshooting a Cisco 6500 crash

I was asked recently to share some knowledge about the support of the Cisco 6500 switches as the information available on the DOC-CD could be fairly overwhelming.

As it happens a clients Cisco-6509 switch fell over yesterday. I was called out to address the issue of the Cisco-6509 that decided it was tired of life by rebooting itself.  I’ll go through some of the steps I did to find the root cause. Obviously note the steps listed here will not find the cause of every possible issue with a 6500 switch, but can be used as a guideline.

Usually the first thing I would do is to see the reason for the reboot with a “sh version”. Look at the highlighted lines.

ndcbbnpendc0103#sh ver
Cisco Internetwork Operating System Software
IOS (tm) s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(18)SXF6, RELEASE SOFTWARE (fc1)
Technical Support:
Copyright (c) 1986-2006 by cisco Systems, Inc.
Compiled Mon 18-Sep-06 23:32 by tinhuang
Image text-base: 0x40101040, data-base: 0x42D90000

ROM: System Bootstrap, Version 12.2(17r)SX5, RELEASE SOFTWARE (fc1)
BOOTLDR: s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(18)SXF6, RELEASE SOFTWARE (fc1)

ndcbbnpendc0103 uptime is 3 hours, 23 minutes
Time since ndcbbnpendc0103 switched to active is 3 hours, 22 minutes
System returned to ROM by s/w reset at 00:14:27 PDT Wed Sep 20 2006 (SP by bus error at PC 0x402DC89C, address 0x0)
System restarted at 09:13:44 ZA Wed Mar 10 2010
System image file is "disk0:s72033-adventerprisek9_wan-mz.122-18.SXF6.bin"

Obviously it is clear that the switch did a software reset caused by ‘bus error at PC 0x402DC89C, address 0x0‘.

Continue reading “Troubleshooting a Cisco 6500 crash”

Troubleshooting BGP

The new focus is of the R&S exam is troubleshooting. And for some reason this is seen as a new topic to study and as a result feared. It is vital to understand why troubleshooting was added to the lab, and why it will possibly be added to other tracks.  Anybody can apply vanilla configs, provided it is done without error, in the correct order, and by avoiding the question pitfalls.

Troubleshooting was introduced by Cisco to give the CCIE certification that edge it needs to seperate the guys that really understand the technologies and those that just learned to configure labs. Troubleshooting is thus NOT a new section! If you understand know each technology, understand its it building blocks, processes and states, troubleshooting should be nothing ‘new’.

That said, once you understand the work, drafting a troubleshooting methodology per technology should be fairly straight forward. A detailed troubleshooting approach is included in each chapter of the Routing-Bits Handbooks.

(Note ‘{ }’ curl-brackets indicate replaceable values, the rest is regex)

Troubleshooting BGP session start-up problems

1- Are you seeing the expected neighbors in a NON ‘idle’ or ‘active’ state?
#sh ip bgp summary

2- Is a sourced telnet to the neighbor address working?
#telnet {peer-ip} 179 /source {src-int-ip}

3- Confirm if the configuration is correct and matching to neighbors configuration?
#sh run | b router bgp

4- If eBGP, is the neighbor directly connected? (Should be 1 hop in the trace)
4.1- If not directly connected is multihop configured?
#trace {peer-ip} source {src-int-ip}
#sh run | i {peer-ip}.*ebgp-multihop

Continue reading “Troubleshooting BGP”