Out-of-Band network

An Out-of-Band management network plays an integral part in supporting any network. Without it when core devices go down, unnecessary time is spend driving to the downed site to fix and correct the problem if remote connectivity in unavailable.

For those that don’t know, an Out-of-Band (OOB) management network is a small support network that usually runs alongside the production network at key locations, with the sole purpose to provide console level access to core devices remotely. This access can be vital to assure downtime is minimized.

The usual OOB requirements are:

1. Low implementation cost since it is used only for support.
2. Low monthly cost for the same reason.
3. OOB should not depend on any existing infrastructure.
4. Should be easily accessible from remote locations.
5. Must be secure, since it connects to the core devices.

ISDN and dialup technologies are most commonly used, due to the low monthly line costs. But ISDN and Dialup has the inherit cost problem if the line is connected for extended periods (days), either due engineer negligence or configuration troubles. I have also seen 64k Diginet links used, which is really not the best option cost wise, when the OOB network spans different geographical regions.

I was recently task to fix a OOB design that were using Diginet links. I looked at the design, and I cancelled all the serial links days later due to insanely high monthly costs.

Instead, to address all the required points above, I proposed a new design similar to the diagram below. (This diagram only depicts one site though)

Firstly you need an access-server providing console access to the routers. This can be provided by any of the following:

• A Cisco 2509 has 8 built-in async ports (connects up to 8 routers)
• A Cisco 2511 has 16 built-in async ports (connects up to 16 routers)
• A Cisco 1841 with a HWIC-8A (8 async ports)
• A Cisco 28xx with either a HWIC-8A or  HWIC-16A (16 async ports)
• One or two 8-port Octal cables (CAB-HD8-ASYNC) to connect the access-server to every other router.

Luckily I had enough of these already at each site.

The second portion would be the actual router that provides the remote connectivity whilst securing access to the network. Let’s call this the gateway router. Here I used Cisco 1841 routers which were readily available at this client. The only part that needed to be ordered was x amount of HWIC-3G-GSM to provide 3G/HSDPA connectivity.

3G/HSDPA is the best option to be used for remote connectivity. The monthly connection cost is really minimal. The amount of data required to manage an OOB network is also very little. And there are no big usage costs as with ISDN and dialup.

Obviously this technology poses some new challenges, but these will depend entirely on the service offering from the cellular provider. Some of the challenges could include:

• If the cellular provider only assigns dynamic public IP’s, with no option for static IP’s.
• If the cellular provider assigns IP’s using RFC1918 address space, or
• If the cellular provider restricts inbound access to 3G clients.

The easiest solution is obviously receiving a static IP from the cellular provider. Getting around receiving public dynamic IPs are easy enough to get around. But the other options might make this difficult.

The options available if the 3G router only receives a dynamic public IP:

• Dynamic DNS via an online DNS company like http://www.dyndns.org would be used.
• Using an EEM script, the Cellular interface IP can be mailed to a public mail box every so often.  (Don’t use your company email address)
• Alternatively a more secure solution would be tweet the Cellular interface IP to a locked private twitter account. (This is my recommended method, as it adds a third layer of security). Refer to a earlier post I did on this here.

Now that the access-server is sorted out, the gateway router providing connectivity. All that is left is security.

This design has 4 layers of security (if the Twitter method is used):

1. Twitter IP (optional)- Without having access to the locked twitter account how would anyone connect to the OOB gateway router without knowing the IP?
2. SSH access to the gateway router- Upon successful authentication via VTY to the gateway router, the SSH session would be terminated and in the background a dynamic ACL entry would temporarily allow SSH access to the access-server located behind the gateway router. If an outsider by some miracle chance guesses the cryptic SSH username and password, he will be confused why gets disconnected. He also will have no idea about the dynamic ACL entry that opened. This was done on purpose.
3. SSH access to the access-server– ONLY if the previous SSH attempt was successful , in a 20 second window, will SSH access to the access-server be allowed.
4. AAA Access or Local Authentication – Once VTY access is gained onto the access-server, console access to the Core routers is still governed by AAA access first and in the event of network failure local authentication will apply.

By using non standard SSH ports, and different usernames and password on each layer the security could be increased that much more.

And if what is listed above is not enough, security could be taken further. Making use of a Easy-VPN, or randomly generated usernames and passwords could be done by using EEM and mailing them to an EMAIL-to-SMS system. It all depends how much you want to play.