The hype machine in Cisco channel land has been working overtime since Cisco started shipping its new USC line of servers and blade center. If you’ve heard what is being said, Cisco is basically claiming to have reinvented the server and is now offering unparalleled performance over their competitors. Their one large claim to fame on the onset was their VMware VMark scores, but as of this writing HP has bested them in every category by small margins. The other key selling point is Cisco’s Extended Memory Technology which allows an increased amount of physical RAM in UCS servers aimed at providing greater virtual machine density.

Cisco, in my view,  has never been a company overly concerned about sexiness in their hardware or software, although they certainly tried harder than usual with their king Nexus 7000 switch. The UCS C200 servers I have acquired will be used to power a new virtualized Unified Communications infrastructure (Call Manager) which is another major advancement in Cisco’s product offerings. So while my use case will not push these servers to their theoretical performance limits, I will still get down and dirty with this new hardware platform. 

Under the hood

My first impression of the C200 is that it looks remarkably similar to an older lower-end Dell PowerEdge or SuperMicro white box server. Aesthetically pretty vanilla, at this level anyway. That said, the layout is simple and gets the job done in true minimalist fashion. All internal components are OEM’d from the usual suspects: Intel, Samsung, Seagate, LSI… Getting the cover off of this thing is truly a pain requiring a lot of hard release button mashing and downward forceful pushing. Both of my C200’s were like this so definitely not a fluke.

Cisco Integrated Management Controller (CIMC)

CIMC is the remote out-of-band management solution (IPMI) provided with Cisco servers. With the very mature HP ILO and Dell DRAC remote management platforms around for years, Cisco’s freshman attempt in this space is very impressive indeed. All of the basic data and functionality you would expect to find is here plus a lot more. Access to the CIMC GUI requires Adobe Flash via a web browser which is visually pretty but disappointing to see in an enterprise platform. They certainly aren’t the only major vendor to start trending this direction (read: VMware View 4.5).

Performance is a bit slow for tabs on some pages where the hardware has to be polled and display data refreshed. But when that data eventually trickles in, the level of detail is dense.

 

That was all just from the Inventory page! More great detail is revealed in the Sensors section with multiple readings and values for each core component of the server.

Other user, session, logs, and firmware management options include all the usual settings and variables. One other neat option in the Utilities sub menu is the ability to reboot CIMC, reset it to factory default as well as import configurations! That’s huge and will make managing multiple servers much more coherent. All told and bugs aside, the potential of CIMC is very impressive.

Call Manager -  the virtual edition

A major shift for Cisco, now available in CUCM Version 8.x, is the ability to deploy the enterprise voice architecture inside of VMware ESXi. Call manager, and it’s sister voice mail service Unity Connection, are just Linux servers (RHEL 4) after all so this makes perfect sense. You can now deploy Call Manager and Unity clusters inside of a virtual space while leveraging the HA provided by VMware as well.

This of course doesn’t come without its caveats. Currently Cisco does not support VMs living outside of Cisco servers and that includes storage. So you will have to buy a Cisco server to deploy this solution as well as keep the VMs on Cisco disk, not your own corporate SAN. You can use your own VMware licensing and vCenter at least which is a good thing. Once Cisco has established a comfortable foothold in the enterprise server market, look for these policies to ease a bit. Right now they need to sell servers!

Not purely Cisco-related, but a minor observation that others have noticed as well is that ESXi incorrectly reports the status of Hyper-Threading support on non-HT Intel-based servers. My C200 is equipped with Xeon E5506 CPUs which do not support HT. Not a big deal, just an observation. If HT was available in this CPU I would definitely enable it as ESX(i) 4.1 can now schedule much more efficiently with the new Intel CPU architectures.

Wrap

All in all there’s a lot to like about the new Cisco offerings. A commitment to virtualization and hardware optimized to run virtual workloads are smart investments to make right now. There are some physical design choices that I don’t particularly care for but this model server is at the bottom of the platform stack, so maybe more consideration was paid to the platforms at the top? CIMC was carefully constructed and, although buggy right now, shows some real innovation over competing platforms in this space. More companies that would not have otherwise been able to buy into a full-blown Call Manager cluster configuration can now do so with reduced hardware investments.

References:
Cisco OVA templates

The Cisco AnyConnect 2.5 client in conjunction with ASA ASDM version 6.3 bring several new features to Cisco’s SSL VPN implementation. Some of the more interesting features are:

• Post Log-In Always-on VPN – VPN session is auto-established after user logs into PC.
• Connect Failure Policy – Determines if user is allowed to access internet if Always-On VPN is enabled and AnyConnect cannot establish VPN session.
• Captive Portal Hotspot Detection/Remediation – Detects “pay for internet access” types of portals seen in hotels or airports and allows users timed access to establish connectivity.
• Quarantine – Restricts client PCs that are out of compliance and allows remediation options (AV updates, etc).
• AnyConnect Profile Editor – Provides GUI-based profile editing via ASDM instead of XML-only editing.

Here is my setup:

• 2 x ASA 5510’s in active/active cluster
• ASA version: 8.2(1)
• ADSM version: 6.3(1)
• Security Plus license
• AnyConnect Essentials license (total VPN peers: 250)

Building the client profiles is easy enough, in ASDM you just select the Remote Access VPN tab and drill down to AnyConnect Client Profile under Network (client) Access. Select Add, give the profile a name, and configure your policy.

Once the XML file is created on the ASA you can set your policy options:

The Always on piece is set on the Preferences(Cont) page. Here you can get really restrictive by having the AnyConnect client detect the PC’s network state and apply actions based on it. If the client detects a trusted network (defined in the profile), VPN will disconnect, if the network is untrusted it will connect (Trusted Network Detection/TND). You can allow or disallow disconnection of the VPN session by the user and set a connect failure policy. When the connect failure policy is set to closed the user will be unable to even browse the internet from their home network unless VPN is established. Setting to Open reverses this behavior. If you build split tunnels into your group policy then only traffic destined to the specified corporate resources will flow over the VPN tunnel. Anything internet related can go out whatever internet connection they are connected to. You can, of course, force all traffic over the VPN tunnel.

The part where this breaks is when you attempt to assign the profile to a group policy via ASDM[unless, as someone pointed out in the comments, you are running ASA 8.3(1)]. This screenshot actually shows the correct sequence of commands being executed although I have seen it try to apply the profile from the global webvpn sub menu. In any case this has not worked for me through ASDM after multiple attempts with different profiles and group policies [because I am on ASA version 8.2(1)].

Assigning your ASDM-configured client profile via the ASA CLI works fine. ADSM is essentially trying to do the same thing but it just doesn’t work [for me].

1) Register the client profile XML as a global profile package:

ASA1(config)# webvpn
ASA1(config-webvpn)# svc profiles TestProfile disk0:/TestProfile.xml
ASA1(config-webvpn)# exit

2) Confirm the package is available to the group policy “VPN_GroupPolicy”:

ASA1(config)# group-policy VPN_GroupPolicy attributes
ASA1(config-group-policy)# webvpn
ASA1(config-group-webvpn)# svc profiles value ?
config-group-webvpn mode commands/options:
Available configured profile packages:
TestProfile

3) Assign the profile package to the group policy:

ASA1(config-group-webvpn)# svc profiles value TestProfile

Once the user logs into the appropriate group on their AnyConnect client they will pull down the profile XML which gets stored in c:\ProgramData\Cisco\Cisco AnyConnect VPN Client\Profile. Per the enabled settings above this is what the client state looks like in a trusted and untrusted network:

As you can see the result of the closed Connect Failure Policy on the right, the user will have to establish VPN before being allowed to even get out to the internet on a home or public network.

Microsoft has a similar solution with their DirectAccess implementation. DirectAccess abilities are built into Windows 7 Enterprise and Ultimate so require no additional clients. I started to go down that road first but got scared away by the list of requirements and infrastructure changes I would have to make on the backend to implement. AnyConnect is an excellent alternative that comes with the peace of mind of Cisco security plus there is no additional infrastructure to set up. SSL VPN is not free, however, and comes in two flavors: premium and essentials. The essentials license for my 5510 cluster, which supports a max of 250 VPN peers, was under $100. Premium lics are several times more expensive.

*It should be noted that if you intend to upgrade your ASA to 8.3(1) then you may need to also increase the amount of DRAM in your appliance. Cisco upped the default amount in all devices purchased after Feb 2010. Link.

*A quick note about TND, it can indeed be spoofed. The 2 criteria that the AC client is looking for is the trusted domain name and DNS server address list. If you configure these 2 items to match what is expected in the AC client profile, the AC client will think the network is trusted. If you change or add DNS servers to your corporate DHCP server scopes, this will break TND. The AC client profile must match what is given to the client via DHCP (or statically) for this to work.

Here is my client profile XML, for reference:

<?xml version="1.0" encoding="UTF-8"?>
<AnyConnectProfile xmlns="http://schemas.xmlsoap.org/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://schemas.xmlsoap.org/encoding/ AnyConnectProfile.xsd">
<ClientInitialization>
<UseStartBeforeLogon UserControllable="false">false</UseStartBeforeLogon>
<AutomaticCertSelection UserControllable="true">false</AutomaticCertSelection>
<ShowPreConnectMessage>false</ShowPreConnectMessage>
<CertificateStore>All</CertificateStore>
<CertificateStoreOverride>false</CertificateStoreOverride>
<ProxySettings>Native</ProxySettings>
<AutoConnectOnStart UserControllable="true">true</AutoConnectOnStart>
<MinimizeOnConnect UserControllable="true">true</MinimizeOnConnect>
<LocalLanAccess UserControllable="true">true</LocalLanAccess>
<AutoReconnect UserControllable="false">true
<AutoReconnectBehavior UserControllable="false">ReconnectAfterResume</AutoReconnectBehavior>
</AutoReconnect>
<AutoUpdate UserControllable="false">true</AutoUpdate>
<RSASecurIDIntegration UserControllable="false">Automatic</RSASecurIDIntegration>
<WindowsLogonEnforcement>SingleLocalLogon</WindowsLogonEnforcement>
<WindowsVPNEstablishment>AllowRemoteUsers</WindowsVPNEstablishment>
<AutomaticVPNPolicy>true
<TrustedDNSDomains>domain.com</TrustedDNSDomains>
<TrustedDNSServers>10.10.10.10,20.20.20.20</TrustedDNSServers>
<TrustedNetworkPolicy>Disconnect</TrustedNetworkPolicy>
<UntrustedNetworkPolicy>Connect</UntrustedNetworkPolicy>
<AlwaysOn>true
<ConnectFailurePolicy>Closed
<AllowCaptivePortalRemediation>true
<CaptivePortalRemediationTimeout>20</CaptivePortalRemediationTimeout>
</AllowCaptivePortalRemediation>
<ApplyLastVPNLocalResourceRules>false</ApplyLastVPNLocalResourceRules>
</ConnectFailurePolicy>
<AllowVPNDisconnect>true</AllowVPNDisconnect>
</AlwaysOn>
</AutomaticVPNPolicy>
<PPPExclusion UserControllable="false">Disable
<PPPExclusionServerIP UserControllable="false"></PPPExclusionServerIP>
</PPPExclusion>
<EnableScripting UserControllable="false">false</EnableScripting>
<EnableAutomaticServerSelection UserControllable="false">false
<AutoServerSelectionImprovement>20</AutoServerSelectionImprovement>
<AutoServerSelectionSuspendTime>4</AutoServerSelectionSuspendTime>
</EnableAutomaticServerSelection>
<RetainVpnOnLogoff>true
<UserEnforcement>AnyUser</UserEnforcement>
</RetainVpnOnLogoff>
</ClientInitialization>
<ServerList>
<HostEntry>
<HostName>ac.domain.com</HostName>
<HostAddress>ac.domain.com</HostAddress>
</HostEntry>
</ServerList>
</AnyConnectProfile>

Metro-Ethernet

There is a lot to like about metro-Ethernet offerings. It’s fast, it’s Layer 2, and comparably it is significantly cheaper than its DS-x counterparts for similar speed offerings with less complexity.  The caveats are lighting the fiber and any construction costs associated with building into your facility. If you are too far from access to the provider’s fiber ring, construction could be a deal breaker.

In the Metro-E space the most utilized offerings include a private solution (managed) and a standard offering that is fiber pairs handed off at the demarc. The key difference here is that in a private model the service provider will install switches in your premise and “light” the fiber from end to end so that they can manage and monitor it. You simply cross-connect into their switch. While this is a very nice option it comes at a steep price. The alternative via a standard type of offering is where the provider simply hands off pairs of fiber to your facility, they light the fiber to you, and you are responsible to light it back to them (Tx/Rx). The catch here is that you need to have your own optics capable of lighting the fiber possibly over a great distance so the required optics will be expensive (ZX).

In either case redundancies will be costly as ideally in a dark/lit fiber scenario you would employ multiple paths to the provider ring through at least two entrances to your facility, literally. Construction costs double as do MRCs but if you absolutely cannot afford for your internet or site-to-site connection to be jeopardized, at any cost, then this is the solution for you. Leading data center providers (Savvis, Rack Space, etc) all have multiple data providers each with multiple entrances to their facilities.

Fiber type and connecting modules are extremely important, pay attention to these details. It may seem obvious but when you’re buying, connecting, and configuring three dozen other pieces of infrastructure at the same time these details can be overlooked. The Metro-E provider will be using single-mode fiber cabling as is the standard for shooting light great distances. Your optics MUST match the fiber type. So if you will need to buy ZX modules for your own Cisco switches be sure to get the ones that match the handed off fiber type. These modules come in both single-mode and multi-mode flavors. Your cross-connect cables must match as well.

Whether standard or private, your service provider will demarc the connection in a location of your choosing, most likely a cabinet in your datacenter. The installation should include a fiber distribution box that they will install and terminate. Your provider will most likely hand off SC connectors on the other side of the distribution box. Your Cisco ZX module will require LC connectors so you will need to acquire a cable with different ends. Having just gone through this, my provider handed off duplexed SC connections so I had to remove the clips on the SC side of the cable to split the SC connections.

Questions to ask:

• What are the differences between a private managed service and a standard service?
• Do you need a point-to-point connection between 2 or more locations, or just internet access?
• What is the type of fiber being handed-off by your provider? Single-mode (SMF)? multi-mode (MMF)?
• Which standard SX? LX? ZX?
• What are the service SLAs?

Fiber Cabling

There are 2 primary types of fiber optic cabling in play in today's datacenters: single-mode (SMF) and multi-mode (MMF). SMF is capable of transmitting almost infinite bandwidth over hundreds of kilometers so is used mainly in backbone fiber rings. MMF can transmit light a shorter distance so is found in campus location use. SMF is also very expensive and less flexible than MMF. Of these 2 types there are a few different options. MMF is found in 50/125µm or 62.5/125µm and SMF is found in 8.3/125µm. These numbers, measured in microns (µm), indicate the diameter of the glass core that carries the light, and the outer cladding which confines the light to the core. 50 micron is the most common flavor of multi-mode fiber and is commonly used for both Ethernet and fiber-channel.

62.5 micron is the older standard, 50 micron is the newer. You can read more about both in the linked documents in the reference section below. A third specification groups the different transmission quality of the fiber types.  The performance codes for MMF are OM1, 2, and 3, for SMF they are OS1 and 2. These codes indicate the bandwidth capabilities for different light wavelengths. See the attachment “Understanding OM1, OM2…” below. For example, I just installed 6 strands of 50 micron OM3 MMF between my datacenter and user floor to run 3 x 10GigE links.

Just like CATx cabling, fiber cabling can be purchased from many “big name” companies like CablesToGo, or Belkin at steep premiums. I strongly recommend against buying from these companies as better quality can be found by smaller e-tailers that make and certify their own cables. My favorite vendors right now are fibercablesdirect.com and monoprice.com.

References:

Cabling and Test Considerations for 10Gige LANs [via fiberoptic.com]

Understanding OM1,OM2,OM3,OS1,OS2 Fiber for Data Centers [via Cisco Learning Network]

Multimode Fiber: 50 micron vs 62.5 micron [via netoptics.com]

One of the advantages I have in my current job role is being able to research, test, and deploy all facets of enterprise technology. I am by trade a “server guy” but I have some fairly extensive experience on the network side of things as well, L2/L3 switching and firewalling mainly. I have had VOIP running at home for many years now and did a very small implementation for a startup years ago. This is my  first enterprise installation.

Call Managers

The two top contenders in the enterprise voice world are Cisco and Avaya. People swear by each, it depends who you talk to. My perception is that Cisco is the “Cadillac” option and there’s a saying in IT, “No one’s ever been fired for buying Cisco”. While both vendors have similar offerings and options with regard to call managers and IP phones, Avaya is typically cheaper all things considered. I’m building out a brand new datacenter infrastructure as part of my efforts which means I need new switches and firewalls. This is where the Cisco voice option makes the most sense. Serious discounts are to be had by buying voice gear bundled with core infrastructure. Cisco is very serious about not losing out on voice deals and make it very hard for competitors while pricing new build-outs.

Cisco has four essential options when it comes to voice platforms, all falling under variations of the moniker Cisco Unified Communications Manager. I’m still partial to Cisco’s original product name Call Manager, but I digress. CM can be purchased as:

• Express (CME)
• Unified Communications 500
• Business Edition (CMB)
• and full blown Communications Manager (CM)

Here is the  short take on the differences: CME is a voice enabled router with a CME vWIC (add-on card). The basic CM functions are handled but this solution is intended for very small environments. UC 500 is a stackable appliance that tops out at 64 phones. CMB is a hardened appliance (an OEM’d HP DL320 painted Cisco teal) that is capable of running the full functionality that CM can offer but is locked to a single appliance. This level in particular has a leg up on Avaya as CMB contains the Unity Connection voice mail system baked into the appliance. Avaya’s comparable IP Office product requires a separate server to handle this function. Full blown CM has the same essential feature sets as CMB but it can be run on various “Media Convergence Servers” (MCS) that vary in specs, and it’s clusterable. The CMB appliance is based on the MCS 7828 platform.

Getting to the PSTN

We have a few options when it comes to providing connectivity for our voice services. Yes VOIP is voice over IP but this does not mean you can simply run it over your main data circuits like you do your Vonage service at home. Telephony is an important aspect of business operation and not only deserves a dedicated solution, but requires it. There are two primary means for connecting to the PSTN (Public Switched Telephone Network), PRI and SIP.

Primary Rate Interface circuits (PRI) are channelized versions of traditional T1/T3 type circuits. A T1 contains 24 channels, 23 of which can be used for placing calls, the 24th is used for signaling. Oversubscription is common and practical so for architecture purposes you have to consider how many simultaneous outbound calls your environment is likely to experience. Internal calls are switched through CM and do not cross the PRI. So for 100 users you could probably safely plan for the likelihood that 40 people might be on the phone at once. Call centers would be thought of differently, obviously. To satisfy those 40 calls, however, you will need 2 PRIs (T1’s).

Session Initiation Protocol (SIP) is purchased as a “SIP trunk” which is an IP connection that permits voice traffic to be routed over the backbone of the carrier. The advantage of SIP is that it’s purchased by bandwidth increments (3Mbps, 5Mbps, etc) which can be increased easily and is normally cheaper than an equivalent PRI. It can also be used for voice and data traffic. The same basic principle of the PRI applies to SIP when planning for oversubscription. 1 x PRI (1 T1) = 1.5Mbps so 1 x 3Mbps SIP trunk is equivalent to 2 x PRI’s.

Voice experts that I’ve spoken with at Cisco and Avaya still think a pure SIP solution is too immature today in terms of call quality. Both recommend at least a partial PRI solution for primary calls and leveraging SIP for call spillover/ redundancy. I’ll go into this deeper in the design. Another important consideration is to ensure that the voice carriers are the same for SIP/PRI, especially between sites. If different carriers are chosen and the primary circuit fails, carrier B would not be set up to receive calls for the company’s DID’s (phone numbers).

Design

My design consists of a two-phase approach providing primary voice services for a main campus location, with acceptable levels of redundancy, for phase one and DR voice services for phase two. Cisco guarantees a 4-hour response time but we can’t have a 4-hour voice outage. Of the choices available CMB makes the most sense for my design. I need to connect 100-125 phones, sustain at least 40 concurrent outbound calls, and, barring facility complications beyond my control, guarantee 99.9% onsite uptime. CMB is a feature rich appliance with redundant hardware features and carries a price point that fits my budget.

For onsite redundancy with CMB there are three viable options.

• Cold spare appliance
• SRST-enabled router
• CME-enabled router

Configs can be backed up and imported into a stand-by appliance which is fast and easy from a recovery stand point. The problem is that this solution isn’t cost effective. Survivable Remote Site Telephony (SRST) can be leveraged in a number of ways. It can be used to provide high availability for the main campus or to protect branch offices from a CM failure in the main campus. Connected IP phones will register the SRST router as a call processor so if there is a problem with CM, SRST will take over telephony operations seamlessly. Functionality is reduced to basic call processing in SRST mode but that is what’s most important after all. CME is also a viable option but isn’t as cost effective as SRST.

Sizing your main campus SRST router is important to ensure that it can sustain the same level of calls that CM did. For my design I’ll be using a voice-enabled 2851 router for the main campus. For DR considerations you can’t have a remote SRST router handling calls for the main campus in the absence of CM, this is a per site limitation. To provide that level of redundancy we would need to deploy another CM in DR. For my DR purposes I can deploy a smaller (cheaper) 2801 router which will serve as a voice gateway for CM in the main campus. What this design buys me is network redundancy should something happen to my 2851, PRI, or SIP circuits in the main campus. CM will route, via IP, to DR and use the SRST router there as the gateway to the PSTN. Budgeting prohibits me from making my design completely bullet-proof so I have to account for as many scenarios as I possibly can.

Leveraging both PRI and SIP circuits in the main campus accomplishes three things. Should the PRI circuits fill up, i.e. caller 47 picks up the phone, this call would be routed out the SIP trunk. Should the PRI circuits fail for any reason, CM will route calls out the SIP trunk for the duration of the outage. Should the primary internet data circuit fail, the SIP trunk will serve as a backup for internet access.

My architecture will satisfy the following objectives:

• HA for CMB in the main campus via SRST
• Redundancy for main campus voice circuits via deployment of PRI’s and SIP
• DR to protect against main campus PRI, SIP, or SRST router failures

I will talk more about switching, security, IP phones, and data circuits in part 2 of this topic. Here is a high level view of my overall architecture: