CCNP Enterprise Core (350-401) Cisco Packet Forwarding

Switching Database Manager (SDM) Templates

The number of MAC addresses that a switch may need to store in comparison to the number of routes it needs to hold depends on where it is deployed in the network.

The memory used for TCAM tables is limited and allocated during the boot-up process on the switch. When a section of hardware resources becomes full the packet processing is sent to the CPU which can seriously impact the performance of the switch.

The ratios of allocation between Layer 2 and Layer 3 features are stored and modified using Switching Database Manager (SDM) templates. The SDM template can be changed on Cisco 9000 series switches with the command sdm prefer vlan or sdm prefer advanced in global configuration mode. If the switches are stacked then every switch is required to be configured with the same SDM template.

The current template can be viewed with the command show sdm prefer

CCNP Enterprise Core (350-401) Cisco Packet Forwarding

Stateful Switchover (SSO)

Routers designed for high availability (HA) include hardware redundancy such as dual power supplies (PSU) and router processors (RP).

The RP is responsible for for learning the network topology and building the route table (RIB).

If the RP fails it can trigger routing protocol adjacencies to reset resulting in network instability.

During a RP failure it may be more desirable to hide the failure from the rest of the network and allow the router to continue forwarding packets using the previously programmed CEF table entries rather than drop packets and wait for the secondary router processor to rebuilt the forwarding table and re-establish adjacencies.

Stateful Switchover (SSO) is a redundancy feature that allows a Cisco device with two route processors to synchronise router configuration and control plane state information.

The processing of mirroring this information between route processors is known as checkpointing. Stateful Switchover enabled routers will always checkpoint line card operation and Layer 2 protocol states.

In a triggered failure event the standby route processor will immediately take control and prevent basic problems such as line interface flapping. Layer 3 forwarding is disrupted without additional configuration as the switchover will trigger a routing protocol adjacency flap that clears the route table.

When the route table is cleared, CEF entries will be purged and traffic no longer routed until the topology is relearned by the new route processor.

Enabling additional configuration in the form of nonstop forwarding (NSF) or nonstop routing (NSR) will keep CEF entries for a short duration in a failover event to keep packet forwarding in an event of an RP failure until the control plane has time to recover.

CCNP Enterprise Core (350-401) Cisco Packet Forwarding

Hardware Cisco Express Forwarding (CEF)

ASICs in hardware platforms are expensive to design, produce and troubleshoot and limited in their functionality yet they allow for very high packet rates due to only being programmed for specific tasks.

By LiveWireInnovation – Own work, CC BY-SA 3.0

Routers can also be equipped with network processing units (NPUs). NPUs have an advantage over ASICs as they are programmable meaning their programming and firmware can be changed with ease.

Distributed forwarding architectures with hardware CEF allow packet throughput to be greatly increased by offloading packet switching responsibilities to one or more line cards. Packet switching accomplished in distributed platforms is done via dCEF (Distributed Cisco Express Forwarding). dCEF allows CEF data structures to be downloaded to ASICs and the CPUs of all line cards so they can all participate in packet switching. The main advantage of this is switching can be done at a distributed level and increasing the packet throughput of the router.

CCNP Enterprise Core (350-401) Cisco Packet Forwarding

Software Cisco Express Forwarding (CEF)

Software CEF, more commonly known as software Forwarding Information Base (FIB) consists of two parts:

Forwarding Information Base

The forwarding information base is built from the routing table on the local device and contains the next hop IP for each destination that the device knows on the network. When a routing or topology change occurs on the network the IP routing table is updated and these changes are also reflected in the forwarding information base. Cisco Express Forwarding uses the forwarding information base to make prefix-based switching decisions.

Adjacency Table

Also known as the adjacency information base (AIB), the adjacency table contains the directly connected next-hop IP addresses and their corresponding next-hop MAC addresses, as well as the egress interfaces MAC address. It is populated with data from the ARP table or other Layer 2 protocols.

In software CEF switching, when the device receives an IP packet, the Forwarding Information Base is checked for a valid entry.

If the entry is missing, this is known as a ‘glean’ adjacency in CEF. Glean means that the IP packet should go to the general purpose CPU because CEF is not able to handle it. Missing entries invoke the ARP process and once resolved a complete CEF entry can be added. There is a rate limiter in place to prevent the CPU from being starved of serving other essential operations.

If the entry is valid it is continued to be checked by examining the adjacency table for each packets destination IP address.

As part of the forwarding process the packets headers are overwritten. The receiving hardware:

  • overwrites the destination MAC address of a packet with the next-hop routers MAC address from the adjacency table
  • overwrites the source MAC address with the MAC address of the egress layer 3 interface
  • Decrements the time-to-live (TTL) counter field by one, and is dropped if it reaches below 0.
  • Recalculates the checksum of the IP header
  • Delivers the packet to the next-hop router
CCNP Enterprise Core (350-401) Cisco Packet Forwarding

Distributed Forwarding

Distributed forwarding is when there are line cards that can make decisions without the input of a route processor.

When a packet is received on the ingress line card, it is transmitted to the local forwarding engine. The forwarding engine performs a packet lookup and determines if the outbound interface is local.

If the outbound interface is local it will forward the packet out of a local interface.

If the outbound interface is not local and located on a different line card, the packet is sent across the switch fabric, which is also known as the back plane, directly to the egress line card and bypassing the route processor.

CCNP Enterprise Core (350-401) Cisco Packet Forwarding

Centralised Forwarding

With the low cost of general purpose central processing units, the price of software based routers is become more affordable at the expensive of total packet throughput.

When a route processor (RP) is equipped with a forwarding engine so it can make all of the packet switching decisions, this is known as a centralised forwarding architecture.

If line cards are equipped with forwarding engines so they can make packet forwarding decisions without the intervention of the route processor, this is known as distributed forwarding architecture.

In a centralised forwarding architecture, when a packet is received on the ingress line card it is transmitted to forwarding engine of the route processor. The forwarding engine examines the packets headers and sends out the packet via a port via the egress line card.

CCNP Enterprise Core (350-401) Cisco Packet Forwarding

Ternary Content Addressable Memory (TCAM)

A platforms TCAM allows for the matching and evaluation of a packet on more than one field.

TCAM is an extension of the content addressable memory (CAM) architecutre but is enhanced to allow for upper layer procerssing such as identifying the Layer 2 and 3 source and destination addresses, protocol, QoS markings and more.

TCAM provides more flexibility in searching over CAM which only returns a binary result. The TCAM search provides three results on a search, 0 for true, 1 for false and X for do not care.

TCAM entries are stored in Value, Mask, and Result format.

The Value indicates the field that should be searched, examples would be the IP address and protocol fields.

The Mask indicates the field that is of interest and should be queried.

The Result is the action that should be taken with a match on the value and mask. Multiple actions can be selected along instead of just dropping or allowing traffic such as redirecting a network flow to a Quality of Service policer or specifying a pointer to a different entry in the routing table.

Many switches contain multiple TCAM entries so inbound/outbound security, quality of service, and Layer 2 and Layer 3 forwarding decisions can all take place at once.

TCAM operates on a hardware level, providing faster processing and scalability instead of processing switching.

TCAM allows some features such as access control lists (ACL) to process at the same speed regardless of whether of how big they are.

CCNP Enterprise Core (350-401) Cisco Packet Forwarding

Cisco Express Forwarding (CEF)

A Cisco proprietary switching mechanism, Cisco Express Forwarding has been the default switching mechanism on most Cisco platforms that do all their packet switching and forwarding using the general purpose CPU since the 1990s, and is the default switching mechanism used by all Cisco platforms that use specialised ASICs and Network Processing Units (NPUs) for high packet throughput.

The general purpose CPU on both software and hardware based platforms are similar and perform all the same functions. There is a difference on software based platforms that the general purpose CPU is in charge of all operations including CEF (based in software) and hardware based platforms do CEF switching in specialised ASICs, tenary content addressable memory (TCAM) and NPUs (Hardware CEF).

CCNP Enterprise Core (350-401) Cisco Packet Forwarding

Process Switching

Process switching can also be referred to as software switching or slow path switching.

Process switching is when the general central processing unit (rather than any dedicated ASIC chip) takes charge of packet switching on the hardware device. Within Cisco IOS the process ip_input runs on the general CPU for incoming network packets that can not be switched by Cisco Express Forwarding (CEF).

Process switching in the general CPU is significantly slower than switching done in the hardware (CEF). The router process switching is designed to handle a very small percentage of overall traffic, with the preference of the bulk of the routing to be done in hardware.

The reasons why packets that may go to the ip_input process rather than to CEF:

  • Packets that are sourced from or destined to the router (Using control traffic or routing protocols)
  • Packets that are too complex for the CEF hardware to handle (Packets with particular IP options)
  • Packets that require extra processing or contain information that is not known (ARP)

CCNP Enterprise Core (350-401) Cisco Packet Forwarding

Forwarding Architectures

In the beginning Cisco routers on recieving a packet would remove the Layer 2 information and verify that the route existed for the destination address on that packet. If a matching route could not be found on the router, the packet would be dropped. Otherwise if a matching route was found, new Layer 2 information would be added to the packet and forwarded. This was called process switching.

As Cisco router technology has advanced the routers no longer remove the addressing but simply rewrite it. IP packet switching or packet forwarding is a faster process than waiting to recieve a packet, read it, and determine whether it needs to be dropped or forwarded. This became knoiwn as fast switching and later Cisco Express Forwarding (CEF)