Wednesday, November 30, 2011
Tuesday, November 22, 2011
Sunday, November 20, 2011
Fulcrum's Formula
Fulcrum Microsystems is proud of its FM4000 family of switches which come in a variety of port configurations and package sizes that are tailored for key applications ranging from server clustering to port aggregation. But I am interested in talking about its FM4224 model which provides ultra low latency switching. To understand how Fulcrum does this we need to understand the two very common Ethernet switching paradigms first: cut-through and store-and-forward.
Both Store-and-forward and Cut-through switches follow the same forwarding decision process which is based on the destination MAC address of data packets. It is the series of steps in which they do this forwarding decision that differentiates the both processes. Ideally store-and-forward switch makes a forwarding decision on a data packet after it has received the whole frame and checked its integrity, a cut-through switch engages in the forwarding process soon after it has examined the destination MAC (DMAC) address of an incoming frame. The DMAC address is found in the first 6 bytes of a frame so as per above definition a cut-through switch should screen only those first 6 bytes but in practise they wait until a few more bytes of the frame have been checked before they make any decision.
Let’s look at cut-through switching in more detail. When a cut-through switch receives and invalid or bad packet it does not drop the packet like a store-and-forward switch but the best it can do is flag it and send it. So all the invalid packets are sent to the other segments of the network where the frame is examined and dropped by the destination switch. So to make a more intelligent decision cut-through switches examine more than just the DMAC bytes when a packet comes in. And the more interesting thing is that the number of bytes the switch is going to read can be predetermined based on the value of the EtherType field. For more information on the EtherType field of a frame please visit http://en.wikipedia.org/wiki/EtherType .
So what’s the major advantage of the cut-through switches over store-and-forward switches? Well you have guessed it right. It’s the speed at which they forward the packets. And for some high performance applications where latencies are very critical cut-through switches dominate over the store-and-forward switches. But cut-through switches are not all that great. They have some drawbacks too. Check out the link which shows a presentation from Fulcrum’s website. The presentation compares different switching paradigms and their drawbacks.The comparision is based on the memory architechture.
But my question is what is fulcrum's formula? If you checked the above link you will see that Fulcrum uses RapidArray memory and Nexus crossbar to improve it buffer speed. So Nexus is a highly efficient fully-connected non-blocking crossbar circuit, providing the highest capacity of over a Terabit in TSMC’s 130nm FSG process technology, the lowest latency of less than 3ns, the smallest die area and the most efficient power profile where power consumption is directly related to activity. Nexus has been fabricated and fully characterized in process technologies ranging from 0.25um to 65nm. It is being leveraged by Fulcrum in its latest generation switch products, and licensed to partners as an efficient on-chip interconnect infrastructure.
Fulcrum has designed its own SRAM and TCAM technology, to deliver sophisticated multi-ported SRAM blocks that operate at more than twice the speed of standard ASIC-based memories, and consume less power (only when active). The circuit complements Nexus in switch systems where fast memory and efficient switching go hand-in-hand. Similar to Nexus, RapidArray has been fabricated and fully characterized in various process nodes and block sizes, the largest to date being well over 9MB of SRAM and 500KB of TCAM in the TSMC 65nm process.
But my question is what is fulcrum's formula? If you checked the above link you will see that Fulcrum uses RapidArray memory and Nexus crossbar to improve it buffer speed. So Nexus is a highly efficient fully-connected non-blocking crossbar circuit, providing the highest capacity of over a Terabit in TSMC’s 130nm FSG process technology, the lowest latency of less than 3ns, the smallest die area and the most efficient power profile where power consumption is directly related to activity. Nexus has been fabricated and fully characterized in process technologies ranging from 0.25um to 65nm. It is being leveraged by Fulcrum in its latest generation switch products, and licensed to partners as an efficient on-chip interconnect infrastructure.
Fulcrum has designed its own SRAM and TCAM technology, to deliver sophisticated multi-ported SRAM blocks that operate at more than twice the speed of standard ASIC-based memories, and consume less power (only when active). The circuit complements Nexus in switch systems where fast memory and efficient switching go hand-in-hand. Similar to Nexus, RapidArray has been fabricated and fully characterized in various process nodes and block sizes, the largest to date being well over 9MB of SRAM and 500KB of TCAM in the TSMC 65nm process.
Together as shown in the above diagram Fulcrum provides single shared memory for highly efficient cut-through switching.
Sunday, October 30, 2011
GLBP caught up in the Broadcast Storm
We all know that broadcast storms can degrade network performance. I have seen one such case last week. A Unix guy complained that his server was losing connections. He showed logs from his server. There were flaps on his nics. The server is connected in a U shaped network which means he has two NICs in bonding and each of them is connected to a diffrent switch. I didn't find any issues on the access layer swithes. There were no logs, no errors on the interfaces, no CPU utilization, nothing! I logged into the gateway 6500 switches and found GLBP flaps for that particular VLAN. I checked the tracking interface which was fine. No other logs on the devices other than the continuous GLBP flaps. There were several vlan interfaces in there and the GLBP for them were fine. The issue resolved on its own( luckily). Later when I pulled SevOne report for that particular interface there was a high rise in the broadcast traffic on that VLAN and this was causing the issue. So how is the broadcast storm affecting GLBP? I guess the stom blocked/delayed the hello packets of GLBP causing continous failovers. Let me know if you have any thoughts on this
Tuesday, October 18, 2011
Optera Training
1. Things that you should know:
=======================
a. DWDM: DWDM (Dense Wave Division Multiplexing) allows multiple light waves to travel on the same optical fiber, thereby considerably increasing its capacity. Different types of traffic can be assigned to each wavelength.
More info on DWDM can be found on wiki
b. Fiber Types: There are basically two types of optical fiber: Multi-mode and Single Mode. While Multi-Mode can be either step-index or graded-index, most modern multi-mode fibers types today are graded-index.
c. Connector types: There are four types of connectors commonly in use today:
FC - Commonly used on OCLD cards. These are keyed connectors with a screw-on head. Make sure the key is inserted before the connector is screwed on.
MU – Also referred as “little” connectors. Used on cards where space is an issue. Used on multi-port SRM cards to connect cards to the distribution panel.
SC – Common OC3 fiber connector. It is a keyed connected. Used on Passport OC3 card, Optera OCI cards and Cisco routers.
ST –Twist-lock type connector. Not very common.
d. Site types: There are three types of sites: Terminal, OADM and OFA
A Terminal site is a terminating site. Band and channels terminate at the Terminal site and they do not pass through. Terminal sites are implemented by wiring in the OMX module.
In an OADM (Optical Add-Drop Multiplexer) some band are dropped while some other bands are allowed to pass-through. OADM sites are implemented by wiring in the OMX module.
An OFA (Optical Fiber Amplifier) site is made up of a 5200 shelf that contains C and L amplifiers. The OFA site is required whenever the signal needs to be boosted. Note that an OFA shelf can be co-located with OADM sites.
2. Lets check the hardware:
====================
There are 20 slots in a 5200 shelf. In R3, the position of each card is fixed.
The shelf is divided in a West (cards 1-4) and East section (cards 15-18). The OCLDs in the East section of one shelf connect to the OCLDs in the West section of the next shelf in the network.
OCI circuit pack: The optical-channel interface (OCI) circuit packs provide an interface between subtending equipment and the OPTera Metro 5200 system. You can install a maximum of eight OCI circuit packs in an OPTera Metro 5200 shelf. Some OCI circuit packs are designed to support only particular protocols.
OCLD circuit pack: Optical channel laser and detector (OCLD) circuit packs are identified by wavelength band (BAND 1 to BAND 8) and by channel within the wavelength band (CH1 to CH4). Channels of the OCLD circuit packs relate to the wavelength band of the OMX modules in the shelf. Specific channels of OCLD circuit packs have fixed positions in the OPTera Metro 5200 shelf. Up to eight OCLD circuit packs can be installed in the shelf.
OCM circuit pack: The optical channel manager (OCM) circuit pack bridges the data channel between the OCLD and OCI circuit packs.
SP circuit pack: The shelf processor (SP) circuit pack manages communication functions for OPTera Metro 5200. There is one SP circuit pack in an OPTera Metro 5200 shelf.
OMX modules
Each OPTera Metro 5200 OADM or terminal shelf has an optical multiplexer (OMX) tray that holds two OMX modules. Each OMX module has one wavelength band filter, and one channel filter. The OMX wavelength band must be the same for both OMX modules installed in the same shelf. The OMX wavelength band also determines the wavelengths of the optical channel laser and detector (OCLD) cards that you install in the shelf.
Each OMX module contains passive optical filters that add and drop up to four channels in the wavelength band assigned to the OPTera Metro 5200 shelf.
Other channels pass through the OMX unchanged.
The OMX module in OPTera Metro 5200 has two functional areas:
• optical add section
• optical drop section
The optical add section contains a band filter (ADF) and a channel multiplexer (MUX). The optical drop section contains a band filter and a channel demultiplexer (DEMUX). The ADF drops specific wavelengths while allowing other wavelengths to pass through the filter.
Signal Flow: Let's follow the signal
==========================
OTR/MOTR
–The client connects to the OTR/MOTR at 1350Nm or 850Nm the card then transforms this signal
to a single WDM wavelength(with out using the chassis back or management) in the case of the
MOTR multiple signals are time division multiplexed into the single output signal this is then
passed externally (via a fiber patch) to the OMX.
=======================
a. DWDM: DWDM (Dense Wave Division Multiplexing) allows multiple light waves to travel on the same optical fiber, thereby considerably increasing its capacity. Different types of traffic can be assigned to each wavelength.
More info on DWDM can be found on wiki
b. Fiber Types: There are basically two types of optical fiber: Multi-mode and Single Mode. While Multi-Mode can be either step-index or graded-index, most modern multi-mode fibers types today are graded-index.
c. Connector types: There are four types of connectors commonly in use today:
FC - Commonly used on OCLD cards. These are keyed connectors with a screw-on head. Make sure the key is inserted before the connector is screwed on.
MU – Also referred as “little” connectors. Used on cards where space is an issue. Used on multi-port SRM cards to connect cards to the distribution panel.
SC – Common OC3 fiber connector. It is a keyed connected. Used on Passport OC3 card, Optera OCI cards and Cisco routers.
ST –Twist-lock type connector. Not very common.
d. Site types: There are three types of sites: Terminal, OADM and OFA
A Terminal site is a terminating site. Band and channels terminate at the Terminal site and they do not pass through. Terminal sites are implemented by wiring in the OMX module.
In an OADM (Optical Add-Drop Multiplexer) some band are dropped while some other bands are allowed to pass-through. OADM sites are implemented by wiring in the OMX module.
An OFA (Optical Fiber Amplifier) site is made up of a 5200 shelf that contains C and L amplifiers. The OFA site is required whenever the signal needs to be boosted. Note that an OFA shelf can be co-located with OADM sites.
2. Lets check the hardware:
====================
There are 20 slots in a 5200 shelf. In R3, the position of each card is fixed.
The shelf is divided in a West (cards 1-4) and East section (cards 15-18). The OCLDs in the East section of one shelf connect to the OCLDs in the West section of the next shelf in the network.
The cards are:
OCI circuit pack: The optical-channel interface (OCI) circuit packs provide an interface between subtending equipment and the OPTera Metro 5200 system. You can install a maximum of eight OCI circuit packs in an OPTera Metro 5200 shelf. Some OCI circuit packs are designed to support only particular protocols.
OCLD circuit pack: Optical channel laser and detector (OCLD) circuit packs are identified by wavelength band (BAND 1 to BAND 8) and by channel within the wavelength band (CH1 to CH4). Channels of the OCLD circuit packs relate to the wavelength band of the OMX modules in the shelf. Specific channels of OCLD circuit packs have fixed positions in the OPTera Metro 5200 shelf. Up to eight OCLD circuit packs can be installed in the shelf.
OCM circuit pack: The optical channel manager (OCM) circuit pack bridges the data channel between the OCLD and OCI circuit packs.
SP circuit pack: The shelf processor (SP) circuit pack manages communication functions for OPTera Metro 5200. There is one SP circuit pack in an OPTera Metro 5200 shelf.
Filler cards: Filler cards are installed in slots of the OPTera Metro 5200 shelf that do not have active circuit packs.
There are three types of filler cards:
• OCLD filler cards
• OFA filler cards
• blank filler cards
There are three types of filler cards:
• OCLD filler cards
• OFA filler cards
• blank filler cards
OMX modules
Each OPTera Metro 5200 OADM or terminal shelf has an optical multiplexer (OMX) tray that holds two OMX modules. Each OMX module has one wavelength band filter, and one channel filter. The OMX wavelength band must be the same for both OMX modules installed in the same shelf. The OMX wavelength band also determines the wavelengths of the optical channel laser and detector (OCLD) cards that you install in the shelf.
Each OMX module contains passive optical filters that add and drop up to four channels in the wavelength band assigned to the OPTera Metro 5200 shelf.
Other channels pass through the OMX unchanged.
The OMX module in OPTera Metro 5200 has two functional areas:
• optical add section
• optical drop section
The optical add section contains a band filter (ADF) and a channel multiplexer (MUX). The optical drop section contains a band filter and a channel demultiplexer (DEMUX). The ADF drops specific wavelengths while allowing other wavelengths to pass through the filter.
Signal Flow: Let's follow the signal
==========================
OCLD
–The client connects to an OCI circuit pack at 1350Nm or 850Nm this card transforms the signal
from optical to electrical and passes it across the chassis backplane (via the OCM) to an OCLD.
–The OCLD then transforms the electrical signal to a WDM wavelength. and passes it externally
(via a fiber patch) to the OMX.
–The OMX then Multiplex's this WDM wavelength into a channel of four, then adds them to a single
fiber cable carrying another three possible Channels.
–The client connects to an OCI circuit pack at 1350Nm or 850Nm this card transforms the signal
from optical to electrical and passes it across the chassis backplane (via the OCM) to an OCLD.
–The OCLD then transforms the electrical signal to a WDM wavelength. and passes it externally
(via a fiber patch) to the OMX.
–The OMX then Multiplex's this WDM wavelength into a channel of four, then adds them to a single
fiber cable carrying another three possible Channels.
OTR/MOTR
–The client connects to the OTR/MOTR at 1350Nm or 850Nm the card then transforms this signal
to a single WDM wavelength(with out using the chassis back or management) in the case of the
MOTR multiple signals are time division multiplexed into the single output signal this is then
passed externally (via a fiber patch) to the OMX.
–The OXM then Multiplex's this WDM wavelength into a channel of four, then adds them to a
single fiber cable carrying another three possible Channels
single fiber cable carrying another three possible Channels
Sunday, October 9, 2011
Monday, October 3, 2011
Socket Programming
Although I hate programming so much I find socket programming quite interesting. I recommend Beej’s Guide to Network Programming. It's a fun way to start if you are scared of programming like I am.
Saturday, September 17, 2011
Wednesday, September 7, 2011
My TSHOOT Exam!
I was surprised by the format.. I expected the format to be like the ROUTE and SWITCH exam with good proportion of all types of question. But when the exam started the exam instruction said that there will be 4 questions and pass marks is 790! I read it twice and clicked next. The first 3 questions were multiple choice questions and the fourth one was a topology based question with 14 trouble tickets in it. The exam was brilliant. It was so much fun solving those trouble tickets. Absolutely scary too because I was running out of time. But it was great experience. The TSHOOT exam rocks..
Tuesday, September 6, 2011
CCNP Accomplished!!
Everyone, I finished my CCNP TSHOOT exam today and with it I have I have finished all 3 exams for CCNP! I will get my certification kit soon. Can't stop laying hands on it. Will update a blog soon about the TSHOOT exam. It was the best exam I ever took!
Tuesday, August 9, 2011
Things that you might miss when studying HSRP
- HSRP group can contain more than two members; One is active member, one is standby router and the others will be in listening HSRP state.
- HSRP sends hello messages to 224.0.0.2 (All Routers Multicast Address) using UDP port 1985.
- HSRP group numbers can be [0,255]
- HSRP groups are locally significant on the interface. This is very important point to understand. For example, HSRP group 1 on the VLAN 10 interface is not same as the HSRP group 20 on VLAN 11 interface.
- HSRP router election:
Active Router is the one with the highest priority
If there is a priority tie then the router with the highest HSRP Ip address wins.
6. HSRP States: Disabled> Init> Listen> Speak> Standby> Active.
7. Only standby member monitors the hello messages. Listening members do not monitor them.
8. Default hello message every =3s; Hold time=10s
9. The actual interface address and the standby address must be configured to be in the same IP subnet.
9. The actual interface address and the standby address must be configured to be in the same IP subnet.
Sunday, July 17, 2011
Understanding your NetFlow
I was using SevOne at office the other day to pull some reports of the nonunicast traffic in a vlan to troubleshoot a particular GLBP issue. And then I wondered what will I do without such tools. Then I became curious and wanted to know how it actually knows all that information.
SevOne consolidates granular performance data from various data sources such as SNMP, NetFlow, VoIP, IP SLA, NBAR and WMI onto a unified dashboard. I want to particularly focus on NetFlow in this paticular article.
Netflow is a network protocol that was developed by Cisco in 1996. It was designed to collect IP traffic information. Soon it became an industry standard for traffic monitoring. There has been several versions of Netflow developed over the years and its current state is known in the industry as Flexible NetFlow.
The flow is defined by factors such as Source IP address, Destination IP address, Source port, Destination Port, Layer 3 protocol type. The version5 which is the most common version in use has 18 such fields. Version 5 is great if you are just looking for regular IPv4 traffic. It does not provide in depth analysis of the traffic but provides a very good overview of the composition of your traffic flow. The Later versions such as v7 and v8 were extensions of v5 and had features like router-based aggregation and reduced NetFlow export data volume.
The problem with these versions was that they used fixed export formats that were not flexible and adaptable. This caused the customers to re-engineer for each new version. So they built a more flexible and extensible export format called version 9. This was done by introducing the notion of template. Templates provide an extensible design to the record format, a feature that should allow future enhancements to NetFlow services without requiring concurrent changes to the basic flow-record format. This new feature supports additional technologies such as MPLS or Multicast.
We also have Internet Protocol Flow Information Export (IPFIX) coming up in the near future which is based on NetFlow Version 9 but acts as a more universal industry standard.
SevOne consolidates granular performance data from various data sources such as SNMP, NetFlow, VoIP, IP SLA, NBAR and WMI onto a unified dashboard. I want to particularly focus on NetFlow in this paticular article.
Netflow is a network protocol that was developed by Cisco in 1996. It was designed to collect IP traffic information. Soon it became an industry standard for traffic monitoring. There has been several versions of Netflow developed over the years and its current state is known in the industry as Flexible NetFlow.
The flow is defined by factors such as Source IP address, Destination IP address, Source port, Destination Port, Layer 3 protocol type. The version5 which is the most common version in use has 18 such fields. Version 5 is great if you are just looking for regular IPv4 traffic. It does not provide in depth analysis of the traffic but provides a very good overview of the composition of your traffic flow. The Later versions such as v7 and v8 were extensions of v5 and had features like router-based aggregation and reduced NetFlow export data volume.
The problem with these versions was that they used fixed export formats that were not flexible and adaptable. This caused the customers to re-engineer for each new version. So they built a more flexible and extensible export format called version 9. This was done by introducing the notion of template. Templates provide an extensible design to the record format, a feature that should allow future enhancements to NetFlow services without requiring concurrent changes to the basic flow-record format. This new feature supports additional technologies such as MPLS or Multicast.
We also have Internet Protocol Flow Information Export (IPFIX) coming up in the near future which is based on NetFlow Version 9 but acts as a more universal industry standard.
Thursday, July 7, 2011
Saturday, June 4, 2011
EIGRP Stub
When using the EIGRP Stub Routing feature, you need to configure the distribution and remote routers to use EIGRP, and to configure only the remote router as a stub. Only specified routes are propagated from the remote (stub) router. The router responds to queries for summaries, connected routes, redistributed static routes, external routes, and internal routes with the message "inaccessible." A router that is configured as a stub will send a special peer information packet to all neighboring routers to report its status as a stub router.
Any neighbor that receives a packet informing it of the stub status will not query the stub router for any routes, and a router that has a stub peer will not query that peer. The stub router will depend on the distribution router to send the proper updates to all peers.
Wednesday, June 1, 2011
Passive Interface in EIGRP and OSPF
Tip: Configure the loopbacks under OSPF point-to-point network. Otherwise OSPF will make them a /32 network no matter what the actual subnet is.
Now we will check the effect of passive-interface command in OSPF.
1. Enable wireshark capture on R3 fa0/0. Also turn on debug commands on routers.
2. Make fa0/0 on R2 a OSPF passive-interface
3. OSPF drops off and EIGRP kicks in.
*Mar 1 13:39:58.352: %OSPF-5-ADJCHG: Process 1, Nbr 120.120.120.120 on FastEthernet0/0 from FULL to DOWN, Neighbor Down: Interface down or detached
R2#
*Mar 1 13:39:59.788: %SYS-5-CONFIG_I: Configured from console by console
R2#
*Mar 1 13:40:03.868: IP-EIGRP(Default-IP-Routing-Table:1): route installed for 110.110.110.0 ()
*Mar 1 13:40:03.872: IP-EIGRP(Default-IP-Routing-Table:1): route installed for 120.120.120.0 ()
*Mar 1 13:40:03.872: IP-EIGRP(Default-IP-Routing-Table:1): route installed for 100.100.100.100 ()
Sh ip route output before:
Gateway of last resort is not set
100.0.0.0/32 is subnetted, 1 subnets
O IA 100.100.100.100 [5/11] via 23.23.23.2, 00:00:06, FastEthernet0/0
33.0.0.0/24 is subnetted, 1 subnets
C 33.33.33.0 is directly connected, Loopback3
23.0.0.0/30 is subnetted, 1 subnets
C 23.23.23.0 is directly connected, FastEthernet0/0
22.0.0.0/24 is subnetted, 1 subnets
C 22.22.22.0 is directly connected, Loopback2
110.0.0.0/24 is subnetted, 1 subnets
O IA 110.110.110.0 [5/11] via 23.23.23.2, 00:00:06, FastEthernet0/0
10.0.0.0/24 is subnetted, 1 subnets
C 10.10.10.0 is directly connected, Loopback0
11.0.0.0/24 is subnetted, 1 subnets
C 11.11.11.0 is directly connected, Loopback1
120.0.0.0/24 is subnetted, 1 subnets
O IA 120.120.120.0 [5/11] via 23.23.23.2, 00:00:08, FastEthernet0/0
Sh ip route output after:
Gateway of last resort is not set
100.0.0.0/32 is subnetted, 1 subnets
D 100.100.100.100 [90/409600] via 23.23.23.2, 00:02:21, FastEthernet0/0
33.0.0.0/24 is subnetted, 1 subnets
C 33.33.33.0 is directly connected, Loopback3
23.0.0.0/30 is subnetted, 1 subnets
C 23.23.23.0 is directly connected, FastEthernet0/0
22.0.0.0/24 is subnetted, 1 subnets
C 22.22.22.0 is directly connected, Loopback2
110.0.0.0/24 is subnetted, 1 subnets
D 110.110.110.0 [90/409600] via 23.23.23.2, 00:02:21, FastEthernet0/0
10.0.0.0/24 is subnetted, 1 subnets
C 10.10.10.0 is directly connected, Loopback0
11.0.0.0/24 is subnetted, 1 subnets
C 11.11.11.0 is directly connected, Loopback1
120.0.0.0/24 is subnetted, 1 subnets
D 120.120.120.0 [90/409600] via 23.23.23.2, 00:02:22, FastEthernet0/0
R3 fa0/0 is still receiving hello packets fro R3 fa0/0.. Since R3 is the DR router but there is no response from the other end..
Subscribe to:
Posts (Atom)