Carrier Supporting Carrier Architectures

Going through the CSC questions in IEWB1 I noticed that I didn’t really understand the difference between the three types.. I guess just like the Inter-AS options the lines between the different architectures have blurred. In summary the CSC options are;

#1 (IP) – MPLS is only used on PE/CE (of the supporting carrier)
#2 (MPLS) – MPLS is used between the supporting-carrier PE and supported-carrier PE
#3 (HVPN) – MPLS is used right up to the supported carrier’s PE and CE

I guess once you get to hierarchical VPN it is actually CSC#3 with sub option #1 or #2.. I guess the method you use really depends on what kind of traffic the lowest tier wants to send over the backbone.

The first option simply means that the service provider does not need to peer with the customer using BGP (useful for customers who want the whole internet in their VRF). The second allows the supported carrier to deploy any MPLS application over the supported carrier’s backbone (AToM, VPN, IPv6 etc). The third option just means that the supported carrier can hierarchically support another carrier for either of the previous.

Looking closer at OSPF Sham-links

I was already aware what Sham-links were supposed to be for and might have even configured them once or twice. It does seem there are a few things that can go wrong. I noticed a lot of this doing one of the IEWB1 labs. Some of the following is pretty obvious and some I do not recall seeing mentioned anywhere;

• The addresses used in the sham-link command must be internal to the VRF of that OSPF instance
• A sham-link cannot be established if the source/destination address’s next-hop is not via the MPLS Cloud (the address is learned over the PE-CE link through the backdoor link)
• Default of behavior for MPLS VPNs when a PE redistributes BGP into OSPF is to advertise the LSAs with type 3. As the routes are redistributed into BGP at the ingress PE the advertising router ID is changed to self by the PE. If a sham-link exists however the LSAs with the same advertising router ID will not be propogated as LSA type 1-2.

The third point there is what I found most interesting. Of course you wouldn’t be able to have a backdoor topology using the MPLS VPN as the primary link without this extra little feature of sham-links. This is because the first attribute which OSPF uses to choose the best route is LSA type.

O > O IA > O Ex

And here’s the proof

R6#sh ip ro
10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks
O IA    10.1.8.8/32 [110/12] via 10.1.46.4, 00:03:27, GigabitEthernet0/0
C       10.1.6.6/32 is directly connected, Loopback0
O E2    10.1.4.4/32 [110/1] via 10.1.46.4, 00:03:27, GigabitEthernet0/0
O E2    10.1.3.3/32 [110/1] via 10.1.46.4, 00:03:27, GigabitEthernet0/0
C       10.1.46.0/24 is directly connected, GigabitEthernet0/0
O IA    10.1.38.0/24 [110/2] via 10.1.46.4, 00:03:27, GigabitEthernet0/0
O IA    10.1.68.0/24 [110/65546] via 10.1.46.4, 00:03:27, GigabitEthernet0/0

Then add the sham-link (the other side was already done)

R4(config-router)# area 0 sham-link 10.1.4.4 10.1.3.3 cost 1000
*Mar  1 01:36:20.811: %OSPF-5-ADJCHG: Process 68, Nbr 10.1.38.3 on OSPF_SL0 from LOADING to FULL, Loading Done

Looking at the same routing table you can see all the same routes are still learnt via the same neighbor yet the LSA type has changed

R6#sh ip ro
10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks
O       10.1.8.8/32 [110/1012] via 10.1.46.4, 00:00:06, GigabitEthernet0/0
C       10.1.6.6/32 is directly connected, Loopback0
O E2    10.1.4.4/32 [110/1] via 10.1.46.4, 00:00:06, GigabitEthernet0/0
O E2    10.1.3.3/32 [110/1] via 10.1.46.4, 00:00:06, GigabitEthernet0/0
C       10.1.46.0/24 is directly connected, GigabitEthernet0/0
O       10.1.38.0/24 [110/1011] via 10.1.46.4, 00:00:06, GigabitEthernet0/0
O       10.1.68.0/24 [110/66546] via 10.1.46.4, 00:00:06, GigabitEthernet0/0

One thing which can make this simple config go horribly wrong is the sham-link addresses. You can see in my example they are external for the CE router, this just happened to be the way the question asked for the task to be done.

It was not a requirement for the PE loopbacks to be in OSPF area 0. If however it was then the difficulty increases very quickly just to make the sham-link come up (due to bullet point two above). You would potentially have to fight the weight and administrative distance to make it work.

Bottom line: if the question doesn’t require you to have the PE addresses in OSPF area 0 then don’t! This will save a lot of time tweaking route-maps and other things..

Slow progress over Christmas break

I’ve already booked my lab for 21/06/2010 (the earliest date I could get). Shame it is so far away I think I could attempt it a lot earlier..

Anyway just got through “IPExpert – Volume I” and now will be moving onto “Internetwork Expert Volume I”.

Fun stuff!

FRoMPLS Lab

Was doing an IPExpert lab for L2VPN and it was all L2TP which was a little disappointing. I guess it was all for the best because I had never configured a non-VPDN application of L2TP. It’s usually extremely easy to implement because the configs are basically the same as for AToM, what I did find difficult was getting traffic from a frame-relay DLCI onto my AToM tunnel.

interface Serial1/0
encapsulation frame-relay
frame-relay interface-dlci 100 switched

connect DLCI100-FRoMPLS Serial1/0 100 l2transport
xconnect 10.0.0.2 123 encapsulation mpls

This seems to be the only way you can implement FRoMPLS on the given IOS in the lab, I thought it was worth mentioning because it is not easy to guess if you haven’t seen it before due to the connect command. You can actually check out the configuration guides for an AToM application and pretty much duplicate the config on L2TP except you might need a pwclass with “ip local interface []“.

1. http://www.cisco.com/cisco/web/psa/default.html?mode=prod
2. Product > IOS > 12.2 > 12.2T
3. Feature Guides > AToM > Configuring

I don’t know whether it actually matters in the lab but IPExpert seem to love best practices stuff like if you provision a L2VPN you should increase the MPLS MTU in the core to allow the customer reach their full MTU for the given medium.

The topology above is what I did this with, the potential for MPLS to enhance the scalability of a service provider who sells a lot of frame-relay is just amazing. In this lab the idea is that you don’t necaserily have administration over the frame-relay switches so you can use MPLS or L2TP to solve the problem by placing a pseudowire between the two serial interfaces of the PEs.

It really is a shame that frame-relay just isn’t that popular anymore, makes learning a lot of this pointless…

Typical Cisco tactics – default label protocols

Was doing a lab for FRoMPLS and thought it would be a good idea to have either PE as a different platform. I chose the “C3640-JSX-M” and “C7200-K91P-M” which are both images listed for the lab. When doing the basic setup for mpls during the verification I saw OSPF as up but LDP was not, even after some time;

PE1-7206#show ip ospf neighbor

Neighbor ID     Pri   State           Dead Time   Address         Interface
10.0.0.2          1   FULL/DR         00:00:32    10.0.12.2       FastEthernet0/0
PE1-7206#show mpls ldp neighbor

PE1-7206#show mpls ldp discovery
Local LDP Identifier:
10.0.0.1:0
Discovery Sources:
Interfaces:
FastEthernet0/0 (ldp): xmit

You can see from the above verification that yes MPLS is turned on for the interface for which the OSPF neighbor is on. The same output was shown on the alternate PE with one minor difference;

PE2-3640#show mpls ldp discovery
Local LDP Identifier:
10.0.0.2:0
Discovery Sources:
Interfaces:
FastEthernet0/0 (tdp): xmit

Although for show commands it doesn’t seem to matter whether you use the mpls or tag-switching version it does for the actual communication over the link. This is due to the fact that they use different ports for communication (646 / 711 for LDP / TDP respectivly). A debug reveals that the packets actually use the same source and destination port.

PE2-3640#debug ip packet detail
IP packet debugging is on (detailed)
*Mar  1 00:15:24.315: IP: s=10.0.12.2 (local), d=255.255.255.255 (FastEthernet0/0), len 48, sending broad/multicast
*Mar  1 00:15:24.319:     UDP src=711, dst=711

This is important for two reasons, one if you are configuring access-lists the best practice answer would be “permit udp any eq 711 any eq 711″ and two similar to BGP a router will not even listen on a given port unless the port is opened by turning the protocol on.

*Mar  1 00:19:51.527: IP: s=10.0.12.1 (FastEthernet0/0), d=224.0.0.5, len 80, rcvd 0, proto=89
PE2-3640(config)#mpls label protocol ldp
*Mar  1 00:19:54.591: IP: s=10.0.12.1 (FastEthernet0/0), d=224.0.0.2, len 62, rcvd 0
*Mar  1 00:19:54.591:     UDP src=646, dst=646

From the above it can be seen that until UDP 646 is opened by changing the label protocol to LDP (which is non-default for this IOS/platform), the packets are not even inspected at layer four (it seems).

All in all, I will be watching out for this one as it is an easy way to kill your score if you don’t verify as you go. I hope someone reads this and it does the same for them some day!

Success feels good

Just got back from my second attempt (free retake) at 350-029 and I’ve passed 839/1000! I am still extremely irritated by the questions they put in it, quite shocking.. but I won’t have to do another one for two years anyway.

Now back onto the lab prep, at the moment I’m thinking to aim for mid next year..

Looking deeper into OSPF as a PE-CE protocol

Default behavior of redistribution on VRF aware OSPF will lead to the MPLS VPN cloud trying to emulate OSPF. This is done using a “Super Backbone” which is hierarchically above area zero. RFC4577 explains in great detail how this should ideally work.

The whole point of this behavior is to mask the VPNv4 BGP from the customer networks. Traditionally if you redistribute any protocol into OSPF it will be created as an LSA type 5 (external). In Cisco’s implementation all LSAs of type 1-3 will be advertised out the egress PE(s) as type 3.

Given the topology below assume that the link from CE1-CE2 is not active (for the moment). All Ethernet links have been set to a network type of point-to-point for simplification.

Looking at the link state database we can see the summary prefixes (type 3) all have the router ID of the nearby PE. This means that the “Super Backbone” is working, converting area zero LSA type 1 into type 3 at the egress PE. It is important to make sure that the process IDs match between PEs. The process ID configured on the PE is attached in the form of an extended community, under the ospf process you can manually set the value to be attached to the ext-community;

OSPF DOMAIN ID:0×0005:0×0000007B0200

router ospf 123 vrf cust
domain-id type 0005 value 000000000123

OSPF DOMAIN ID:0×0005:0×000000000123

An issue which can occur when implementing OSPF as the PE-CE protocol, which I have seen first hand in a production network is that the “Super Backbone” cannot be treated as a transit network as it would in a layer two VPN. I can imagine that enterprises and even the people who design/architect for enterprises (not trying to have a dig a CCIE RS holders) do not take into consideration how OSPF MPLS VPNs actually work. You cannot just make the MPLS VPN cloud a single area, even if all the PE-CE links are in area zero all LSAs will be type three after they have crossed the cloud.

CE1#show ip ospf 123 database router adv 172.0.0.2

OSPF Router with ID (172.0.0.1) (Process ID 123)
Router Link States (Area 0)

Adv Router is not-reachable
LS age: 481
Options: (No TOS-capability, DC)
LS Type: Router Links
Link State ID: 172.0.0.2
Advertising Router: 172.0.0.2
LS Seq Number: 80000004
Checksum: 0xD09A
Length: 60
Number of Links: 3

The LSAs (from CE2) are inside the database (on CE1) but they will not ever be able to be used for forwarding because the topology has broken the “Area 0 must be contiguous” rule. CE2 is in fact sending LSA type 1 for both Area0 and Area1 but all LSAs from Area0 are marked “Adv Router is not reachable” above.

Another thing worth noting is that in regular OSPF the “advertising router” for an LSA stays the same within an area and is changed at the border (by an ASBR/ABR). This is consistant with the “ever router must know every other router in the area” concept. This is also applied to the Superbackbone. The “advertising router” for an LSA will be changed as it is advertised out the egress PE. This combined with some of the above shows how the Superbackbone truely does act as an area hierachically above area zero.

GNS3 .net file (rename to *.rar)

Score report: Me 2, Cisco 0

I’m not going to post too many details because I don’t want to violate the NDA or upset anyone at Cisco but basically after my first attempt on the 350-029 exam I failed by one question and due to several items being incorrect in the exam I logged a case with Cisco Certsupport.

They had the “Exam Content Owner” look at questions I had left comments and after seeing this they offered me a retake voucher.

By the way the score is 2:0 because there was another incident where questions in the test exams on the Cisco Learning Network were incorrect and I pointed this out and the content owner agreed.

Configuring ATM switches in Dynamips

Really getting into the layer two technology labs as my big weakness is stuff like ATM SVCs of which there is even some in the IPExpert focus labs. Since no one can afford to by an LS1010 and the respective interfaces for devices this leaves us with Dynamips. I have been struggling with the ATM switches in Dynamips and just assumed that it was bugged but I have just realized the error of my ways.

Now I am one of those people whose mother always said “Don’t do that, read the instructions first!”. The person who developed GNS has the VPI/VCI backwards. The letters P and C stand for Path and Channel. The idea being a single path can carry multiple channels. Even in the ATM header it is VPI/VCI. Now the default settings don’t even allow you to change the VCI. You must first check the “Use VCI” box. I find this utterly ridiculous and I strongly doubt that people who use GNS3 primarily use ATM switches to make PVPs between devices.. if it is even capable of that – LAME!

IPExpert Workbook Volume 1 (Focus Labs)

Although I haven’t actually passed my written I do not desire forking our another $411 until Cisco certsupport tell me to bugger off. For this reason I decided I might as well start lab study as I am just wasting time. I have decided to go with IPExpert as I have heard it is more complete.

They give you GNS3 files which don’t work. I did manage to modify them and save myself some time. Basically I had to add an extra two hypervisors (totaling four) and spent about an hour tweaking the idlepc values. It doesn’t seem to matter whether you pick a low number, a high number or something with an asterisk. You just have to keep changing it until your screen looks like this;

My PC specs are;
2.85Ghz Quad Processor
4GB RAM

At this point you can yell at your monitor “ALL THUNDERBIRDS ARE GO!!” or something similar