Transparent Ethernet over an IP network

In today's corporate networks the layer-2 domain is collapsed down to the access layer - at least it should be, in most cases -, and there is virtually no requirement for a contiguous layer-2 domain passed the access layer switches.

However there are corner cases where extending the layer-2 domain across a layer-3 network is a requirement. For example server visualization may require the storage network to be on the same layer-2 domain. Or when a layer-2 interconnect circuit is backed up by a cheap(er) layer-3 circuit. 

I'll discuss various technologies to accomplish transparent Ethernet over an IP network and present sample configurations, for them.

In the topology we will be using for the sample configurations are two routers (i.e R1 and R2) connected back-to-back. R1 will have a loopback interface configured with the IP 1.1.1.1 and R2 will have a loopback with the IP 2.2.2.2. The physical back-to-back interfaces are configured on the 10.0.0.0/30 subnet, R1 has the IP 10.0.0.1 and R2 - how surprising - has 10.0.0.2. The routing is done through a static route, pointing to the IP address of other router's physical interface.

GRE Tunnel

Generic Routing Encapsulation (GRE) is a mechanism for encapsulating any network layer protocol over any other network layer protocol, developed by Cisco. The general specification is described in RFC 2784 (obsoletes earlier RFC 1701 and RFC 1702). 

Configuration example:

R1:

!

bridge irb

!

interface Loopback0

 ip address 1.1.1.1 255.255.255.255

!

interface FastEthernet0/0

 no ip address

 bridge-group 1

!

interface Tunnel0

 no ip address

 tunnel source Loopback0

 tunnel destination 2.2.2.2

 bridge-group 1

!

interface Serial0/0

 ip address 10.0.0.1 255.255.255.252

!

ip route 2.2.2.2 255.255.255.255 10.0.0.2

!

R2:

!
bridge irb
!
interface Loopback0
 ip address 2.2.2.2 255.255.255.255
!
interface FastEthernet0/0
 no ip address
 bridge-group 1
!
interface Tunnel0
 no ip address
 tunnel source Loopback0
 tunnel destination 1.1.1.1
 bridge-group 1
!
interface Serial0/0
 ip address 10.0.0.2 255.255.255.252
!
ip route 1.1.1.1 255.255.255.255 10.0.0.1
!

considerations

• Supports Dot1Q, and ISL VLAN trunking, depending on the routing platform.
• Integrated Routing and Bridging does NOT support transparent tunneling of CDP, STP, VTP, LLDP, PAgP, LACP, and UDLD.
• The MTU of the sending device may need to be adjusted, Integrated Routing and Bridging does NOT support fragmentation.
• Native support for tunneling over an IPv6 network.

More information about Generic Routing Encapsulation on the Cisco website can be found here.

DLSw+ circuit

Data-Link Switching is an almost forgotten technology, originally developed to tunnel non-routable, non-IP based protocols such as IBM Systems Network Architecture (SNA) and NetBIOS/NetBEUI across IP networks, described in RFC 1434 (updated by RFC 2166). 

DLSw+ is Cisco's proprietary implementation of DLSw, with following extensions:

• Choice of transport option, including TCP, FST (fast-sequenced transport), and direct encapsulation.
• Scalability enhancements through: peer groups, on-demand peers, explorer firewalls, and location learning.
• Media conversion between local and remote LANs and SDLC or Ethernet.

One of the more popular uses of DLSw+, was bridging Token ring networks over other media. 

DLSw+ is a flexible, though dated technology that can transport of Ethernet frames across an IP network.

Configuration example:


R1:

!
bridge irb
!
dlsw local-peer peer-id 1.1.1.1
dlsw remote-peer 0 tcp 2.2.2.2
dlsw bridge-group 1
!

interface Loopback0

 ip address 1.1.1.1 255.255.255.255

!

interface FastEthernet0/0

 no ip address

 bridge-group 1

!

interface Serial0/0

 ip address 10.0.0.1 255.255.255.252

!

ip route 2.2.2.2 255.255.255.255 10.0.0.2

!

R2:

!
bridge irb
!
dlsw local-peer peer-id 2.2.2.2
dlsw remote-peer 0 tcp 1.1.1.1
dlsw bridge-group 1
!

interface Loopback0

 ip address 2.2.2.2 255.255.255.255

!

interface FastEthernet0/0

 no ip address

 bridge-group 1

!

interface Serial0/0

 ip address 10.0.0.2 255.255.255.252

!

ip route 1.1.1.1 255.255.255.255 10.0.0.1

!

considerations

• Supports Dot1Q, and ISL VLAN trunking, depending on the routing platform.
• Integrated Routing and Bridging does NOT support transparent tunneling of CDP, STP, VTP, LLDP, PAgP, LACP, and UDLD.
• The MTU of the sending device may need to be adjusted, Integrated Routing and Bridging does NOT support fragmentation.
• NO native support for tunneling over an IPv6 network.
• DLSw+ is process switched.
• NO support for DLSw+, on any Cisco IOS XE, IOS RX, or NX-OS platform.
• Support for DLSw+ is only available on legacy Multi-Layer Switches, for example the CAT4500, and the CAT6000.

More information about Data-Link Switching (DLSw) and Data-Link Switching Plus (DLSw+) on the Cisco website can be found here.

L2TP Tunnel

Layer-2 Tunneling Protocol, described in RFC 2661, is based on primarily two older tunneling protocols for Point-to-Point communication: Cisco's Layer 2 Forwarding Protocol (L2F) and USRobotics Point-to-Point Tunneling Protocol (PPTP). The original L2TP implementation uses UDP for transport. The newer L2TP version 3, described in RFC 3931 (updated by RFC 5641), uses IP as transport - just like GRE -, that can be implemented on any IP-aware network. L2TPv3 has two 'modes' point-to-point and point-to-multipoint. We will be focusing exclusively on L2TPv3 point-to-point mode tunneling.

Configuration example:

R1:

!

pseudowire-class TEST

 encapsulation l2tpv3

 ip local interface Loopback0

!

interface Loopback0

 ip address 1.1.1.1 255.255.255.255

!

interface FastEthernet0/0

 no ip address

 no cdp enable

 xconnect 2.2.2.2 1 encapsulation l2tpv3 pw-class TEST

!

interface Serial0/0

 ip address 10.0.0.1 255.255.255.252

!

ip route 2.2.2.2 255.255.255.255 10.0.0.2

!

R2:

!

pseudowire-class TEST

 encapsulation l2tpv3

 ip local interface Loopback0

!

interface Loopback0

 ip address 2.2.2.2 255.255.255.255

!

interface FastEthernet0/0

 no ip address

 no cdp enable

 xconnect 1.1.1.1 1 encapsulation l2tpv3 pw-class TEST

!

interface Serial0/0

 ip address 10.0.0.2 255.255.255.252

!

ip route 1.1.1.1 255.255.255.255 10.0.0.1

!

considerations

• Cisco Express Forwarding must be enabled. The xconnect configuration mode is inaccessible until CEF is enabled.
• Native support for tunneling over an IPv6 network.
• The MTU of the sending device may need to be adjusted, to prevent fragmentation. The ingress router receives an Ethernet frame and encapsulates the packet by removing the preamble, the start of frame delimiter (SFD), and the frame check sequence (FCS), inserts the frame into a L2TPv3 packet and transmits the packet to its peer.
• You may need to configure the l2protocol-tunnel command to enable or disable tunneling of Layer 2 protocols (e.i. CDP, STP, VTP, LLDP, PAgP, LACP, and UDLD) depending on the platform. In my test environment (GNS3, c3725, IOS 124-15.T8) CDP was tunneled transparently.

More information about Layer Two Tunnel Protocol (L2TP) on the Cisco website can be found here.

EoMPLS over GRE Tunnel

Ethernet over Multi Protocol Label Switching is a mechanism to encapsulate and transport Ethernet Protocol Data Units (PDUs) over an MPLS network, and is described in RFC 4448 (updated by RFC 5462). Basically a frame is received on a switch, MPLS labels the frame, transportes it via MPLS labels over an IP routed network, and the receiving switch removes the MPLS label, and switches the frame out the appropriate port(s).

EoMPLS is the most common method Internet Service Providers use to implement Virtual Private Wire Service (VPWS) and Virtual Private LAN Service (VPLS), on their networks.

MPLS requires every hop in the path to be an MPLS router. Here is where the GRE Tunnel comes in to play. We will use GRE to tunnel the MPLS traffic, end-to-end.

Configuration example:

R1:

!
ip cef
!
mpls ip 
!
interface Tunnel0
 ip address 172.16.1.1 255.255.255.252
 mpls ip
 tunnel source Serial1/0
 tunnel destination 192.168.100.4
!
interface FastEthernet0/0
 no ip address
 no cdp enable
 xconnect 172.16.1.2 1 encapsulation mpls
!
interface Serial1/0
 ip address 192.168.100.1 255.255.255.0
!
mpls ldp router-id Tunnel0 force
!

R2:

!
ip cef
!
mpls ip 
!
interface Tunnel0
 ip address 172.16.1.2 255.255.255.252
 mpls ip
 tunnel source Serial1/0
 tunnel destination 192.168.100.1
!
interface FastEthernet0/0
 no ip address
 no cdp enable
 xconnect 172.16.1.1 1 encapsulation mpls
!
interface Serial1/0
 ip address 192.168.100.4 255.255.255.0
!
mpls ldp router-id Tunnel0 force

!

NOTE: In the configuration above I used the 172.16.1.0/30 IP address range on the tunnel interface. Also the Tunnel interface on the platform used defaults to tunnel mode gre ip.

considerations

• Cisco Express Forwarding must be enabled. The xconnect configuration mode is inaccessible until CEF is enabled.
• Native support for tunneling over an IPv6 network.
• The MTU of the sending device may need to be adjusted, to prevent fragmentation. The ingress router receives an Ethernet frame and encapsulates the packet by removing the preamble, the start of frame delimiter (SFD), and the frame check sequence (FCS), inserts the frame into a MPLS packet and transmits the packet to its peer.
• You may need to configure the l2protocol-tunnel command to enable or disable tunneling of Layer 2 protocols (e.i. CDP, STP, VTP, LLDP, PAgP, LACP, and UDLD) depending on the platform. In my test environment (GNS3, c7200, IOS 150-1.M) CDP was tunneled transparently.

More information about Multi Protocol Label Switching (MPLS) on the Cisco website can be found here.

Conclusion

Both the 'vanilla' GRE Tunnel and the DLSw+ circuit use the Integrated Routing and Bridging feature to span the Ethernet across the layer-3 network. And therefore these two methods carry the same limitations as a single router bridging two layer-2 domains, using the IRB feature. You can either bridge a normal Ethernet connection, a single VLAN or a Trunk, per bridge group. Since the router is not a switch it does not support DTP or VTP, nor will it transparently carry those and other layer-2 protocols (e.g. CDP, STP) - since the router is a layer-2 hop, just like a switch.

For more information about Integrated Routing and Bridging (IRB) see the Cisco website here.

As mentioned earlier DLSw+ in not supported on any of the Cisco IOS XE, IOS RX, or NX-OS platforms. Also the protocol is process switched which may require extensive tuning, you can find an excellent post about tuning DLSw performance here, on the Packetpushers website.

Of all the methods described, L2TPv3 and EoMPLS over GRE are the most powerful and flexible, providing truly transparent Ethernet, allowing you to implement it as a trunk or per VLAN. The per VLAN method allows you to transport different VLANs to different endpoints, or even mix layer-2 and layer-3 features on a single port, using sub-interfaces. Big downsides are - yes, there are downsides - that It also may also tempt you as a network engineer to be too clever; creating a real unholy mess of layer-2 over layer-3 confusion, which is not fun to troubleshoot.