WHITE

PAPER | JULY 2015

COMPARING HYBRID ACCESS NETWORKS SOLUTIONS

ABSTRACT

To cope with growing user appetite for bandwidth, telecommunication network operators are exploring various
solutions to combine different access network technologies together. This will pave the way for hybrid access
networks, i.e. access networks that combine very different technologies such as DSL and LTE. In this white paper, we
compare different technical solutions that can be used to create such hybrid access networks. In particular, we
compare the benefits of the Multipath TCP-based solutions to network-layer solutions. Furthermore, our
measurements done on three widely deployed DSL access routers show that a software-based solution using Multipath
TCP can achieve high bandwidth on existing devices.

 TESSARES WHITE PAPER
July 2015

Introduction
Network operators have invested a lot to deploy their broadband and mobile access networks during the last decade.
Although the performance of these networks has continued to improve, the user and application needs have also
continued to grow. This has lead to users demanding higher bandwidth while in several countries governments are
pushing to ensure that a minimum broadband performance is available for all users throughout the entire country. In
cities and densely populated areas, deploying high bandwidth networks is both feasible and cost effective. However, in
rural areas and less densely populated regions, the cost of high bandwidth networks may be much higher than the
revenue the network operators can collect from the users.

To cope with this problem, many network operators are investigating Hybrid networks to provide higher bandwidth in
some parts of their footprint by combining two very different network access technologies: fixed network (typically
xDSL) and cellular network (typically 3G or 4G wireless network technologies). Hybrid access networks seem rather
straightforward at first glance simply equip the home DSL router with a cellular antenna to enable it to connect to
both networks at the same time. This is illustrated in the figure below.

Fixed&
Fixed&
network
network

Cellular&
Mobile&
network

Figure 1: Hybrid access on DSL router

Unfortunately, in reality such a Hybrid access network is not only a hardware challenge, i.e. equipping a DSL router with
an antenna. It is mainly a software challenge because the DSL router (also called HCPE Hybrid CPE) must include
software that enables it to efficiently send and receive information over any of the connected networks.

There are several possible solutions to solving this important problem. To understand the pros and cons of these
solutions it is useful to go back to networking basics. For this we refer to the five layer reference model shown in most
networking textbooks. We will use it as our guideline to analyse in which layer a Hybrid access network solution would
best fit.

layers"

5" Applica)on" Browsing,"le"transfer,"email,"voice,"video,"etc"

4" Transport" TCP,"UDP,"etc"

3" Network" IPv4,"IPv6,"etc""

2" Data"Link"
Fixed:"DSL,"Cable,"Fiber,"etc"
Cellular:"WiFi,"WiMAX,"3G,"4G/LTE,"etc"
1" Physical"

Figure 2: The 5 layer reference model for networks

www.tessares.net COMPARING HYBRID ACCESS NETWORKS SOLUTIONS 2/8

 TESSARES WHITE PAPER
July 2015

The physical layer
The bottom layer is the physical layer. It deals with the physical transmission of data and delivers a stream of bits.
There has been a lot of innovation in this layer over the years enabling high bandwidth over DSL and wireless links. If
the two links to be combined use the same link layer technology, e.g. VDSL2, then a pure physical layer solution is
feasible and some products already support this. However, if the two links use different physical layer technologies,
then it becomes impossible to combine them using a pure physical layer solution due to the incompatibilities between
the physical layers used by, for example, DSL and a cellular network such as 4G/LTE.

The datalink layer

This layer includes important networking technologies such Ethernet, WiFi or PPP. The problems of grouping two links
using the same datalink layer are well known and several solutions exist and have been deployed. In Ethernet
networks, various link bonding techniques group several Ethernet links together. These solutions are typically used
between switches, but sometimes on server interfaces. Multilink PPP is another example of a technology that has been
used to combine different serial links such as dialup modems or ISDN channels.

For the hybrid access network, the drawback with these datalink layer solutions is that the most frequent candidates
for hybrid access networks, namely LTE and DSL, do not use the same link layer technology. In theory, it could be
possible to create a (Multilink) PPP session over the DSL link and a (Multilink) PPP session over the LTE network and
terminate them in the core network. Unfortunately, this is difficult to implement because PPP was mainly targeted at
low-speed networks and has not been optimised for higher bandwidth links. Furthermore, terminating the PPP session
in the wireless network would likely require running the PPP sessions over a tunnel such as GRE over IP, which would
lead to a large overhead.

The network layer

The network layer contains two key protocols in todays Internet: IPv4 and IPv6

Given the scarcity of IPv4 addresses, operators deploy IPv4 by allocating at most one address per DSL user and then rely
on Network Address Translation (NAT) on the DSL router to enable all hosts connected in the home to share the same
IP address. Each host in the home uses a private address, which is translated by the NAT.

With IPv6, many operators opt for a different deployment. Since an operator can easily obtain a large block of public
IPv6 addresses, it can delegate a small block of addresses to each home router. The benefit of IPv6 from an
architectural viewpoint is that each home user has a unique IPv6 address and the home router does not need to include
any NAT function.

www.tessares.net COMPARING HYBRID ACCESS NETWORKS SOLUTIONS 3/8

 TESSARES WHITE PAPER
July 2015

Two families of hybrid access network solutions are possible in the network layer:

Stand-alone Hybrid CPE

Hybrid CPE combined with an Hybrid Access Gateway in the operators network

The stand-alone Hybrid CPE is simply a router equipped with two different access links (WAN) as shown in the figure
below.

LAN$ WAN$

WiFi$ LTE$
host$ HCPE$ @$
Ethernet$ DSL$

Figure 3: Hybrid CPE DSL + LTE

In practice, this simple scenario does not allow a host to efficiently use the available networks. The main problem is the
IP address that is used by the hosts in the home (LAN). With IPv4, they use a private IP address with one address
allocated to the LTE link and one address to the DSL link. When the Hybrid CPE receives a packet from a host in the
home network, it must translate its private IP address into either the address of the DSL link or the address of the LTE
link. If the router wants to send packets over both the LTE and the DSL link, it must translate their addresses
intelligently. It cannot simply send the even packets over the DSL link and the odd packets over the LTE interface. This
would break many TCP connections. Instead, the Hybrid CPE must maintain per-flow state and send all the packets that
belong to a given flow (e.g. a TCP connection/session or a UDP flow) over the same interface. This implies that one
flow can only use one interface. Furthermore, for some applications, maintaining per-flow state is not sufficient. For
example, an online banking application often opens several TCP connections. If these connections originate from
different addresses (e.g. one from the DSL link and one form the LTE link), many online banks will suspect a security
problem and will block the communication. For these applications, it is important to send all flows over the same
interface. Unfortunately, it is not easy to know in advance which applications perform such security checks. In addition,
if one interface breaks, it is impossible with this solution to continue the existing flows since they are bound to one
address that is now unreachable. When a link fails, all established flows have to be re-established by the applications.
This could affect applications like some SSL VPNs that use long-lived connections.

When IPv6 is used, the situation is different. The host will have one IPv6 address assigned by the Hybrid CPE that acts
as a regular router. The Hybrid CPE receives this prefix from either the DSL or the LTE network. Lets assume for
simplicity that the prefix is assigned through the DSL network. The Hybrid CPE will receive another IPv6 address from
the LTE network. When sending a packet from the host, the router will forward the packet over the DSL interface
because security reasons prevent it from sending a packet over the LTE network whose source address was not
assigned by the LTE network. This implies that it will be difficult to use the LTE interface. An alternative would be to
assign an IPv6 prefix to the DSL router and advertise, e.g. through routing protocols, this prefix over both the LTE and
the DSL network. This solution would work in a lab setting and would allow packets to be sent over both the LTE and
the DSL network, but injecting one route for each customer in both the LTE and the DSL networks is not a scalable
solution that can support a large number of users.

www.tessares.net COMPARING HYBRID ACCESS NETWORKS SOLUTIONS 4/8

 TESSARES WHITE PAPER
July 2015

The second family of network-layer solutions combines a Hybrid CPE (HCPE) with a Hybrid Access Gateway (HAG)
installed in the operators network. Their main objective is to hide to the DSL and LTE networks the IP addresses used
by the endhosts. The reference network is described in the figure below.

LAN$ WAN$

WiFi$ LTE$
host$ HCPE$ HAG$ @$
Ethernet$ DSL$

Figure 4: HCPE and Hybrid Access Gateway

Several scenarios are possible with this deployment. A simple but realistic scenario is to use tunnels (e.g. GRE) on the
HCPE. Let us assume that this HCPE has two addresses: A1 and A2. A1 is the IP address assigned on the DSL link while
A2 is the address assigned on the LTE interface. Let us now analyse what happens when host H sends many packets
towards destination S. The first packet is received by the HCPE which decides to forward it over the DSL interface. For
this, the packet must be translated to use the A1 source address. Upon reception of this packet, the destination
associates the flow to source address A1. Now, let us consider how the second packet sent by the host will be
forwarded by the HCPE. There are three possibilities:

1. The HCPE sends the packet with source address A1 over the DSL interface. The packet reaches the destination,
but only one path is used for this flow.
2. The HCPE sends the packet with source address A1 over the LTE interface. This is the simplistic approach that
many users would expect. Unfortunately, it does not work because the HCPE cannot send a packet over the
LTE interface whose source address does not match the address allocated to this interface. This restriction is a
best current practice of source address filtering that is required for obvious security reasons.
3. The HCPE takes the packet sent by the host, translates the source address into address A1 (the same as for the
initial packet) and sends the packet over the LTE interface inside a packet whose source address is A2 and
destination address is the HAG. The HAG knows that the HCPE has two different addresses and when it
receives the encapsulated packet, it removes the outer encapsulation and forwards the packet (with source
address A1) towards the final destination.

The main benefit of the third approach is that the final destination does not know that the HCPE and the HAG are using
encapsulation to use two different links. All the packets received by the final destination appear to have originated
from address A1.

Thanks to encapsulation, the HCPE can use the two available interfaces. However, this is usually not sufficient to
achieve a good user experience. The DSL and LTE links have different delays and different bandwidth. Furthermore,
the LTE network is a shared network used by a varying number of users. This implies that the bandwidth of the LTE link
varies dynamically over time. In this context, let us now consider a sequence of packets that are sent over the two
interfaces. Given the variability of the bandwidth on the LTE network, it is difficult for the HCPE/HAG to estimate
accurately the fraction of the traffic that can be sent over the LTE network. In particular, for many applications most of
the traffic is sent from the HAG towards the HCPE, and while the HCPE may have some information about its current
allocated bandwidth, it is difficult for the HAG to track the bandwidth allocated to the thousands of HCPEs that it
serves. Given the delay difference, which also varies over time since it depends on the load on the LTE network, a
sequence of packets sent over the two interfaces will not reach the HAG or the HCPE in the same order. This is a major

www.tessares.net COMPARING HYBRID ACCESS NETWORKS SOLUTIONS 5/8

 TESSARES WHITE PAPER
July 2015

problem for TCP because many TCP implementations assume that out-of-order packets indicate losses or congestion
and react to such events by slowing down the transmission. If TCP is used through a HAG that reorders packets, it will
have difficulty in using all the available bandwidth. The only solution to cope with this problem is to introduce buffers
in the HAG and the HCPE to equalise the delays over the different paths, but this will require techniques to estimate the
varying delays and to reorder the packets. This obviously increases the complexity of the solution.

From a performance viewpoint, hiding the existence of the two paths to protocols such as TCP that carry more than
90% of the total Internet traffic is not a good approach. TCP includes a congestion control mechanism that continuously
monitors the quality of the available path and adapts its transmission rate accordingly. But if packets from a single
connection are sent over different paths, and thus experience different delays and different losses, TCP may over-react
since it cannot distinguish between the two paths and degrade performance.

The transport layer

On the Internet, the key transport protocols are UDP and TCP. TCP is used by more than 90% of the total Internet
traffic while UDP is mainly used by multimedia applications (VoIP, IPTV), games, DNS and some VPN services.

As explained in the network layer section, one of the main difficulties with network layer solutions is that they hide the
hybrid access networks with different characteristics to TCP and thus can cause interferences that will lower the
performance of TCP. However, in contrast, a recently standardised TCP extension called Multipath TCP is an ideal
protocol in such a network since it is capable of efficiently using different paths. Though Multipath TCP is not
implemented as standard on regular hosts or Internet servers, it is a possible to deploy it without waiting for these
hosts and servers to be upgraded by adding Multipath TCP functionalities on both the HCPE and the HAG.

LAN$ WAN$

WiFi$ LTE$
bonding$

host$
HCPE$
(MPTCP)$
HAG$
(MPTCP)$ @$
Ethernet$ DSL$

TCP$ MPTCP$ TCP$

Figure 5: MPTCP enabled HCPE & HAG

Let us consider in detail how a Multipath TCP-based solution can resolve the Hybrid access network bonding problem.
As before, assume that the HCPE has two addresses: A1 and A2 respectively assigned to the DSL and the LTE interfaces.
When a host creates a TCP connection, it sends a packet to the HCPE. The HCPE decides over which interface the first
packet of the TCP connection will be transported, e.g. the DSL interface. The packet header is translated by the HCPE
and appears as originating from address A1. In addition to translating the source address, the HCPE also adds
Multipath TCP options inside the packet. The HAG captures the packet before it reaches its final destination, notices
that this is a new Multipath TCP connection, removes the Multipath TCP option and sends it to the final destination.
The final destination replies and the reply goes through the HAG which inserts the Multipath TCP option before
forwarding it to the HCPE. The reply confirms the establishment of the connection to the HCPE. Data can then flow
over the DSL link. To use the LTE link to also forward data, the HCPE (respectively the HAG) creates a sub flow over the

www.tessares.net COMPARING HYBRID ACCESS NETWORKS SOLUTIONS 6/8

 TESSARES WHITE PAPER
July 2015

LTE interface. This is done by creating a TCP connection that is terminated on the HAG (respectively the HCPE). At this
point, data can flow over both the DSL and the LTE interfaces. The Multipath TCP congestion control scheme
automatically measures the performance of the two interfaces and adjusts the distribution of their load based on
performance criteria or by enforcing the operators defined policies. If one interface fails, the connection continues via
the other without breaking the data transfer.

Compared with network layer solutions, Multipath TCP brings two important benefits from a performance viewpoint.
First, the overhead of Multipath TCP (20 bytes per packet) is lower than the overhead of network layer solutions which
rely on encapsulation (e.g. 36 bytes for GRE for IPv4). More importantly, thanks to its subflows, Multipath TCP knows
over which access link each data item is transmitted. This enables both the HCPE and the HAG to continuously measure
the performance of these access links and distribute the data between them based on their real time performance. In
contrast, a network layer solution hides the existence of the two access links to the transport layer such that it cannot
then efficiently adjust its transmission rate. In the lab, this difference is not important because the bonded links have
stable capacity. In real networks, capacity fluctuates and the ability to efficiently adjust the transmission rate is key to
maximising performance and perceived quality of experience.

Multipath TCP has other benefits from a deployment viewpoint. Like other transport protocols, it is software based.
This implies that the required Multipath TCP functions can be added as a software upgrade to the CPEs that are already
deployed in the field.

In regard to the HCPE, operators may fear that Multipath TCP will not be able to achieve a good throughput due to the
low-end CPUs used on existing CPEs and for this reason prefer network layer solutions that are easier to accelerate with
existing hardware. However, measurements performed with the Tessares optimised Multipath TCP stack on three
different HCPEs based on the Broadcom BCM63168 chipset have demonstrated that Multipath TCP can achieve high
performance on widely deployed CPEs. The figure below shows the results of detailed experiments conducted while
bonding different links: DSL and LTE with different bandwidths and different delays. It shows that Multipath TCP bonds
the two links very efficiently without saturating the HCPEs CPU. Millions of such routers are deployed in existing
networks. These measurements show that operators could provide new services to their current users with just a
software upgrade.

Theore;cal"Aggregated"Downstream" Measured"Aggregated"Downstream"

120"

100"
Applica;on"goodput"[Mbps]"

Combined"
80" throughput"(TP)"
always">"90%"of"
the"theore;cal"
60" TP"(sum"of"DSL"TP"
and"LTE"TP)""
40"

Always">"95%"of""
20" the"sum"of"TCP"
goodputs"

0"
10Mbps"(DSL)"+"25Mbps"(LTE)" 30Mbps"(DSL)"+"25Mbps"(LTE)" 70Mbps"(DSL)"+"40Mbps"(LTE)"

Figure 6: DSL + LTE bonding with > 10ms delay difference

www.tessares.net COMPARING HYBRID ACCESS NETWORKS SOLUTIONS 7/8

 TESSARES WHITE PAPER
July 2015

Measurements with next generation HCPEs that are not yet deployed in the field show that they can achieve even
higher throughput (several hundred Mbps) without saturating their CPUs.

The application layer

Some researchers have tested application layer approaches to solve the hybrid link bonding challenge. These solutions
leverage Multipath TCP as well.

For web traffic, a first solution is to install a web proxy on the HCPE that uses Multipath TCP to interact with another
proxy installed in the operators network. This solution allows the use of two available links, but has two important
drawbacks. First, it only supports HTTP traffic. While HTTP is an important application nowadays, this is not the only
application layer protocol that is used by end-users. Therefore, restricting a solution to support only one application is
not a generic approach. Second, it is difficult to achieve high performance on existing DSL routers that usually have
limited CPU and memory to efficiently support a high performance web proxy.

A second solution is to rely on applications such as OpenVPN that allow tunnelling any type of traffic over TCP.
OpenVPN is a popular VPN application that was initially designed to operate over UDP. It can also operate over TCP,
e.g. when firewalls block UDP. Some users have experimented with OpenVPN over Multipath TCP. This approach
works in the laboratory, but when deployed in the field, packet losses cause performance problems that are an
inherent consequence of the utilisation of TCP as a tunnelling protocol. To understand the problem, let us consider a
very simple scenario where one host transfers data over a TCP connection through a TCP-based tunnel between two
routers. When a packet loss occurs between the two routers, the TCP tunnel will detect the loss and retransmit the
data. The end hosts will observe an increase in delay but no loss. For TCP, the losses are the primary indication of
congestion. If the end hosts do not observe losses, they will try to transmit data faster, which would cause long delays
in the routers that cannot absorb the data as quickly as required.

Conclusion
In this document, we have qualitatively compared the different techniques that can be used by network operators to
deploy hybrid networks. We have explained the limitations of network-layer solutions and have shown the benefits
that the Multipath TCP protocol brings in such environments. Being a software solution, Multipath TCP can be deployed
on existing CPEs without requiring a costly hardware upgrade.

About Tessares

Tessares is an innovative startup created by several of the key developers of the Multipath TCP protocol and of its
reference implementation in the Linux kernel. Tessares leverages this technical expertise to design, develop and deploy
software-based solutions to enable network operators to improve network performance and reliability by combining
different access network technologies for the benefit of their customers. Tessares is a privately-owned company that
received significant first round funding from both Proximus (previously called Belgacom) and VIVES (vivesfund.com).

Contact: contact@tessares.net / +32 10 392 254 / +32 497 07 65 42

www.tessares.net COMPARING HYBRID ACCESS NETWORKS SOLUTIONS 8/8