Internet Service Provider Networks and Network Services

Optical Transport Network Transport

Nodes

Optical Control Plane

Core Network Edge data

center

Edge data center

Residential access network

…

Mobile access network

!  #

Rest of the Internet

Subscribers

Over-the-top content providers

$

Service edge

Service edge

NodesEdge

NodesCore SDN Control Plane

Subscriber Access

Software-defined Multicast Residential

Network Access

IPTV

VoIP Forwarding

SDN Protocol InterfaceVirtualization SDN

Controller

Traffic Engineering Unicast

Forwarding Core Fabric Services

Figure 4.1: A technological view on ISP networks as used in this thesis.

A technical overview of ISP networks is given in Figure4.1. The core network is made up to two parts: an optical transport network that provides optical network links and an IP network using these links for interconnection. This is important, because in contrast to the core network, in edge and data center networks the network devices operate their links directly, without the help of an additional optical control plane. This means that the capabilities of a link are determined by its endpoints, which are SDN devices.

Optical transport networks, although they often could be called software-defined for their use of remotely configurable equipment such as ROADMs, are unlikely to adopt a standard SDN protocol. This is because their equipment is much closer to the latest research. Therefore, the technology is often proprietary and not standardized, which prevents the application of a standardized SDN protocol. Hence, this part of ISP networks is therefore not included in our investigation.

The network edge is assumed to be organized as small data centers, e.g., as envisioned by Peterson et al. in their Central Office Re-architected as a Data Center (CORD) approach [Pet+16]. These CORD data centers are also expected to host the service edge, which is the customer-facing part of the network.

4.1 ����������� ��� ������������ 37

The conceptual overview of the relevant areas of ISP networks is given in Figure4.2.

We investigate the core and edge parts of the network, as well as data center networks. In these areas, network services are implemented and provided to the customers. Therefore, these are the areas where SDN is expected to provide the highest efficiency improvements.

Access networks often use specialized network technology. This is because they have a different technological focus that the other parts of the network: providing connectivity at the lowest possible cost. However, the costs in this area of the network are dominated by traditional construction costs and government regulation. Network management and traffic steering are restricted to forward network traffic from subscribers to the closest edge data center. Therefore, it is not relevant for our focus on areas of ISP networks where SDN is expected to improve the management and control efficiency.

Business Customers

Residential Customers Mobile

Customers Data

Centers

The Internet Direct &

IXPs/SDXs

Edge Access

Core

Control plane load high

medium very low

Figure 4.2: ISP network areas color coded by control plane load.

The network areas in Figure 4.2are colored by their expected control plane load.

Access networks are expected to be static, with connecting and disconnecting links of customers being the only events that cause control plane load. The elements of the core network, both packet switching and the optical transport elements, are expected to be modified in existing networks only in case of an element failure or for traffic engineering.

Both events are not expected very often, and intervals between events are assumed to be in the magnitude of hours. Finally, the network edge is the area of the network where control plane applications are active to implement network services. Fundamental services like unicast connectivity are provided at the edge by translating IP destination to paths through the core network fabric. Customers are authenticated, and their contracts are enforced at the edge. We, therefore, expect considerable control plane load in this network area. Reasons for that might be inter-autonomous system (AS) routing updates, customer access events, or interactions of customers with network services. An example for the latter is when a customer switches a channel in multicast-based IPTV, which

requires the network edge to update its multicast packet delivery configuration. In the remainder of this work, we will generalize mobile, residential, and business customers simply as customers. From a technical perspective, they represent networks that are connected to the ISP through the access area.

We observed two main characteristics for network services that are implemented by control plane applications and operated in ISP networks: control plane activity is expected to be low on average with sudden spikes of activity and an inherent need for prioritization of network services. The first observation is a natural outcome of the design approach for network infrastructure, which is to design for peak load instead of the average load. Peak load events for network services, e.g., can be the failure and subsequent re-activation of a device at the network edge. An example is the failure of a mobile network access node, which must re-activate thousands of customer connections on other devices as quickly as possible. In contrast to that, the average load of a mobile network access service is expected to consist of handovers for moving customers and few customers that switch their mobile devices on and off[Jin+13]. The same is true for the unicast routing service. This service is designed to handle the failure or connection event of a Border Gateway Protocol (BGP) router that provides routes from and to neighboring ASs. In such an event, a BGP router usually receives the whole Internet routing table with about 732,000 unique IPv4 prefixes�. Depending on the significance of the neighboring BGP router, such an event might result in up to the same amount of routing entry modifications in the data plane.

The need for prioritization of network services is caused by two effects: first, dependen-cies between services and second, commercial interests. Services such as the core network forwarding, or unicast routing are essential for all other services provided by an ISP as depicted in Figure4.1. Therefore, these services should always take precedence over other, less critical services. The latter argument is already an example for commercial interests of the ISP. In cases where the control plane load is higher than the available processing capacities, the network operator has to decide which network services should continue to operate, and which should be impaired. In such cases, the ISP will let those network services be impaired that generate the smallest economic loss. For example, the core network is the foundation for many other services and should therefore never be impaired. Add-on services that generate only a small part of the ISP’s revenue are better candidates to be starved of resources. Therefore, prioritizing network services and, subsequently, control plane applications are essential for efficient and reliable ISP network operations. Furthermore, these situations are examples where low priority applications need to be able to adapt to unexpected resource starvation on certain or all data plane elements. Finally, control plane applications can fail or contain errors. The controller must ensure that a failed or erroneous application cannot cause a failure of the controller or degrade the performance of other control plane applications.

1 GeoffHuston.CIDR REPORT. Accessed: 2018-8-27. Aug. 2018. url:https://www.cidr-report.org/as2.

0/.

4.1 ����������� ��� ������������ 39

Another essential aspect of ISP and the closely related carrier networks is their opera-tional complexity. Due to the importance of the network to the ISP companies, the general approach to the deployment of new technology is conservative [Lev+14]. Furthermore, to ensure the operating staffcan maintain the system, the network management must be understandable for humans. One negative example is Generalized Multi-Protocol Label Switching (GMPLS) [RFC3945] which aimed at combining the widely used MPLS for packet networks with optical transport network features. Unfortunately, GMPLS ended up being an overly complex platform [Liu+12;Far10;DPM12] with no gradual update path from existing technology. Therefore, although GMPLS was developed for a long time and by many partners, it was not widely adopted by the carrier community [DPM12]. Hence, we to prevent our design to be deemed too complex, we require them to be as understandable and maintainable for their human operators as possible.

We summarize the information requirements for control plane application as follows:

To achieve resource efficiency for applications, the controller must collect information on their resource consumption. To enable the controller to ensure reliable operations, in addition knowing all performance-relevant information, it must ensure that operating a control plane application does not have unexpected side effects. Furthermore, appli-cations need to receive information on the performance of the control path to handle bottlenecks in a controlled manner. Finally, the entire system should be designed to be comprehensible by human operators to allow them to intervene in the case when the automatic reliability measures are not sufficient.