High-Performance Networking in the University Campus Edwin Hoffman, CSA Case Summary 1 Case Summary A university campus can benefit in many areas from high-performance networking systems. Among these are: Computer IT departmentsWhere modern technology produces students with cutting-edge skills Physics and mathematics departmentsHigh-performance computing (HPC) in science departments Campus connectivityHigh-performance campus connectivity requirements Campus entertainment delivery servicesCost savings or incremental income from campus entertainment delivery Extended and distance learningIncremental income and marketability of the university using technology to expand popular courses both on campus and off campus COMPUTER IT DEPARTMENTS The computer IT department must stay current with modern trends of the computing world, even when the computer department network is the least thought of and most important of this departments assets. Sun Microsystems' motto used to be the The Network is the Computer. This is an even more germane statement today. Following the trends in networking is essential in any computer department, but even more so in the successful ones to maintain their high standards. PHYSICS AND MATHEMATICS DEPARTMENTS Whether it is experimental analysis of material performance, theoretical particle physics, or massive computing array mathematics, the network that connects HPC systems together needs to be fast, reliable, and cost effective. The trend is toward 10-Gigabit speeds, but few can afford the technology, therefore compromise is always the watch word. CAMPUS CONNECTIVITY In campus networks, the goal is usually to provide the maximum performance with the maximum resiliency for the lowest cost. Considering that the entire lifeblood of a university is riding on such a campus network, it is essential to get the best performance possible. When linking the university campus together, obtaining the cheapest system that works can be exactly the wrong approach. State university systems can benefit from the efficiencies that high-performance campus interconnects can bring. CAMPUS ENTERTAINMENT DELIVERY SERVICES When Northwest University decided to bring cable TV on campus, the price for running coaxial cable throughout the campus was the deciding factor. The university decided to deliver their cable TV over an Ethernet network saving over $12 million in cable costs alone. Northwest missed the opportunity to offer Video-on-Demand (VoD) services from campus sources that could have brought a steady stream of income for the university. Many universities have also implemented cable TV on campus, but have also missed the same opportunity. HPC Computing In the last few years, high-speed networks have become a dominant factor in proliferating High-Performance Computing (HPC) facilities. Anonymous. Raptor Solution 2 EXTENDED AND DISTANCE LEARNING Students and their families select universities based on the quality of their courses, teaching methods, and the fame of their instructors. Usually these popular courses are fully booked quickly after being opened for selection, and some universities now have invitation-only courses that are pre-booked. Using high-performance networks universities can expand these courses to include students who are not in the auditorium at all, but are in dorm rooms, homes, offices on or off campus. Correctly placed video and audio pickups in auditoriums can relay the instructor's words and actions, as well as the audience reactions, questions, and so on from both auditorium and remote sites. These popular courses can also be sold to smaller universities close by that may want to join and offer the course. This is true distance learning. Given the present state of high-performance networking, the high cost discourages education establishments from anything but a minor implementation to cover specific requirements. Raptor Solution Raptor Networks Technologies has resolved to change this situation. Raptor Adaptive Switch Technology (RAST) allows the user to implement high-performance and high resiliency networks at much lower prices. Raptor Networks provides 1-Gigabit and 10-Gigabit Layer 2/3/4 switching with some unique features that allow standards-based Ethernet devices to operate in unique ways. RAST creates a distributed switch fabric, a virtual chassis, which can be located in geographically diverse sites and offer a single-switch solution at the same time. RAST operates at Layer 1/2, providing high levels of redundancy and resiliency with complete interoperability with other Ethernet-based systems. EXTENSIBLE VLANS Using RAST, a VLAN can be created that exists in all of the connected RAST- compatible switches. Other switch systems only create the VLAN in a single switch and trunk it to the other switches in the network. Figure 1 shows four buildings connected together with Raptor ER-1010 switches and a single VLAN, which exists on and is owned by all the switches. With this configuration, It is possible to create a Layer 2 network dedicated to storage. FIGURE 1. Four-Building VLAN Configuration VID 11 VID 11 VID 11 VID 11 VLAN ID 11 can now be used as a Layer 2 switched LAN extending over the entire campus/city. This example shows storage over IP solutions with mirrored storage available in each building a highly redundant and disaster-tolerant solution! Raptor Solution 3 TRUE PARALLEL COMPUTING Parallel computing systems such as Oracle Parallel Server (OPS) systems cannot be used effectively due to network bandwidth limitations. Most networks support OPS severs only when connected to massive core switches. Parallel systems are not very resilient systems. Figure 2 shows the OPS servers in their own VLAN with QoS parameters that they can really use and are geographically redundant. Even a major disaster such as a fire or building collapse cannot stop these servers. Inter-VLAN routing allows users to access the OPS system using Access Control Lists (ACLs) to make them secure. FIGURE 2. Parallel Server Configuration If video delivery (unicast/multicast), surveillance, or conferencing is required, it is easy to assign another system-wide VLAN and apply QoS parameters. FIGURE 3. Video Delivery, Surveillance, and Conferencing Network VID 11 VID 11 VID 11 VID 11 VID 21 VID 21 When using a parallel processing application, a VLAN that exists in multiple buildings is just so much easier. Oracle Parallel Servers operating in their own VLAN and with VLAN-based QoS is just simple. Oracle Parallel Oracle Parallel Server Server VID 11 VID 11 VID 11 VID 11 VID 21 VID 21 Oracle Parallel Server Oracle Parallel Server VID 31 VID 31 VID 31 VID 31 192.168.1.x 192.168.18.x 172.168.18.x 192.168.1.x 172.168.18.x 192.168.1.x 192.168.1.x 192.168.18.x 172.168.18.x 192.168.18.x 192.168.18.x 172.168.18.x The video VLAN allows secure access to video resources via ACLs, while also ensuring delivery by applying correct QoS parameters to the video streams. Raptor Solution 4 Finally, all the clients and their attendant mail, print, application servers as well as other system-wide VLAN and control access to all the other VLANs via ACLs are added to the configuration. FIGURE 4. Complete Configuration RAST provides for the switching or routing of traffic depending on where the VLANs meet in the system. Figure 4 shows all the VLANs that are available in any switch. In Building A, if a client needed to route to VLAN (21), the switch directs the traffic to Building B where it then routes into VLAN 21. In Building B, if a client (23.8.1.x) needs to access the same OPS VLAN, it routes in Building B. In Building D, if a client in subnet 23.8.4.x needs to access the mail server in Building A in subnet 23.8.1.x, the traffic switches over the shortest distance between sites and routes in Building A. RAST actually allows what many other systems attempt to accomplish (and fail) to do by allowing VLANs to operate natively over long links and allowing subnets to exist in any or all sites. Resilient Packet Ring (RPR), Q-in-Q (VMAN), VLAN stacking, or even VPLS were introduced to try and create this holy grail and allow a VLAN to exist on multiple sites. When any of these attempts to create a transportable VLAN are made, it causes a major issue that these technologies cannot address. When a VLAN is created the switch it was created on owns the VLAN. If the VLAN is transported to another switch, the VLAN is trunked over using 802.1Q and 801.p tagging so that the remote switch can join the VLAN. The problem with this structure is that when routing between VLANs, the only switch that can route the VLAN is the owner of the VLAN (the creator). Expensive bandwidth is wasted because data is transported back to the VLAN creator to be routed, and perhaps over the same link to the egress point. Note All subnets actually exist in all four buildings. VID 21 VID 11 VID 31 VID 33 VID 33 192.168.1.x 172.168.18.x 23.8.4.x 192.168.18.x 23.8.1.x VID 21 VID 11 VID 31 VID 33 VID 33 192.168.1.x 172.168.18.x 23.8.4.x 192.168.18.x 23.8.1.x 192.168.1.x 172.168.18.x 23.8.4.x 23.8.1.x VID 11 VID 31 VID 33 VID 33 VID 11 VID 31 VID 33 VID 33 192.168.1.x 172.168.18.x 23.8.4.x 23.8.1.x Building A Building B Building D Building C Traffic flows are highly efficient in a Raptor Adaptive Switch Technology (RAST) network configuration. Raptor Solution 5 If VLANs really existed on both sides of the network, then the routing function would occur at both sides. FIGURE 5. VLAN Limitations Without RAST With Raptor Networks RAST, the VLAN does exist on both sides of the network, therefore the subnet exists on both sides of the network, and any routing requirement is carried out locally and does not waste bandwidth. More importantly it does not increase latency. Imagine that happens to latency when data is transported from side to side just to route it! FIGURE 6. RAST VLAN Metro Network VID10 VID10 VID11 VID12 VID13 VID11 VID12 VID13 VLAN Creator VLAN trunk Terminator VID11 VID12 Route Path VID11 VID12 VID13 These subnets can only exits here ! Subnet 192.168.1.x =VID11 Subnet 23.108. 54.x=VI D12 When a system on the terminator side needs to route to another system on the terminator side, but in a different VLAN, the path data MUST take is to ride over the Q-in-Q network to the switch that created the VLANs (remember Q-in-Q simply allows tagged VLAN packets to tunnel through another VLAN) and perform the route at that point. This is costly because each routed packet uses exactly twice the bandwidth (2 times over the metro link) when it should use none! VID11 VID12 VID13 ER-1010 unit 2 VID11 VID12 Route Path VID11 VID12 VID13 ER-1010 unit 1 With a RAST network over dark fiber or managed fiber, the VLANs exist completely in both sites. Therefore when ER-1010 unit 2 is asked to route between VID11 and VID12, the route happens in ER-1010 unit 2. No wasted bandwidth occurs, latencies remain low, routing happens only where it must. Case Conclusion Corporate Headquarters: 1241 E. Dyer Road, Suite 150 Santa Ana, CA 92705 Phone: 949-623-9300 /Fax: 949-623-9400 / Web: www.raptor-networks.com / E-mail: info@raptor-networks.com Raptor Networks Technology, Inc. reserves the right to make changes without further notice to any products or data herein to improve reliability, function, or design. Information furnished by Raptor Networks Technology, Inc. is believed to be accurate and reliable. However, Raptor Networks Technology, Inc. does not assume any liability arising out of the application or use of this information, nor the application or use of any product or circuit described herein, neither does it convey any license under its patent rights nor the rights of others. Raptor Networks Technology, Inc. is a registered trademark and RAST is a trademark of Raptor Networks Technology, Inc. All other trademarks are the property of their respective owners. CS701 02/23/2007 Case Conclusion RAST allows 10-Gigabit speed across distances up to 120 km at prices that are a fraction of the 10-Gigabit Ethernet prices offered by other vendors. RAST provides the most efficient and resilient high-performance network possible. This technology allows a university to implement all the smart applications as needed to provide the broadest possible range of services for the campus.