The design objective of Kerala State Wide Area Network (SWAN) had been to set-up a state-of-the art network, which is highly robust, resilient, scalable, easily available, and provide equal access to all users irrespective of their physical location
Centre for Development of Advanced Computing (CDAC), Thiruvananthapuram, known as ER&DC at that time, conducted a seminar on e-Governance in December 2001. The theme of the seminar was emerging trends in networking and data centre in the context of development of the State of Kerala. The recommendation of the seminar was that Kerala should have a Government Data centre and a backbone network for e-Governance applications. It was also suggested that the free bandwidth offered by the private bandwidth providers in return to the right-of-way given to them for cable laying, also be tapped for e-Governance activities. CDAC undertook the design of the e-Governance Data centre and the State Information backbone for Kerala, and subsequently the Kerala State Wide Area Network (KSWAN).
Network architecture It was decided to have a three layer architecture with a backbone ring forming the first layer, distribution from the backbone ring to the district headquarters and then to the block headquarters forming the second layer and the last mile connectivity from the district and block POPs forming the third layer.
The network backbone connected three cities — Trivandrum (southernmost district), Kochi (central district) and Kozhikode (northern district) — configured in a resilient ring. For all practical purpose, this is a dual ring with dual redundant routers in each of the Network Operating Centres (NOCs) at Trivandrum, Kochi and Kozhikode. The plan is that the network can grow from the NOCs of the backbone to the districts and then to the sub-district level in hierarchical star topology, thus providing a highly scalable and resilient network. Redundancy in the next levels can be designed with district-to-district links as well as multiple links in the same route (NOC to district or district to block).
The decision on the technology for the backbone was straightforward. There would be voice, video and data traffic, some of them with stringent specifications on the delay, packet loss and jitter, to be integrated in the optimum method. And, the all pervasive IP needs to be preserved. The obvious cost-effective solution was Multi Protocol label Switching (MPLS), because of its superior performance advantages, which included higher security, betterQuality of Service and better traffic management.
Since MPLS works as an overlay protocol to IP, the two protocols can coexist in the same cloud without interference. The encapsulation of IP packets with labels enhances data security. The ability to classify traffic and allocate specified bandwidth for customers and applications as well as selecting the specified link (of the multiple links) for a specific application enhances the control of the MPLS network many times. This makes more sense in the Kerala scenario, where bandwidth from multiple bandwidth providers has been used.
The backbone makes use of the free bandwidth offered to Government by the bandwidth providers (in lieu of the Rightof- Way given to them), providing multiple levels of redundancy. Serial links from various providers have been bundled to form a multilink so that failure of a link would not affect the applications. One of the link is provided with Ethernet interface, which has not been bundled with the serial links for various reasons.
MPLS Traffic engineering was tested in the backbone for detecting the bandwidth overflow and dynamic selection of alternate path. For testing this, we created two tunnels — one between Trivandrum and Kochi (in the shortest path) and the other between Trivandrum and Kochi through Kozhikode. 3 Mbps bandwidth was reserved for both the tunnels. We used two serial links, each of 2 Mbps (to form 4 Mbps multilink) between Trivandrum, Kochi and Kozhikode NOCs. Heavy traffic was forced between Trivandrum and Kochi. All the traffic flowed through the Trivandrum-Kochi shortest route, as expected. We then removed one of the E1 links from the Trivandrum-Kochi bundle. It was observed that the traffic moves to the alternate route only for the next session,which means that load balancing is sessions based and not packet based. Dedicated bandwidth reservation and utilisation of unused bandwidth was also tested in the MPLS backbone. Two users, one in business class and the other in premium class, were created for this.
Design for high availability
Apart from the ring topology and multiple links from various bandwidth providers, redundancy has been built in the core routers, access routers, switching engines, firewalls etc to ensure a highly available backbone. The network centres have been provided with diesel generators, redundant UPS, precision airconditioning in n+1 mode etc. All the telecom companies have terminated their fibers at the network centres to ensure high uptime.
The backbone network is being extended to all the 14 districts, growing from the three network centres of the backbone ring. The network is designed over MPLS up to the districts, with multiple E1 links and Ethernet connectivity from various providers. ROW (Right of Way) bandwidth from the bandwidth providers would be made use of in the distribution layer also, up to the districts. These links are terminated in fiber so that the reliability is high. Provision to connect to adjacent districts has also been provided, to ensure high availability of the district links. Firewall and IPS protection, RADIUS authentication, MPLS VPN and encryption support, VLAN support are some of the security protections given. In the next level, all the 152 block headquarters in the State would be linked to the corresponding district headquarters through leased lines. Here also, provision to connect to the adjacent block headquarters is provided to enhance the network uptime.
The district POPs are designed to provide for access through leased line, ISDN, PSTN and wireless. The block POPs would support leased line and wireless access. Wireless connectivity takes the minimum time for setting up, if high masts are not required to establish line of sight. There is no recurring cost for wireless (assuming license free band). Offices, which do not have line of sight with the base stations, get connected through leased lines. Those offices, which require only occasional connectivity, could go for dial up connectivity, either through PSTN or ISDN. Internet connectivity has been provided at the Government data centre, which is collocated at the Trivandrum NOC of the State network backbone.
Kerala is not new to wireless networks. The Akshaya network in Malappuram district, linking more than 500 Akshaya centre spread over 3500 sq kms, has been implemented entirely in wireless, making it the first district to be networked fully through wireless. The confidence of this network, which has been implemented in a district with very unfriendly terrain, was behind the decision to go for wireless access as a primary last mile technology. The technology selection for wireless was a difficult one. WiFi is a proven technology for both indoor and outdoor applications.
There are many proprietary technologies, which have proved their worth in many networks both in India and abroad. And, WiMax is coming with a bang. At the time of formulating the network specifications, the WiMax standards were only evolving. There was no clarity on the frequency channels. Commercial products were also not available. Added to this, spectrum availability and frequency licensing in India was also not clear. So, WiMax was ruled out for last mile access.
Proprietary technologies were not encouraged for the reason that the offices wanting last mile access from the base station would have to procure radios from the same manufacturer, if proprietary technology is accepted. So, the only option for wireless access was 802.11b/g, the heavily used technology worldwide.
It was decided that the wireless base station in the district POP and the block POP should support a minimum of 60 remote radios, within a range of 10 kms so that majority of the land in Kerala could be covered using the KSWAN wireless umbrella. 802.11b and g technologies have limitations on the range as well as the number of remote radios that can be supported by a point-to-multipoint radio, because of the issues like near-far problem and hidden-node problem. In order to circumvent this and to provide for higher throughput for each link, it was decided that 3 or 4 base station radios shall be used in each base station, with 1200/ 900 coverage from each radio. The radios support linkwise bandwidth management, WPA2 and AES encryption,MAC based authentication, remote management and Quality of Service. The offered products for KSWAN were tested against the specifications. The test set up consisted of a wireless base station set up at a central location and eight remote radios at other locations. Four remote radios were set up at about 1 km aerial distance from the base station and the remaining four remotes at an aerial distance of about 10 kms. Data rate was measured through FTP, to get the real application throughput. Two FTP servers were set up at the base station. The upload (remote to base station) and download (base station to remote) speeds of the remote radio was tested using FTP, with only one remote radio installed. The number of remote radios increased gradually and performance recorded in each set up. The thick vegetation and the difficult terrain in Kerala demand high mast, both at the base stations and the remote stations. Hence, it was decided to use selfsupport mast at the base stations and guy wire-supported mast at the remote offices.
High level of physical and data security has been provided at the backbone. Smart card/ biometric access control and CCTV based surveillance have been installed in the three NOCs, apart from physical security. Two levels of firewall, network based IDS, Host based IDS, VPN support, RADIUS authentication, Zoning and VLAN support, customisable Access Control list, Antivirus solution etc are available in the NOCs. The wireless network will have WEP/ WPA/ AES as well as MAC based authentication. The whole network is to be certified as per BS7799/ ISO 27001 before being operational.
Network management and monitoring
Network management is centralised and is managed from Trivandrum NOC. Moni-toring of the network is being done through enterprise management/ monitoring system, which integrates the element management systems of the various products deployed. All the components to be used are to be SNMP and MIB II compliant. Web based monitoring with dedicated monitoring portals for each customer can be configured. Apart from the availability of the components, various performance parameters and traffic usage are also being monitored. Monitoring system also has been configured to generate offline reports on various parameters to various granularities. The conformance to the SLA agreed upon by the operator is being verified through the reports from the monitoring system.
Integration with others
The KSWAN backbone has been co-located at the Government Data centre at Trivandrum. The backbone has been integrated to the Data centre with adequate security. KSWAN network is also to be integrated with Akshaya network, currently functional in Malappuram district. It is envisaged that the KSWAN connectivity to the block level can strengthen the Akshaya network by providing a redundant route in case of a link failure in Akshaya network backbone. The IP scheme for the Data centre has been designed and is operational. This is to be extended to the whole network, to suit the geographical as well as the departmental requirements.
The KSWAN backbone is complete in all aspects and is operational for the last one year. The distribution layer is partly complete up to the district level, which is mainly being used for video conferencing. Various e-Governance applications are currently making use of the infrastructure. The complete network would be functional by April 2007.