NTU creates unique hybrid cloud environment for High Performance Computing Center

Source: NetApp. Professor Soh Yeh-Chai, the Founding Director of the High Performance Computing Center at Nanyang Technological University, Singapore.

Nanyang Technological University (NTU) in Singapore is one of the top universities in Asia, ranking 7th in the QS University Rankings for Asia in 2014. Its High Performance Computing Center (HPCC) provides high-end compute-intensive IT resources for compute-heavy academic research, including from the Singapore Center of Environmental Life Sciences Engineering and the Earth Observatory of Singapore. 

The HPCC reached its limit a several years ago and was forced to rethink and revamp its architecture, especially in the light of unpredictable demand. 

Source: NetApp.

"We don't know when there will be a surge in the demand for higher computing power," said Professor Soh Yeh-Chai, the Founding Director of the HPCC at NTU. "Our challenge is to figure out how to meet a virtually unlimited and unanticipated demand for computing power that allows our scientists, engineers, animators and students to carry with their work without interruption."
NTU opted for a hybrid cloud infrastructure as the architecture would provide a secondary data centre site as part of regulatory requirements. The HPCC team also wanted to provide users with a seamless experience, so that additional requests for power would not slow down the work for existing users. 

NTU subscribed to Amazon Web Services (AWS) to handle resource-heavy spikes and meet unexpected needs smoothly. Data was kept secure through an AWS Virtual Private Cloud (VPC) that allowed virtual machines to remain isolated from other AWS users. 

The university also built a private cloud infrastructure with virtualisation and platform management using Red Hat OpenStack Cloud Infrastructure. Traditionally, an organisation would have had to buy completely new equipment to add more computing power, as parts of existing hardware which are sitting idle could not be easily 'unlocked' and assigned to new work. With the Red Hat setup, NTU can make use of the unused resources through virtualisation - building 'virtual' CPUs, virtual memory and virtual machines to make better use of existing hardware and facilities. 

Given the nature of the research that the HPCC supports, the team needed to find a storage infrastructure solution that allowed them to maintain the confidentiality of their research data. Even though AWS had partitioned NTU's cloud requirements into a separate Virtual Private Cloud, NTU did not want to store everything in a public cloud. 

What NTU did was to design its own 'public cloud' for storage, that would talk to its private cloud, but whose infrastructure was wholly managed by NTU. "We took several ideas and innovated them together to come up with an infrastructure that replicated the function of public cloud servers with minimal latency. It was designed to be interoperable with our existing private cloud environment and managed by our HPCC team in Singapore," said Professor Soh. 

"Many people forget that there are still traditional applications that cannot sit and run at 100% performance in the cloud. We deal with many scientific applications that require traditional computing resources but we also have many apps like email that can sit and be managed in the cloud."

The HPCC hybrid cloud infrastructure comprises three parts — a private cloud within NTU, a public cloud powered by AWS and a third private storage set-up at Equinix, which provides data centre space, and which is powered by NetApp. The setup extends the advantages of a private cloud — compliance and security — into the public cloud and provides a unified user interface for all users accessing both private and public resources. 

NTU worked with NetApp, Equinix and Red Hat, the university's private cloud provider, to set up private storage hosted at an Equinix data centre using NetApp's FAS 3220HA systems. Red Hat's cloud management software managed all components within the private and public cloud environments. 

Today, NTU's HPCC team has gained multiple benefits with its new hybrid infrastructure including fulfilling its regulatory need to have an affordable secondary site for disaster recovery. The storage data is replicated to and from the NetApp systems, hosted by Equinix. The Equinix data centre is also linked to AWS, which allows it to serve as a backup site in the event of a disaster. 

With the new design the HPCC team has saved costs without having to procure new hardware or add members to the team to manage spikes in performance demand. Moreover, the cloud model is one that can be replicated in other sites with similar demands. 

The list of NetApp solutions at the HPCC are:

• NetApp FAS 3220HA
• NetApp SAN (FC and iSCSI) with CIFs and NFS
• NetApp Clustered Data ONTAP operating system
• NetApp FlexClone technology
• NetApp OnCommand management suite
• NetApp FlexVol technology
• NetApp deduplication and compression
• NetApp SnapMirror replication technology
• NetApp thin provisioning

Source: NetApp. IHLs are institutes of higher learning.