Order-of-magnitude improvements for research data throughput

With ballooning dataset sizes across a variety of research areas and the continued increase in the collaborative nature of research, the ability of researchers to share data becomes paramount. AARNet focuses many of its efforts on increasing the real throughput rates of research data flows, preferably without requiring changes at the end user site. The aim is to increase the ability of researchers at its member institutions to compete and collaborate globally. Over recent months AARNet has deployed the Science DMZ architecture at key data-storage nodes across Australia, and is currently extending this to additional Australian institutions. This talk will describe the particular measures required to achieve production throughput of linerate 10Gbps and beyond over distances of over 5,000km across the national MPLS backbone, as well internationally across over 15,000km of undersea cable. This includes key learnings in tuning the network, the data-transfer systems and the perfSonar test servers, and describes the deployment of appropriate tools to end-users to allow them to share and transfer data. This talk will also present work currently underway on using the 10+Gbps network capabilities to extend Cloudstor Plus, AARNet’s Owncloud-based cloud storage platform, further out to the customer for improved performance and efficiency whilst not sacrificing security. Other similar efforts underway elsewhere will be compared and contrasted. To finish, some future directions will be presented. In particular, some work underway using the AARNet SDN testbed to integrate the Neutron and Swift components of OpenStack into the Science DMZ switches utilising Openflow to signal network reconfiguration dynamically. The aim is to provide users a more transparent method of moving data between their data centres and out to external cloud providers.

Speakers

Part of session

Big Data and the Cloud

Related documents