Quartzite

MRI: Development of Quartzite, a Campus-wide, Terabit-Class, Field-Programmable, Hybrid Switching Instrument for Comparative Studies

This project, building a campus wide ultra high-speed optical fiber network that supports scientific application and experiments of high volume data, develops an experimental next-generation instrument to efficiently investigate and compare campus-scale terabit-class lambda network architectures that span from optical-circuits-only to packet-switched-only networks (and a range of hybrid combinations in between). Current commercial approaches to storage systems do not scale in either performance levels or data abstractions. The proposed approach builds on the foundation of the shared-nothing compute cluster emerging from data systems, visualization walls, and high-end instrument interfaces, having raw horsepower to serve and ingest high volumes of data required by applications. Constructing a next generation switch for simultaneously switching 10Gbs streams efficiently, the work aims at building a 21st century photonic instrument to explore the practical tradeoffs of network and application design in bandwidth-rich infrastructure. Supporting large scientific problems and enabling big simulations, the project constructs Quartzite, the experimental, next-generation instrument. While fostering comparative studies, Quartzite, a data-intensive application breadboard, enables stitching together resources, bringing them virtually in. Thus, this wavelength-selective switch creation, communication and delivery project, adds hybrid-networking structure to a unique campus-scale platform and enables the study of network architecture and application design in a band-width-rich infrastructure and the sharing of large data sets across clusters. The work involves high risk, with a promise of even higher impact, since data intensive scientific exploration can be brought into the scientists’ lab, by using on-demand high-speed data flows to harness campus- to international-scale resources.

• The work explores the following issues:
• How surplus of on-demand bandwidth can be exploited by end user applications,
• How distributed systems can be best architected,
• When is a non-shared packet network needed,
• How should control of a hybrid fabric be handled,
• Can applications truly exploit a high-speed parallel infrastructure,
• Is dynamic reconfiguring of campus network to meet transient capacity demands practical,
• Is it beneficial to expose direct circuits to individual endpoints, and
• Do novel packet scheduling strategies for shared links dramatically improve the capacity.
• Broader Impact: The Quartzite-enabled comparisons will influence the network structure of future research university networks, greatly increasing the capability for data-intensive research throughout the country. Working with industrial partners, the hybrid Quartzite core system and software will service us all.