[CSG Winter 2007] Storage at Indiana University

Advertisements

Managing very large files in research computing at IU.

Task force two years ago on research cyberinfrastructure had recommendations concerning storage – Continuing to deliver centralized facilities to support research computing as well as dependable archival storage were identified as important. Large file storage is just a piece of the storage strategy for IU.

They have about a petabyte of spinning disk available for researchers, as well as 4 petabytes of archival storage (the Massive Data Storage System). The “Data Capacitor” captures data from instrumentation.

Data Capacitor uses Lustre OS.

MDSS designed to provide a deep store for large files. Runs HPSS. Interfaces include FTP, Samba, and tar. Radiology is one of the biggest users. Also working with digital library programming. They give the researchers 500 GB for free, and after that they want to discuss it.

Preservation, curation, and long term management of data is a big issue – need to link librarians, computer supporting, and IT professionals. Serge notes that finding ways of accomplishing persistent URIs for data is important.

Backup with mirroring is if you accidentally delete something or introduce bad data in big data sets is a serious problem.

Technorati Tags: , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s