Scientific DataSet (SDS) is a managed library for reading, writing and sharing array-oriented scientific data, such as time series, matrices, satellite or medical imagery, and multidimensional numerical grids.
- Rich metadata to create self-descriptive data packages.
- Support for several common data formats, such as comma-separated values (CSV), network common data form (NetCDF), and hierarchical data format (HDF5).
- The ability to scale up from simple text files to multi-terabyte Windows Azure archives.
- Concurrent access to the data from multiple computing agents in multicore and distributed settings.
- Consistency checks and transactional updates.
Source code of the core library and more information about SDS library and tools can be found on CodePlex project site here: http://sds.codeplex.com.
A lighter package that treats NetCDF library as an external dependency is at http://github.com/predictionmachines/sdslite. This package is also available via NuGet.
The library was developed in collaboration between MSR Computational Science lab in Cambridge, UK and the Computer Science department, Moscow State University.
You can read more about the library and how to get started in the Introduction to Scientific DataSet document.