Project HSD: Holographic Storage Device for the Cloud

Project HSD: Holographic Storage Device for the Cloud


News & features

News & features


Holographic storage research testbed at Microsoft Research Cambridge

Holographic storage research testbed at Microsoft Research Cambridge

The enormous growth in demand for cloud storage has highlighted the need to rethink our storage systems from the media up. NAND flash and spinning hard disk drives are the mainstays of today’s warm cloud storage but are no longer improving capacity exponentially; in addition they face  reliability and performance challenges due to the mechanical moving parts in hard disk drives and the declining endurance of flash cells.

Project HSD is a collaboration between Microsoft Research Cambridge and Microsoft Azure to re-imagine an old idea – holographic storage – as a cloud-first design. We are capitalizing on the recent exponential improvement and commoditization in optical technologies such as smartphone cameras, as well as the unique opportunity to design at cloud scale.

Our interdisciplinary team comprises experts in physics, optics, machine learning, and storage systems; and our mission is to design mechanical-movement free, high-endurance cloud storage that is both performant and cost-effective. This has led to new research challenges and breakthroughs in areas ranging from materials to machine learning.

Project HSD was publicly announced at Ignite 2020 by Mark Russinovich, CTO of Azure. See Mark talk about HSD and show a demo of it (watch from 1h25 to 1h30) and read the related blog post for more details on HSD.

Project HSD is part of the broader Optics for the Cloud group at MSR Cambridge, which explores the future of cloud infrastructure at the intersection of optics and computer science.

How does Holographic Storage work?

What is Holographic Storage?

Holographic storage was first proposed back in the 1960’s shortly after the invention of the laser. Holographic optical storage systems store data by recording the interference between the wave-fronts of a modulated optical field, containing the data, and a reference optical field, as a refractive index variation inside the storage media. It is this information containing refractive index variation that is the “hologram”. The stored data can then be retrieved by diffracting only the reference field off the hologram to reconstruct the original optical field containing the data.

Holograms can be created in polymer and electro-optic crystalline materials by exposing the material to a modulated optical field i.e. the interference pattern between the data and reference optical fields. In polymers the holograms are stored as a permanent change in the material and provide a Write Once Read Many (WORM) storage solution. Holographic storage in polymers for archival storage has been actively pursued as the successor to blue ray, however, this technology has yet to see commercial success.

In electro-optic crystalline materials the hologram is stored as a spatial variation in the distribution of the electron density inside the host crystal which due to the electro-optic effect, causes the refractive index to also vary spatially. The spatial distribution of the electrons can be changed by exposure to light of a lower energy wavelength, e.g. green light, to write holograms, and reset by exposure to a higher energy wavelength, e.g. UV, to erase the stored holograms. After UV erasure the media can be reused to write further holograms, thus electro-optic materials provide a Rewritable, Read Many media.

The hologram stores information in a 3D volume and thus provides 3 degrees of freedom when it comes to storing information. In polymer materials the thickness of the material is limited by scattering so typically thin layers of material are used which limits the capacity that can be achieved and necessitates the use of spinning media to achieve acceptable capacities e.g. 300GB in a CD size for factor. In electro-optic crystals large volumes (10s of cubic millimetres) can be used which allows the 3D nature of this storage technology to be fully exploited. In 2000 IBM were able to demonstrate impressive storage capacities in an electro-optic crystal, Iron doped Lithium Niobate (LiNbO3:Fe). However for the storage requirements of the time Hard Disk Drives proved to be a more compelling technology.

In Project HSD we are exploring the use of holographic storage in rewritable electro-optic materials for warm data storage to see if this technology makes sense in the cloud era.

How does holographic storage work?

This video shows how holographic storage works, using green light to write data as a persistent hologram inside an optical crystal. The data can then be read back from the hologram using another green light signal. The media is rewritable after erasure with UV light

The physics of hologram formation in iron doped lithium niobate

This video shows what is happening inside the lithium niobate media when a data page hologram is written.


Project Team