Graphic demonstration: Microsoft Silverlight

Some technology has audiences applauding when they see the demos. That tends to be the reaction to Deep Zoom and Photosynth, two of the components of Microsoft Silverlight.

This next generation technology is designed to bring the web, graphics, pictures and text to life; but it could also transform the presentation of data within healthcare. Paul Bray reports. 

Imagine that clinicians could call up a patient's entire medical history at will, not just from electronic clinical records but also from handwritten notes and sheaves of x-rays and scans.

Now imagine that they could instantly drill down to read any paragraph of text or study the tiniest detail of any image. This is the sort of functionality that, until now, only high-end graphics experts could enjoy on very high-powered workstations.

Closing in with Deep Zoom

But what if clinicians could do these things from any location - hospital, GP's surgery, even a mobile device by a patient’s bedside - as quickly as turning a page or holding a magnifying glass over a picture?

That would take us a lot closer to the Holy Grail of enabling patient information to be presented efficiently at the point of care; allowing healthcare professionals to navigate through charts and scans at will. And it is nearer to being a reality, thanks to a new Microsoft technology called Deep Zoom. “You can view a multi-gigabyte image of, say, Bo Diddley's guitar and instantly zoom in to examine the master's fingerprints in forensic detail on the fret-board.”

Originally developed by Seadragon Software, a company acquired by Microsoft in 2007, Deep Zoom is a way of streamlining the delivery of very large high quality images (of a gigabyte or more) over relatively low bandwidth links, such as basic broadband or a 3G phone.

Conventional technology requires a whole image to be downloaded onto the user's workstation before they can pan around it or zoom in. Deep Zoom only delivers the pixels that will actually be displayed after each pan or zoom.

As a result, users can start with a low-resolution thumbnail of an image or group of images, then select any portion, at any resolution, and have just that selection delivered direct to their computer or phone. Because Deep Zoom only delivers one screen-load of pixels at a time, response times are near-instantaneous, even on traditionally “slow” links.

From Bo Diddley to Charles Dickens

If you want to see the power of Deep Zoom in action, go to the Hard Rock Café website. There, you can view a multi-gigabyte image of, say, Bo Diddley's guitar and instantly zoom in to examine the master's fingerprints in forensic detail on the fret-board.

In another early demonstration of Deep Zoom, the founder of Seadragon, Blaise Aguera y Arcas, loaded images of every page of Dickens's novel, Bleak House, onto his display and then zoomed in to view a single chapter, a page, then a paragraph, and finally a single word.

This level of detail could be applied to a patient's entire medical history, even when composed of x-rays or a GP’s handwritten notes.

Shine a (Silver) light

Deep Zoom is now part of Silverlight, Microsoft's next-generation technology to "bring the web alive" and allow rich internet applications (RIAs) to run on virtually any platform, including Windows and mobile devices.

“Rich” means the capacity for graphics and video to be viewed, used and manipulated - sometimes for entertainment, but also in order to make better and more intuitive decisions in the workplace. The medical profession is a prime example.

Making sense of images with Photosynth

Another element of Silverlight with potential in healthcare is Photosynth; a technology that enables a series of two-dimensional images to be stitched together into a 3D model. “Instead of seeing a scan as a series of “slices” a consultant could move around the 3D model and zoom in to study sections in as much detail as the original scans allowed.”

As with Deep Zoom, the applications and pilots that have been put into the public domain so far are mainly for consumers; look on the web and you’ll find walk-throughs of city centres or tourist attractions, such as Notre Dame in Paris.

But Photosynth could be applicable in a medical context. For example, a series of MRI or ultrasound scans could be used to build up a 3D image of a patient's brain or heart. Deep Zoom could then be used to navigate the image, so that instead of seeing a scan as a series of “slices” a consultant could move around the 3D model and zoom in to study sections in as much detail as the original scans allowed.

"It's akin to moving from a green-screen mainframe environment to a graphical interface such as Windows," says Nick Umney, Microsoft's lead technical specialist for health. "It really is a step-change of the same magnitude."

Powerful in isolation, these Silverlight technologies are even more beneficial in conjunction with other applications. For example, a web-based meeting between consultants or a remote training session for students could be enhanced by sharing medical images without requiring the transfer of vast files over a bandwidth-starved network.

A glimpse into the future

Currently Deep Zoom and Photosynth only work on still images. But in time, the technology will do the same for video. This would mean that a clip showing, say, a faulty heart valve could be panned-and-zoomed in 3D. "...the kind of pattern-recognition technology used by Photosynth to stitch overlapping pictures together could ultimately be developed to "read" the content of images."

Another limitation is that Silverlight can't interpret the content of images. This means, for example, that a scanned document can't be searched like text.

However, part of its raison d'etre is to enable multimedia elements to be treated as processable information. So it’s not unreasonable to suppose that the kind of pattern-recognition technology used by Photosynth to stitch overlapping pictures together could ultimately be developed to "read" the content of images.

This could enable an oncologist, for example, to tell the computer, "Here's what an early-stage tumour looks like; go and search through these x-rays and see if you can spot any more."

Good news for IT managers

Because they are designed to use very modest bandwidth, Silverlight and associated graphical technologies should require very little in the way of major infrastructure to implement, whether users are in-house or using mobile or home links.

They could even be used to provide information directly to patients. Yet they have the potential to revolutionise the way that complex information is presented to health care professionals, and to empower them to navigate around it in new ways.

Find out more

Installation of Silverlight on the desktop is fast and takes only a few moments, so have a look at these websites for more opportunities to put Silverlight through its paces:

- Silverlight Showcase

- Silverlight Schwarz interactive

- BBC's Big Weekend