Information for designers and interactive developers - Issue 2 - May 2008
home  |  articles  |  register for the newsletter  |  Issue 1  |  Issue 3  |  Issue 4  |  Issue 5  |  Issue 6
Media Technologies in Silverlight:
Windows Media and VC-1
Windows Media
New media features in Silverlight 2
Silverlight 2 brings several new media features to the platform, which will be described in more detail below:




  • Windows Media Audio 10 Professional
  • Silverlight DRM, Powered by PlayReady
  • Server-Side Playlists
  • Performance and quality improvements
  • Move Networks support
VC-1
While Silverlight supports the older Windows Media Video 7 and 8 codecs, that's mainly for backwards compatibility with existing content. For content created for Silverlight, the VC-1 codec is what should be used. Microsoft designed VC-1 under the name Windows Media Video 9. The broader video industry was excited about what the technology could deliver outside of just Windows Media files (in the original tests for high-definition discs by the DVD Forum, VC-1 beat both H.264 and MPEG-2 for output quality). So Microsoft worked with SMPTE (the Society of Motion Picture and Television Engineers) to standardize WMV 9, which SMPTE designated VC-1.

Windows Media Video 9 and VC-1 are just different names for the same technology. Any .wmv file encoded with "VC-1" will play anywhere that same profile of WMV 9 would play back. All VC-1 profiles play back in Silverlight, except for VC-1 Advanced Profile encoded as interlaced (Silverlight only supports playback of progressive scan content).

Windows Media name SMPTE name WMP version introduced Supported in Silverlight? Recommended use
WMV 9 (Simple Profile) VC-1 Simple Profile WMP 9 Yes Phones and PDAs
WMV 9 (Main Profile) VC-1 Main Profile WMP 9 Yes Broad compatibility with WMP and other players
WMV 9 Advanced Profile VC-1 Advanced Profile WMP 11 Yes (not interlaced) Silverlight and WMP11 playback

VC-1 for Quality
VC-1 had two primary design goals: quality and efficiency. When we talk about how "good" a codec is, we're typically referring to its ability to provide the best quality at a given data rate, or "bang for the bit." Another aspect of an advanced codec is how well it can deliver quality at higher bitrates; some codecs are competitive at lower bitrates, but a maximum quality ceiling is hit past which higher data rates don't help quality much, and the output quality doesn't match the quality of the source.

Microsoft's VC-1 implementation is recognized in Hollywood as being the professional codec most able to preserve the fine detail of film grain and smooth transitions in shadows.

VC-1 for Performance
Another important part of VC-1 is its relatively light CPU requirements for playback. Compared to the other standardized advanced codec, H.264, VC-1 is able to push about 2x as many pixels per second to the screen when using equivalent profiles. This means using VC-1 instead of H.264 can cut CPU requirements in half, or double the frame rate or frame size.

Audio
Windows Media Audio 9.2
The original-flavor Windows Media Audio codec has been around for a decade. The current encoder version is WMA 9.2, but that's fully backwards compatible all the way to the WMA 2 decoder from the 90's. WMA is supported in a wide array of devices, from portable music players to car stereos. And, of course, in Silverlight, as both .wma files and as the audio codec in a WMV file.

WMA is a codec from the MP3 era, and offers superior compression efficiency than MP3 with similar features; it's limited to mono or stereo 16-bit. WMA is the only supported codec in Silverlight 1.0, so will need to be used for any project not exclusively targeting Silverlight 2 (note PowerPC Macs require Silverlight 1.0).

Windows Media Audio 10 Professional
The Windows Media Audio 10 Professional is a high-performance audio codec originally focused on professional and enthusiast audio modes like multichannel 5.1 audio and higher bit depths (24-bit) and sample rates (up to 96 KHz). While WMA Pro offered great quality at 128 Kbps, it wasn't used for web audio much as it didn't support lower data rates. It was introduced with Windows Media Player 11 (available for XP and included with Vista).

WMA 10 Pro added new "Low Bit-Rate" (LBR) modes from 32-96 Kbps that dramatically outperform competing codecs, including WMA 9.2. These can be up to twice as efficient as WMA, yielding dramatic bitrate reductions or improvements in quality over WMA 9.2. I recommend that all content targeting Silverlight 2 and/or Windows Media Player 11 use WMA Pro. It can really deliver danceable music at 48 Kbps.

MP3
Silverlight also supports standard .mp3 files for playback. This is useful for legacy content, but WMA 9.2 and particularly WMA 10 Pro outperform MP3, and should be used for content specifically targeting Silverlight.

Tip: Encode audio at 44.1 KHz
Silverlight always processes audio at 44.1 KHz internally. If you have audio sources at 48 KHz, you might as well convert it to 44.1 KHz when compressing. That'll save a little processing time on playback by avoiding the need for Silverlight to convert it before playback.

Content Protection
Being able to protect high-value content is an important feature of Silverlight, which provides a number of ways to make sure ad-supported content is viewed with the ads, and content can be secured from unauthorized use.

Web Playlists
Web Playlists are a new feature of the IIS 7 Media Pack, a free download for Windows Server 2008. They dynamically generate an ID for each media asset, ensuring that media is always played in order, including advertisements.

Silverlight DRM, Powered by PlayReady
Silverlight 2 adds support for digital rights management, using Microsoft's PlayReady technology. The Silverlight 2 client can use two forms of DRM: the traditional Windows Media Digital Rights Management 10, and the new PlayReady with enhanced AES encryption. This enables existing WMDRM-encrypted content to play back within both Silverlight and other Windows Media playback software that supports DRM, and Silverlight-only media to offer a higher degree of security. Either way, a new PlayReady license server will be required (which can be provisioned inside an organization, or via a DRM service provider).

Unlike other DRM technologies, Silverlight DRM doesn't work only in streaming mode. Silverlight DRM can be applied to live streams, on demand streams, progressive download content, and even a local file (with internet connectivity only needed to access the license server).

Delivery mechanisms for Silverlight
Progressive Download via server
For content that doesn't need real-time streaming, Silverlight supports reading files from any web server. Silverlight's support goes beyond what was in Windows Media Player, including lower start-up latency.

IIS 7 Media Pack
One new feature added in the free IIS 7 Media Pack for Windows Server 2008 is Bit Rate Throttling. This fixes the problem with traditional progressive download of users pulling down a video file much faster than needed. For example, a user with a fast connection might try to play back an hour-long 500 Kbps file over a 5000 Kbps connection. They may only watch the first six minutes of the file before they move on to something else, but in that six minutes, the entire hour of the video has been transmitted, costing extra bandwidth for content that'll never be watched.

With Bit Rate Throttling, progressive download includes an initial burst down to the client at high speed to enable immediate playback, but then ratchets down to send bits at the peak data rate of the file, preserving bandwidth while continuing to provide a high-quality experience.

Streaming via Windows Media Services
Real-time streaming of WMV files requires Windows Media Services (WMS), included in Windows Server 2003 and available as a free download for all versions of Windows Server 2008. Real-time streaming offers the fastest random access for on-demand content, and is required for live streaming.

Streaming Playlists
Streaming from Windows Media Services also enables allows the use of WMS server-side playlists, which allow features like can provide providing a list of failback servers if the original server isn't available, or define defining and enforce ad insertions, and obfuscate the URLs to the actual content. Silverlight 2 will provide support for a number of server-side playlist features.

Local playback
It's also perfectly possible to play back Silverlight from local storage, as long as the application itself doesn't require access to web services. For example, the video player templates in Expression Encoder play back fine off a local directory without any network access. Silverlight 2's support for the .NET CLR/DLR makes it possible to do richer client-side logic, but for media playback, the Silverlight 1.0 ECMAScript support can do quite a lot.

Move Networks
Move Networks is the company that provides the great HD streaming experience for high-profile broadcasters like ABC, the CW, Discovery, ESPN, and Fox. At this year's MIX08 event, Microsoft announced that Move is adding support for Silverlight to their delivery platform. The Move Networks technology provides seamless adaption to the current connection speed of each user, providing them the best quality their bandwidth can handle without any "buffering" messages. At NAB2008, Move Networks showed the first ever technology demo of Silverlight-enhanced experiences, further raising the bar for video delivery.

Codec modes and features for Silverlight
Once a project is determined to be either progressive download or streaming, the core decisions about compression modes can be made. There are basically two choices for best practices for Silverlight-targeted content: CBR or peak-limited VBR.

Constant Bitrate Encoding (CBR)
In CBR encoding, the average and peak bitrate of the content are the same. Basically, within a defined window of time, the data rate can't ever exceed the average. CBR can be thought of as varying the quality of the content in order to keep bitrate constant. CBR provides best results for real-time streaming, since that is fundamentally limited by peak bitrate.

In VC-1 Encoder SDK-based apps, 1-pass CBR is at least as good as the older 2-pass CBR mode, and should be used instead.

Live encoding always needs to be 1-pass CBR.

Variable Bitrate Encoding (VBR)
With VBR, the bitrate goes up or down throughout the file as the difficulty of the video for encoding changes. VBR can be thought of as varying the bitrate to keep the quality constant. Since we generally think of the quality of a file as being how good the worst parts look, VBR can be a lot more efficient in total file size than CBR, thus saving on bandwidth costs compared to streaming.

VBR is a great fit for both video and audio. However, only a limited range of audio bitrates support VBR mode. However, the quality improvements of VBR can be dramatic enough that it can be better to use a lower bitrate in VBR than higher in CBR (for example, WMA 10 Pro 64 Kbps VBR can often sound better than WMA 10 Pro 80 Kbps CBR).

The main VBR mode that should be used for Silverlight is VBR Peak Constrained, which lets both average and peak bitrate be specified. Average bitrate determines the overall file size, and peak bitrate determines how difficult the hardest point of the video can be to decode and transmit. The peak bitrate should be at least 50% higher than the average.

Windows Media ecosystem
VC-1 Encoder SDK
The new kid on the block for WMV content creation is the VC-1 Encoder SDK, which offers big improvements in quality, performance, and control compared to the previous Format SDK. The VC-1 Encoder SDK is free for the "Pro" version, with a commercial "Enterprise" version also available for products with high-end requirements for live broadcasting or multi-machine encoding.

Inlet's Fathom and Spinnaker were the first professional products shipping with the VC-1 Encoder SDK, recently followed by Microsoft's own Expression Encoder 2.0. A number of partners have announced they will be releasing their own products incorporating the SDK, including Anystream, Envivio, Inlet Technologies, Memory-Tech, Rhozet (Harmonic Inc.),Tarari, and ViewCast. Further companies are working on VC-1 Encoder SDK based products they have not yet announced.

About Ben Waggoner
Ben Waggoner has been compressing video for nearly 20 years. He cofounded the pioneering digital media services company Journeyman Digital, launched Terran Interactive's consulting services division, and was principal of Interframe Media. Ben joined Microsoft in 2005, and is now Principal Video Strategist for Microsoft's Silverlight platform. He is the author of "Compression for Great Digital Video" from Focal Press.
MIX essentials - see it all online
All sessions are available on-demand on our online video platform called Chopsticks. Review sessions you missed or see it all for the first time if you did not make it to the event. In addition to the recorded sessions, all PowerPoint presentations are also available for download (click the Resources tab next to the video).

Follow us on Twitter @mixessentialsbe, we'll keep you posted on any news regarding MIX essentials, Silverlight and Expression.
DreamSpark
The Microsoft DreamSpark student program makes available, at no charge, a broad range of development and design software for download. This includes free access to several commercial products, including Visual Studio 2008 Professional, the Expression suite and XNA Game Studio.
Expression Studio 2 is now available
Download a trial of the new Expression Studio. New features in Blend 2: Silverlight support, vertex animations and an improved user interface with a new split design/XAML view. Expression Web 2: now with PHP support, Photoshop import functionality, improved web standards compliancy, and ASP.NET Ajax server control support and more.
Silverlight rehab video
Folks at the Microsoft Silverlight team in Redmond are quite creative, or are they really in need of rehab? Watch and have fun.
Microsoft