E-mail us with your comments and feedback about this article.
Abstract
Bandwidth on the Internet and intranets is highly variable and unreliable, two properties that can make streaming digital media content difficult. To help maintain a continuous, unbroken stream of data, Windows Media technologies use intelligent streaming. This article describes how intelligent streaming works and shows you how to create content that takes advantage of this feature.
Bill Birney
Microsoft Corporation
May 2003
Applies to:
Microsoft® Windows Media® Encoder 9 Series
Microsoft Windows Media Services 9 Series
Microsoft Windows Media Player 9 Series
Introduction
Intelligent streaming is a set of features in Microsoft® Windows Media® Technologies that automatically detects network conditions and adjusts the properties of a stream to maximize the quality of the user experience. Today's Internet connections are highly variable in terms of actual throughput achieved for any specific connection and range of possible connection speeds. For example, a user with a laptop computer can connect to an Internet service provider (ISP) with a 300-kilobit-per-second (Kbps) Digital Subscriber Line (DSL) connection at home, a 1.5-megabit-per-second (Mbps) T1 connection at work, and a 56.6-Kbps modem connection while traveling on business. Furthermore, the actual throughput achieved in each of these scenarios is likely to vary. This is especially important for low-bandwidth modem connections, where the connection speed can vary by 50 percent or more of the maximum, depending on network and ISP congestion.
In a Windows Media streaming system, the Windows Media server and the client (typically running Windows Media Player) communicate with each other to establish actual network throughput and make a series of adjustments to maximize the quality of the stream. Intelligent streaming offers dramatic benefits to the user. It maximizes use of available bandwidth; in a DSL or local area network (LAN) environment, users receive content tailored to their connection speed. It greatly improves the user experience; users connected by modem immediately notice the presentation is smoother, less jerky, and of generally higher quality.
The most difficult task of streaming audio and video over a network is maintaining a continuous presentation to the user in a highly changeable environment. Buffering is the biggest problem of streaming digital media. It is caused when the client runs out of data in memory, called the buffer, and must wait for more to arrive. The client will always run out of data if the bit rate of the incoming stream exceeds the current available bandwidth.
Unpredictability of bandwidth is taken for granted on the Internet. For example, just because a user connects to an ISP at 56 Kbps does not mean there is 56 Kbps of bandwidth available to the user at all times. The actual bandwidth at any given point is determined by network conditions and traffic on the Internet, which is constantly fluctuating, causing bandwidth to drop to 18 Kbps one moment, and then increase to 40 Kbps the next. If a user attempts to view video being streamed at 50 Kbps, the presentation suffers considerably when bandwidth is squeezed.
To ensure a continuous presentation, Windows Media technologies use intelligent streaming, which adjusts the bit rate of the content stream to counteract changes in available bandwidth. When a user first connects to a stream, intelligent streaming ensures that the client receives content at the bit rate that is most appropriate for the current bandwidth. As the user continues to play a stream, intelligent streaming dynamically and seamlessly adjusts the bit rate of the streaming content as the available bandwidth changes.
Multiple-Bit-Rate Encoding
To take full advantage of intelligent streaming, content must be encoded as a multiple-bit-rate stream. In multiple-bit-rate encoding, a number of discrete, user-definable audio and video streams are encoded into a single Windows Media stream. The streams are encoded from the same content, but each is encoded at a different bit rate. When Windows Media Player connects to a Windows Media server to receive a multiple-bit-rate Windows Media file or broadcast stream, the server only sends the set of audio and video streams that is the most appropriate for current bandwidth conditions. The process of selecting the appropriate stream is completely transparent to the user.
Intelligent Bandwidth Control
Intelligent streaming uses a series of strategies to modify the bit rate of the stream so it plays continuously on the client regardless of the current bandwidth. As bandwidth fluctuates between server and client, the server detects the changes and adopts the best strategy. The strategies are as follows:
The server and client automatically determine the current available bandwidth, and then the server selects and delivers the stream with the most appropriate bit rate.
During transmission, if the bandwidth decreases, the server automatically detects the change and switches to a stream with a lower bit rate. If bandwidth improves, the server switches to a stream with a higher bit rate, but never higher than the original bit rate.
If the bandwidth can no longer support streaming video, the client and server intelligently degrade image quality in order to avoid buffering. This method of decreasing the bit rate to accommodate the current bandwidth is called stream thinning. The server decreases the video frame rate first. If the bit rate is still too high, the server stops sending video frames altogether.
After the server stops sending video frames, it uses intelligent streaming to attempt to maintain a continuous audio stream. If audio quality starts to degrade, the client reconstructs portions of the stream to preserve quality.
When bandwidth is at its best, the server employs the first strategy. As conditions worsen, the server tries each strategy in the previous list of options one by one until the bit rate is optimized for the current bandwidth.
Intelligent Image Processing
The client post-processes the video stream to enhance quality even at very low bit rates. Windows Media technologies include an intelligent filtering technology, which works in conjunction with the Windows Media codecs and the Player to smooth blockiness and remove ghosting artifacts, significantly improving the overall appearance of the video.
Blockiness also occurs during the decoding of high-bit-rate streams, but it is not as noticeable. A streaming media codec, such as the Windows Media Video codec, encodes a video image by breaking it up into pixels. The lower the bit rate, the fewer the pixels. When too few pixels are used to create an image, the pixels become visible as blocks. The client post-processing filter used in intelligent streaming smoothes the edges of the blocks and erases certain other artifacts, such as ringing, so the resulting image is more pleasing to the eye.
To encode content that takes advantage of intelligent streaming, you can simply use one of the multiple-bit-rate destinations provided in Windows Media Encoder. For greater control, you can manually select the bit rates for each audio and video stream within a multiple-bit-rate stream. Client post-processing and intelligent bit rate optimization are all automatic, on-the-fly features. Best of all, you only need to create and manage a single file to handle multiple bit rates.
To understand the following topics, you should be familiar with the operation of Windows Media Encoder 9 Series. For more information about encoding, see Windows Media Encoder Help.
Setting Up the Encoder
To set up a multiple-bit-rate encoding session, do the following:
Open the Session Properties dialog box in the encoder, and click the Compression tab.
In Destination, click Windows Media server (streaming).
In Audio and Video, click the type of audio and video you want to encode.
Select more than one of the bit rate check boxes. The number of bit rate choices changes depending on the audio and video type you chose.
If you want more manual control of the encoding session, on the Compression tab, click Edit.
Optionally, in the Custom Encoding Settings dialog box (shown in the following figure), you can add and configure custom target bit rates, and then modify the settings of each bit rate by clicking the corresponding tab. Multiple-bit-rate encoding is only available if you select the constant-bit-rate (CBR) encoding mode.
If you want to use intelligent streaming with multiple-bit-rate streams, the content must be hosted on a Windows Media server and played in Windows Media Player. You can create files that contain multiple-bit-rate streams and play them locally or from a Web server, but only the highest bit rate streams will play.
For more information about editing or creating profiles that encode with multiple bit rates, see Windows Media Encoder Help.
Viewing the Process
After you have configured an encoding session for capturing multiple-bit-rate content and have connected and adjusted the video and audio streams, start encoding. If your computer meets the requirements for encoding high-bit-rate content, you can capture and encode multiple-bit-rate content. The more streams you encode, however, the faster the CPU and more memory your computer requires.
To monitor the encoder as it is working, view the Monitor panel.
Figure 2. Encoder Monitor panel
Watch the CPU load counter. When CPU usage is at 80 percent or less, the encoding computer has enough processing power to handle rapid increases in frame detail. If the encoding computer is too slow, however, the processor may reach maximum capacity. On a computer with a 400 megahertz (MHz) processor, for instance, CPU usage might immediately climb to 100 percent when encoding starts. Although this level of processor usage is normal when encoding content from one file to another, high CPU usage when capturing content indicates that the encoder is more likely to drop or discard frames.
Certain fields in this panel change continuously as encoding proceeds. Under Encoding, the Total bit rate counter can change dramatically as the amount of video detail per second fluctuates. This variation illustrates how the encoder continuously adjusts parameters to maintain the current bit rate as closely as possible for the bandwidths selected in the profile. The Total bit rate counter shows the combined bit rates of all the streams being encoded at a particular point in time.
A multiple-bit-rate Windows Media file is larger than a single-bit-rate file of the same length because of the extra streams. Likewise, when encoding a live multiple-bit-rate stream, the bandwidth of the connection between the encoder and the server must be larger. In the previous illustration, three bit rates were selected for a total average bit rate of around 524 Kbps. To handle the output of the encoder, therefore, the bandwidth between the encoder and server must be at least 524 Kbps, with enough additional bandwidth for other network traffic.
When a client attempts to connect to the server to receive a broadcast or file, the server determines the current bandwidth of the connection. For example, suppose a user connects with a 56-Kbps analog modem, but does so at a time when network traffic is particularly high. When the user connects, the server might determine the actual available bandwidth to be 40 Kbps. At that bandwidth, a 53-Kbps stream is too high, so the server streams a 31-Kbps video stream and the audio stream.
After 10 minutes, network congestion increases, and bandwidth suddenly falls to 32 Kbps. The client's frame rate begins to decrease, and some packets are lost, but the server reacts immediately by switching to a 19-Kbps stream. The user notices some degradation in image quality, but audio is continuous and disruption of the presentation is minimized. Then, 20 minutes later, bandwidth worsens again, decreasing to 14 Kbps. This is even too low for the lowest video stream, so the Windows Media server stops streaming all video to the client. The user notices the loss of video but is still able to listen to an uninterrupted audio stream, which only requires 8 Kbps of bandwidth. A few minutes later, bandwidth improves a great deal, and the server again is able to send the 31-Kbps video stream.
The negotiation between server and client is handled automatically and seamlessly. There are no manual adjustments necessary on either end. If multiple-bit-rate streams are available to the server, it uses them. The only thing you have to do as producer is to make sure the multiple-bit-rate streams are available. If one of the multiple-bit-rate destinations is not exactly right for your needs, you can edit or create a new profile.
Monitoring Performance
In the multiple-bit-rate environment, the most important concern is CPU speed and memory on the encoding computer. The greater the CPU speed, the more processing power is available to the encoder, and the more streams it can encode. For information about system requirements, see the Windows Media Encoder 9 Series section of the Microsoft Web site.
Monitoring the performance of your CPU and memory resources is a simple way to monitor the quality of your encoding. The more streams, the more frames per second, the larger the image size, and the higher the quality that you specify, the more tasks per second the CPU must perform. Windows Media Encoder automatically adjusts its task load to the given bandwidth and to the limits of the CPU. For example, if you enter a high frame rate (30 frames per second) and a low bit rate (28.8 Kbps), and then use CBR encoding, Windows Media Encoder maintains a constant bit rate and attempts to achieve the requested frame rate by lowering image quality.
Your CPU can become overburdened, especially during multiple-bit-rate encoding. When the number of tasks per second is too great for the processing power of your CPU, the encoder adjusts to the environment by dropping frames. An occasional dropped frame during a high-action sequence may not be that noticeable, but image quality and frame rate can be degraded when your CPU usage is at 100 percent most of the time. For the best quality, reduce the number of streams, the image size, or the frames per second until usage is no higher than 80 percent.
When encoding content from one file to another, Windows Media Encoder adjusts to the processor speed. Because time is not an issue, the encoder takes as long and uses as much CPU bandwidth as necessary to render the digital media without compromising quality. A 30-second file can take five minutes to encode on one machine and 10 seconds on another depending on processor speed. The encoder maximizes use of the CPU to keep encoding time to a minimum. You can see that the CPU is at 100 percent, but unlike live encoding, this does not mean frames are being dropped.
Controlling Intelligent Streaming
For the most part, intelligent streaming works automatically. However, you can control how a Windows Media server delivers a stream to a client by adding a modifier to the URL that the client uses to access a stream. You add a modifier the same way you add a query string to a URL. After the URL string, add a question mark character (?), followed by the modifier and modifier value.
For example, to disable stream thinning, which in turn disables intelligent streaming, you would use the WMThinning modifier with a value of 0, as shown in the following URL:
mms://WMServer01/File.wmv?WMThinning=0
Modifiers can be used anywhere a source value is specified, such as a src attribute in a server-side playlist, a client-side playlist, or even directly in the Open URL dialog box in Windows Media Player 9 Series. The following table describes three modifiers.
Modifier
Value
WMThinning
Specifies whether stream thinning is enabled. A value of 1 specifies that stream thinning is enabled; 0 disables stream thinning and intelligent streaming.
WMContentBitrate
Specifies the maximum bit rate that can be streamed from a source, in bits per second. You can use this modifier with the WMBitrate modifier to manually control which stream in a multiple-bit-rate stream is selected when intelligent streaming is not desired.
WMBitrate
Specifies the bit rate that the Windows Media server cannot exceed when delivering a stream, in bits per second.
For more information about all the modifiers and how to use them, see Windows Media Services 9 Series Help.
Intelligent streaming is, for the most part, completely automatic. The interplay between client and server takes place behind the scenes. If you have added multiple-bit-rate streaming, the server can intelligently adjust the bit rate according to the current bandwidth, so the user receives a high-quality presentation.
The goal when using intelligent streaming is to make bit rates and bandwidth transparent to the user; the user should only be aware of the content, not all the processes that work together to deliver it. By using Windows Media technologies, you can create a single Windows Media file or encode a single live stream for all users to enjoy, regardless of connection speeds and network conditions.
Back to the top
Back to the top
