Sign in

    Developing a Streaming Pipeline Component for BizTalk Server

    This paper shows how to address issues with high memory consumption and latency by taking a streaming approach to pipeline component development.
    • Note:There are multiple files available for this download.Once you click on the "Download" button, you will be prompted to select the files you need.


      File Name:



      Date Published:


      File Size:

      161 KB

      488 KB

        Orchestrations are often the first port of call for most BizTalk developers and are the feature most identified with the product. However, many scenarios can be implemented very well using messaging alone, harnessing the power of port configuration and pipelines. Any serious messaging implementation is bound to involve one or more custom pipeline components, and these are not difficult to develop. Nonetheless, too often developers ignore the streaming fashion of the pipeline and write components that read the entire message into memory. Although this approach does work, and admittedly is somewhat easier to grasp, it has two significant downsides:

        • Higher memory consumption – the entire message has to be loaded into memory, whatever the message size is. In addition, if the .NET XML DOM is used (via the XmlDocument class, for example) the memory footprint of the component can be considerably bigger than the message size.

        • Latency – any further processing of the message has to wait until the current component has finished processing the entire message.

        Both of these downsides can be addressed by taking a "streaming" approach to pipeline component development, which is what this paper attempts to demonstrate. The sample provided with this paper is designed to demonstrate both the non-streaming and the streaming ways of achieving the solution.
    • Supported Operating System

      Windows Server 2003, Windows Server 2008, Windows Vista

        To view the document, use Microsoft Word. To run the sample, see the directions in Appendix B of the paper.
        1. Click the Download button on this page to start the download.
        2. Do one of the following:
          • To start the installation immediately, click Open or Run this program from its current location.
          • To copy the download to your computer for installation at a later time, click Save or Save this program to disk.