Thursday, April 19, 2007

Peeping Inside BizTalk


Developers can easily develop Interfaces using BizTalk. On the Stage the BizTalk Show looks cool but have you thought about who writes the Scripts in background. Its only Drag and Drop for developers and once the solution is deployed and it's up to be tested. At times one must have thought what happens when I drop a message, where it goes and how the whole mechanism works. Just go through the artifacts below ….its worth having a look and I bet you would say….”Hey I dint knew this at all”.
Message
1.      Information about the promoted properties is extracted and stored in the bts_documentSpec table in the Management database.
2.      Items that are marked with PropertyField annotations in your message schema will lead to the pipeline disassembler putting a Promoted property in the context. Items that are marked with DistinguishedField annotations in your message schema will lead to the pipeline disassembler putting a Written property into the context.
3.      One of the benefits of promoted properties is that the value of the element that is promoted is available in the context of the message. This means that retrieving that value is inexpensive, as it does not require loading the message into memory to execute an XPath statement on the message. Instead, a simple property bag can be used along with a key to get the value.
4.      Writing a value into the context with the same name and namespace that were used previously to promote the property causes that property to no longer be promoted. The write essentially overwrites the promotion.

Suscriptions:-Instance, Activation
1.      An activation subscription is one specifying that a message that fulfills the subscription should activate, or create, a new instance of the subscriber when it is received. Examples of things that create activation subscriptions include send ports with filters or send ports that are bound to orchestrations, and orchestration receive shapes that have their Activate property set to true. An instance subscription indicates that messages that fulfill the subscription should be routed to an already-running instance of the subscriber. Examples of things that create instance subscriptions are orchestrations with correlated receives and request/response-style receive ports waiting  for a response from BizTalk Server.

2.      The difference between the two types of subscription at the information level is that an instance subscription includes the unique instance ID, stored in the subscription table in the master MessageBox database. When an  orchestration instance or receive port completes processing, instance subscriptions are removed from the MessageBox while activation subscriptions remain active as long as the orchestration or send port is enlisted.
ReceiveSubscription

Picture above shows what Biztalk actually does when a message comes in or is going out.We dont see it in Biztalk architecture diagram.

What Adapter does with the Message
1.      The adapter creates a message (an implementation of the Microsoft.BizTalk.Message.Interop.IBaseMessage interface), adds a part to it (an implementation of the Microsoft.BizTalk.Message.Interop.IBasePart interface), and provides the  stream of data as the part content, the adapter writes and promotes into the message context properties related to the location, adapter type, and others related to the adapter. After the message and its context have been created, the adapter passes the message to the Endpoint Manager. The message is then processed through the receive pipeline, which has been configured for the receive location. After the message has been processed by the pipeline, a map may  be used to transform the message into the format desired before the Endpoint Manager publishes the message with the  Message Agent.

Out of Memory Problems: Why?
1.      Routing only If BizTalk Server is only used only for routing messages based upon promoted message properties, then  the message is streamed into the Messagebox database using the .NET XmlReader interface, and message parts are not  individually loaded into memory. In this scenario, out of memory errors are not an issue and the primary  consideration is the amount of time that is required to write very large messages (over 100 MB) into the Messagebox  database.The BizTalk Server development team has successfully tested the processing of messages up to 1 GB in size   when performing routing only.

2.       Mapping Transforming a document with a map is a memory-intensive operation. When a document is transformed by a map, BizTalk Server passes the message stream to the .Net XslTransform class, which then loads the document into a .NET  XPathDocument object for processing. Loading the document into the .NET XPathDocument can potentially expand the original file size in memory by a factor of 10 or more. This expansion may be more pronounced when mapping flat files because flat files must be parsed into XML before they can be transformed.

Solution:-
1.       Adjust the message size threshold above which documents are buffered to the file system during mapping. To modify the size threshold, create a DWORD value named TransformThreshold at the following location in the BizTalk Server  registry:
        HKLM\Software\Microsoft\BizTalk Server\3.0\Administration\TransformThreshold
        After you have created this value, enter a decimal value with the number of bytes to set the new threshold to. For  example, enter a decimal value of 2097152 to increase the message size threshold to 2 MB (from the default of 1 MB). Increase this value on systems with a large amount of available memory to improve throughput. Buffering documents to disk conserves memory at a slight cost to overall throughput.
All Inputs above are Courtesy to Microsoft.