I was just about to write my own PowerCLI script to update the timeout setting across multiple hosts/clusters when I ran across this post with the script already written. You saved me some time, thanks!

ICT-Freak.nl

When I tried to Storage vMotion a VM to another LUN I got an “operation timed out error”:

After analyzing the vpxd.log, I found the following lines:

The best part I found out, is that the VM was rolled back to it’s old location without losing data or getting corrupt.

This problem is documented in KB1010045. The following solution will resolve the timeout problem:

Increase the Storage VMotion fsr.maxSwitchoverSeconds setting in the virtual machine configuration file to a larger value.

The default is 100 seconds.

To modify fsr.maxSwitchoverSeconds:

1. Right-click the virtual machine and click Edit Settings.
2. Click the Options > Advanced > General.
3. Click Configuration Parameters.
4. From the Configuration Parameters window, click Add Row.
5. For the Name field,  fsr.maxSwitchoverSeconds and for the Value field enter the new timeout value.
6. Click OK twice to save.
7. Restart the virtual machine for the changes to take effect.

You…

View original post 86 more words

Item template for creating custom functoid classes

It’s hard to find useful Biztalk-related templates for VisualStudio especially one as useful as this item template that Sandro Pereira brought to my attention for creating Custom Functoid Classes. Thanks Sandro.

Sandro Pereira BizTalk Blog

Following my previous post here’s another small contribution: Item Template for Visual Studio 2010: Custom BizTalk Functoid.

The New Functoid Class item template for Visual Studio 2010 allows you to create new Functoids for BizTalk Server 2010 without having to code the entire class and therefore allowing you to focus on what really matters, the functoid behavior. Just start a new Visual Studio C# library project and add a functoid class to start building your BizTalk Server 2010 custom Functoid.

How can I install the Item Template?

To use this item template in your C# library projects just download the project (Don´t unzip) and copy the compressed file to C:\Users\UserName\Documents\Visual Studio 2010\Templates\ItemTemplates\VisualC#.

Now when you add a new item to your project you will now see a new option: “Custom FunctoId Class”

I really like to engage others people in this small projects, this time I was lucky to…

View original post 44 more words

Randal van Splunteren has written an excellent article, Dealing with base64 encoded XML documents in BizTalk which I think is one of the most useful Biztalk write-ups that I have run across to date.  He covers several topics which are often very complex and yet breaks it down into simple steps that make it sound easy enough that even a novice could do including Base64 encoding/decoding, extracting/reinjecting a binary payload using pipelines so as to keep it out of the MessageBox, and more. 

— From http://biztalkmessages.vansplunteren.net/articles/dealing-with-base64/ —

When you work with BizTalk for a while there is a big chance you come across a scenario where you need to process a XML document that contains a base64 encoded document like for example a .tiff or .PDF.

Below you find a very simple example of such a document:

The ‘Document’ node in the above example contains the base64 encoded contents. For readability I truncated the contents of this node. The inefficient nature of base64 encoding causes the size of the node contents to be enormous.

Most of the “base64 scenarios” I have seen have a common set of steps that need to be executed in order:

1. Receive XML document with embedded base64 encoded document in BizTalk.
2. Process the XML part of the document by an orchestration.
3. Send the base64 encoded document to a back-end system.

Also in most cases there is no actual need for the base64 encoded document to travel along with its XML container message through BizTalk. Bringing such documents in BizTalk (especially in an orchestration) comes with a performance penalty and is resource intensive. Other XML nodes in the message are needed and processed by for instance an orchestration but the base64 encoded document is just sent to a back-end system. A typical example is a scenario where you need to feed a document management system using a web service.

Although I’ve seen companies use different solutions they all come down to the same basic pattern:

• Strip off, decode and store the base64 encoded document in a temporary store (file system) when received by BizTalk.
• Process the XML message (without bae64 encoded document) in an orchestration.
• Load, encode and add the document back in to the XML message just before sending to a back-end system.

Key things that should be taken into account when implementing this pattern:

• Strip off the base64 content as soon as it is received by BizTalk. In other words use a receive pipeline. This will keep the document out of the message box.
• Add the document back to the content at the last possible moment. In other words use send pipeline.
• Use a streaming approach in the pipelines to prevent from high memory footprints and optimize through put.
• Optionally decide not to decode and encode the base64 document but just store it encoded in the temporary file store to optimize performance.

The following figure illustrates the pattern:

I’m aware that there has been written a lot about BizTalk, streaming and pipelines. I couldn’t find anything on the above topic however. In particular there are no samples. Besides that I’ve seen people make nasty mistakes in their attempt to implement above pattern .

Because of the above mentioned reasons and because I recently had to implement this pattern myself for a customer I decided to provide a downloadable sample.

Receiving side:

In order to to remove and store the content of the document we need to write a pipeline component. The component needs to redirect the destination stream to a file on the right moment and switch back to the original stream after that. The component needs some configuration information to determine where to store the temporary document and where it can find the document in the XML message. Per instance pipeline configuration is used to configure and store this information.

The pipeline component execute method is very simple:

The method does a couple of things. It first generates a filename. This means it replaces the %Guid% macro in the filename with a new guid. Next a custom XmlReader class is instantiated. This class does the actual work and is a wrapper around the XmlTextReader class (see below). I use the XmlTranslator class to ‘convert’ to reader back into a stream. Note that this implementation does not actually do anything with the message it just connects the streams and waits for the stream to be pulled. Finally this method writes the name of the generated document file on the context. It does this using the configured key and namespace.

The custom XmlReader is wrapper class around the XmlTextReader class with a specialized implementation of the ‘Read’ method:

The method checks to see whether or not the element configured in the pipeline configuration is found. I know it would be better when the Read method would not just check the local name but also checks if the namespace of the element

If the element is found the method ‘Base64DecodeDocument’ is called to write the contents (chunk by chunk) to a file stream using the BinaryReader class. In fact this method pulls the stream until it reaches the end of the base64 encoded content.

After the pipeline is executed the XML message will contain an empty node and a decoded file is stored in the temporary storage location.

Sending side:

The sending side needs to do the reverse of the receiving side. It has to set the source stream to a file instead of original stream and switch back on the right moment. This is also implemented as a pipeline component. In this case the execute looks like:

The pipeline component has the following configuration information:

The ‘ElementTo…’ properties tell the component which element must be created to put the base64 encoded contents in. The ‘FilePath..’ properties are used to tell the component what context property to use to get the filename of the stored document. The ‘PreceedingElementName’ is the name of the element after which the new element will be created. An alternative could be to have a map create a new and empty node. The pipeline component than only has to fill the contents of that node.

The execute method as indicated above first gets the filename of the document from the context. After that it instantiates a ‘RetrieveAndAddDocumentStream’ class and passes the configuration information.

The ‘RetrieveAndAddDocumentStream’ is a subclassed version of the ‘XmlTranslatorStream’ class. The ‘TranslateEndElement’ method of this class will detect the element configured in the variable ‘PreceedingElementName’ and put the class in the state ‘ElementFound’. After this the new element is created and the contents is not read from the XML stream but from the file stream. This happens until the complete file is read. Finally a closing tag is written.

Sample:

I’ve put together a sample that receives a XML message with base64 encoded contents. It strips off the contents, decodes it and saves it to a file. After that the message without document is send to the message box. A send port picks up the message and puts the document back in. There is no orchestration that processes the XML message. I just want to show the working of the pipeline components redirecting the streams back and to the file.

The sample source code can be downloaded from here.

A working .MSI file can be downloaded from here. Please read the setup instructions for this package first!

In order to run the sample you need to build both solutions, put the pipeline components in the GAC and deploy the pipeline project to BizTalk. A binding file is provided that will create the necessary ports. If you extract the .zip file to ‘C:\Sample.Pipelines\’ the binding file will have the correct paths.

Additional considerations:

• Make sure the temporary storage location is reliable. You can for example use a clustered or mirrored file system.
• Think about error handling and resubmit. If something goes wrong while processing the XML message be aware that the message is not the same as the message received by BizTalk. The base64 content is missing in the first message
• Also think about the right moment to remove the file from the temporary storage location. You probably want to do this when document has been put back into the XML message.

Envelope Schemas in BizTalk

Reblogged from BizTalk Envelopes explained – Robert Kokuti:

Sunday, May 16, 2010 10:46 AM

Recently I’ve been trying to get some order into an ESB-BizTalk pub/sub scenario, and decided to wrap the payload into standardized envelopes. I have used envelopes before in a ‘light weight’ fashion, and I found that they can be quite useful and powerful if used systematically. Here is what I learned.
The Theory

In my experience, Envelopes are often underutilised in a BizTalk solution, and quite often their full potential is not well understood. Here I try to simplify the theory behind the Envelopes within BizTalk.

Envelopes can be used to attach additional data to the ‘real’ data (payload). This additional data can contain all routing and processing information, and allows treating the business data as a ‘black box’, possibly compressed and/or encrypted etc. The point here is that the infrastructure does not need to know anything about the business data content, just as a post man does not need to know the letter within the envelope.
BizTalk has built-in support for envelopes through the XMLDisassembler and XMLAssembler pipeline components (these are part of the XMLReceive and XMLSend default pipelines). These components, among other things, perform the following:
XMLDisassembler

1. Extracts the payload from the envelope into the Message Body
2. Copies data from the envelope into the message context, as specified by the property schema(s) associated by the envelope schema.

Typically, once the envelope is through the XMLDisassembler, the payload is submitted into the Messagebox, and the rest of the envelope data are copied into the context of the submitted message. The XMLDisassembler uses the Property Schemas, referenced by the Envelope Schema, to determine the name of the promoted Message Context element.

XMLAssembler

1. Wraps the Message Body inside the specified envelope schema
2. Populates the envelope values from the message context, as specified by the property schema(s) associated by the envelope schema.

Notice that there are no requirements to use the receiving envelope schema when sending. The sent message can be wrapped within any suitable envelope, regardless whether the message was originally received within an envelope or not. However, by sharing Property Schemas between Envelopes, it is possible to pass values from the incoming envelope to the outgoing envelope via the Message Context.
The Practice
Creating the Envelope

Add a new Schema to the BizTalk project:

Envelopes are defined as schemas, with the <Schema> Envelope property set to Yes, and the root node’s Body XPath property pointing to the node which contains the payload.
Typically, you’d create an envelope structure similar to this:

Click on the <Schema> node and set the Envelope property to Yes.

Then, click on the Envelope node, and set the Body XPath property pointing to the ‘Body’ node:

The ‘Body’ node is a Child Element, and its Data Structure Type is set to xs:anyType. This allows the Body node to carry any payload data. The XMLReceive pipeline will submit the data found in the Body node, while the XMLSend pipeline will copy the message into the Body node, before sending to the destination.
Promoting Properties

Once you defined the envelope, you may want to promote the envelope data (anything other than the Body) as Property Fields, in order to preserve their value in the message context. Anything not promoted will be lost when the XMLDisassembler extracts the payload from the Body. Typically, this means you promote everything in the Header node.
Property promotion uses associated Property Schemas. These are special BizTalk schemas which have a flat field structure. Property Schemas define the name of the promoted values in the Message Context, by combining the Property Schema’s Namespace and the individual Field names.
It is worth being systematic when it comes to naming your schemas, their namespace and type name. A coherent method will make your life easier when it comes to referencing the schemas during development, and managing subscriptions (filters) in BizTalk Administration. I developed a fairly coherent naming convention which I’ll probably share in another article.
Because the property schema must be flat, I recommend creating one for each level in the envelope header hierarchy.
Property schemas are very useful in passing data between incoming as outgoing envelopes. As I mentioned earlier, in/out envelopes do not have to be the same, but you can use the same property schema when you promote the outgoing envelope fields as you used for the incoming schema. As you can reference many property schemas for field promotion, you can pick data from a variety of sources when you define your outgoing envelope. For example, the outgoing envelope can carry some of the incoming envelope’s promoted values, plus some values from the standard BizTalk message context, like the AdapterReceiveCompleteTime property from the BizTalk message-tracking properties. The values you promote for the outgoing envelope will be automatically populated from the Message Context by the XMLAssembler pipeline component.

Using the Envelope

Receiving

Enveloped messages are automatically recognized by the XMLReceive pipeline, or any other custom pipeline which includes the XMLDisassembler component. The Body Path node will become the Message Body, while the rest of the envelope values will be added to the Message context, as defined by the Property Shemas referenced by the Envelope Schema.

Sending

The Send Port’s filter expression can use the promoted properties from the incoming envelope.

If you want to enclose the sent message within an envelope, the Send Port XMLAssembler component must be configured with the fully qualified envelope name:

One way of obtaining the fully qualified envelope name is copy it off from the envelope schema property page:

The full envelope schema name is constructed as <Name>, <Assembly>

The outgoing envelope is populated by the XMLAssembler pipeline component. The Message Body is copied to the specified envelope’s Body Path node, while the rest of the envelope fields are populated from the Message Context, according to the Property Schemas associated with the Envelope Schema.

That’s all for now, happy enveloping!

I am currently working on two different Biztalk Solutions which need to perform multiple dynamic manipulations preferably BEFORE anything is stored to the Message Box, but I wasn’t sure about the execution order of pipeline components vs. Inbound/Outbound Maps on ports. Fortunately, I have found that answer from yet another one of Thiago Almeida’s excellent post’s.

[INACTIVE BLOG] Connected Thoughts - Thiago Almeida

 

For future reference to all those that want to remember in what order maps and pipelines get executed on receive and send ports:

One-way Receive Port:

End system –> Receive Adapter –> Receive Pipeline –> Inbound Map –> Message Box

Request Response Receive Port:

End system –> Adapter –> Receive Pipeline –> Inbound Map -> Message Box-> Outbound Map –> Send Pipeline –> Adapter –> End System

One-way Send Port (static and dynamic):

Message Box –> Outbound Map –> Send Pipeline –> Send Adapter –> End System

Request Response Send Port (static and dynamic):

Message Box-> Outbound Map –> Send Pipeline –> Adapter -> End system –> Adapter –> Receive Pipeline –> Inbound Map -> Message Box

 

See the pattern? Pipelines always get executed right before or right after the end system adapter. And maps are always executed right before or right after the…

View original post 3 more words

BizTalk Server: Basics principles of Maps – Testing and Validation of maps (at design time) (Part 6)

Sandro Pereira posted the 6th of a series titled, ‘BizTalk Server: Basics principles of Maps’ which is probably the best explanation I have seen since I started working with Biztalk Server.

BizTalk Server: Basics principles of Maps – Testing and Validation of maps (at design time) (Part 6).

Here is a great explanation of how to use polling with the WCF-SQL adapter.

Richard Seroter's Architecture Musings

A few years back now (sheesh, that long already??) I wrote a post about debatching messages from the classic BizTalk SQL adapter.  Since that time, we’ve seen the release of the new and improved WCF-based SQL adapter.  You can read about the new adapter in a sample chapter of my book posted on the Packt Publishing website.  A blog reader recently asked me if I had ever demonstrated debatching via this new adapter, and to my surprise, I didn’t found anyone else documenting how to do this.  So, I guess I will.

First off, I created a database table to hold “Donation” records.  It holds donations given to a company, and I want those donations sent to downstream systems.  Because I may get more than one donation during a WCF-SQL adapter polling interval, I need to split the collection of retrieved records into individual records.

After creating a new…

View original post 571 more words

Nested FOR XML results with SQL Server’s PATH mode

Richard Dingwall offers a simple explanation of how to use the SQL Server’s FOR XML in PATH mode here: http://richarddingwall.name/2008/08/26/nested-xml-results-from-sql-server-with-for-xml-path/

Multiple Stored Procedure Calls using the BizTalk SQL Adapter without a loop shape

Thiago Almeida has written a nice article which details how to call a stored procedure multiple times without using a loop shape here: http://connectedthoughts.wordpress.com/2008/09/09/multiple-stored-procedure-calls-using-the-biztalk-sql-adapter-without-a-loop-shape/

Biztalk – Orchestration Handling of Suspended Messages

Richard Seroter has written an excellent article which describes the behavior of resuming a suspended message in Biztalk Server at http://seroter.wordpress.com/2007/01/02/orchestration-handling-of-suspended-messages/