Increasing BizTalk map performance by mapping in C#
I have been struggling with some performance issues with mappings last year. The received messages could get up to 200 MB and a lot of data had to be looked up from other records in that same request message during the mapping. The only way to improve performance was by using XSLT, on which I wrote a post last year. Eventually even that was not enough and I had to resort to using C#. I used Linq to query all the records I needed and performed the mapping one-by-one, and eventually assembled the message again after at the end. I totally agree… not the nicest solution, but I needed something quickly (I didn’t want to reinvent the complex map) and it worked. I reduced the mapping time from running all night (over 6 hours!) to approx 20 minutes.
But it made me think. There should be another, better, way to do this and do it nicely. In Mule mappings are mostly coded in Java. Only because the mapping module for Mule isn’t really up to anything more difficult than an easy 1-on-1 mapping.
You could take the same approach to speed up the mappings in BizTalk:
- De-serialize the source message
- Perform the map in C# code
- Serialize the target message
The Microsoft integration team actually setup a new way of mapping for Microsoft Azure BizTalk Services. Those maps were actually based on .Net and not on XSLT, which increased performance, but it never caught on. For the Enterprise Integration Pack, Microsoft went back to XSLT again. Don’t know which approach I liked more…
Code Sample
You can download the code sample here “Mapping in BizTalk with C#”
Step 1: The BizTalk project
First setup a BizTalk project that contains two XSDs: the source schema and the target schema. Also add an orchestration, to be used later to call the C# map.
Step 2: Setting up serialization
After the XSDs in the BizTalk project are in place, the serialization project can be setup. That’s a C# class library project that holds the classes needed for serialization.
Last week I posted a blog about a code sample that handles (de)serialization of your messages. The code sample demonstrates on how to setup automatic serialization of XSDs. It works with Text Templates (T4 Templates). I’ll shortly go over it again, since that’s the basis of this entire exercise.
Instead of using XSD.exe to generate the serialization classes, a Text Template (T4 Template) is being used in this sample. This will enable automatic generation of the C# serialization classes. The Text Template will generate serialization classes for all XSDs found in the same project.
After creating the C# class library, the Text Template can be added. Make sure to add the normal Text Template and not the Runtime Text Template.
Now copy the code of the Text Template inside the code sample to the Text Template you just added. Below is an excerpt of that code:
<#@ template language="C#" hostSpecific="true" debug="True" #>
<#@ assembly name="System.Core" #>
<#@ assembly name="System.Data.Linq" #>
<#@ assembly name="EnvDTE" #>
<#@ assembly name="System.Xml" #>
<#@ assembly name="System.Xml.Linq" #>
<#@ import namespace="System" #>
<#@ import namespace="System.CodeDom" #>
<#@ import namespace="System.CodeDom.Compiler" #>
<#@ import namespace="System.Collections.Generic" #>
<#@ import namespace="System.IO" #>
<#@ import namespace="System.Linq" #>
<#@ import namespace="System.Text" #>
<#@ import namespace="System.Xml.Serialization" #>
<#@ import namespace="System.Xml.Schema" #>
<#@ import namespace="Microsoft.CSharp" #>
<#@ import namespace="Microsoft.VisualStudio.TextTemplating" #>
<# var manager = Manager.Create(Host, GenerationEnvironment); #>
//------------------------------------------------------------------------------
// T4 template originally posted by Rupert Benbrook
//
// <auto-generated>
// This code was generated by a tool.
// Runtime Version:<#=Environment.Version.ToString()#>
//
// Changes to this file may cause incorrect behavior and will be lost if
// the code is regenerated.
// </auto-generated>
//------------------------------------------------------------------------------
//
// This source code was auto-generated by XmlSerialization.tt.
//
using System.Xml.Serialization;
using System.Collections.Generic;
using System.Linq;
using System;
<#
IServiceProvider hostServiceProvider = (IServiceProvider)Host;
EnvDTE.DTE dte = (EnvDTE.DTE)hostServiceProvider.GetService(typeof(EnvDTE.DTE));
EnvDTE.ProjectItem templateProjectItem = dte.Solution.FindProjectItem(Host.TemplateFile);
EnvDTE.Project project = templateProjectItem.ContainingProject;
XmlSchemas xsds = new XmlSchemas();
foreach (EnvDTE.ProjectItem projectItem in GetAllItems(project.ProjectItems.Cast<EnvDTE.ProjectItem>()))
{
string path = projectItem.get_FileNames(0);
string directory = Path.GetDirectoryName(path);
if (path.EndsWith(".xsd"))
{
using (FileStream stream = File.OpenRead(path))
{
XmlSchema xsd = XmlSchema.Read(stream, null);
xsds.Add(xsd);
}
}
}
xsds.Compile(null, true);
XmlSchemaImporter schemaImporter = new XmlSchemaImporter(xsds);
CodeNamespace codeNamespace = new CodeNamespace((string)System.Runtime.Remoting.Messaging.CallContext.LogicalGetData("NamespaceHint"));
XmlCodeExporter codeExporter = new XmlCodeExporter(codeNamespace);
List<XmlTypeMapping> maps = new List<XmlTypeMapping>();
foreach (XmlSchema xsd in xsds)
{
foreach(XmlSchemaType schemaType in xsd.SchemaTypes.Values)
{
maps.Add(schemaImporter.ImportSchemaType(schemaType.QualifiedName));
}
foreach(XmlSchemaElement schemaElement in xsd.Elements.Values)
{
maps.Add(schemaImporter.ImportTypeMapping(schemaElement.QualifiedName));
}
}
foreach(XmlTypeMapping map in maps)
{
codeExporter.ExportTypeMapping(map);
}
CodeGenerator.ValidateIdentifiers(codeNamespace);
CSharpCodeProvider codeProvider = new CSharpCodeProvider();
using(StringWriter writer = new StringWriter())
{
codeProvider.GenerateCodeFromNamespace(codeNamespace, writer, new CodeGeneratorOptions());
GenerationEnvironment.Append(ConvertToList(writer.GetStringBuilder().ToString()));
}
manager.Process(true);
#>
// THE COMPLETE SAMPLE CAN BE FOUND IN THE CODE SAMPLE ITSELF...
After the Text Template has been put in place, the XSDs created in the BizTalk project should be added to the serialization project. This can be achieved by adding them as a link. There’s really no need to add them as a copy, since then we always need to update those schemas every time they change.
Now. The last step in this project is generating the C# serialization classes. For that a custom tool must be run. Right click the Text Template and choose Run Custom Tool. After the custom tool ran, you should see the classes that represent your XSDs.
Step 3: Develop a C# map
The mapping project is the project in which the C# mapping will be developed. In this sample there’s just one easy mapping and a helper class used to serialize the result.
The class itself can be found below and basically contains the code used for the mapping.
using BizTalkCSMapping.Serialization;
using Microsoft.XLANGs.BaseTypes;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Xml;
using System.Xml.Serialization;
namespace BizTalkCSMapping.Mapping
{
public static class OrderRequest_x_LegacyOrder
{
/// <summary>
/// Executes the map
/// </summary>
/// <param name="OrderRequestMessage">The source message</param>
/// <returns>The target message</returns>
public static XmlDocument ExecuteMap(XLANGMessage OrderRequestMessage)
{
OrderRequest inputOrder = (OrderRequest)OrderRequestMessage[0].RetrieveAs(typeof(OrderRequest));
// Get the addresses from the input to make life easier during the mapping itself
var billingAddress = inputOrder.AddressInfo.Where(x => x.addressType == "BILLING").FirstOrDefault();
var shippingAddress = inputOrder.AddressInfo.Where(x => x.addressType == "SHIPPING").FirstOrDefault();
// Create the legacy order (output/target) message
// Add everything we already can add
LegacyOrder outputOrder = new LegacyOrder()
{
ORDERNR = inputOrder.OrderNr,
BILLTOADDRESS = new LegacyOrderBILLTOADDRESS()
{
NAME = inputOrder.Client.Name,
STREET = billingAddress.StreetName + " " + billingAddress.StreetNumber,
ZIPCODE = billingAddress.ZipCode,
CITY = billingAddress.City,
COUNTRY = billingAddress.Country
},
SHIPTOADDRESS = new LegacyOrderSHIPTOADDRESS()
{
NAME = inputOrder.Client.Name,
STREET = shippingAddress.StreetName + " " + shippingAddress.StreetNumber,
ZIPCODE = shippingAddress.ZipCode,
CITY = shippingAddress.City,
COUNTRY = shippingAddress.Country
},
ORDERLINE = new List<LegacyOrderORDERLINE>()
};
// Iterate per orderline and add them to the output
foreach (var orderLine in inputOrder.OrderLines)
{
outputOrder.ORDERLINE.Add(new LegacyOrderORDERLINE()
{
ARTICLENUMBER = orderLine.SKU,
QTY = orderLine.Quantity
});
}
// Convert the object to an XMLDocument and return it
return ConvertObjectToXmlDocument(outputOrder);
}
/// <summary>
/// Serializes an object to an XmlDocument
/// </summary>
/// <param name="obj">The object to be serialized</param>
/// <returns>XmlDocument representing the object</returns>
private static XmlDocument ConvertObjectToXmlDocument(object obj)
{
XmlDocument xmlDoc = new XmlDocument();
XmlSerializer ser = new XmlSerializer(obj.GetType());
System.Text.StringBuilder sb = new System.Text.StringBuilder();
using (System.IO.StringWriter writer = new System.IO.StringWriter(sb))
{
ser.Serialize(writer, obj);
xmlDoc.LoadXml(sb.ToString());
return xmlDoc;
}
}
}
}
Step 4: Calling the mapping from your BizTalk project
In this sample an orchestration is being used to call the map. It could also be called from a pipeline component if you’d like. The code should be called from the Construct Message shape and it will return an XmlDocument, which can then be assigned to the message being created within the orchestration.
Calling the code is very straightforward. It’s one line of code, inside the Construct Message shape.
MsgLegacyOrder = BizTalkCSMapping.Mapping.OrderRequest_x_LegacyOrder.ExecuteMap(MsgOrderRequest);
Step 5: Deploy and test the sample
The code can be tested by deploying to BizTalk runtime the old-fashioned way.
- First build the entire solution.
- After a successful build, make sure to install the C# assemblies, containing the serialization classes and mapping, inside the Global Assembly Cache (e.g. using Gacutil.exe or add them as a resource in BizTalk Administrator).
- Now deploy the BizTalk project. Right-click the project and choose Deploy.
- Import the bindings found inside the Bindings folder in the BizTalk project.
- Now adjust your Receive Location and Send Port to your needs and test whether the sample works. You can use the OrderRequest.xml file as an input to test the mapping.
Input:
<ns0:OrderRequest xmlns:ns0="http://BizTalkCSMapping.BizTalk.Schemas.OrderIn">
<ns0:OrderNr>000001A</ns0:OrderNr>
<ns0:Client>
<ns0:Id>120182</ns0:Id>
<ns0:Name>Scrooge McDuck</ns0:Name>
</ns0:Client>
<ns0:AddressInfo>
<ns0:Address addressType="BILLING">
<ns0:StreetName>Money Bin Lane</ns0:StreetName>
<ns0:StreetNumber>1a</ns0:StreetNumber>
<ns0:City>Duckburg</ns0:City>
<ns0:ZipCode>81727</ns0:ZipCode>
<ns0:Country>USA</ns0:Country>
</ns0:Address>
<ns0:Address addressType="SHIPPING">
<ns0:StreetName>Money Bin Lane</ns0:StreetName>
<ns0:StreetNumber>1b</ns0:StreetNumber>
<ns0:City>Duckburg</ns0:City>
<ns0:ZipCode>81727</ns0:ZipCode>
<ns0:Country>USA</ns0:Country>
</ns0:Address>
<ns0:Address addressType="CUSTOMER">
<ns0:StreetName>Money Bin Lane</ns0:StreetName>
<ns0:StreetNumber>1c</ns0:StreetNumber>
<ns0:City>Duckburg</ns0:City>
<ns0:ZipCode>81727</ns0:ZipCode>
<ns0:Country>USA</ns0:Country>
</ns0:Address>
</ns0:AddressInfo>
<ns0:OrderLines>
<ns0:OrderLine>
<ns0:SKU>0001</ns0:SKU>
<ns0:ArticleName>Penny</ns0:ArticleName>
<ns0:PricePerUnit>1</ns0:PricePerUnit>
<ns0:Quantity>12</ns0:Quantity>
<ns0:TotalPrice>12.00</ns0:TotalPrice>
</ns0:OrderLine>
<ns0:OrderLine>
<ns0:SKU>0025</ns0:SKU>
<ns0:ArticleName>Quarter</ns0:ArticleName>
<ns0:PricePerUnit>25</ns0:PricePerUnit>
<ns0:Quantity>5</ns0:Quantity>
<ns0:TotalPrice>125.00</ns0:TotalPrice>
</ns0:OrderLine>
</ns0:OrderLines>
</ns0:OrderRequest>
Expected output:
<?xml version="1.0" encoding="utf-16"?> <LegacyOrder xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://BizTalkCSMapping.BizTalk.Schemas.OrderOut"> <ORDERNR xmlns="">000001A</ORDERNR> <BILLTOADDRESS xmlns=""> <NAME>Scrooge McDuck</NAME> <STREET>Money Bin Lane 1a</STREET> <CITY>Duckburg</CITY> <ZIPCODE>81727</ZIPCODE> <COUNTRY>USA</COUNTRY> </BILLTOADDRESS> <SHIPTOADDRESS xmlns=""> <NAME>Scrooge McDuck</NAME> <STREET>Money Bin Lane 1b</STREET> <CITY>Duckburg</CITY> <ZIPCODE>81727</ZIPCODE> <COUNTRY>USA</COUNTRY> </SHIPTOADDRESS> <ORDERLINE xmlns=""> <ARTICLENUMBER>0001</ARTICLENUMBER> <QTY>12</QTY> </ORDERLINE> <ORDERLINE xmlns=""> <ARTICLENUMBER>0025</ARTICLENUMBER> <QTY>5</QTY> </ORDERLINE> </LegacyOrder>
Download the code sample
I have uploaded the code sample to the MSDN gallery for code samples.
Click here to go there now and download the sample.
