Ahmed Farrag Biztalk Developers’ Lounge

Application Infrastructure: Cloud Benefits Delivered

2010-05-13T09:39:00.000-07:00

http://www.appinfrastructure.com

Want to bring the benefits of the cloud to your current IT environment? Cloud computing offers a range of benefits, including elastic scale and never-before-seen applications. While you ponder your long-term investment in the cloud, you can harness a number of cloud benefits in your current IT environment now.

Join us on May 20 at 8:30 A.M. Pacific Time to learn how your current IT assets can harness some of the benefits of the cloud on-premises—and can readily connect to new applications and data running in the cloud. As part of the Virtual Launch Event, Gartner vice president and distinguished analyst Yefim Natis will discuss the latest trends and biggest questions facing the Application Infrastructure space. He will also speak about the role Application Infrastructure will play in helping businesses benefit from the cloud.

Plus, you’ll hear some exciting product announcements and a keynote from Abhay Parasnis, GM of Application Server Group at Microsoft. Parasnis will discuss the latest Microsoft investments in the Application Infrastructure space aimed at delivering on-demand scalability, highly available applications, a new level of connectivity, and more. Save the date!

add it to your calender http://www.appinfrastructure.com/Save%20the%20Date%20Application%20Infrastructure%20Virtual%20Launch%20Event.ics

Regards,

Ahmed Farrag

Biztalk 2009 Benchmark and Assesment Questions

2010-02-24T05:16:00.000-08:00

There have plenty of enhancements in the performance of Biztalk over the last versions, this has mainly been due to 1) the enhancements in the underlying platform, mainly the SQL Server, and 2) to the tuning of the Biztalk engine itself with the cumulative updates happening from one version to another..

Biztalk 2009 really sets a really impressive standard for performance with it's underlying SQL Sever 2008,, all details for the scenario used and the numbers that were achieved could be found in the below articles

http://msdn.microsoft.com/en-us/library/ee377068(BTS.10).aspx
http://msdn.microsoft.com/en-us/library/ee377036(BTS.10).aspx
http://msdn.microsoft.com/en-us/library/ee377057(BTS.10).aspx

" With these results, the Biztalk Server Product Team was able to demonstrate that a single Biztalk Server computer and a single SQL Server computer can support over 94 million messages in a messaging scenario and 37 million orchestrations during a 24-hour period. By scaling the Biztalk Server and SQL tiers to the optimal configuration available in our environment, we were able to process over 181 million messages per day and over 86 million orchestrations. The results were performed in a sand boxed environment by using the class of hardware deployed in many enterprises" ... quoted from the above results..

I would just like to highlight that the above results were done in a sand boxed environment with certain types of adapters and hardware ,, so please have a look above on the exact scenario used and how they achieved these results.

Also, performance could vary a lot depending on the type of adapters you are using, amount of tracking you are doing, complexity of the orchestrations, messages sizes , extra..

so generally when a customer asks me , "How many Biztalk server would I need ? " ,, my answer is always "it depends ! " .. to make life easier for my customers, i give them these simple questions to fill for each application that will be connected to Biztalk...
Application Name :

Platform and Database
Does it have APIs? , if yes, what is the protocol and format supported? (for example XML over HTTP, method calls over TCP/IP, flat files uploaded to FTP Folder)
Average Number of Incoming Messages / second
Average Number of Outgoing Messages/Sec
Average Message Size in Kilobytes
Messaging Pattern (Synchronous /Asynchronous)
Required Latency in Millisecond (in case of synchronous messaging)
Is there batch processing , if yes, what is the size of the batch in megabytes

By answering the above questions, we will be in a situation to know whether we are in Low Latency scenario type of integration or more on a high throughput asynchronous type, or maybe even something in between! ... also, we could identify if there is FTP, POP3 or MSMQ adapters being used, and hence the possible need to cluster the Biztalk instances carrying hosting these adapter..

Also, the batch sizes, will give us a guidance on whether to tailor for Higher RAM capacity in the Biztalk Boxes or not,, if we are looking at smaller messages, then the normal RAM recommendation is sufficient.

By combining the numbers of all the applications that will be connected to Biztalk, you will be able to have a rough idea on how many servers that you would need...

again, this is not an official way of sizing the capacity planning for your Biztalk environment, you should always refer to the published "Performance and Capacity Planning" available http://technet.microsoft.com/en-us/library/aa577523(BTS.10).aspx, however, this is just my way to make my life easier when you are still at very early stages of assessment..

Cheers

Ahmed Farrag

Moved to a New Blog

2008-08-12T05:24:00.000-07:00

http://blogs.msdn.com/ahmedfarrag

see you there :)

Ahmed Farrag

Best Practices Analyzer v1.1 Shipped

2008-05-09T05:43:00.000-07:00

last month, we shipped the Best Practices Analysers ..

here are some more details about it if you didn't know it before !!

Overview
The BizTalk Server Best Practices Analyzer performs configuration-level verification by reading and reporting only. The Best Practices Analyzer gathers data from different information sources, such as Windows Management Instrumentation (WMI) classes, SQL Server databases, and registry entries. The Best Practices Analyzer uses the data to evaluate the deployment configuration. The Best Practices Analyzer does not modify any system settings, and is not a self-tuning tool.

What’s new:
V1.0 has 77 rules, v1.1 has 220 with 143 new rules added!

Here are some relevant rules
1. Added a warning rule for stopped orchestrations
2. a check on MaxLogSize MSDTC
3. Verify that TempDB and BizTalkMsgBoxDb data and log files are placed on separate drives
4. Check that the Max Degree of Parallelism setting is set to 0 in all other BizTalk databases
5. Verify that the MaxWorkerThreads in the SQL Server UMS configuration setting is set to XXX
6. Check that the Maximum Server Memory is set to XXX GB
7. System Time Diff between SQL and BizTalk Machines
8. Orphaned SPIDs
9. SQL Server Service Pack Installed for DB : xxxxxx
10. Ignore Duplicate Key check
11. Check Biztalk SQL Server JObs running or not

Where to Download:
http://www.microsoft.com/downloads/details.aspx?FamilyId=DDA047E3-408E-48BA-83F9-F397226CD6D4&displaylang=en

- Ahmed Farrag

Biztalk Operational Guide Released !!

2008-03-03T01:54:00.000-08:00

i'm very excited about this new operational guide that was released a couple of weeks ago, it's really a must read for any Biztalk developer/administrator, it covers all the topics related to deployment, management and monitoring aspects of any Biztalk implemenation.

i have seen many customers who have deployed Biztalk but for some reason has missed on having a concrete strategy for the versioning/management and monitoring parts of the solution. the operational guide sheds the light over a lot of these issues that need to be tackled on any production environment.

you can find it here .

The guide can be downloaded from here: http://www.microsoft.com/downloads/details.aspx?FamilyID=5251f103-329d-4f8d-95b5-64f99f2f6dfc&DisplayLang=en.

Cheers
Ahmed

configuring MS DTC service on virtual images

2008-01-21T06:56:00.000-08:00

i was trying to build a biztalk lab today and i was basically connecting a couple of virtual images on a virtual server and trying to assign a Biztalk Role for each image, i ran into the standard MSDTC settings that you have to manage with Biztalk, which entails a couple of standard steps,,, after doing everything , the MS DTC didn't work !!! .. it said that the machines have the same identifier and therefore the DTC won't work, it seems that both images were a clone from each other !!

i had to unistall and insall the MS DTC on the main server and it worked after that like a charm : )

P.S: you will have to reboot the machine after you uninstall the MSDTC service using the msdtc.exe -uninstall

Cheers :)

Ahmed

Best Practices for the Biztalk SQL Server Adapter

2007-12-25T06:34:00.000-08:00

http://download.microsoft.com/download/5/0/f/50f397a7-13c7-4807-bd51-66db5f5b0908/best%20practices%20for%20the%20sql%20adapter.doc

Now @ Microsoft

2007-11-30T14:14:00.000-08:00

Dear all -

this week is my second week as a Microsoft Employee :)
my title in Microsoft in a really Long one ,, it goes like this

"
Regional Technology Solution Professional
Process Platform Solutions
Middle East and Africa (MEA HQ)
"
i guess it will be challenging to get this on the business card :)

i'm really excited about the new Role, i'll be basically working with the pre-Sales team in Microsoft on Biztalk opportunities in the region, conducing technical Architecture sessions and delivering proof of concept solutions.

cheers : )

Ahmed

Tips : Consuming Orchestrations as WebService

2007-10-27T06:01:00.000-07:00

as i promised those who attended my training last Thursday, i'd post the issue that we faced together and the solution we found during consuming Orchestrations exposed as Webservices from infopath.

it started with the event viewer throwing an exception that the "NT Service Account" doesn't have privileges to access the Biztalk Database and we found another entry about the "Isolated Host" user doesn't have privileges to register the SOAP receive port.

well, the solution was simple, basically, when you expose an orchestration as a webservice, the wizard automatically creates the webservice with it's code for you. however, when you try to call this webservice, from a client like InfoPath, the AppPool under which this site runs normally uses the default NT System Service account, all what you need to do is to assign a different identity for that AppPool that has privilege to access the SQL Database.

also you need to do the same for the user running the Isolated Biztalk Host Instance.

That's it :)

Cheers :)

Another Deep Technical Biztalk Training

2007-10-26T04:13:00.000-07:00

last Wednesday i have started a deep technical training on Biztalk 2006, the training took place at ITWorx and was attended by more than 25 different software developers from different seniority levels,

we had open discussions all through the two days plus many comprehensive demos, most of the attendees had little or no idea about Biztalk so we started by discussing the feasibility of building enterprise integration software, we went through the possible design alternatives for building such application, everyone started recommending components in such a solution.

the design proposed by the attendees included a messaging component, business rules component, and different "agents" to handle different communication protocols :)

then we started to map the proposed design by everyone to the actual Biztalk Components, realizing that BizTalk Does all this "out-of-the-box" .. i believe this helped the attendees a lot in realizing what exactly BizTalk does. :)

these sessions were video taped , i'll try to do my best to get these videos and post them here.

for those who attended the session, please feel free to drop by and send me your feedback. :) , i hope you have enjoyed it as much as i did.

Cheers :)

Ahmed Farrag

MEA MVP Open Day (My Last Event as a MVP)

2007-10-21T16:53:00.000-07:00

Today was a really nice day at the MEA MVP Open Day in Egypt, i got to deliver a session on BizTalk titled "Real world SOA and Biztalk", in the session i have tried to focus on what it takes to build SOA solutions in the Real World and all the myts and facts and SOA.

also i have touched base about the Biztalk Architecture and how it fits in the SOA world and how you could easily use Biztalk as the core engine of your SOA solution.

unfortunately i have planned to discuss a lot about ESB (Enterprise Service Bus Guidance ) that is published on codeplex ..www.codeplex.com/esb but i didn't have much time left in the session so i focused only on SOA and Biztalk.

I'll upload the PowerPoint presentation tomorrow as i have just forgot the updated version of it on the presentation laptop !!!

This event is very special to me, as it will be the last event for me as a MVP since i'll be joining Microsoft MEA team after 2 weeks, and as you all know, once you are in Microsoft, you can't be a MVP no more, I'll be working as a Regional TSP in the process group, focusing mainly on Biztalk, it's a challenging position and I'm really looking forward to it.

anyhoow, if you have attended the session, i hope you have liked it and feel free to give me your feedback.

cheers :)

Ahmed Farrag

Biztalk Server 2006 R2 RTM is Ready !!!

2007-10-04T17:57:00.000-07:00

The Biztalk Team has Recently released the RTM Version of the very anticipated Biztalk Server R2, the New R2 version is available in evaluation version for registered users, you download the new release from here
The New version contains major enhancements, my favorites are the support for MOSS 2007 adapters, WCF Adapters and the all new RFID Support, I have been playing with the betas for quite a while and i seen the patterns and practices team doing a great job with there ESB guidance that was based on specific build of the R2 Release. i believe with this new release and with the upcoming release version of the ESB guidance, developers will have all what they need to build real SOA solutions, if you need to know more about the ESB Guidance project, you can check it at www.codeplex.com/esb.

Regards,
Ahmed Farrag
MVP Biztalk Server

ESB CTP 3 UddiPublisher Error

2007-08-20T04:22:00.000-07:00

i have been playing around with the ESB CTP 3 and i found the following error when trying to run the "PreProcessingCORE.vbs" ,, it started by creating the Virtual directory and then stopped at the step of running the uddipublisher project giving the following exception

"Error Creating UDDI Entries... Class:Microsoft.Practices.ESB.UDDI.UddiClient Method: Void .ctor : Exception: System.ServiceModle.FaultException'1[uddiorg.api_v2.dispositionReport]: "

the error is clearly an authentication error, so after trying several options for a while, i discovered that the error was in the Uddi publisher settings, the user name must be defined as /.

Cheers :)

Biztalk Build or Buy ???

2007-08-05T13:10:00.000-07:00

I have been asked a lot, "where is the power of Biztalk?" and that’s comes from experienced Software engineers, they always tend to say "Well, we can do this!!”, and in fact that's true, if you took a bird’s eye view on what Biztalk does, it's primarily two things,, Business Process Management and Messaging Solutions, and this involves basically handling messages from different sources with different formats.

So what about the engineer who said that he can do this, well the answer is simple, Yes, you can do custom code to query a web service and get the response and then transform the message using XSLT and possibly apply some business rules checks. However when you try to deploy this solution in an enterprise, you will start facing some problems that you will have to address.
First, you will need to worry about the online vs. offline support of your application, what if the web service is not available? And you need to re-try the call; you will hence need to develop some sort of service that does this for you. Also, what about the business rules that you have applied; you need to give access to the business users to change them at any time, so you'd probably create a separate business rules store with its own configuration and database.

What about the throughput of the messages, what if the load on the server increased, could your application be load balanced? , you'd then have to create some sort of a load balancing service that is could be distributed on multiple servers...!!

You will find so many "what if" in this application that you will eventually end up building a complete framework just to support your initially small application!!
This framework of tools is what Microsoft calls Biztalk Server!!!

Biztalk Server comes with out-of-the-box support for long-running transactions; Rules engine storage, scalable infrastructure, tracking capabilities and much more; basically all what you need to host an integration/BPM solution.

So if you are targeting enterprise level solutions, I believe the equation turns in the favor of "Buy" not "Build”.

2007-07-09T01:56:00.000-07:00

I found out about a week ago that I received the Microsoft MVP Award for BizTalk. I am really pleased to be part of this community and I am looking forward to continuing to help the BizTalk developer community. Thanks to everyone who nominated and supported me (special thanks to Mohamed Wahby (Microsoft Egypt) and Mohamed Karam (ITWorx) ).

Biztalk Server and Windows Server 2003 Service pack 2

2007-06-01T15:26:00.000-07:00

i faced today a weird problem with the installation of Biztalk Server...

After installing BizTalk server 2006, I configured both “Enterprise SSO” and “Group” Successfully.
When trying to configure “BizTalk Runtime”, it fails with the following error:

“Could not deploy 'Microsoft.BizTalk.KwTpm.StsDefaultPipelines' on computer 'WS2003-VM-03'. Please check if BizTalk is installed correctly. (SeedCfg)”

I tried un-install then re-install BizTalk server and still getting the same error.

Here is my configuration:
OS: Windows 2003 Server + SP2
BizTalk Server 2006 – BizTalk databases are on remote SQL Server.

the problem was fixed when i installed the Biztalk Server on a Windows 2003 with SP1 .. it seems like there is something with SP2 that couldn't register that assembly..

Cheers ,,

Ahmed

BiztalkersEgypt Site Launched

2007-05-27T09:10:00.000-07:00

in the past couple of month i was thinking about creating a Biztalk Server community in Egypt, I'm already a board member in a technical club called Cutting-Edge where we organize technical sessions and seminars for software developers.

i have started a Special interest group inside the club focused on BizTalk technologies and called it "Biztalkers", i have also delivered 2 sessions in the past month to introduce Biztalk to the club members...

i believe the next step is to have an online community for biztalk developers/architects in Egypt and the Middleast where we could all learn from each other and provide video tutorials on how to start biztalking :) , that's where the idea of BiztalkersEgypt Came from ...

The site is now up and running http://www.biztalkersEgypt.com and users are starting to register in it. i have also put a forum on the site where users could submit their questions and concerns..

i believe this is a very important step in promoting the awareness for this product in Egypt and the MiddleEast in a very simplified yet efficient manner.

A Must Read Troubleshooting guide ..

2007-04-12T10:32:00.000-07:00

i have been using this troubleshooting guide for a while right now and i believe its a must read for all biztalk developers, it covers sections about debugging orchestrations and monitoring the performance and lots more ...

you can find the link here ..

http://download.microsoft.com/download/3/7/6/376a6f6c-8c97-4ab5-9d5a-416c76793fbb/bts06developerstroubleshootingguide.doc

Enjoy Biztalking :)

'BizTalk Server: 8 Tips And Tricks For Better BizTalk Programming

2007-04-12T03:47:00.000-07:00

i have came across this really interesting tips about biztalk programming, it really summarizes most of the pitfalls that most developers fall into, and as we always say, it's all about design, you can do lots of things with Biztalk, and you can do them in many different ways, the most important thing is how to do things the right way... :)

Enjoy Biztalking :)

'BizTalk Server: 8 Tips And Tricks For Better BizTalk Programming,'bloggerForm','scrollbars=no,width=475,height=300,top=175,left=75,status=yes,resizable=yes

Any Element in Biztalk Schema

2007-02-21T05:51:00.000-08:00

we had a working orchestration today that needs to send notification for the failed messages, one problem came around, the schema of the failed message doesn't match anything to given to the orchestration !!! ,

the solution was so simple, we created a new generic schema, that contained the same root node and under it the Magical "any" element node type.!!!

so now the notification orchestration will only check the root node , and it will accept the message no matter what elements come under the root... and that was IT :)

Cheers :)

Ahmed

(Cutting-Edge ) Get To Know Biztalk Session

2007-01-31T07:37:00.000-08:00

I have uploaded the material for the demo i made in the cutting-Edge club biztalk session ...

click here

please feel free to contact me if you have an questions.

Cheers

Ahmed

Handling Different Flat Files in the Same Pipeline

2007-01-24T03:12:00.000-08:00

I have been asked a while ago about what to do if you have different Flat file schemes arriving at the same location each with a different schemas.

converting flat file into XML files is done by the flat file schema wizard which requires that you create a custom pipeline to put the disassembling schema in it, so the first thing that pops up in your mind is , what if I receive different schemas on the same port ?? I can't make a custom pipeline for each type of schemas; So what I should do,, ??

actually, some prefer to take this "trick" to the orchestrations by passing the file as "passthrough" and the orchestration will decide which transformation map to use, but I'm personally against this option because this enforces tight coupling between the message types and the deployed orchestrations, also it will raise several issues in the deployment and versioning of the whole solution.

I have seen a solution Gilles' Web log where it extends the Flat file disassembler to create a new one that detects the schema type and applied the right transformation ..
here is a link

we have tried this solution and it worked nicely, it's not a simple solution though but it preserve the service orientation of the biztalk solution .

keep biztalking

Anti Patterns -- ( The worst you can do !!! )

2007-01-23T05:05:00.000-08:00

I came across this really interesting article about the Worst you can do with Biztalk, it really sheds the light about the real working problems that I have seen in Biztalk Project, everybody asks about the business logic and where to place it correctly in Biztalk, I believe this article shows the best way to place your business logic

Click here

Keep up the patterns and Enjoy Biztalking :)

Old Lady Pwnz Mercedes Guy

2007-01-02T14:25:00.000-08:00


Really Funny

Calling .NET Assembly from Biztalk Server 2006 Orchestration

2006-12-26T10:52:00.000-08:00

I have working for a while on multiple Biztalk Server project and I have also recently delivered a workshop on this great product,,

I have decided to share the samples I have made during the workshop for those who are interested in learning this product.

one of the these samples shows how to call .NET assembly from an orchestration, all you have to do is simple, declare a variable and choose it's type to be from the class you created, initialize this file and then call it's method,,, !!! Very simple, isn't it.. ?

the class has to be a "serializable" class since all the objects in Biztalk are XML objects,

to download the complete application with source code Click here

have a nice time Biztalking :)

Cheers,
Ahmed

Enterprise Library Famous Security Exception

2006-10-07T17:34:00.000-07:00

Today , i was working on packaging an ASP.NET Web Application that uses the Microsoft enterprise Library, for some reasons after making the web setup project the application gave the famous security Exception of the Enterprise Library it said " Denied access to the registry " !!! ..

after googling for a while i came across a very neat tool published by"Mohamed Sharaf" on his blog- to install the DLLs of the EL using the installutil.exe tool.

the link is found here

Press Here

i modified the application and made in inside an installer class and put it in the web setup project, and it worked like a charm !!!

Storing the ViewState in Session variables !!

2006-07-13T06:57:00.000-07:00

Hi all,

i was working today on optimizing a page that took lots of time to load, the first thing anyone will think about is the images sizes, viewstate and caching .. i tried to remove the view state from the controls that didn't need it , but still , it didn't help alot, besides some controls like the datagrid needs the viewstate to function well, so i have decided to find another place to store the viewstate .. !!!

when you override the functions "LoadPageStateFromPersistenceMedium" and "SavePageStateToPersistenceMedium" in the page, you can then save the viewstate to a session variable, that way the viewstate will not be a burden on the size of the page no more :) , so finally i got the perfomance better :D ..

he is a code that does this.

protected override object LoadPageStateFromPersistenceMedium ()
{
string key = Request.RawUrl + "_VIEWSTATE";
object state = Session[key];
return (state == null) ?
base.LoadPageStateFromPersistenceMedium () : state;
}

protected override void
SavePageStateToPersistenceMedium (object viewState)
{
string key = Request.RawUrl + "_VIEWSTATE";
Session[key] = viewState;
}

Enjoy :)

BSM 2005 Unleased

2006-04-16T08:03:00.000-07:00

Hi All ,,

i have been investigation in the Bussiness ScoreCard Manager 2005 for quiet a while, i'll start a pilot project using this tool very soon..

BSM 2005 is used to create business score cards, the source of data for these scorecards could be multidimensional data or relational data, the BSM builder (the tool that you use to create reports) conncet to two type of datasource
Multidimesional
ODBC.

i have tried to apply a sample today for the building of a ScoreCard based on an Excel sheet.

I have to say, it's not a straight forward task, however, there is a very usefull blog for the installation tips and tricks.

http://blogs.msdn.com/bsm

so good luck with your BSM work !!

Welcome To My Blog .. !!!

2006-03-26T04:17:00.000-08:00

Dear All,

I have been working in the field of software development for quiet a long time now, and recently I had the chance to implement some new technologies, so I decided to start my blog where I would be able to share my experience with all my friends and teammates.

I’ll try to post as many articles about some technologies I have been exposed to including the Microsoft Content Management Server 2002, Biztalk Server 2004 and Business Scorecard Manager; besides the traditional ASP.NET/Web Services applications.. So if you have any specific topics that you would love to read about it more in this blog, please share them

My aim is to make this blog “The (Relaxing) Lounge" where developers could share their questions/concerns/likes/dislikes etc. And I believe knowledge is not something to be owned by someone, it's made to be shared, so let's start sharing It.!!!

Regards,
Ahmed