Saving original HL7 message with MLLP.
I like to share my research work with HL7 in case some of you would be interested to share some thoughts.
The curious issue : Where is the original HL7 message? The ___messages are not tracked as an original flat-file HL7___ when using HAT with the MLLP adapter.
And the real deal:
1 Context
An integration solution with BizTalk Server 2006/2006 R2 is deployed in production. A new version of the solution was completed and ready to be deployed. However it is not possible to fully assert its quality without running messages in production. I know, please do not "5whys" me on this (http://en.wikipedia.org/wiki/5_Whys). For now, it must be seen as a constraint. The pre-production environment cannot have external systems that would reliably reproduce the production environment. Production messages are highly valuable and must not be ruined by the system. A backup plan is mandatory.
The nature of the communication, HL7 exchange, yield to another functional constraint: the receiving sequence of the messages must be maintain for the resubmission.
If a system back out must be held, no production messages should be lost. Moreover, the messages must be re-injected once the back out is completed. BizTalk Server can route ingoing messages to a dedicated database of its tracking system.
2 Problem
How this system performs in a critical scenario and how could it be used to resubmit messages?
3 Forces
# The archiving mechanism does not involve a new development;
# The receiving sequence is known and it can be reproduced during the resubmission;
# The archiving mechanism is not permanent and can be de-activated;
# The original message is not modified and can be inspected as-is by an administrator;
4 Current Finding
4.1 The Test set-up
To perform a test, an existing in-production application was used in DEV environment. This application already has a suite of performance tests written with loadgen [http://www.microsoft.com/downloads/details.aspx?FamilyID=c8af583f-7044-48db-b7b9-969072df1689&displaylang=en]. However, the tests does not clearly identify the sequence of messages that is sent to the BizTalk engine since it is using a file adapter on its receiving side. A new test suite was created which runs from a batch file and works with a MLLP receive port, an adapter that supports ordered delivery. The batch file launches the mllpsend tool shipped with the HL7 accelerator.
4.2 Exploration One: Using HAT UI
* Tracking is enabled from the receive port;
* The messages can be saved from the UI with a right-click. However, there is two major problem:
** the ___original messages is not saved___ in the Tracking database. Only the XML version is available;
** the UI does not allow a multi-selection. In a scenario where there is many messages (> 50), this is simply not workable on an administration perspective;
* A tool on Codeplex, a plugin for HAT [http://btsviewerhatplugin.codeplex.com] solves the previous multi-selection problem. However, after few tests, the plugin does not work as documented on the site. The messages are simply not shown in the UI for no clear reason;
4.3 Exploration Two: Using Custom UI
* Tracking is enabled from the receive port;
* The UI client was download from Thiago Almeida blog [http://connectedthoughts.wordpress.com/2008/04/02/3-ways-of-programatically-extracting-a-message-body-from-the-biztalk-tracking-database/]. See an image here [http://connectedthoughts.files.wordpress.com/2008/04/trackedmessagewindowsform.jpg];
* However, to see the message, the user must know the message guid;
* No multi-selection available from the UI but the article presents how messages can be retrieved from custom code;
** The tracking system offers theses views :
*** [BizTalkDTADb].[dbo].[btsv_Tracking_Parts] : this view holds the data for the message (the body) in its imgPart column;
*** [BizTalkDTADb].[dbo].[btav_MessagesFacts] : this view contains all the information related to the message like the adapter type [Event/Adapter], the port name [Event/Port], the timestamp [Event/Timestamp] (which could be used for the sequencing), [Event/Direction]
** The tracking system offers stored procedures for tracked messages:
*** [BizTalkDTADb].[dbo].[GetTrackedMessage]
*** [BizTalkDTADb].[dbo].[GetTrackedMessageFragments]
*** [BizTalkDTADb].[dbo].[GetTrackedMessageParts]
5 Early Conclusions
5.1 Where is the original HL7 message?
The big issue is that ___messages are not tracked as an original flat-file HL7___. This issue may discard the BTS tracking system as a potential backbone for an archiving-and-resubmission feature.
5.2 Multiple is not :
Out-of-the-box, HAT UI cannot provide a solution for message. The messages are indeed archived in its database and are not so hard to reach individually. However, it is not a workable solution for multiple instances.
5.3 Resubmission needs thoughts :
Both HAT UI and Codeplex Tracked Message UI do nothing for resubmission. In the best case, messages can be saved to a folder. Nevertheless, these files are useless since messages retrieved from the FILE adapter cannot support ordered delivery. However, the tracking views in the BizTalkDTADb database can be queried to retrieve an ordered sequence of messages for a given adapter or port name. Theses messages could then be sent to an ordered aware adapter.
Any comments?
The curious issue : Where is the original HL7 message? The ___messages are not tracked as an original flat-file HL7___ when using HAT with the MLLP adapter.
And the real deal:
1 Context
An integration solution with BizTalk Server 2006/2006 R2 is deployed in production. A new version of the solution was completed and ready to be deployed. However it is not possible to fully assert its quality without running messages in production. I know, please do not "5whys" me on this (http://en.wikipedia.org/wiki/5_Whys). For now, it must be seen as a constraint. The pre-production environment cannot have external systems that would reliably reproduce the production environment. Production messages are highly valuable and must not be ruined by the system. A backup plan is mandatory.
The nature of the communication, HL7 exchange, yield to another functional constraint: the receiving sequence of the messages must be maintain for the resubmission.
If a system back out must be held, no production messages should be lost. Moreover, the messages must be re-injected once the back out is completed. BizTalk Server can route ingoing messages to a dedicated database of its tracking system.
2 Problem
How this system performs in a critical scenario and how could it be used to resubmit messages?
3 Forces
# The archiving mechanism does not involve a new development;
# The receiving sequence is known and it can be reproduced during the resubmission;
# The archiving mechanism is not permanent and can be de-activated;
# The original message is not modified and can be inspected as-is by an administrator;
4 Current Finding
4.1 The Test set-up
To perform a test, an existing in-production application was used in DEV environment. This application already has a suite of performance tests written with loadgen [http://www.microsoft.com/downloads/details.aspx?FamilyID=c8af583f-7044-48db-b7b9-969072df1689&displaylang=en]. However, the tests does not clearly identify the sequence of messages that is sent to the BizTalk engine since it is using a file adapter on its receiving side. A new test suite was created which runs from a batch file and works with a MLLP receive port, an adapter that supports ordered delivery. The batch file launches the mllpsend tool shipped with the HL7 accelerator.
4.2 Exploration One: Using HAT UI
* Tracking is enabled from the receive port;
* The messages can be saved from the UI with a right-click. However, there is two major problem:
** the ___original messages is not saved___ in the Tracking database. Only the XML version is available;
** the UI does not allow a multi-selection. In a scenario where there is many messages (> 50), this is simply not workable on an administration perspective;
* A tool on Codeplex, a plugin for HAT [http://btsviewerhatplugin.codeplex.com] solves the previous multi-selection problem. However, after few tests, the plugin does not work as documented on the site. The messages are simply not shown in the UI for no clear reason;
4.3 Exploration Two: Using Custom UI
* Tracking is enabled from the receive port;
* The UI client was download from Thiago Almeida blog [http://connectedthoughts.wordpress.com/2008/04/02/3-ways-of-programatically-extracting-a-message-body-from-the-biztalk-tracking-database/]. See an image here [http://connectedthoughts.files.wordpress.com/2008/04/trackedmessagewindowsform.jpg];
* However, to see the message, the user must know the message guid;
* No multi-selection available from the UI but the article presents how messages can be retrieved from custom code;
** The tracking system offers theses views :
*** [BizTalkDTADb].[dbo].[btsv_Tracking_Parts] : this view holds the data for the message (the body) in its imgPart column;
*** [BizTalkDTADb].[dbo].[btav_MessagesFacts] : this view contains all the information related to the message like the adapter type [Event/Adapter], the port name [Event/Port], the timestamp [Event/Timestamp] (which could be used for the sequencing), [Event/Direction]
** The tracking system offers stored procedures for tracked messages:
*** [BizTalkDTADb].[dbo].[GetTrackedMessage]
*** [BizTalkDTADb].[dbo].[GetTrackedMessageFragments]
*** [BizTalkDTADb].[dbo].[GetTrackedMessageParts]
5 Early Conclusions
5.1 Where is the original HL7 message?
The big issue is that ___messages are not tracked as an original flat-file HL7___. This issue may discard the BTS tracking system as a potential backbone for an archiving-and-resubmission feature.
5.2 Multiple is not :
Out-of-the-box, HAT UI cannot provide a solution for message. The messages are indeed archived in its database and are not so hard to reach individually. However, it is not a workable solution for multiple instances.
5.3 Resubmission needs thoughts :
Both HAT UI and Codeplex Tracked Message UI do nothing for resubmission. In the best case, messages can be saved to a folder. Nevertheless, these files are useless since messages retrieved from the FILE adapter cannot support ordered delivery. However, the tracking views in the BizTalkDTADb database can be queried to retrieve an ordered sequence of messages for a given adapter or port name. Theses messages could then be sent to an ordered aware adapter.
Any comments?
Comments