Scott Klueppel's Blog - Transactions

The remote server or share does not support transacted file operations

Scott Klueppel — Sat, 08 May 2010 15:32:25 GMT

Despite its pitiful adoption in the developer community, I am implementing Transactional NTFS (TxF) transactions using the Microsoft.KtmIntegration.TransactedFile class. This allows me to reap the benefits of TransactionScope and distributed transactions for file operations (e.g. creates, updates, deletes). This is the only missing piece for typical transactional business applications. With the “KTM” and “KtmRm for Distributed Transactions” services, available only on Vista, Windows 7, and Windows Server 2008, file operations will roll back if the TransactionScope is not completed.

There’s just one problem… Transactional NTFS does not work with file shares. I can’t remember the last time I put a “C:\FileStore” reference in a config file. A friendly share like “\\server\FileStore” is always preferred, especially since DFS came about. Attempting to use a share results in the following error message:

The remote server or share does not support transacted file operations

Don’t read this as “your remote server” or “your remote share”, but rather “all remote servers and shares”. As mentioned in this MSDN article, TxF is not supported by the CIFS/SMB protocols. The error was probably written with the expectation that one day some remote servers and shares would support TxF. I emailed Microsoft about it and received a response fairly quickly. The response was simply:

“We understand the need and have plans to eventually support TxF over SMB2, but we’re not there yet and are not ready to announce if or when this will be supported. When it is the documentation will be updated.”

I’m not getting my hopes up, but Windows Server 2011 looks to be our only hope before .NET changes beyond recognition and TxF is a distant memory. Until then, I wrapped up all of my TxF code in a WCF service and install that service on the server with the FileStore folder.

MSDN article – When to Use Transactional NTFS

http://msdn.microsoft.com/en-us/library/aa365738(v=VS.85).aspx

TxF Sandbox – Sample Projects (including Microsoft.KtmIntegration.TransactedFile)

TxFSandbox.zip

The flowed transaction could not be unmarshaled - Untrusted Domains update

Scott Klueppel — Tue, 16 Mar 2010 02:54:48 GMT

In a previous post, I discussed solutions to the dreaded “The flowed transaction could not be unmarshaled” error commonly experienced when using MSDTC transactions with WCF, SQL, TxF, etc. I have once again experienced the un-trusted domain scenario, and can now report with certainty that adding hosts file entries on both machines will correct the problem. Testing this solution with DTCPing.exe between the two machines proves that making only the hosts file change acquaints the client and server and allows distributed transactions to occur.

You will find many blog and forum post non-solutions. Adding the hosts file entry or the equivalent domain redirects are the only solutions when working with two machines in disparate, un-trusted domains. Some of the non-solutions you’ll find go so far as to say to change your SQL connection string to prevent current (ambient) transaction enlistment. Not quite a complete solution as your first rollback unit test will fail.

Enable WS-AT on 64-bit Vista or Windows 7

Scott Klueppel — Thu, 08 Oct 2009 05:51:21 GMT

Thank you, Mikael Sand (and Bing)!

http://blogical.se/blogs/mikael_sand/archive/2009/08/28/configuring-ws-atomic-transaction-support-in-windows-7-64-bit.aspx

Code Camp Session Files

Scott Klueppel — Thu, 03 Sep 2009 03:34:34 GMT

The 2009 Jacksonville Code Camp was a great success. Many thanks to Bayer, Brandy, and everyone else that made it happen. The bar has been set really high for future Jacksonville code camps, and for the rest of Florida too.

My session on Transactional WCF Services went well. Many great questions and compliments after the session. If you attended and have any unanswered questions, please email me.

You can download the session files below. It contains staged versions of all of the transaction modes we discussed. It also contains a tracing solution and tracing result files to view the client and host tracing files in Client/Service mode. Also see my previous post on using the Service Trace Viewer. It also contains a few demo projects that we didn't get to in the one-hour session.

Files/Solutions included in Session Archive:

PowerPoint slides
Transaction Promotion Code Snippet
Testing database backup
Testing SQL script (query and cleanup between tests)
IDesign ServiceModelEx Project (used by all included Solutions)
Code Demo Solutions

Code Demos include:

1.	TransactionScope - Shows how single/multiple resource managers affect which Transaction Manager is chosen to handle the scoped transaction. Also gives first look at transaction promotion detection.
2a.	Mode None - WCF transaction mode with which no transactions are created or flowed from the calling client.
2b.	Mode Service - WCF transaction mode with which no transactions are flowed from the calling client, but a transaction is created for your service operation.
2c.	Mode Client - WCF transaction mode with which a transaction is required to be flowed, and the service will only use the client transaction.
2d.	Mode Client/Service - WCF transaction mode with which a client transaction will be flowed and used by the service, if available. If no client transaction is flowed, a transaction will be provided automatically for the service operation.
3.	Explicit Voting - Shows how explicit voting with a session-mode service is performed using OperationContext.Current.SetTransactionComplete().
4a.	Testing Various Resource Managers - Shows how a client can use a single TransactionScope to call several services (some transactional, some non-transactional), a database stored procedure, and an IDesign volatile resource manager Transactional<int> .
4b.	Testing Services - Provides a host project for a transactional service and a non-transactional service used in 4a.
5a.	Tracing - Same as 2d. modified with the additional app.config settings in the client and host projects to allow for service tracing to .svclog files.
5b.	Tracing Results - Stored results from executing 5a. in case you don't want to load the database and actually run the projects. The .stvproj file can be opened directly in the Service Trace Viewer. On the "Activity" table, click on the activity "Process action 'http://services/gotjeep.net/GpsTrackServiceContract/SubmitTrack'" then click on the "Graph" tab. You will see that the client and host activities where the arrow moves from client to host (send and receive message, respectively) show the OleTxTransaction in "Headers." The next activity in the host reads "The transaction '5bd25b08-848c-409d-9163-6303b9138382:1' was flowed to operation 'SubmitTrack'."

Download the session files:
TransactionalWCF.zip (854 KB)

The flowed transaction could not be unmarshaled.

Scott Klueppel — Thu, 29 Jan 2009 02:39:52 GMT

“Do not anticipate trouble, or worry about what may never happen. Keep in the sunlight.”

- Benjamin Franklin

It’s obvious that Mr. Franklin didn’t have to deal with bad data, bad code, or even the occasional hiccup. In the real world, there is “trouble” and we need to not only anticipate that trouble, but also worry about what may never happen. Simply “keeping in the sunlight” won’t maintain data or application state integrity. In the real world we need transactions!

Since .NET 2.0, we’ve had the privilege of using System.Transactions.TransactionScope to manage our transactions with very few headaches. One of the headaches that almost everyone experiences is MSDTC. One of the oldest and most elusive topics on the web. There are tons of blog and forum posts directing our fellow developers to check their firewall settings for every MSDTC problem. The latest MSDTC hiccup I have seen comes in the beautifully packaged error message:

The flowed transaction could not be unmarshaled. The following exception occurred: Communication with the underlying transaction manager has failed.

The what could not be what? You can read some MSDN documentation on the topic which will probably cause you more pain. If you are seeing this error message, there’s only a few things that may be wrong.

MSDTC Settings
- Check the MSDTC settings on the machine that is initiating the transaction. If “Allow Outbound” is not checked, then it won’t allow the transaction in progress to be flowed to the next machine in the transaction chain. Check the box and restart MSDTC… it should work.
Un-Trusted Domains
- I have seen this error when you are trying to flow transactions between machines that are in un-trusted domains. Machines in different domains that do not trust each other block the antiquated, yet necessary, WINS resolution between the two machines. MSDTC relies on WINS resolution. I have fixed this problem before by adding host file entries on both machines pointing to the other machine. I can’t guarantee that this will work in all cases. Both of those machines are no longer under my control.
Imaged Servers
- The most recent, and most blogged about problem is surprisingly caused by two machines created from the same image. Cloning or imaging servers is quite common. Building a server from scratch is not a fun activity, so we build one, create an image, and put that image on every server we want to build after that. Once again, MSDTC is standing in our way because of the way it detects the sending and receiving application’s unique identifier. Each machine has a GUID in the registry that identifies it uniquely as a participant in an MSDTC transaction. Imaged machines have the same GUID. I’ll talk about the detection and resolution of this for the remainder of this post.

Running DtcPing.exe between the two machines will actually tell you that the machines are using the same GUID. Output window text from DtcPing shown below.

DTCping log file: C:\DTC Ping\THRESHER4160.log
Firewall Port Settings:
Port:5150-5250
RPC server is ready
Please Start Partner DTCping before pinging
WARNING:the CID values for both test machines are the same
Please send following LOG to Microsoft for analysis:
Partner LOG: SCORPION6128.log
My LOG: THRESHER4160.log

Tucked away in the last step of a Microsoft Knowledge Base article titled "How to troubleshoot MS DTC firewall issues" is a reference to this problem. Use regedit.exe to look at the registry on both machines. Locate the HKEY_CLASSES_ROOT\CID key in the registry.

Find the CID key that has a description value of “MSDTC”. If they are the same, the transaction cannot flow.

WARNING: Back up your registry before making any changes!

Solution 1 - Replace the offending CID keys/values on one of the machines: In this case, you will need to find all keys/values with GUID 28b81f1c-2afb-4ee2-ad85-5bc62dad1647 in your registry and replace it with a new GUID (using GuidGen). There is likely to be 3 places this GUID appears. It is also important to note that the offending GUID appears in the DtcPing log file generated during the ping test.

Solution 2 – Use msdtc command line tool to re-install MSDTC: The commands are simply:

msdtc -uninstall
msdtc -install

After making the registry changes or running the msdtc command line tool, you must restart your server for the changes to take effect.

What happens to your transaction with different data access methods

Scott Klueppel — Fri, 05 Dec 2008 02:50:25 GMT

Unless you are working on a extremely simple or read-only application, transactions are a must. Using the System.Transactions namespace is the easiest and most efficient way to maintain system consistency when dealing with multiple calls or multiple resources. Although System.Transactions arrived in .NET in the 2005 product, it is still a relatively unknown part of the framework. System.Transactions was designed by the Indigo team in preparation for WCF. It is not compatible with Win98 or WinME, but most people are incompatible with Win98 and WinME so it works out just fine.

Before System.Transactions, we only had access to System.Data.SqlClient.SqlTransaction or a true SQL transaction using BEGIN/ROLLBACK/COMMIT TRAN. Using SQL transactions, you are stuck with only being able to update DB records as part of your transaction. If you wanted to change a cached value in your app in addition to the SQL updates in the same transaction then you would be out of luck. This also required a lot of transaction code in your stored procedures, writing stored procedures that can be called independently or part of transaction made for very messy stored procedures and often led to multiple stored procedures that served the same purpose.

Using the SqlTransaction class was also messy. The most important restriction is that you need to have all database calls inside the same SqlConnection. This does not work well for a well-designed N-tier application. The other problem is that you need to handle your own non-DB transaction logic inside the same SqlTransaction and conditionally commit/rollback as necessary. This all tends to lead to several try-catch statements within one SqlTransaction. Handling the plumbing to manually add each SqlCommand to the transaction gets old quickly too.

Using SqlTransaction

   27 string connectionString = ConfigurationManager.ConnectionStrings["Testing"].ConnectionString;

   28 using (SqlConnection con = new SqlConnection(connectionString))

   29 {

   30     SqlTransaction tran = null;

   31     try

   32     {

   33         con.Open();

   34         tran = con.BeginTransaction();

   35         using (SqlCommand cmd = new SqlCommand("usp_ErrorLog_Insert", con))

   36        
{

   37             cmd.Transaction = tran;

   38             cmd.CommandType = System.Data.CommandType.StoredProcedure;

   39             cmd.Parameters.AddWithValue("Message", "Testing
1");

   40             cmd.Parameters.AddWithValue("UserID", 5150);

   41             cmd.ExecuteNonQuery();

   42        
}

   43 

   44         using (SqlCommand cmd = new SqlCommand("usp_ErrorLog_Insert", con))

   45        
{

   46             cmd.Transaction = tran;

   47             cmd.CommandType = System.Data.CommandType.StoredProcedure;

   48             cmd.Parameters.AddWithValue("Message", "Testing
2");

   49             cmd.Parameters.AddWithValue("UserID", 5150);

   50             cmd.ExecuteNonQuery();

   51        
}

   52 

   53         tran.Commit();

   54     }

   55     catch

   56     {

   57         if (tran != null) tran.Rollback();

   58     }

   59     finally

   60     {

   61         con.Close();

   62     }

   63 }

System.Transactions liberated us from the mundane SqlClient code and repetitive try-catches. simply wrapping your old ADO.NET with a using (TransactionScope) { } is all you need to do. You will typically add a transactionScope.Complete() statement as the last line in the TransactionScope using block is really all you need. Any exception thrown before this point will break out of scope, implicitly aborting the transation. Much better.

System.Transactions uses the LTM (Lightweight Transaction Manager) when dealing with single resources or machines. The transaction is automatically promoted to MSDTC (Microsoft Distributed Transaction Coordinator) when another resource is enlisted in a transaction. A lot of people struggle with MSDTC because it is difficult to setup, requires special firewall considerations, and doesn't really work well for smart client applications since you have to install MSDTC on every client machine.

I'll show one transaction performed three different ways and show what happens with the LTM and MSDTC for each of them. I will also demonstrate an excellent reason to migrate to Enterprise Library.

1) Executing two ADO.NET SqlCommands in different SqlConnections

  122 using (TransactionScope scope = new TransactionScope())

  123 {

  124     string connectionString = ConfigurationManager.ConnectionStrings["Testing"].ConnectionString;

  125     using (SqlConnection con = new SqlConnection(connectionString))

  126     using (SqlCommand cmd = new SqlCommand("usp_ErrorLog_Insert", con))

  127     {

  128         cmd.CommandType = System.Data.CommandType.StoredProcedure;

  129         cmd.Parameters.AddWithValue("Message", "Testing
1");

  130         cmd.Parameters.AddWithValue("UserID", 5150);

  131         try

  132        
{

  133             con.Open();

  134             cmd.ExecuteNonQuery();

  135        
}

  136         finally

  137        
{

  138             con.Close();

  139        
}

  140     }

  141 

  142     Console.WriteLine("Local
Transaction ID: {0}", 

  143         Transaction.Current.TransactionInformation.LocalIdentifier);

  144     Console.WriteLine("Distributed
Transaction ID: {0}", 

  145         Transaction.Current.TransactionInformation.DistributedIdentifier.ToString());

  146 

  147     using (SqlConnection con = new SqlConnection(connectionString))

  148     using (SqlCommand cmd = new SqlCommand("usp_ErrorLog_Insert", con))

  149     {

  150         cmd.CommandType = System.Data.CommandType.StoredProcedure;

  151         cmd.Parameters.AddWithValue("Message", "Testing
2");

  152         cmd.Parameters.AddWithValue("UserID", 5150);

  153         try

  154        
{

  155             con.Open();

  156             cmd.ExecuteNonQuery();

  157        
}

  158         finally

  159        
{

  160             con.Close();

  161        
}

  162     }

  163 

  164     Console.WriteLine("Local
Transaction ID: {0}", 

  165         Transaction.Current.TransactionInformation.LocalIdentifier);

  166     Console.WriteLine("Distributed
Transaction ID: {0}", 

  167         Transaction.Current.TransactionInformation.DistributedIdentifier.ToString());

  168 

  169     scope.Complete();

  170 }

This writes the following to the command line:

Local Transaction ID: e90f47f4-df80-496b-a9c0-0c45b2f452c4:2
Distributed Transaction ID: 00000000-0000-0000-0000-000000000000
Local Transaction ID: e90f47f4-df80-496b-a9c0-0c45b2f452c4:2
Distributed Transaction ID: 1fad8108-ddae-496a-a7da-ce92df175e40

You'll notice that the first command creates a transaction using LTM as indicated by the Local Transaction ID. After the second command is executed, the transaction is promoted to DTC as indicated by the Distributed Transaction ID. This is expected because there are two distinct SqlConnections. Even though the connection string is the same, TransactionScope treats these ADO.NET objects as unique resources.

This has additional implications when connection pooling comes into play. After I close the first connection, it is returned to the pool and is available for use. If this connection is requested for use, it will no longer be available to commit or abort this transaction, and you will see the dreaded MSDTC error "Communication with the underlying transaction manager has failed."

2) Executing two ADO.NET SqlCommands in the same SqlConnection

   69 string connectionString = ConfigurationManager.ConnectionStrings["Testing"].ConnectionString;

   70 using (TransactionScope scope = new TransactionScope())

   71 using (SqlConnection con = new SqlConnection(connectionString))

   72 {

   73     using (SqlCommand cmd = new SqlCommand("usp_ErrorLog_Insert", con))

   74     {

   75         cmd.CommandType = System.Data.CommandType.StoredProcedure;

   76         cmd.Parameters.AddWithValue("Message", "Testing
1");

   77         cmd.Parameters.AddWithValue("UserID", 5150);

   78         try

   79        
{

   80             con.Open();

   81             cmd.ExecuteNonQuery();

   82        
}

   83         finally

   84        
{

   85             con.Close();

   86        
}

   87     }

   88 

   89     Console.WriteLine("Local
Transaction ID: {0}", 

   90         Transaction.Current.TransactionInformation.LocalIdentifier);

   91     Console.WriteLine("Distributed
Transaction ID: {0}", 

   92         Transaction.Current.TransactionInformation.DistributedIdentifier.ToString());

   93 

   94     using (SqlCommand cmd = new SqlCommand("usp_ErrorLog_Insert", con))

   95     {

   96         cmd.CommandType = System.Data.CommandType.StoredProcedure;

   97         cmd.Parameters.AddWithValue("Message", "Testing
2");

   98         cmd.Parameters.AddWithValue("UserID", 5150);

   99         try

  100        
{

  101             con.Open();

  102             cmd.ExecuteNonQuery();

  103        
}

  104         finally

  105        
{

  106             con.Close();

  107        
}

  108     }

  109 

  110     Console.WriteLine("Local
Transaction ID: {0}", 

  111         Transaction.Current.TransactionInformation.LocalIdentifier);

  112     Console.WriteLine("Distributed
Transaction ID: {0}", 

  113         Transaction.Current.TransactionInformation.DistributedIdentifier.ToString());

  114 

  115     scope.Complete();

  116 }

This writes the following to the command line:

Local Transaction ID: e90f47f4-df80-496b-a9c0-0c45b2f452c4:1
Distributed Transaction ID: 00000000-0000-0000-0000-000000000000
Local Transaction ID: e90f47f4-df80-496b-a9c0-0c45b2f452c4:1
Distributed Transaction ID: becac9c9-e15f-4370-9f73-7f369665bed7

This is not expected because both commands are part of the same connection. Of course I am closing the connection to simulate an N-tier app where the data access layer is maintaining it's own SQL access, opening and closing its connection as it should. If I did not close the connection, you would not see a Distributed Transaction ID after the second command.

3) Executing two Enterprise Library commands

  176 using (TransactionScope scope = new TransactionScope())

  177 {

  178     Database db = DatabaseFactory.CreateDatabase("Testing");

  179     DbCommand cmd = db.GetStoredProcCommand("usp_ErrorLog_Insert");

  180     db.AddInParameter(cmd, "Message", System.Data.DbType.String, "Testing
1");

  181     db.AddInParameter(cmd, "UserID", System.Data.DbType.Int32, 5150);

  182     db.ExecuteNonQuery(cmd);

  183 

  184     Console.WriteLine("Local
Transaction ID: {0}", 

  185         Transaction.Current.TransactionInformation.LocalIdentifier);

  186     Console.WriteLine("Distributed
Transaction ID: {0}", 

  187         Transaction.Current.TransactionInformation.DistributedIdentifier.ToString());

  188 

  189     Database db1 = DatabaseFactory.CreateDatabase("Testing1");

  190     DbCommand cmd1 = db.GetStoredProcCommand("usp_ErrorLog_Insert");

  191     db1.AddInParameter(cmd1, "Message", System.Data.DbType.String, "Testing
2");

  192     db1.AddInParameter(cmd1, "UserID", System.Data.DbType.Int32, 5150);

  193     db1.ExecuteNonQuery(cmd1);

  194 

  195     Console.WriteLine("Local
Transaction ID: {0}", 

  196         Transaction.Current.TransactionInformation.LocalIdentifier);

  197     Console.WriteLine("Distributed
Transaction ID: {0}", 

  198         Transaction.Current.TransactionInformation.DistributedIdentifier.ToString());

  199 

  200     scope.Complete();

  201 }

This writes the following to the command line:

Local Transaction ID: 6737b756-2d5b-4eff-902d-15f9ccd5c26f:3
Distributed Transaction ID: 00000000-0000-0000-0000-000000000000
Local Transaction ID: 6737b756-2d5b-4eff-902d-15f9ccd5c26f:3
Distributed Transaction ID: 00000000-0000-0000-0000-000000000000

Whoa! How cool is that? No DTC promotion. Enterprise Library is intelligently deciding to keep the connection open when it is part of the transaction. This will save a lot of wasted time as the promotion to DTC adds a noticeable delay. If I wasn't using Enterprise Library already, I'd switch now.

Useful links: