In a previous post, I discussed solutions to the dreaded “The flowed transaction could not be unmarshaled” error commonly experienced when using MSDTC transactions with WCF, SQL, TxF, etc. I have once again experienced the un-trusted domain scenario, and can now report with certainty that adding hosts file entries on both machines will correct the problem. Testing this solution with DTCPing.exe between the two machines proves that making only the hosts file change acquaints the client and server and allows distributed transactions to occur.
You will find many blog and forum post non-solutions. Adding the hosts file entry or the equivalent domain redirects are the only solutions when working with two machines in disparate, un-trusted domains. Some of the non-solutions you’ll find go so far as to say to change your SQL connection string to prevent current (ambient) transaction enlistment. Not quite a complete solution as your first rollback unit test will fail.