Luis Guerrero EN

Run elevated task at any point of the Azure Rol lifetime.

by on Apr.30, 2011, under .NET, Azure, wcf

There are two mostly common used for Windows Azure, starting a new 100% .NET project on Windows Azure, or migrate a legacy project to the cloud. Talking from this point legacy project could be the most difficult project of all, because can use COM components during him lifetime. As you may know all the Windows Azure roles are clean machine so it’s means that you need to install manually all your component COM. Normally this issue is solved by placing on the startup node of the Role configuration file, a .cmd file with all the registration of the COM components.

But what happens if during role’s lifetime you need to execute a task elevated? You can’t do that because the hosting process of the worker role and web role is not executing with elevated rights. So we have to find a place where we can execute task with elevated rights, and this places is the roll’s startup node. So what we have to do is have a sentinel program up and running at roll’s startup and make this sentinel program to accept requests to execute process.

So this is exactly what we are going to do, using WCF to open a Windows pipe to enable communication between two different process in the same machine, enabling the process to send message with the necessary information to run this elevated task.

Let’s do it!

Service definition.

Since all we need is a WCF service be exposed on NetNamedPipeBinding, first of all we have to define a service contract with the operation contract we need. In this case we only need one operation, ExecuteTask.

[ServiceContract(Namespace = "http://azure.plainconcepts.com/schemas/04/2011/azure/executionhost")]
public interface IExecutionHost
{
    [OperationContract]
    void ExecuteTask(ProcessTask host);
}

Once we define the service contract now is the time to create the service itself, who is responsible to finally execute the tasks.

[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single)]
public class ExecutionHostService : IExecutionHost
{
    public void ExecuteTask(ProcessTask host)
    {
        Process process = new Process();
        process.StartInfo = host.StartInfo;
        process.Start();
    }
}

When we have done this, the next step is host the server itself on the sentinel process that will handle the requests, as we said before we’re going to use the NetNamedPipeBinding.

public class ExecutionHostServiceManager
{
    public ExecutionHostServiceManager()
    {
        service = new ExecutionHostService();
        ServiceHost host = new ServiceHost(service);

        string address = "net.pipe://PlainConcepts/Azure/ExecutionHost";
        NetNamedPipeBinding binding = new NetNamedPipeBinding(NetNamedPipeSecurityMode.None);
        host.AddServiceEndpoint(typeof(IExecutionHost), binding, address);

        // Add a mex endpoint
        long maxBufferPoolSize = binding.MaxBufferPoolSize;

        int maxBufferSize = binding.MaxBufferSize;

        int maxConnections = binding.MaxConnections;

        long maxReceivedMessageSize =
            binding.MaxReceivedMessageSize;

        NetNamedPipeSecurity security = binding.Security;

        string scheme = binding.Scheme;

        XmlDictionaryReaderQuotas readerQuotas =
            binding.ReaderQuotas;

        BindingElementCollection bCollection = binding.CreateBindingElements();

        HostNameComparisonMode hostNameComparisonMode =
            binding.HostNameComparisonMode;

        bool TransactionFlow = binding.TransactionFlow;

        TransactionProtocol transactionProtocol =
            binding.TransactionProtocol;

        EnvelopeVersion envelopeVersion =
            binding.EnvelopeVersion;

        TransferMode transferMode =
            binding.TransferMode;
        host.Open();
    }

    private ExecutionHostService service;
}

And that is!

Now we have to put this together in a console application and wait forever.

class Program
{
    static void Main(string[] args)
    {
        new ExecutionHostServiceManager();
        Thread.Sleep(Timeout.Infinite);
    }
}

Making call to the service

We now have the service contract, the service definition and the hosting process, now it’s time to define the client and make call to the servicer. Since we’re using WCF we need to define a class that will handle the request to the pipe. To accomplish this it’s necessary to inherit from ClientBase<T>, and T need to the service contract of the service.

public class ExecutionHostClient : ClientBase<IExecutionHost>
{
    static ExecutionHostClient()
    {
        string address = "net.pipe://PlainConcepts/Azure/ExecutionHost";
        NetNamedPipeBinding binding = new NetNamedPipeBinding(NetNamedPipeSecurityMode.None);
        binding.CloseTimeout = TimeSpan.MaxValue;
        binding.ReceiveTimeout = TimeSpan.MaxValue;
        binding.SendTimeout = TimeSpan.MaxValue;
        EndpointAddress endpoint = new EndpointAddress(address);
        client = new ExecutionHostClient(binding, endpoint);
    }

    public ExecutionHostClient(Binding binding, EndpointAddress remoteAddress) :
        base(binding, remoteAddress)
    {
    }

    public void ExecuteTask(ProcessTask task)
    {
        Channel.ExecuteTask(task);
    }

    public static void ExecuteRemoteTask(ProcessTask task)
    {
        client.ExecuteTask(task);
    }

    private static ExecutionHostClient client;
}

It’s important to create this proxy with the same configuration of the server, because we’re not using the default Visual Studio code generated proxy, and it’s our responsibility to create the binding and the address of the endpoint.

Invoking services

In this example we are registering COM components by calling the regsvc32.exe in code. We looking for dll files on a known folder of our Windows Azure Solution and invoke the service to make the elevated call.

public class RegisterComHelper
{
    public RegisterComHelper()
    {

    }

    public void Register()
    {
        // hay que buscar la localizacion en el servidor de azure de donde estan los ensamblados
        // como no sabemos donde estan los ficheros tenemos que buscar el modulo 
        // Habitania.RegisterCom.dll que es especifico para este ejemplo
        // así nos aseguramos que estamos buscando la dll correcta
        Process current = Process.GetCurrentProcess();
        var found = (from p in current.Modules.Cast<ProcessModule>().ToList()
                     where p.ModuleName == "PlainConcepts.Azure.WorkerRoleDemo.dll"
                     select p).FirstOrDefault();

        if (found != null)
        {
            // a partir de la locacion del modulo cargada por el proceso 
            // somos capaces de encontrar la informacion del directorio y buscar
            // la carpeta dlls que contiene la lista de dlls que queremos registar
            string directoryLocation = Path.GetDirectoryName(found.FileName);

            string dllPath = Path.Combine(directoryLocation, "V3COM30");

            string[] files = Directory.GetFiles(dllPath);

            foreach (var item in files)
            {
                if (item.EndsWith(".dll"))
                    RegisterComObject(item);
            }

            dllPath = Path.Combine(directoryLocation, "V3COM");

            files = Directory.GetFiles(dllPath);

            foreach (var item in files)
            {
                if (item.EndsWith(".dll"))
                    RegisterComObject(item);
            }
        }
    }

    private void RegisterComObject(string filePath)
    {
        ProcessStartInfo info = new ProcessStartInfo();
        info.FileName = Path.Combine(
            Environment.GetFolderPath(Environment.SpecialFolder.System),
            "regsvr32.exe");
        info.Arguments = string.Format("/i {0}", filePath);
        info.UseShellExecute = false;

        ExecutionHostClient.ExecuteRemoteTask(new ProcessTask()
        {
            StartInfo = info
        });
    }
}

This way of invoking task on Windows Azure seems a little bit complicated, but once is done can be super flexible and permit to extend this demo with custom task and background tasks.

The complete source code can be downloaded from here.

Luis Guererro.

1,829 Comments :, , , , , more...

ByeByeBrain – Tower defence game from Plain Concepts

by on Sep.06, 2010, under Windows Phone 7, Xna

 

We proudly present at Plain Concepts Game Studios, ByeByeBrain. A zombie tower defence game for incoming Windows Phone 7. The game was developed with XNA using 3D for gameplay, with 5 different types of towers and 4 types of zombies including mini game for special attacks. The game combines the classic tower defence with exciting mini games and the possibility of an interactive camera during game play. The game have social integration with Facebook and Twitter to publish your scores and some screenshot of the game.

You can see the full video preview at http://games.plainconcepts.com/ at HD with Silverlight and this video in video (http://vimeo.com/14723559) and full desktop background with some concept art.

Also we are in Gizmodo home page. http://gizmodo.com/5631096/bye-bye-brain-says-hello-to-windows-phone-7

 

spanker1920x1200cleaner1920x1200colonel1920x1200fuzz1920x1200nerd1920x1200

  • Has a built in Dependency Property system based on WPF and Silverlight ones.
  • Has a 2d animation engine like BeginAnimation and SingleAnimation.
  • Approx. 20Mb of XAP file.
  • 61139 lines of code (Visual Studio 2010 metric system)
  • The game was developed in more than three locations during this 2 months developments
  • One member of the team is donnetes addict Smile

Luis Guerrero.

2,431 Comments :, , , , , more...

How to access to Keyboard in XNA for Windows Phone 7

by on Jul.17, 2010, under Windows Phone 7

One of the great thinks about developing on XNA is that you have access to low level graphics to draw your sprites and your 3d mesh, but sometimes you need some high level component to provide some functionality to the user like for example a keyboard.

If you plan the user on Windows Phone 7 to type the name to start a new game you need to create your own keyboard and textbox support. This can be a little bit hard, so that why Microsoft include a simple API to show a Task in the Phone in order the user write some text and return this string back to you.

image

Guide.BeginShowKeyboardInput(
       PlayerIndex.One,
       "You Win",
       "Insert your name",
       "",
       new AsyncCallback(OnEndShowKeyboardInput),
       null);

With this code you are invoking the Keyboard Task, you need to keep in mind that your application will be deactivated and If you have code there will be executed, once the user finish to type the name and tap on accept button will return back to your application.

image

private void OnEndShowKeyboardInput(IAsyncResult result)
{
    name = Guide.EndShowKeyboardInput(result);
}

Here is where you get the string that the user typed.

You can download simple demo showing this functionality from here.

Luis Guerrero.

2,405 Comments :, , more...

Azure worker role throw a FileLoadException on load

by on May.11, 2010, under Azure, Debugging

I’m working now on a Smooth Streaming Silverlight player for the world cup and we using Azure to host some services for the player. Recently I started the development of all those services on a Web project and when I’m trying to run the project inside the Azure Simulation Environment the web role throw me a FileLoadException.

[runtime] Role entrypoint could not be created:
System.IO.FileLoadException: Could not load file or assembly 'Microsoft.Silverlight.MediaPlayer.Web, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a10dc7bdece43c5c' or one of its dependencies. The given assembly name or codebase was invalid. (Exception from HRESULT: 0x80131047)
File name: 'Microsoft.Silverlight.MediaPlayer.Web, Version=1.0.0.0, Culture=neutral, PublicKeyToken=a10dc7bdece43c5c' ---> System.IO.FileLoadException: Could not load file or assembly 'file:///C:\Plain\Microsoft.SmoothPlayer\Source\Azure\Microsoft.Silverlight.MediaPlayer.Online\bin\Debug\Microsoft.Silverlight.MediaPlayer.Online.csx\roles\Microsoft.Silverlight.MediaPlayer.Web\approot\bin\Microsoft.Silverlight.MediaPlayer.Web.dll' or one of its dependencies. The given assembly name or codebase was invalid. (Exception from HRESULT: 0x80131047)
File name: 'file:///C:\Plain\Microsoft.SmoothPlayer\Source\Azure\Microsoft.Silverlight.MediaPlayer.Online\bin\Debug\Microsoft.Silverlight.MediaPlayer.Online.csx\roles\Microsoft.Silverlight.MediaPlayer.Web\approot\bin\Microsoft.Silverlight.MediaPlayer.Web.dll'

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

   at System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection)
   at System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
   at System.Reflection.Assembly.LoadFrom(String assemblyFile)
   at Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.CreateRoleEntryPoint(RoleType roleTypeEnum)
   at Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.InitializeRoleInternal(RoleType roleTypeEnum)

I double check that my local assembly Microsoft.Silverlight.MediaPlayer.Web.dll is there and have enough permission to read and load it. Also I enabled Fusion Log to catch the assembly fail load but I can’t find any references to my assembly… so I decided it’s time to WinDBG.

I attach to the process WaWebHost.exe (Microsoft Windows Azure Web Host), loaded all symbols and sos. The first command I wrote was !threads and I found this:

0:000> !threads
ThreadCount: 6
UnstartedThread: 0
BackgroundThread: 5
PendingThread: 0
DeadThread: 0
Hosted Runtime: no
                                              PreEmptive                                                Lock
       ID OSID        ThreadOBJ     State   GC     GC Alloc Context                  Domain           Count APT Exception
XXXX    1 1ba0 000000000014c4a0   2008220 Enabled  00000000053a1c10:00000000053a1fd0 0000000002f182a0     1 Ukn System.IO.PathTooLongException (0000000005305a80)
XXXX    2  294 00000000000e46d0      b220 Enabled  00000000053d09f8:00000000053d1fd0 0000000002f17990     0 MTA (Finalizer)
XXXX    3 13b0 00000000022f9350   a802220 Enabled  0000000000000000:0000000000000000 0000000002f17990     0 MTA (Threadpool Completion Port)
XXXX    4 1a80 0000000002fbc080    80a220 Enabled  0000000000000000:0000000000000000 0000000002f17990     0 MTA (Threadpool Completion Port)
XXXX    5 1ae0 0000000002fbe650      1220 Enabled  0000000000000000:0000000000000000 0000000002f17990     0 Ukn
   0    6  7e4 0000000002fe22c0      1020 Enabled  00000000053c69b0:00000000053c7fd0 0000000002f182a0     0 Ukn System.IO.FileLoadException (00000000053a7ca8)

As you can see threre are two applications domain inside this process and one of them (0000000002f182a0) have to exception in thread #1 and #6.

The exception in Thread #6 is familiar but the other thread has a System.IO.PathTooLongException.

0:001> !pe 0000000005305a80
Exception object: 0000000005305a80
Exception type: System.IO.PathTooLongException
Message: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
InnerException: <none>
StackTrace (generated):
    SP               IP               Function
    00000000031BD360 000007FEED2D5A8F mscorlib_ni!System.IO.Path.NormalizePathFast(System.String, Boolean)+0xd0f
    00000000031BD420 000007FEED239CA6 mscorlib_ni!System.IO.File.Exists(System.String)+0x96

StackTraceString: <none>
HResult: 800700ce
The current thread is unmanaged

So it’s seem that my path it’s too long to be loaded by the runtime!

Hope this help!

Luis Guerrero.

2,670 Comments more...

Detecting memory issues with Silverlight

by on Apr.30, 2010, under Debugging, Silverlight

Hi all!

Today we’re going to talking about a memory leak issue found in Silverlight 4 and report on Silverlight’s forum by tsheflin As you know Silverlight is a .NET technology, so it’s mean you can use C# or Visual Basic to build your Silverlight’s applications and you have all the goodness of .NET, automatic memory management, garbage collector, type safety and so on.

But sometimes something goes wrong and we found that there are memory leaks on a dot net application; Silverlight is not free about this issues. So we’re going to identify how to find this memory issues with WinDBG and sos for Silverlight.

I’m going to use the same example provided by the user of the Silverlight.net forum. You can download from here.

First of all we need to build and run our application in release mode, this is important because there are a few differences between debug and release mode, then run the application without debugging and now its WinDBG time!

If your browser is Windows Internet Explorer 8 and you running Windows 7 or Vista, you may know that IE runs each tab in a separate process so you need to identify first witch process is running your code, in my example I found my process is 7958.

With WinDBG open (you can download from here http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx#a), go to File -> Attach to process (F6), and select your process id (in my machine is 7958), immediately appear output window with all the basic debug information. Since our application is a managed application and WinDBG is designed to debug native application we need a debugger extension to debug managed application. What we need is SOS (Son of Strike). SOS is located in my machine here C:\Program Files (x86)\Microsoft Silverlight\4.0.50401.0\sos.dll for every version of Silverlight there is a SOS version, so depending the Silverlight version you’re debugging you need to load this extension.

Once we have this file, SOS can be loaded writing

.load C:\Program Files (x86)\Microsoft Silverlight\4.0.50401.0\sos.dll

And now we have all the power of SOS in WinDBG.

Next we need to find a valid memory address for our type SilverlightVisualTreeRemovalFail.SilverlightControl1 so we’re going to use !dumpheap command to dump all heap types and filter by this type writing this:

!dumpheap -type SilverlightVisualTreeRemovalFail.SilverlightControl1

0:005> !dumpheap -type SilverlightVisualTreeRemovalFail.SilverlightControl1

Address MT Size

087aad6c 04a45230 92

087ab460 04a45230 92

087ab6f0 04a45230 92

087ac01c 04a45230 92

087ac710 04a45230 92

087ace04 04a45230 92

087ad4f8 04a45230 92

087adbec 04a45230 92

087ae2e0 04a45230 92

087ae9d4 04a45230 92

087af0c8 04a45230 92

087af7bc 04a45230 92

087afeb0 04a45230 92

087b05a4 04a45230 92

087b0c98 04a45230 92

087b138c 04a45230 92

087b1a80 04a45230 92

087b2174 04a45230 92

087b2868 04a45230 92

087b2f5c 04a45230 92

087b3650 04a45230 92

087b3d44 04a45230 92

087b4438 04a45230 92

087b4b2c 04a45230 92

087b5220 04a45230 92

087b5914 04a45230 92

087b6008 04a45230 92

087b66fc 04a45230 92

087b6df0 04a45230 92

087b74e4 04a45230 92

087b7bd8 04a45230 92

087b82cc 04a45230 92

087b89c0 04a45230 92

087b90b4 04a45230 92

087b97a8 04a45230 92

087b9e9c 04a45230 92

087ba590 04a45230 92

087bac84 04a45230 92

087bb378 04a45230 92

087bba6c 04a45230 92

087bc160 04a45230 92

087bc854 04a45230 92

087bcf48 04a45230 92

087bd63c 04a45230 92

087bdd30 04a45230 92

087be424 04a45230 92

087beb18 04a45230 92

087bf20c 04a45230 92

087bf900 04a45230 92

087bfff4 04a45230 92

In output window we found some addresses for this type, selecting one 087b1a80 we need to find witch object is referencing this object (this is what is causing not to be recoleted) writing this:

!gcroot 087b1a80
0:005> !gcroot 087b1a80

Note: Roots found on stacks may be false positives. Run "!help gcroot" for

more info.

Scan Thread 5 OSTHread 10e8

Scan Thread 22 OSTHread 1f8

Scan Thread 23 OSTHread 1588

DOMAIN(07134BE0):HANDLE(Pinned):4a912f8:Root: 09784260(System.Object[])->

08796c78(System.Collections.Generic.Dictionary`2[[System.IntPtr, mscorlib],[System.Object, mscorlib]])->

09788260(System.Collections.Generic.Dictionary`2+Entry[[System.IntPtr, mscorlib],[System.Object, mscorlib]][])->

087b1e90(System.Windows.Controls.ControlTemplate)->

087b1a80(SilverlightVisualTreeRemovalFail.SilverlightControl1)

We found that this object (087b1a80) is reference by other object and on the top of the list there is a HANDLE(Pinned):4a912f8 so its means that the garbage collection will always found a path to this object causing not to be garbage collected.

Now we identify the problem, but for now we can’t do anything because this is a Microsoft issue, because Microsoft’s code is referencing our control so we can’t fix this by ourselves. If this code is from us what we need to is simply remove references from this dictionary and then the control will be garbage collected.

We can also dump all the heap to a file and then open this file with CLRProfiler, type !traverseheap and file output and the open with CLRProfiler.

All the best.

Luis Guerrero.

2,858 Comments more...

Silverlight player is now Microsfot’s Case Study

by on Apr.29, 2010, under Silverlight

Hi all!

Working as a developer sometimes is hard; clients are always busy and want their software delivered on time! So it’s exiting when you discover that software you made is a success for your client, so here we are.

Few months ago I started a Silverlight video player for Spanish’s national broadcaster Tele5. They want more flexibility to deliver their content to users. So we help them to build a Silverlight powered video player. This was before MEF (Managed Extensibility Framework) exists, so I implement by myself a plug-in arquitecture for Silverlight with some similar concepts as MEF have.

So, what’s this post about? To tell everyone that this project is now a Microsoft Case Studies and you can read all about this exciting project on Microsoft’s web page. Of course you can enjoy the player itself on Tele5’s news.

http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?casestudyid=4000007022 (EN)

http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?casestudyid=4000007045 (ES)

http://www.informativos.telecinco.es/

All the best.

Luis Guerrero.

Leave a Comment more...

Concurrent programming and Managed Extensibilty Framework

by on Mar.04, 2010, under .NET, Concurrent, MEF

Here they are! Last month I have been talking about concurrent programming on .NET Framework with Task Parallel Library and MEF (Managed Extensibility Framework), so here you have all the resources, the presentation in PowerPoint format and some demos.

clip_image001

Task Parallel Lirarty demos in Visual Studio 2010 RC format

There are no demos from MEF but you can download some from http://www.codeplex.com/mef/

2,135 Comments more...

Writing high performance parallel code with monitors

by on Nov.29, 2009, under .NET, Concurrent

Multithread application is going to be next issue for developer, and we need to be ready for this big change. In .NET Framework 4, Microsoft introduced Task Parallel Library, a set of API that helps developer creating concurrent applications. That’s means that you don’t have to take care about all those concurrent issues anymore, its mean now is easier to create concurrent code.

Currently I’m working in an application that makes a hard use of all TLP code. What I do is creating a lot of Task objects to manage the executing of a set of rules that have to be executed for each Uri. I maintain a list with all the Task object to observer during time, how much Task are now running, and make some statistics about software speed.

This means I have UI, created in WPF that need to communicate with this code to get all running Task and display this information in the UI. Since I’m using a simple List<Task> to get a reference to all task created during my application execution, I need a way to lock and add Tasks to this list, enumerate this list and remove Tasks. But I want this lock to be shortest one and only lock to important operations like add and remove items when Task are finished.

So I have two important operations, create new Task and add to this list, and when the Task is finish removes this Task from the list, and for that I need to wait if the lock is acquired by other thread, because multiple Tasks can finish at the same time.

Creating task is done by a timer inside my application and this timer is set to run each second to check if the running Task in the system are lower that minimum Task running. If this is true is the moment create more Task and add to the list. Since this timer is executed every second I’ll need to lock my list of Task, do my work and then release the lock. I’m using the list as lock object. But this timer can wait more than one second until the lock is released, because other Task are now executing and finish work and remove items from my list, so It’s mean I’m creating a classic convoy in my application. Every time my timer method is executed and wait until the lock to release, this time is bigger that timer elapsed time, so I need a way to wait only a certain period of time.

What I do in this method is using the new API for Monitor in .NET 4 that attempts, for a number of milliseconds, to acquire an exclusive lock and atomically set a value that indicates whether the lock was taken or not. In my case I set these milliseconds in 400, because in this method I do more stuff than work with my list of Task. If the lock is bigger than 400ms the lock will not be acquired.

bool taken = false;
try
{
    Monitor.TryEnter(task, 400, ref taken);
    if (taken)
    {
        if (!cancelationTokenSource.IsCancellationRequested)
        {
            var count = (from p in task
                         where p.Status == TaskStatus.Running || p.Status == TaskStatus.WaitingToRun
                         select p).Count();
        }
    }
}
finally
{
    if (taken)
    {
        Monitor.Exit(task);
    }
}

As you can see in this code I’m calling Monitor.TryEnter inside a try/catch block, and then in the finally block if the lock was acquired I release the lock by calling Monitor.Exit. I do in that way because if during the execution of my code an exception is throw, the call to Monitor.Exit at the end of the code will not be executed anymore, causing that I have a lock that will never be released.

My other scenario is when I want to get all Task to count how much task are now running, I do the same inside get accessor of the property Task, but in this case I only wait for 250ms instead of 400ms. If the lock is acquired I copy the collection into a new list and there I can safety count how much Task are running.

public List<Task> Tasks
{
    get
    {
        bool taken = false;
        List<Task> list = null;
        try
        {
            Monitor.TryEnter(task, 250, ref taken);
            if (taken)
            {
                list = new List<Task>(task.ToArray());
            }
        }
        finally
        {
            if (taken)
            {
                Monitor.Exit(task);
            }
        }
        return list;
    }
}

This solution is simple and keeps your code with the lowest contention rate giving boots to those important operations that need to wait until the lock is released.

I could also create a wrap over List<T> list using ReaderWriterLockSlim to control read and write operation during lock, but ReaderWriterLockSlim is focus on multiple readers and only one writer, and I my application I have multiple writers and only one reader.

All the best.

Luis.

2,578 Comments more...

Dealing with Heap Corruptions in .NET

by on Nov.16, 2009, under Debugging, WinDBG

One of the best features of the .NET Framework is automatic memory management, its means you don’t need to worry about memory leaks, memory corruptions and other memory issues. But this is not always true, I want to show an issue I having now with a .NET 4 application.

As may know clr have a managed heap, and this heap contains all the object you’ve creating across your code, and the garbage collector take care on dispose, free memory and sort all the content of the heap. There are some useful commands to show the content of the managed heap with SOS (Son of strike) WinDBG’s extension, one of them is !dumpheap.

During the execution of my application randomly throw me a System.ExecutionEngineException. As you can imagine this is a fatal error in the runtime itself, not in my code so there is no chance for catch the exception and my application will close.

0:281> !threads
ThreadCount:      325
UnstartedThread:  0
BackgroundThread: 268
PendingThread:    0
DeadThread:       56
Hosted Runtime:   no
                                           PreEmptive                                                   Lock
       ID  OSID        ThreadOBJ     State GC       GC Alloc Context                  Domain           Count APT Exception
   0    1   f74 00000000001b47b0      6020 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 STA
   2    2  3ddc 00000000001baaf0      b220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Finalizer)
   5    3  3f44 0000000000fb8ee0   100a220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
   7    4  2acc 000000001c75c120   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
   8    5   f2c 000000001c75c830   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
   9    6  1bf4 000000001c75cf40   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  10    7  1e10 000000001c75ea80   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX    9       000000001c7f8dd0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  12    b  1a9c 000000001c814830   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  13    d  1f50 000000001df31e80   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  14    a  21b4 000000001c7fdd10   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  15    f  2d54 000000001df31160   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  18   10  3edc 000000001df4ef50      b220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     1 MTA
  19   12  3ffc 000000001df60bd0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  20    e   958 000000001df612e0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  21    c  2be0 000000001df62100   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  22   14  1920 000000001df62f20   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  23   11  39f0 000000001df619f0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  24   15  2e00 000000001df63630   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   16       000000001df63d40   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  25   17   734 000000001df64450   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  26   18  2c78 000000001df64b60   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   19       000000001df65270   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 Ukn (Threadpool Worker)
  27   1a  2330 000000001df62810   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  28   1c  20f4 000000001df667a0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   13       000000001df66090   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   1e       000000001df675c0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  29   1b  360c 000000001df66eb0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  30   1d  1f04 000000001df67cd0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  31    8   ce0 000000001df65980   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  32   20  3a94 000000001df34070   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  33   21  2ce8 000000001df34780   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  34   22  3dd4 000000001df355a0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  35   24  31c8 000000001df363c0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  36   26  15d0 000000001df371e0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  37   23  3aec 000000001df35cb0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  38   27  33b8 000000001df378f0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  39   1f   afc 000000001df34e90   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  40   28   9b8 000000001df683e0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  41   29  1ba8 000000001c81e7c0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   2a       000000001c81f5e0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  42   25  3cbc 000000001c81eed0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  43   2c  20fc 000000001df36ad0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  44   2d  290c 000000001c8573d0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (GC) (Threadpool Worker) System.ExecutionEngineException (0000000002871228)
  45   2b  2f58 000000001c81fcf0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  46   2f  33b4 000000001c8581f0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   2e       000000001c858900   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 Ukn (Threadpool Worker)
  47   31  1020 000000001c859010   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   32       000000001c859720   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 Ukn (Threadpool Worker)
  48   33   ad8 000000001c859e30   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  49   34  3230 000000001c85a540   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  50   35  1a74 000000001c85ac50   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  51   36  3650 000000001c8064f0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  52   37  2890 000000001c806c00   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   38       000000001c807310   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  53   39  1c20 000000001c807a20   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   3a       000000001dfe2860   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  54   3b  1694 000000001dfe2f70   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  55   3d  37b8 000000001dfe3d90   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   30       000000001dfe44a0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   3e       000000001dfe4bb0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  56   3f  18b8 000000001dfe52c0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  57   40  11b0 000000001dfe59d0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  58   41  1ee0 000000001dfe60e0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   42       000000001c857ae0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  59   43  27e8 000000001e0c80b0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  60   44   a84 000000001e0c87c0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  61   46  1ba4 000000001e0c95e0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  62   48  2160 000000001e0ca400   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  63   4a  2bdc 000000001e0cb220   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  64   47  12d0 000000001e0c9cf0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  65   49   be4 000000001e0cab10   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  66   3c  186c 000000001e0cb930   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   4b       000000001dfe3680   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  67   45  3498 000000001e0c8ed0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   4d       000000001e19b9a0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  68   4e  1550 000000001e19c0b0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  69   50  313c 000000001e19ced0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   51       000000001e19d5e0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   4f       000000001e19c7c0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  70   52  20bc 000000001e19dcf0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  71   4c  33fc 000000001e19b290   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  72   54  2380 000000001e19eb10   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  73   53   a04 000000001e19e400   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  74   56  2028 000000001e22f050   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  75   58  3df0 000000001e22fe70   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   57       000000001e230580   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  76   55  1d08 000000001e22f760   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   5a       000000001e230c90   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  77   5c  373c 000000001e231ab0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   59       000000001e22e940   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 Ukn (Threadpool Worker)
  78   5d   e90 000000001e2321c0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   5e       000000001e2ba200   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  79   5f  1358 000000001e2ba910   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   5b       000000001e2bb020   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  80   60  1a98 000000001e2bb730   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  81   61  26f4 000000001e2bbe40   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  82   62   abc 000000001e2bc550   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  83   63  19d4 000000001e2bcc60   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  84   64  1220 000000001e2bd370   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  85   65  1e6c 000000001e2bda80   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  86   66  162c 000000001e2313a0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   67       000000001e2de370   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  87   68  3040 000000001e2dea80   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  88   6a  139c 000000001e2df8a0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  89   6b  39f4 000000001e2dffb0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  90   6c  375c 000000001e2e06c0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  91   6d  1548 000000001e2e0dd0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  92   6e  2920 000000001e2e14e0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   6f       000000001e2e1bf0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  93   70  376c 000000001e2e6370   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   71       000000001e2e6a80   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  94   72  14ec 000000001e2e7190   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   75       000000001e2e78a0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  95   73  1930 000000001e2e86c0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  96   76  17dc 000000001e2e8dd0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  97   74   c88 000000001e2e7fb0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  98   77   d0c 000000001e2e94e0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   78       000000001e2e9bf0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   7a       0000000024ddd4c0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
  99   7b  2878 0000000024dddbd0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 100   79  1278 0000000024dde2e0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 101   7c  2358 0000000024ddcdb0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   7d       0000000024dde9f0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 102   7e  28a8 0000000024ddf100   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 103   7f  1778 0000000024ddf810   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 104   80  2990 0000000024ddff20   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 105   81  1864 0000000024de0630   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 106   82  3dc4 0000000024de4db0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 107   83  371c 0000000024de54c0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 108   84  3e80 0000000024de5bd0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 109   85  3634 0000000024de62e0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 110   86  3908 0000000024de69f0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 111   87  2998 0000000024de7100   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 112   88  200c 0000000024de7810   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 113   89  3370 0000000024de7f20   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 114   8a  33dc 0000000024de8630   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 115   8b  1b98 0000000024df2a90   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 116   8c  1378 0000000024df31a0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 117   8d  3e0c 0000000024df38b0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   8e       0000000024df3fc0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 118   8f  3444 0000000024df46d0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 119   90  3044 0000000024df4de0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   91       0000000024df54f0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 120   92  389c 0000000024df5c00   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 121   93  3938 0000000024df6310   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   95       0000000024e8d5b0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 122   96  18e0 0000000024e8dcc0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 123   97  3954 0000000024e8cea0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 124   98  329c 0000000024e8e3d0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 125   94  2cf0 0000000024e8eae0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 126   9a  3d18 0000000024e8f900   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 127   9b   a80 0000000024e8f1f0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 128   9c  13ac 0000000024e90720   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 129   99  2f84 0000000024e90010   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   9e       000000001e0f9180   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   9f       000000001e0f8a70   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 130   a0  2a20 000000001e0f9fa0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   9d       000000001e0f9890   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 131   a3  2544 000000001e0fb4d0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 132   a1  2910 000000001e0fa6b0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 133   a4   a9c 000000001e0fadc0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 134   a5  1880 000000001e0fbbe0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 135   a6  2d28 0000000024f038f0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 136   a7  12d4 0000000024f04710   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   a8       0000000024f04e20   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 137   a9  11c8 0000000024f05530   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 138   ab  2d4c 0000000024f06350   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 139   aa  1bf8 0000000024f05c40   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 140   a2   244 0000000024f04000   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 141   ac  2688 0000000024f06a60   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 142   ad  27e0 0000000024f07170   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 143   ae  315c 000000001e0fc2f0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   af       000000002501d670   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 144   b0   7bc 000000002501dd80   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 145   b3  1294 000000002501f2b0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 146   b4  3e94 000000002501f9c0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 147   b5   fc8 00000000250200d0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 148   b6   d48 00000000250207e0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 149   b7  1f84 0000000025020ef0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 150   b8  3138 0000000025021600   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 151   b9  3214 0000000025021d10   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 152   ba  175c 0000000025022420   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 153   bb  3a24 0000000025022b30   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 154   bc  340c 0000000025023240   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   bd       0000000025023950   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 155   be  27b4 0000000025024060   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 156   bf  2c54 0000000025024770   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 157   c0  3698 0000000025024e80   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 158   c1  20f0 0000000025064a30   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 159   c2   1dc 0000000025065140   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 160   c3  2378 0000000025065850   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 161   c4  1444 0000000025065f60   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   c5       0000000025066670   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   c6       0000000025066d80   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 162   c7   b0c 0000000025067490   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 163   c8  30f4 0000000025067ba0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 164   c9  3d60 00000000250682b0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 165   ca  108c 00000000250689c0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 166   cb  37b0 00000000250690d0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   cc       00000000250697e0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 167   b1   dc0 0000000025069ef0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 168   b2  3ed4 000000002506a600   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   cd       000000002506ad10   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 169   ce   c38 000000002506b420   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 170   cf  24cc 000000002506bb30   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   d0       000000002506c240   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 171   d1  1d80 000000002501e490   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 172   d2  1edc 000000002501eba0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 173   d3  304c 000000002513cbf0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 174   d4  3b08 000000002513d300   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 175   d7  3adc 000000002513e830   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   d8       000000002513ef40   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 177   69  2178 000000002513da10   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 178   d6  3b40 000000002513e120   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 179   d9   f28 000000002513fd60   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 180   da  382c 0000000025140470   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   db       0000000025140b80   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 181   dc   b88 0000000025141290   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 182   dd  22ac 00000000251419a0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 183   df  1aac 00000000251427c0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 184   e0  353c 0000000025142ed0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   d5       000000002513f650   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 185   de  319c 00000000251420b0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   e1       00000000251435e0   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 186   e2  3678 0000000025143cf0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 187   e3  13fc 0000000025144400   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 188   e4  3038 000000001e2df190   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 189   e5  3cc0 0000000024e5ae20   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 190   e6  3384 0000000024e5b530   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 191   e7  301c 0000000024e5bc40   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 192   e8  3448 0000000024e5c350   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 193   e9  2bcc 0000000024e5ca60   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   ea       0000000024e5d170   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   eb       0000000024e5d880   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX   ec       0000000024e5df90   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 194   ed  1d04 0000000024e5e6a0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 195   ee   c54 000000002523fed0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 196   ef  2220 00000000252405e0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 197   f0  2b60 0000000025240cf0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 198   f1  2d88 0000000025241400   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 199   f2  3ee8 0000000025241b10   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 200   f3  3f2c 0000000025242220   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 201   f4  1240 0000000025242930   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 202   f6  23dc 0000000025243750   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 203   f7   a74 0000000025243040   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 204   f8  3f40 0000000025243e60   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 205   f9  2f94 0000000025244c80   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 206   fa   fa4 0000000025245390   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 207   fb  2c7c 0000000025245aa0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 208   f5  2a30 0000000025244570   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 209   fe  102c 0000000025246fd0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX  100       0000000025384550   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 210   fd   c28 00000000252461b0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 211   ff  233c 0000000025384c60   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 212  101  3c34 00000000252476e0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX  102       0000000025385a80   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 213  104  1ad4 00000000253868a0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 214   fc  2ef0 0000000025385370   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 215  103  16bc 0000000025386190   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 216  105   990 0000000025386fb0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 218  107  3b38 0000000025387dd0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 219  108   974 00000000253884e0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 220  109  10e4 0000000025388bf0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 221  10a  1280 0000000025389300   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 222  10b  1494 00000000253876c0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 223  106  25ac 0000000025389a10   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 224  10e  2b94 000000002538af40   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 225  10c  3218 000000002538a830   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 226  110  2a5c 000000002538bd60   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 227  10d  2d90 000000002538b650   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 228  10f   e10 00000000252468c0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 229  112  21bc 0000000025464400   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 230  113  13e4 0000000025464b10   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 231  114  2d84 0000000025465220   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 232  115  3e98 0000000025465930   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 233  116  115c 0000000025466040   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 234  117  1228 0000000025466750   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX  118       0000000025466e60   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 235  119  3054 0000000025467570   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 236  11a  10f4 0000000025467c80   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 237  11b  18b4 0000000025468390   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 238  11c  3a1c 0000000025468aa0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 239  11d  187c 00000000254691b0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 240  11e  342c 00000000254698c0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 241  11f  12f4 0000000025469fd0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 242  120  2a04 000000002546a6e0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 243  121  36dc 000000002546adf0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 244  122   ffc 000000002546b500   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 245  123   c70 000000002546bc10   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 246  124  39f8 000000002546c400   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 247  125  34c4 000000002546cb10   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX  126       000000002546d220   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 248  127   ae0 000000002546d930   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 249  129  17fc 000000002546ee60   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 250  12b  2288 000000002546f570   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 251  111  2fa0 000000002546e750   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX  12a       000000002546fc80   1019820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 252  128  3160 0000000025470aa0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 253  12c  1fd8 000000002546e040   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 256  12d  13bc 0000000025472df0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 258  12e   fc0 00000000254718c0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 259  12f  2d34 0000000025471fd0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 260  130  366c 00000000254726e0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 261  131   af0 00000000254711b0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 262  132  2bbc 0000000025470390   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 263  133   ae8 0000000025473500   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 264  134  37b4 0000000025473c10   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 265  135  1e54 000000002538a120   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 266  136  3fa0 0000000033dbb290   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 267  137  2698 0000000033dbb9a0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 268  138   ec0 0000000033dbc0b0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 269  139   d30 0000000033dbc7c0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 270  13a   cc8 0000000033dbced0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 271  13b  29f4 0000000033dbd5e0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 272  13c  2c74 0000000033dbdcf0   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 273  13d  3150 0000000033dbe400   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 274  13e  27c4 0000000033dbeb10   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 275  13f  1ee8 0000000033dbf220   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 276  140  1dec 0000000033dbf930   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 277  141  3398 0000000033dc0040   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 278  14a  274c 0000000033e073f0   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 279  14b  1d50 0000000033e08210   3009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
 280  149  1258 0000000033e07b00   1009220 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 MTA (Threadpool Worker)
XXXX  148       0000000033e06ce0     19820 Enabled  0000000000000000:0000000000000000 00000000001a85b0     0 Ukn

As you can see on this output from WinDBG at thread 44 the exception object is (GC) (Threadpool Worker) System.ExecutionEngineException (0000000002871228) and the address at the end is my System.ExecutionEngineException. Let’s start my investigation from there.

0:281> !pe 0000000002871228
Exception object: 0000000002871228
Exception type:   System.ExecutionEngineException
Message:          <none>
InnerException:   <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131506
The current thread is unmanaged

Now let’s examine managed stack and native stack.

0:044> !clrstack
OS Thread Id: 0x290c (44)
Child SP         IP               Call Site
000000002191d9b8 000007fef3fecca3 [HelperMethodFrame: 000000002191d9b8]
000000002191dc30 000007fef2f394b7 System.Text.StringBuilder.ToString()
000000002191dc80 000007feed584193 System.Data.Query.PlanCompiler.CTreeGenerator.ResolveVar(System.Data.Query.InternalTrees.Var)
000000002191dd40 000007feed5845fe System.Data.Query.PlanCompiler.CTreeGenerator.CreateProject(RelOpInfo, System.Collections.Generic.IEnumerable`1<System.Data.Query.InternalTrees.Var>)
000000002191de10 000007feed587e49 System.Data.Query.PlanCompiler.CTreeGenerator.Visit(System.Data.Query.InternalTrees.PhysicalProjectOp, System.Data.Query.InternalTrees.Node)
000000002191deb0 000007feed58582c System.Data.Query.PlanCompiler.CTreeGenerator..ctor(System.Data.Query.InternalTrees.Command, System.Data.Query.InternalTrees.Node)
000000002191df10 000007feed5725f1 System.Data.Query.PlanCompiler.ProviderCommandInfoUtils.Create(System.Data.Query.InternalTrees.Command, System.Data.Query.InternalTrees.Node, System.Collections.Generic.List`1<System.Data.Query.PlanCompiler.ProviderCommandInfo>)
000000002191df60 000007feed536485 System.Data.Query.PlanCompiler.CodeGen.Process(System.Collections.Generic.List`1<System.Data.Query.PlanCompiler.ProviderCommandInfo> ByRef, System.Data.Query.InternalTrees.ColumnMap ByRef, Int32 ByRef)
000000002191dfe0 000007feed4b4d90 System.Data.Query.PlanCompiler.PlanCompiler.Compile(System.Collections.Generic.List`1<System.Data.Query.PlanCompiler.ProviderCommandInfo> ByRef, System.Data.Query.InternalTrees.ColumnMap ByRef, Int32 ByRef, System.Data.Common.Utils.Set`1<System.Data.Metadata.Edm.EntitySet> ByRef)
000000002191e0d0 000007feed54418e System.Data.EntityClient.EntityCommandDefinition..ctor(System.Data.Common.DbProviderFactory, System.Data.Common.CommandTrees.DbCommandTree)
000000002191e270 000007feed62aae4 System.Data.EntityClient.EntityProviderServices.CreateDbCommandDefinition(System.Data.Common.DbProviderManifest, System.Data.Common.CommandTrees.DbCommandTree)
000000002191e2b0 000007feed5cf34d System.Data.Objects.Internal.ObjectQueryExecutionPlan.Prepare(System.Data.Objects.ObjectContext, System.Data.Common.CommandTrees.DbQueryCommandTree, System.Type, System.Data.Objects.MergeOption, System.Data.Objects.Span, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.Generic.KeyValuePair`2<System.Data.Objects.ObjectParameter,System.Data.Objects.ELinq.QueryParameterExpression>>)
000000002191e3b0 000007feed5fec9a System.Data.Objects.ELinq.ELinqQueryState.GetExecutionPlan(System.Nullable`1<System.Data.Objects.MergeOption>)
000000002191e4b0 000007feed610a65 System.Data.Objects.ObjectQuery`1[[System.__Canon, mscorlib]].GetResults(System.Nullable`1<System.Data.Objects.MergeOption>)
000000002191e530 000007feed6117cf System.Data.Objects.ObjectQuery`1[[System.__Canon, mscorlib]].System.Collections.Generic.IEnumerable<T>.GetEnumerator()
000000002191e580 000007feef066e79 System.Linq.Enumerable.First[[System.__Canon, mscorlib]](System.Collections.Generic.IEnumerable`1<System.__Canon>)
000000002191e5e0 000007feef081daf System.Linq.Queryable.First[[System.__Canon, mscorlib]](System.Linq.IQueryable`1<System.__Canon>)
000000002191e640 000007ff00416efa ServerDetection.Data.DataAccess.UpdateSever(ServerDetection.Server)
000000002191e940 000007ff004133b6 ServerDetection.StreamRulesEngine.ExploreServer(ServerDetection.Server)
000000002191eb50 000007ff00412f2c ServerDetection.StreamRulesEngine.<CreateTask>b__15(System.Object)
000000002191eb90 000007fef36796bb System.Threading.Tasks.Task.Execute()
000000002191ebf0 000007fef2f60d3c System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
000000002191ec50 000007fef3679455 System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)
000000002191ecd0 000007fef367d9d0 System.Threading.Tasks.Task.ExecuteEntry(Boolean)
000000002191ed10 000007fef2f96be8 System.Threading.ThreadPoolWorkQueue.Dispatch()
000000002191eda0 000007fef2f96a85 System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
000000002191f3f0 000007fef3e621b4 [DebuggerU2MCatchHandlerFrame: 000000002191f3f0] 
0:044> kb 2000
RetAddr           : Args to Child                                                           : Call Site
000007fe`f3feaf0a : 00000000`00000001 00000000`00000000 000007fe`00000002 000007fe`00000001 : clr!WKS::gc_heap::plan_phase+0x454
000007fe`f3feb6a5 : 000000d4`19ef3257 00000000`2191d629 00000000`00000001 00000000`00000001 : clr!WKS::gc_heap::gc1+0xbb
000007fe`f3feb35e : 00000000`00193900 000007fe`00000000 00000000`00000000 00000000`00000000 : clr!WKS::gc_heap::garbage_collect+0x276
000007fe`f3fe9c18 : 00000000`1c857440 00000000`00000088 000007fe`f3e30000 00000000`09350d00 : clr!WKS::GCHeap::GarbageCollectGeneration+0x14e
000007fe`f3fe929e : 00000000`000000a8 000007fe`f46c3700 00000000`2191d800 00000000`1c857440 : clr!WKS::gc_heap::try_allocate_more_space+0x25f
000007fe`f3ee4a95 : 00000000`00000088 00000000`00193900 00000000`1c857440 00000000`2191dca0 : clr!WKS::GCHeap::Alloc+0x7e
000007fe`f2f394b7 : 00000000`08000001 00000000`09351c00 00000000`030d39f0 00000000`0934f478 : clr!FramedAllocateString+0xb19
000007fe`ed584193 : 00000000`09350e88 00000000`09350d00 00000000`0300b1a8 00000000`09350e88 : mscorlib_ni+0x3694b7
000007fe`ed5845fe : 00000000`09350e08 00000000`093405d0 00000000`000000fa 00000000`00000000 : System_Data_Entity_ni+0x854193
000007fe`ed587e49 : 00000000`0934f478 00000000`09350ce8 00000000`00000000 000007fe`f3e98733 : System_Data_Entity_ni+0x8545fe
000007fe`ed58582c : 00000000`0934f478 00000000`0934bff0 00000000`093382d8 000007fe`ed5734c8 : System_Data_Entity_ni+0x857e49
000007fe`ed5725f1 : 00000000`2191def0 000007fe`f3e701ff 00000000`00000000 00000000`2191e1c8 : System_Data_Entity_ni+0x85582c
000007fe`ed536485 : 00000000`09334420 000007fe`ed3ed8b9 00000000`0934f450 00000000`2191e050 : System_Data_Entity_ni+0x8425f1
000007fe`ed4b4d90 : 00000000`00000000 00000000`00000000 00000000`02de9628 00000000`1288f490 : System_Data_Entity_ni+0x806485
000007fe`ed54418e : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : System_Data_Entity_ni+0x784d90
000007fe`ed62aae4 : 00000000`093341a0 00000000`062e3608 00000000`09333ef8 00000000`09333ef8 : System_Data_Entity_ni+0x81418e
000007fe`ed5cf34d : 00000000`09333f28 000007fe`ed3f0a59 00000000`04f1db88 000007fe`f2f46f83 : System_Data_Entity_ni+0x8faae4
000007fe`ed5fec9a : 00000000`062e3608 000007ff`003d27e8 00000000`02db2aa0 0000635b`00000000 : System_Data_Entity_ni+0x89f34d
000007fe`ed610a65 : 00000000`04f1db88 00000000`00000000 000007ff`002ae568 00000000`2191e0b8 : System_Data_Entity_ni+0x8cec9a
000007fe`ed6117cf : 00000000`04f1dbe8 000007ff`00489730 000007fe`eeefa86f 00000000`00000000 : System_Data_Entity_ni+0x8e0a65
000007fe`ef066e79 : 00000000`062e7bc8 00000000`090f7a68 00000000`090f7a68 000007fe`ed5ff21f : System_Data_Entity_ni+0x8e17cf
000007fe`ef081daf : 000007ff`00489730 000007ff`00489430 00000000`09124478 00000000`08ee3668 : System_Core_ni+0x456e79
000007ff`00416efa : 000007ff`00488e40 00000000`00000000 00000000`08ed3048 00000000`12871030 : System_Core_ni+0x471daf
000007ff`004133b6 : 00000000`02dc41e8 00000000`0603a228 00000000`033d1008 000007fe`f383ea3c : 0x7ff`00416efa
000007ff`00412f2c : 00000000`02db1118 00000000`0603a228 00000000`2191ec88 000007fe`f3e7063c : 0x7ff`004133b6
000007fe`f36796bb : 00000000`02db1118 00000000`0603a228 00000000`0603a3a8 000007fe`f2f61545 : 0x7ff`00412f2c
000007fe`f2f60d3c : 00000000`0603a3a8 00000000`1c8573d8 00000000`2191ec40 000007fe`f3e6ed1e : mscorlib_ni+0xaa96bb
000007fe`f3679455 : 00000000`12871030 00000000`0603a3a8 00000000`2191ec88 00000000`00000001 : mscorlib_ni+0x390d3c
000007fe`f367d9d0 : 00000000`0603a3a8 00000000`033d1018 00000000`0603a3a8 000007fe`f2fe71a9 : mscorlib_ni+0xaa9455
000007fe`f2f96be8 : 000007fe`f2c4e7f8 00000000`2191f140 000007ff`00022020 00000000`0316bf48 : mscorlib_ni+0xaad9d0
000007fe`f2f96a85 : 48cc31a4`460aef00 00000000`09103290 00000000`00000000 00000004`00000005 : mscorlib_ni+0x3c6be8
000007fe`f3e621b4 : 48cc31a4`460aef03 000007fe`f2f66e32 00000000`09103200 00000001`000001bb : mscorlib_ni+0x3c6a85
000007fe`f3e67129 : 00000000`2191f208 00000000`02db11e0 00000000`21910000 00000000`00000000 : clr!CallDescrWorker+0x84
000007fe`f3e671a3 : 00000000`2191ef48 00000000`00000000 00000000`2191ef50 00000000`00000000 : clr!CallDescrWorkerWithHandler+0xa9
000007fe`f3e6c5b5 : 00000000`2191f6a8 00000000`2191f4c8 00000000`1c8573d0 00000000`00000001 : clr!MethodDesc::CallDescr+0x2f4
000007fe`f403c42f : 00000000`2191f6a8 00000000`2191f658 00000000`2191f4c8 00000000`00000000 : clr!MethodDescCallSite::CallWithValueTypes_RetArgSlot+0x35
000007fe`f3ef6c5a : ffffffff`fffffffe 000007fe`f3e33489 ffffffff`fffffffe 00000000`1c8573d0 : clr!QueueUserWorkItemManagedCallback+0x44
000007fe`f3ef6bef : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : clr!SystemDomain::LoadBaseSystemClasses+0x43e
000007fe`f3ef6b5c : 00000000`00159770 00000000`2191f380 00000000`00000000 00000000`2191f3c8 : clr!SString::Equals+0x190
000007fe`f403c2f9 : ffffffff`ffffffff 00000000`1c8573d0 00000000`00000000 000007fe`f3e333eb : clr!CParseUtils::TrimWhiteSpace+0x119
000007fe`f3f04bf5 : 00000000`00000c0a 00000000`00000000 00000000`1c8573d0 000007fe`f3ecd2d0 : clr!ManagedPerAppDomainTPCount::DispatchWorkItem+0xe6
000007fe`f4004377 : 00000000`00000000 00000000`1dfa6501 000000f2`00f200f2 00000000`00000c0a : clr!ThreadpoolMgr::NewWorkerThreadStart+0x49f
000007fe`f4002c22 : 00000000`00000000 00000000`00000000 00000000`2191f7a0 00000000`00000000 : clr!ThreadpoolMgr::WorkerThreadStart+0x3b
00000000`76e9f56d : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : clr!Thread::intermediateThreadProc+0x7d
00000000`770d3281 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0xd
00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d

As you can see on this output from WinDBG at thread 44 the exception object is (GC) (Threadpool Worker) System.ExecutionEngineException (0000000002871228) and the address at the end is my System.ExecutionEngineException.

In my managed stack my last frame is a System.Text.StringBuilder.ToString() so assuming that I’m creating memory from there. If we check native stack this thread was doing a garbage collection.

The sequence is: FramedAllocateString -> GCHeap::Alloc -> gc_heap::try_allocate_more_space -> GCHeap::GarbageCollectGeneration -> gc_heap::garbage_collect -> :gc_heap::gc1

At this point we truly know that the runtime is trying to allocate memory for that string, but since there is no more memory available the runtime raise a garbage collection and there is where the runtime found that there is a corruption in the manage heap.

There is also another useful command in WinDBG !verifyheap is a diagnostic tool that checks the garbage collected heap for signs of corruption on managed heap.

0:044> !verifyheap
-verify will only produce output if there are errors in the heap
The garbage collector data structures are not in a valid state for traversal.
It is either in the "plan phase," where objects are being moved around, or
we are at the initialization or shutdown of the gc heap. Commands related to
displaying, finding or traversing objects as well as gc heap segments may not
work properly. !dumpheap and !verifyheap may incorrectly complain of heap
consistency errors.
object 0000000006fb6ec8: bad member 0000000008f1b750 at 0000000006fb6ef8
curr_object:      0000000006fb6ec8
Last good object: 0000000006fb6e50
----------------

The !verifyheap command is telling us that in that precise moment were occurring a garbage collection and all the structures are not in a valid state, but at the end of the report there is an object that have a bad member, this object is a System.IO.StreamReader and the bad member is System.Char[] 0000000008f1b750 charBuffer.

0:044> !do 0000000006fb6ec8
<Note: this object has an invalid CLASS field>
Name:        System.IO.StreamReader
MethodTable: 000007fef30951a0
EEClass:     000007fef2cce0d0
Size:        96(0x60) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
000007fef30d9518  40001cd        8        System.Object  0 instance 0000000000000000 __identity
000007fef3091d08  4001fda      c80 System.IO.TextReader  0   shared           static Null
                                 >> Domain:Value  00000000001a85b0:NotInit  <<
000007fef30dd718  4001fde       54       System.Boolean  1 instance                1 _closable
000007fef30e3658  4001fdf       10     System.IO.Stream  0 instance 0000000006fb6e50 stream
000007fef30e3968  4001fe0       18 System.Text.Encoding  0 instance 00000000032d6848 encoding
000007fef30ac110  4001fe1       20  System.Text.Decoder  0 instance 0000000006fb6f28 decoder
000007fef30e0d00  4001fe2       28        System.Byte[]  0 instance 0000000008f1a738 byteBuffer
000007fef30db4a8  4001fe3       30        System.Char[]  0 instance 0000000008f1b750 charBuffer
000007fef30e0d00  4001fe4       38        System.Byte[]  0 instance 000000000287ff88 _preamble
000007fef30dc918  4001fe5       40         System.Int32  1 instance                0 charPos
000007fef30dc918  4001fe6       44         System.Int32  1 instance                0 charLen
000007fef30dc918  4001fe7       48         System.Int32  1 instance                0 byteLen
000007fef30dc918  4001fe8       4c         System.Int32  1 instance                0 bytePos
000007fef30dc918  4001fe9       50         System.Int32  1 instance             4096 _maxCharsPerBuffer
000007fef30dd718  4001fea       55       System.Boolean  1 instance                1 _detectEncoding
000007fef30dd718  4001feb       56       System.Boolean  1 instance                0 _checkPreamble
000007fef30dd718  4001fec       57       System.Boolean  1 instance                0 _isBlocked
000007fef30951a0  4001fdd      c88 ...m.IO.StreamReader  0   shared           static Null
                                 >> Domain:Value  00000000001a85b0:NotInit  <<

What we can do with this problem?

Since the exception is part of the runtime there is now way to fix it, so the only option is call Microsoft support service to tell that you have a problem with memory corruptions on .net framework.

Happy debugging. Luis.

2,117 Comments more...

Hello World

by on Nov.16, 2009, under General

Hi everyone! This is my first post, actually is not *really* my first post, because I have another blog but it’s in Spanish.

What is this blog about?

I’m a .NET developer, currently I work in Plain Concepts, a Spanish company but we have offices in London and there is where I work. As part of my job I develop solutions in .net technologies, deliver some training and consulter.

I’ll write about stuff like .NET 4, User Experience (UX), Windows Presentation Foundation (WPF), Silverlight, Microsoft Surface, Task Parallel Library, Advanced Debugging, profiling and so on, so keep tuner.

Luis.

http://www.luisguerrero.net (Spanish version)

http://geeks.ms/blogs/luisguerrero (Spanish version)

http://www.plainconcepts.com/ (Plain Concepts)

http://ux.plainconcepts.com/ (UX Showcase)

838 Comments more...

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!