This article looks at how .NET debugging API works. In theory this API is simple and a joy to use, but there is a lack of a detailed overview as to how to use it in the framework SDK and you have to get some information from looking through the sources to the example debugger that is supplied. I've decided to set down how I believe the debugging services work in order for other people to avoid having to delve through source code as much, and also so that people can point out if I have anything wrong or that I have missed anything.
We'll cover how to write a debugger under .NET, and will also touch on the profiling API support since the two are very closely related.
How the .NET debugger is structured
The CLR debugging services are implemented as a set of COM objects that give applications an opportunity to view and modify the state of a running application. It's a vast improvement over the debugging services on most platforms, as you will see when I start to touch on adding native debugging support, but it does have a few minor niggles. The biggest is the quality of the documentation, which tends to be slightly wrong or incomplete in places and doesn't really give you a clear picture of what you need to do in order to write a debugger. The actual documentation is split over three documents and an example application. All of this information exists in the “Tools Developer Guide” directory in the framework SDK. This is located in the SDK directory under your installation of Visual Studio.NET, or under the installation of the Framework SDK if you have just installed that. The three documents are the files “Debug.doc”, which is the API reference. “DebugRef.doc”, which is an overview of debuggers under the CLR, and “Profiling.doc”, which details the profiling API. In the samples directory is the project “debugger” which has the source for the command line debugger that comes with .NET called CorDbg.
Of course debuggers aren't the only things that you can write with this API. Just as with the profiling API, which is closely related, there is plenty of other information that you can retrieve.
ICorDebug
A debugger starts and ends with a COM object that implements the interface ICorDebug. This is created with a standard call to CoCreateInstance. We must of course initialize COM before we attempt to call CoCreateInstance. Although calling CoInitialize(NULL) seems to work, we call CoInitializeEx because that's Microsoft do in their example code and so I assume that we need the extra support although I haven't seen any documentation to this effect. Note you need to include the line “ #define _WIN32_DCOM ” before you include your header files in order to use this function.
CoInitializeEx(NULL, COINIT_MULTITHREADED);
HRESULT hr = CoCreateInstance(CLSID_CorDebug, NULL, CLSCTX_INPROC_SERVER, IID_ICorDebug, reinterpret_cast<void**>(&debug));
if(FAILED(hr))
{
return hr;
}
At this point it's probably worth mentioning that this is only how you get this object under the real CLR. Rotor also supports the debugging API, and its version is obtained through LoadLibrary on Win32, and the equivalent function on other platforms. It would probably be possible to create a debugger that could support Rotor or the CLR dynamically, but I'll leave that fact as an exercise for the reader to disprove. Being able to do that with Mono, on the other hand, would be useful but unfortunately the Mono developers decided against implementing the debugging API and have their own debugging mechanism. This is a shame because it makes tools such as #Develop a lot more effort to port.
Getting back to the code we now have a valid ICorDebug object, and the first thing we should do to this is call Initialize() on it. For anybody who isn't familiar with COM this is a standard pattern for COM code and is used as a constructor would be in other languages.
The ICorDebug interface gives us everything that we need to control starting and stopping the debuggee, and also provides a way for us to be notified when debugger events occur in the code. There are two important functions, ICorDebug::SetManagedHandler() and ICorDebug::SetUnmanagedHandler() that we are interested in. These both take an interface that provides methods that the CLR will call notifying us of events of interest. The unmanaged handler takes a pointer to an ICorDebugUnmanagedCallback interface, which contains a single callback function that provides a single member with an interface that will be familiar to anybody that has looked at Win32 debuggers before. We are not interested in unmanaged (native) debugging yet and so we can ignore this member for now. Setting the managed handler requires us to pass a pointer to an ICorDbgManagedCallback interface to SetManagedHandler and is much more interesting for us at the moment. This interface provides many member functions that we must implement so that we can receive notifications. In order to use these interfaces we just have to derive from them and provide a default implementation that has some basic code. I have written a very basic class that implements each of these callback member functions.
Each callback on the ICorDbgManagedCallback interface requires us to call Continue on the process, and pass FALSE as the parameter to this call. In order to save a little effort I have used a macro to implement a default implementation of each of these callbacks in my generic debugger class. This macro first of all uses VC++s __FUNCTION__ macro to output the function name to the console so that we can see what as been called, and then it calls process->Continue(FALSE) on our stored process object so that the debugee can continue. Unlike this basic implementation not all debuggers will want to call Continue after every notification because some events such as breakpoint notifications are meant to pause the debuggee. We'll rectify that as we start to add more interesting features.
We are now ready to write our first debugger and so we kick off a new console mode native C++ application and add the debugger class, a few includes for the CLR headers and then we can derive a new class from Debugger, which we call TestDebugger, and then finally in main we will instantiate a TestDebugger object, and call the StartDebugging function on it.
When you run this application you should notice that two of our callbacks should be called, and then the test application should wait for some input. First we were informed that Debugger::CreateProcessA has been called, and the Debugger::CreateAppDomain. Don't worry about the fact that CreateProcess is actually being called CreateProcessA, it's just the fact that the Win32 SDK uses a great many #defines to support Win9x and WinNT (or more specifically Unicode). If you now type some text into our debuggee and press enter then it will inform the world that it's not being debugged, and then two more notifications will trigger in our debugger. These are ExitAppDomain and ExitProcess.
So, according to our debugee we're not actually debugging it, and we were only notified about very high level events. The reason for this is that we are running the application under the debugger, but we actually need to attach to AppDomains before we can debug them. This is simple to do, just override CreateAppDomain in TestDebugger and add the following implementation:
HRESULT TestDebugger::CreateAppDomain(
/* [in] */ ICorDebugProcess *pProcess,
/* [in] */ ICorDebugAppDomain *pAppDomain)
{
std::cout << __FUNCTION__ << std::endl;
pAppDomain->Attach();
pProcess->Continue(FALSE);
return S_OK;
}
A real application would want to add a bit more error handling, but now if you run you will see that you have notifications about two assemblies and two modules being loaded, as well as a thread being created. The last event before the debugee waits for some keyboard input is an event called NameChange, but that is of little interest to us at this time so enter some text into the app to make it exit and trigger the rest of the notifications. You'll see that another thread gets created, and then all our modules and assemblies will unload, the AppDomain will exit, followed by a thread and then the process. Once the process has exited we have stopped debugging.
So there you have it, a simple debugger. A useless debugger too because it doesn't tell you anything except for the number of AppDomains, Modules and Assemblies loaded and that you get more notifications for thread creation than you do for thread exits.
Making the debugger useful
The easiest piece of debugging information that we can see is probably debug messages (System.Diagnostics.Debug.Write*). Of course under .NET there are other ways that we can view these messages without attaching a debugger, but for now we're going to add the line System.Diagnostics. Debug .WriteLine("Hello"); to our test application and run the debugger again. You should notice that we received a new callback, Debugger::LogMessage, and so to handle the message we just need to add the following code to our TestDebugger class:
HRESULT TestDebugger::LogMessage(
/* [in] */ ICorDebugAppDomain *pAppDomain,
/* [in] */ ICorDebugThread *pThread,
/* [in] */ LONG lLevel,
/* [in] */ WCHAR *pLogSwitchName,
/* [in] */ WCHAR *pMessage)
{
std::wcout << L"Debug Message: " << pMessage << std::endl;
GetProcess()->Continue(FALSE);
return S_OK;
}
This implementation doesn't pay any attention to log switches or levels and I'll cover them in a later instalment. If you run the debugger now you'll see that we have the beginnings of a simple version of the simple debug output view like you will find in your favourite IDE. Check out "Part 1" in the download for the full source code so far.
Comments