Interception of Win32 and user DLL API calls using AutoIt and Deviare

AutoIt is a great way to automate 3rd party applications or ones for which the source was lost. AutoIt is great in sending signals to input methods of an application, but in order to know when so send these signals, it needs information about what the application is doing. In many instances the application does not provide a good signal to work with. There ,it is desirable to get very fine grain information and an exellent source of this information are the calls to Win32 and user DLL APIs. Deviare is a facility do to just that. The project itself describes it as: http://www.nektra.com/products/deviare-api-hook-windows/

"hooking engine for instrumenting arbitrary Win32 functions, COM objects, and functions which symbols are located in program databases (PDBs). It can intercept unmanaged code in 32-bit and 64-bit applications"

It is provided as a COM interface, so any application that implements COM can use it. AutoIt does provide a COM interface. However there is little documentation on how to do use it specifically with Deviare. Some questions exist on the AutoIt and Deviare forums, and some code has been posted, but it does not work. And the Autoit Documentation on COM is very nice, but there is not very much of it.

Here are some relevant pieces of information:

https://www.autoitscript.com/autoit3/docs/intro/ComRef.htm

https://www.autoitscript.com/forum/topic/182664-autoit-with-deviare-engine/

http://forum.nektra.com/forum/viewtopic.php?f=9&t=6293

http://forum.nektra.com/forum/viewtopic.php?f=9&t=4866

http://www.nektra.com/products/deviare-api-hook-windows/doc/interface_deviare_1_1_d_hooks_events.html

http://www.nektra.com/products/deviare-api-hook-windows/doc-v2/interface_i_nkt_params_enum.html

So lets make a working example:

The version Windows OS should not really matter here, but I found that the API hooking functioned fine on Win8 x64 and Win7 x64/x32 systems but would not function on two tested WinXP x32 systems...

First, install AutoIt. There are two parts to this. First there is AutoIt proper and then an editor that is specialized for it, SciTE. At the time of this writing, AutoIt is at version v3.3.14.2 and SciTE is at version 3.7.3

So we wind up with a blank slate:

Next, we need to install Deviare. It is open source and available from Github, and precompiled packages are available in the from of a zip file here:

https://github.com/nektra/Deviare2/releases

Here we will use version 2.8.2

In the archive is a bin\ directory with 2 DLLs, DeviareCOM.dll and DeviareCOM64.dll. To install Deviare, these need to be registered. Open a command prompt with administrative rights and run the following command:

regsvr32 DeviareCOM.dll

and if you are on a 64 bit system:

regsvr32 DeviareCOM64.dll

We should now be able to talk to Deviare via COM.

To create variable to access a COM object we will use the ObjCreate function. Deviare registered itself with the COM system as DeviareCOM.NktSpyMgr, so:

$comObject = ObjCreate("DeviareCOM.NktSpyMgr")   ; Create an COM object to access the Deviare interface

Then, to initialize the object

$comObject.Initialize

comObject can pass various types of events on to our script. For example error events in case something does not work right. We won't install an error handler at this point, but we will install a handler than can process regular events coming from Deviare.

$eventObject=ObjEvent($comObject,"Deviare2_","DNktSpyMgrEvents")  ; events from comObject are now passed to eventObject

Above, we are simply stating to pass all events of type DNktSpyMgrEvents to eventObject. The way this is done is by calling functions. So, for example, DNktSpyMgrEvents defines a function called OnCustomDllLoad when certain events occur. That means if we write a function in our script called Deviare2_OnCustomDllLoad, it will now be called by Deviare. The Deviare2_ is an arbitrary prefix. We could also use blahblah_. Its purpose is to avoid naming conflict. If we have several COM objects, each exposing a OnCustomDllLoad function, we need a way to differentiate the functions from one another. Hence the prefix.

The process exits right after this initialization, so to avoid that we run an infinite loop:

While 1

    Sleep(100)

WEnd

From now on the script is strictly event driven and nothing will happen unless Deviare calls one of the functions we have written for it. It will execute until we press Ctrl+Break or go to Tools->Stop Executing in SciTE.

We are now in a position to use Deviare's functionaliy. If we define an Deviare2_OnProcessStarted function, then it will get called every time a new windows process starts, for example.

Volatile Func Deviare2_OnProcessStarted($process)

EndFunc

Lets test that functionality. So far, our code is this:

$comObject = ObjCreate("DeviareCOM.NktSpyMgr")   ; Create an COM object to access the Deviare interface

$comObject.Initialize

$eventObject=ObjEvent($comObject,"Deviare2_","DNktSpyMgrEvents")  ; events from comObject are now passed to eventObject

While 1

    Sleep(100)

WEnd

Volatile Func Deviare2_OnProcessStarted($process)

; for reference on the proc object: http://www.nektra.com/products/deviare-api-hook-windows/doc-v2/interface_i_nkt_process.html

EndFunc

We will now add some code to pop up a message box every time notepad.exe is started.

#include <MsgBoxConstants.au3>

$comObject = ObjCreate("DeviareCOM.NktSpyMgr")   ; Create an COM object to access the Deviare interface

$comObject.Initialize

$eventObject=ObjEvent($comObject,"Deviare2_","DNktSpyMgrEvents")  ; events from comObject are now passed to eventObject

While 1

    Sleep(100)

WEnd

Volatile Func Deviare2_OnProcessStarted($process)

; for reference on the proc object: http://www.nektra.com/products/deviare-api-hook-windows/doc-v2/interface_i_nkt_process.html

  if ($process.Name == "notepad.exe") then

    MsgBox($MB_SYSTEMMODAL, $process.name, "notepad started", 10)

  EndIf

EndFunc

This also allows us to hook into any API calls notepad performs. For example,

Volatile Func Deviare2_OnProcessStarted($process)

; for reference on the proc object: http://www.nektra.com/products/deviare-api-hook-windows/doc-v2/interface_i_nkt_process.html

  if ($process.Name == "notepad.exe") then

    $hook = $comObject.CreateHook("kernel32.dll!CreateFileW", 0)

    $hook.Attach($process, True)

    $hook.Hook()

  EndIf

EndFunc

This will trigger various other functions, similar to OnProcessStarted. All of them will be ignored unless we define them in our script. The most important one is OnFunctionCalled, which now gets called every time notepad.exe performs a CreateFile call.

We can then proceed to create callback functions for all the remaining methods exposed by Deviare. Our code now reads:

#include <MsgBoxConstants.au3>

$comObject = ObjCreate("DeviareCOM.NktSpyMgr")   ; Create an COM object to access the Deviare interface

$comObject.Initialize

$eventObject=ObjEvent($comObject,"Deviare2_","DNktSpyMgrEvents")  ; events from comObject are now passed to eventObject

While 1

    Sleep(100)

WEnd

Volatile Func Deviare2_OnProcessStarted($process)

; for reference on the proc object: http://www.nektra.com/products/deviare-api-hook-windows/doc-v2/interface_i_nkt_process.html

  if ($process.Name == "notepad.exe") then

    $hook = $comObject.CreateHook("kernel32.dll!CreateFileW", 0)

    $hook.Attach($process, True)

    $hook.Hook()

    MsgBox($MB_SYSTEMMODAL, $process.name, "Target Process started and hooked", 10)

  EndIf

EndFunc

Volatile Func Deviare2_OnFunctionCalled($hook, $process, $callInfo)

EndFunc

Volatile Func Deviare2_OnCreateProcessCall($process, $pid, $mainThreadId, $is64BitProcess, $canHookNow)

EndFunc

Volatile Func Deviare2_OnCustomDllLoad( $process, $actionId, $actionResult)

EndFunc

Volatile Func Deviare2_OnCustomDllUnLoad( $process, $actionId, $actionResult)

EndFunc

Volatile Func Deviare2_OnCustomApiCall( $process, $actionId, $actionResult, $apiResult)

EndFunc

Volatile Func Deviare2_OnHookStateChanged( $Hook, $process, $actionId, $newState, $oldState)

EndFunc

Volatile Func Deviare2_OnLoadLibraryCall(  $process, $dllName, $moduleHandle)

EndFunc

Volatile Func Deviare2_OnFreeLibraryCall(  $process, $moduleHandle)

EndFunc

Volatile Func Deviare2_OnProcessTerminated(  $process)

EndFunc

Volatile Func Deviare2_OnAgentLoad(  $process, $errorCode)

EndFunc

Volatile Func Deviare2_OnAgentUnload(  $process)

EndFunc

Volatile Func Deviare2_OnCustomMessage(  $process, $msgCode, $msgParam, $retVal)

EndFunc

Volatile Func Deviare2_OnHookOverwritten(  $Hook, $process)

EndFunc

Next, let us extract a little more info about the CreateFile call when it happens. We are passed this information by OnFunctionCalled via the $callInfo parameter. The parameter is described here:

http://www.nektra.com/products/deviare-api-hook-windows/doc/interface_deviare_params_1_1_i_call_info-members.html

So $callInfo is a structure. It contains a member called Params which has the information we are after, namely the name of the function called and the parameters passed to it.

Params in turn has a member, Count, that states how many parameters are stored in it.

And these parameters, in turn, have a member for their name as well as their value.

To undo this russian nesting doll, we do the following:

Volatile Func Deviare2_OnFunctionCalled($hook, $process, $callInfo)

   $Params = $callInfo.Params

   $Count = $Params.Count

   $First = $Params.First

   $Second = $Params.Next

   $Third = $Params.Next

   $TitleString = "Number of parameters: " & $Count

   $MessageString = "Parameter: " & $First.name & " Value: " & $First.value & @LF

   $MessageString = $MessageString & "Parameter: " & $Second.name & " Value: " & $Second.value & @LF

   $MessageString = $MessageString & "Parameter: " & $Third.name & " Value: " & $Third.value

   MsgBox($MB_SYSTEMMODAL, $TitleString, $MessageString, 10)

EndFunc

This yields the following information:

The CreateFile call documentation has the following prototype:

https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx

HANDLE WINAPI CreateFile(   

                         _In_     LPCTSTR               lpFileName,   

                         _In_     DWORD                 dwDesiredAccess,   

                         _In_     DWORD                 dwShareMode,   

                         _In_opt_ LPSECURITY_ATTRIBUTES lpSecurityAttributes,   

                         _In_     DWORD                 dwCreationDisposition,   

                         _In_     DWORD                 dwFlagsAndAttributes,   

                         _In_opt_ HANDLE                hTemplateFile );

And that matches what we got.