Reverse Engineering Detour

While browsing gamedeception.net out of curiosity, as I often do, I came across various new threads. This led me to think, "Why not create a hook from scratch?". So, I spent hours researching detouring techniques.

A few basics about my project: The goal is to develop a DLL containing function prototypes that will redirect corresponding functions in the target software (Detour). This means that instead of the software executing its function, it first jumps to my custom function from the DLL, allowing me to modify the software's behavior.

To start, create a new project in Visual Studio (preferably a newer version). Select a Win32 project under "cpp", and in the subsequent window, choose DLL and "empty project". This provides a solid foundation.

Since we've opted out of the auto-generated code, we'll need to write some code ourselves. I'm assuming that readers have basic programming knowledge, so I won't delve into the obvious details:

#include <Windows.h>
#include <stdio.h>

BOOL WINAPI DllMain(HINSTANCE hDLL, DWORD dwReason, LPVOID lpReserved) {
    switch(dwReason) {
        case DLL_PROCESS_ATTACH:
            // here we can do something when the DLL is successfully
            // attached to a process

            char strText[50];
            sprintf(strText, "I am now loaded at 0x%x :3", hDLL);
            MessageBoxA(NULL, strText, "Pups!", NULL);

            break;
        case DLL_PROCESS_DETACH:
            // here is the exit door - don't forget to clean up ;3

            break;
    }
    
    return TRUE;
}

Great! With this setup, our DLL will display a notification when attached to a process. For attaching, you'll need a DLL-Injection tool. I've found WinJect-1.7-2009-05-02.rar to be quite reliable.

While this already modifies the original process, our goal is to achieve even more!

To proceed further, we need a foundation for detouring. While we could write our own function to manipulate bytes, a more efficient approach is to use Microsoft's "Detours Express 3.0". Simply install it.

Next, open the "Visual Studio Command Prompt" and navigate to the installation directory of Detours Express 3.0. Execute the "nmake all" command. This will compile Detours for us, making it ready for integration into our project.

To integrate, right-click on your project in Visual Studio and select Properties. Navigate to "VC++ Directories". Here, specify the Detours' Include folder for the Include Directories and the libX86 folder for the Library Directories. Note: Ensure your project is set to x86 mode (which is the default).

Now, you can append the following to the Include section from the initial code snippet:

#include <detours.h>

#pragma comment(lib, "detours.lib")

Next, download a reverse engineering tool. I recommend IDA Pro Free. Install it.

Once installed, run it as an administrator and open the EXE file you wish to modify. For demonstration purposes, I've created a simple test application containing a function named int meineFunktion(int x). The loading time may vary based on the size of the binaries. Once loaded, click on the "Functions" tab at the top.

By double-clicking on the desired function, IDA will navigate to the corresponding section in the Assembly view. Conveniently, this allows us to identify the Calling Convention used by the function. This information is essential when setting up the hook.

On the left, you'll notice the starting address of the function is "0x004011A0". Typically, IDA uses "0x00400000" as the base address, which means our function offset is "0x000011A0" (or 0x11A0). Now, let's modify our DLL to dynamically retrieve the base address of the process (since it could be dynamic) and calculate the memory location of our function using the function offset:

#include <Windows.h>
#include <stdio.h>

#include <detours.h>

#pragma comment(lib, "detours.lib")

// address declarations
DWORD m_dwBaseAddress;
DWORD m_dwMeineFunktion = 0x11A0; // function address offset from IDA

// function pointer to the original function
int (__cdecl *o_meineFunktion)(int) = NULL;

// forward declaration of our hook function
int __cdecl hk_meineFunktion(int x);


// entry point
BOOL WINAPI DllMain(HINSTANCE hDLL, DWORD dwReason, LPVOID lpReserved) {
    HANDLE hThread = GetCurrentThread();

    switch(dwReason) {
        case DLL_PROCESS_ATTACH:
            // retrieve the base address of the process and assign the calculated address
            m_dwBaseAddress = (DWORD)GetModuleHandleA("testprozess.exe");
            o_meineFunktion = (int (__cdecl *)(int)) (m_dwBaseAddress + m_dwMeineFunktion);

            // here we redirect :)
            DetourTransactionBegin();
            DetourUpdateThread(hThread);
            DetourAttach(&(PVOID&)o_meineFunktion, hk_meineFunktion);
            DetourTransactionCommit();

            break;
        case DLL_PROCESS_DETACH:
            // clean up when the DLL is unloaded
            DetourTransactionBegin();
            DetourUpdateThread(hThread);
            DetourDetach(&(PVOID&)o_meineFunktion, hk_meineFunktion);
            DetourTransactionCommit();

            break;
    }
    
    return TRUE;
}

// our hook function
int __cdecl hk_meineFunktion(int x) {
    MessageBoxA(NULL, "I have been called :DD", "Pups!", NULL);

    // call the original function to not disturb the program flow
    return o_meineFunktion(x);
}

And there you have it! We've successfully interfaced with an external process. This technique opens up a myriad of possibilities, from software debugging, enhancing game features, to various other applications in the realm of software development. I might delve deeper into this topic in future posts. Stay tuned!