12 October 2017
Runtime memory patching on x86/x64
In this blog post we will be discussing different ways of modifying code that is being executed by changing the underlying x86/x64 instructions. This will not only allow us to improve iteration times but also increase efficiency while debugging.
These techniques will be demonstrated using C++ on Windows under Visual Studio but should be applicable on any x86/x64 environment. We assume you are familiar with C or C++ but everything else beyond that will be explained.
Imagine you are debugging a game and a C assert is firing every frame. This assert is not even relevant to the problem you are trying to debug! Annoying, right? It is also time consuming since the natural solution is to stop, comment out the assert, and restart the application – unless you use a cool hot reloading system like any of these. But let’s assume you are not *that* cool. One thing you could do is nuke the assert from orbit by patching it to a no operation in memory and resume execution:
Notice how by changing the register’s value, the opcode also changed in the memory window. So now we have a convenient way to NOP out instructions; just keep that register in the watch window at all times and you’re sorted!
You can use this knowledge for other purposes such as changing if statements or breaking out of loops on the fly. For example:
Understanding the sequence of events above is left as an exercise for the reader but have you noticed that when we stepped over the __debugbreak it didn’t fire? This is because the debugger had already been triggered at that instruction while stepping over so it can’t be triggered again. You can read more about it here.
An assert not triggering when it should is certainly not expected so one way to fix this is by replacing it with the __ud2 intrinsic, which generates an undefined instruction. __builtin_trap generates the same instruction for Clang / LLVM. This is slightly more cumbersome because the debugger can’t continue stepping through code after hitting that instruction. This limitation can be bypassed by dragging the yellow arrow to the next line when using Visual Studio, or by manually setting the RIP pointer as shown above.
Follow Nuno on Twitter @nunopleiria