Reverse Engineering

Reverse engineering (RE) is the systematic approach to determine the inner working of an application by inspecting the application’s logic flow and responses to specific inputs. One will often encounter CTF challenges supplying a sample application (such as malware) that must be reverse engineered in order to uncover hidden keys or serial numbers within the applications or in order to unlock hidden execution paths or bypass expected execution paths.

In order to achieve this you must first understand the control flow of the application and how one can analyse a compiled application.

Common RE Activities

Fuzzing

Test the input response of the application. Test how the application responds when presented with unexpected inputs (submitting an Integer in a String field or visa versa). Determine what controls are in place to prevent unexpected user behaviour. Map out these controls to start identifying control flows within the application. Fuzzing could be a viable technique to identify insecure input validation and possible injection paths.

Decompilation

Another common activity during the reverse engineering process is to decompile the source code of an application. Decompiling an application attempts to convert the binary code into the higher order language it was written in. Higher order languages tend to be more human readable but depending on how the source code was obfuscated, the decompiled code might still be difficult to decipher. It is recommended you step through the code blocks to determine the logical flow of the program. This might be combined with the control flows identified during the previous step.

Decompilers:

For Android applications, common decompiling software include: APKTool, Dex2Jar and Java-Decompiler.

If a program cannot be decompiled a Hex Editor can be used to inspect the compiled code. The “strings” command can also be used on a compiled application to extract all strings found within the application, in some cases this might lead to some information regarding error messages and code control checks.

Disassemble

A disassembler tool breaks down a compiled program into machine code.

Disassemblers:

Debugging

A debugging application allows you to perform dynamic analysis and set break points to step through the execution of a program. Another option would be to perform active analysis and step through the code with a debugger like OllyDbg.

Additional Resources