How to reverse

"Reverse engineering is the process of analyzing a subject system to create representations of the system at a higher level of abstraction"

Most people, like I did when I first started, view reversing as tracing the exact flow of a program. Nothing could be further from the truth. You are not a computer, and you have a completely different skill set from a computer, so why are you trying to act like one? Sometimes you'll get lucky and the binaries you are reversing haven't been stripped. 90% of reversing low level stuff is coming up with accurate function names. From a function name, a human can deduce what a function does very quickly.

When reversing many things, you are given one huge advantage. Most programs are written to interact, or at least be debugged by, humans. There are strings left all over many programs. Programs are usually huge, and trying to figure out what every function does is not only impossible, but not what you want. Instead narrow it down by seeing what prints a certain string, and making educated guesses at the surrounding functions. Get in the head of the designer; think about what (s)he was thinking about when writing this code.

Read up on exploit types and why they work. Have a working knowledge of the assembly you are reversing, but keep in mind you don't have to write it. Understand how the stack behaves in your system and how parameters are passed to functions (ARM is registers R0-R3+stack return R0). Learn everything you can about the general system design before you dive into the specifics of what functions do.

Also, enjoy it. It's really cool when things come together, and you finally figure out that one function you had no idea about. Or when you apply a patch that works. Or when you see what could be an exploit, try it, and realize you are running unsigned code. Work at it, and you will become good. I had never opened IDA before 29 June 2007.

~geohot