Intro to Reverse Engineering

Reverse Engineering

• Figuring out what a binary does
• Usually don't have access to source code
• Can (almost) always get disassembly

Why?

• Bypass copyright protection
  • Hacking the XBox
• Accessing undocumented OS APIs
  • http://undoc.airesoft.co.uk/
• Malware analysis
• Exploit development
• Jailbreaking/Figuring out how jailbreaks work
  • http://geohot.com/e7writeup.html

Whirlwind tour of x86 Assembly

CPU Architecture

Graphic from https://github.com/isislab/Hack-Night/blob/master/2014-Fall/slides/sotirov-re-fall2011.pdf

Moving Data

mov eax, ebx
mov byte ptr [var], 5

push eax
push [var]

pop edi
pop [ebx]

lea edi, [ebx+4*esi]
lea eax, [var]

mov

push

pop

lea

Arithmetic/Logic

add eax, 10
add BYTE PTR [var], 10

add

sub al, ah
sub eax, 216

sub

dec eax
inc DWORD PTR [var]

inc/dec

imul eax, [var]
imul esi, edi, 25

imul

idiv ebx
idiv DWORD PTR [var]

idiv

and eax, 0fH
xor edx, edx

and/or/xor

shl eax, 1
shr ebx, cl

shl/shr

not BYTE PTR [var]
neg eax

not/neg

Addressing Memory

mov eax, [ebx]

mov [var], ebx

mov eax, [esi-4]

mov edx, [esi+4*ebx]

Move 4 bytes in memory at address in ebx into eax

Move contents of ebx into the 4 bytes at memory address var

Move 4 bytes at memory address ESI-4 into eax

Move 4 bytes at memory address ESI+4*ebx into eax

Control Flow

jmp begin

cmp eax, ebx
jle done

cmp DWORD PTR [var], 10
jeq loop

call <label>
ret

jmp

je/jne/jz/jg/jge/jl/jle

cmp

call/ret

Caller Rules

• Save eax, ecx, edx
• Push parameters onto the stack, last parameter first
• Call the subroutine

Callee Rules

• Before executing the function body
  • Push ebp onto the stack, copy esp into ebp
  • Allocate local variables by decrementing esp appropriately
  • Save ebx, edi, esi
• After executing the function body
  • Save the return value in eax
  • Restore ebx, edi, esi
  • Deallocate local variables
  • Restore ebp by popping it off the stack
  • Return with ret

CMU Bomb Demo