Buffer Overflow Basics


When you take a course on security, probably one of the more fascinating topics is getting a shell on a remote system.  In most cases the user who pops a shell on a remote system takes on the access levels of the application that was violated.

Buffer Overflow

The concept behind a Buffer Overflow attack is that there’s an application running on a system.  This application has a vulnerability where memory is not being managed correctly.  For example, perhaps the application is an FTP server and it accepts input for username.  The application has perhaps set aside 250 characters for a username.  What happens if you pass 251?

If the program isn’t handling this memory condition (passing in more data than it expects), it will start writing the next byte of data outside allocated memory.  This creates a Buffer Overflow condition and will crash the application.


That’s not the worst of it though – once memory is written outside the allocated sectors, a clever individual could pass in their own code and divert the flow of the application to execute it.  Code that is written like this must be specific to the CPU/OS… this is called shellcode.

Shellcode is written in Machine Code.  The fantastic security suite Metasploit will generate a lot of shellcode for different systems.  If you know the target, you can craft shell code that can instruct the system to open and serve a shell to the attacker (giving them access to the remote machine.)



A common method of discovering Buffer Overflows is to ‘fuzz’ inputs.  Fuzzing refers to the process of sending strings of data of varying size to all the inputs.  To be efficient, this would be a scripted event.  The script might make a connection to your locally installed software that you’re testing, and try and send 1 character, then 2, then 4, then 16, etc.

If there’s a size limitation vulnerability, then at some point the script will kill the software after a threshold his reached.

Exploit Creation

This is the tricky part.  An attacker will want to occupy the EIP register (on a 32 bit machine) or RIP (on 64 bit machines.)  This register (EIP/RIP) is a pointer to the next instruction.  If you can control it, you can divert the flow of the application to a new spot in memory.

How it is controlled, is in the overflow itself.  As data leaks out of the memory containment and it begins writing to the top of the stack, it writes over the EIP/RIP.  You can see this if you are debugging an application (using Olyydb, gdb, etc.)  You’ll see that the data you sent in the test writes into this memory register and then the application crashes.  So if you were doing the typical attack flow and sending “AAAAAAA…” you would see “AAAA” occupy this register.

When an attacker knows the exact point, that the Buffer Overflow occurs (i.e. sending 251 characters into the input field), then a script can be crafted.  The exploit script would send the amount of data to create the vulnerability, after that amount, the script then passes in a dummy value to test the theory… perhaps “BBBB” which a debugger will display in HEX byte code.

By checking the debugger on each attempt and modifying the script as needed, it allows the attacker to know the exact bytes needed to occupy the EIP/RIP register.

Next the script will pass in some shellcode, which in the debugger it would appear to occupy the ESP (or RSP) registers once the overflow occurs.

Once EIP/RIP is owned and the attacker script is modified to push shellcode into the ESP register (RSP on 64 bit machines)… what’s required is to connect the two points together.  The EIP/RIP will point to the next instruction and if that instruction is a Memory Pointer that will Jump to the ESP/RSP.

The details of all this is beyond the scope of this overview… Suffice to say, tools are utilized to scan for the a pointer that has a valid instruction to Jump to the ESP/RSP register (where the exploit is located.)  Once finalized, the script would:

  • Fuzz till a overflow condition is created
  • pass extra data into the ESP / RSP register
  • Send 4 bytes of data representing the register that has an instruction to Jump to ESP/RSP
  • Then the shellcode is executed in memory


Many such exploits are logged with exploit database sites and / or made part of the Metasploit attack vectors.  Metasploit is a utility that is used in security audits (or by blackhats) – it has a plethora of attack vectors at its disposal.  Such Buffer Overflows as described above are also part of its arsenal.

Metasploit is a searchable utility that could simply be queried for a software package (like an SSH utility), and if an exploit for a version exists, it will load it up. The attacker can load a target type (windows 8, Windows 7, Solaris, Linux, etc.) that will be used for the shellcode generation.

In such a case Metasploit does most of the work… and yeah, it’s kinda scary.

Leave your vote

0 points
Upvote Downvote

Total votes: 0

Upvotes: 0

Upvotes percentage: 0.000000%

Downvotes: 0

Downvotes percentage: 0.000000%

About Admin 326 Articles
I work for a Telecom company writing and testing software. My passion for writing code is expressed through this blog. It's my hope that it gives hope to any and all who are self-taught.