1

What is Buffer Overflows? Security Risks, Preventive Measures and Exploitation

Abstract Programs written in C are innately unprotected against to buffer overflows. Buffer overflow is the result of writing more data into a buffer than the buffer can hold. In C parameters are passed to functions barely without any concealing up their sizes. Despite numerous security technologies manufactured to oppose to buffer overflow defenselesless, buffer overflows going on to be the controlling form of software security vulnerability.According to internet break in statistics 35% of them are due to buffer overflow exploits. In this paper buffer overflows detailed properties and how it can be will be happened explained. Buffer overflows caused security weaknesses, this paper will touch on security risks occurred by this vulnerability. An exploit is a program, usually written in C, that exploits some problem that another program has. The exploit will allow you to run arbitrary code that will let you do something that you shouldn't be able to do in your normal status on the system. Buffer overflows can be used while writing exploit. Against exploits,for closing security opens and preventing attacks some techniques are used. 1.1 What is Buffer Overflows? A buffer is a contiguous allocated piece of memory, such as an array or a pointer in C. In C and C++, there are no automatic bounds checking on the buffer, which means a user can write past a buffer. For example: int main () { int buffer[10]; buffer[20] = 10; } The above C program is a valid program, and every compiler can compile it without any errors. However, the program attempts to write beyond the allocated memory for the buffer, which might result in unexpected behavior. This example shows that C has not effective array structure. Programs written in C are innately unprotected against to buffer overflows. Buffer overflow is the result of writing more data into a buffer than the buffer can hold. In C parameters are passed to functions barely without any concealing up their sizes. void main() { char buyuk_array[256]; memset(buyuk_array, 'A', 255); fonksiyon(buyuk_array); } In this program array capacity is 16 byte and we try to write it 255 byte. Therefore part of memory is filled by A in the program. If we compile this program, we take segmentation

2 fault error.This faults reason is programs using of memory which is not allocated for it. So when program address which will be run can be put the return address, by this it can be start to run. Using functions which use copy functions or another ways, return address can be changed using buffer overflows.Below there is an example which return address is changed. void fonksiyon(int a, int b, int c) { char foo[6]; int *ret; ret = foo + 12; (*ret) += 8; } void main() { int x; x = 0; fonksiyon(1, 2, 3); x = 1; printf("%d\n", x); } When these codes are run, x=0 will be written instead of x=0. In the function, ret is pointer to integer, its adddress is defined as foos address+12. When to esp-8 added 12, it shows that return address. With doing (*ret) += 8 means that increasing 8 return adddress. In this way we can change the flow of execution program. This example shows that return adddress can be changed and the program which is wanted can be run. Also memory has shell spawn instructions. With change of return address to these instruction arrays start address keep to shell. Shells advantage is commands can be run in the shell prompt what you want. Buffer overflows can be happened buffers allocated anywhere in memory. These can be stack,heap or static data area. Necessary state of these areas is adjacent to overflowable buffer. Figure 3 shows a simple program cause a static buffer overflow. Statbuf will corrupt the statptr pointer. Figure 4 shows a similar program subject to heap overflows. In addition to threatening the adjacent buffer, this overflow also has the potential to corrupt the malloc data structures. As a result in adjacent memort space buffer overflow risk is high.

2. Security Risks based on Buffer Overflows Programs written in C are innately unprotected to buffer overflow attacks. For array operation pointer manipulation is needed,but this has a crude structure. Since C has no firstclass array type. For isntance, functions are passed the pointers as array parameters. To guarantee that buffers are not overflowed, it is the programmers responsibility to clearly bounds check the buffers and programmer must be careful while using functions. In practice, arrays are often passed without hiding their sizes, as a result bounds checking is often failed to care for or cannot be done. Most of copy functions in the C library such as strcpy(dest, src) have weaknesses by this way. Therefore this makes these functions like a popular point of attack. An attackers purpose in attacking a defenseless program is to obtain that programs privileges. Attackers getting a shell prompt with the programs privileges constituted his aim for manipulating the program. The attacker wants to get the victim program to execute arbitrary code which actually is called to as shell code informally, because the common code performs the semantic equivalent of exec(/bin/sh). To get the program to execute shell code, the attacker desires to modify a code pointer like the code pointer points to code which the attacker would like executed. Firstly the code pointer is adapted, after this point in execution the program dereferences the perverted code pointer, and instead of jumping to the intended code, the program jumps to the attackers shell code. Therefore attacker realize his aim and to take possession the system. Different kind of buffer oveflow attacks are available. But the stack smashing attack is the simplest and the most well-liked between them. The stack smashing attack resembles attacks in the previous paragraph. This technique overwrites the buffer and change return address of a function, so the new return address is the attack code that is infused into the stack by the attacker instead of the justified call point. When the function returns, the control flow is directed to the attack code and starts to process these codes. The stack smashing attack exploits management of stack For instance the stack configuration and the function call mechanism are caught by these pernicious and bad disposed codes . There are other types of buffer overflow attacks that exploit data structures in the heap as well as in the stack. There are several run time solutions that are highly effective without much run time overhead.

4 3. Buffer Overflow Exploits An exploit is a program, usually written in C, that exploits some problem that another program have. The exploit will allow you to run arbitrary code that will let you do something that you shouldn't be able to do in your normal status on the system. Writing more data into a buffer than the buffer can hold causes buffer overflow. If program does not check its buffer boundary, this shows that it is a vulnerable program. In order for a buffer overflow attack to succeed, it needs to achieve the following two goals inject the attack code and force the process to execute the injected code. However, none of these exploits are possible if buffer overflow is prevented. Stack smashing tecnique is the most popular exploitation texchnique. These attack type is explained in security risks based on buffer overflow paragraph. However there are examples about stack smashing tecnique. Figure 2 shows the C source code of the simple example.There is a string has 128 bytes and a buffer has 96 bytes. strcpy() copies from array to buffer , hence an overflow situation occurs. The content of large string change the return address on the stack to the start of the malicious code. The stack layout before and after strcpy() is shown in Figure 3. The shaded area in Figure3(b) shows the content of the overrun buffer after strcpy() which includes buffer and the location of original return address. The first part of the overrun buffer is filled using the code in shellcode and the second part is filled using the address of buffer. The function return address on the stack is overwritten by the address of buffer(figure3 b). When main() returns, it actually transfers control to address B and begins to execute the malicious code. The malicious code executes the execv system call to start a shell /bin/sh.

5 In a real security attack, the malicious code normally comes from an environment variable, user input, or from a network connection. A successful attack on a privileged process such as a daemon process running as root would give the attacker an interactive root shell. 3.1 Return-into-libc: This tecnique resembles stack smashing attack tecnique. The return-into-libc exploit overflows a buffer to overwrite the return address.It uses an existing code rather than a shellcode.This tecnique overwrites the return address with the address of C library function such as system(). 3.2 Other code pointers: In this tecnique code pointers are overwritten instead of return address. For example a function pointer variable, a pointer to a shared library function in the global offset table , the table of pointers to destructor functions, or a C++ virtual function pointer. These exploits cant be catched by StackGuard, StackShield and Libsafe. They both provide guards against return-into-libc attacks, but they can still be exploited. 3.3 Malloc() overflow: Malloc() function allocates memory in the C library. Allocated memory includes user requested block, data which used to manage the heap. Data for managing heap includes size of block, pointer to other blocks etc. The vulnerability is that a heap variable can be overflowed to overwrite those management data. StackGuard, StackShield, Libsafe and Solar Designers stack patch can detect exploits which are occured which this way. 3. 4 Indirect overflow via pointer: With indirect overflow via pointer it is possible to overwrite the return address without altering the StackGuard canary word. This technique overflows a buffer to overwrite a pointer, which is used subsequently to overwrite a code pointer. It is also possible to overwrite a memory area that is far from the overflowed buffer. Stack-Guard, StackShield and Solar Designers stack patch can detect this technique.

4.Preventing Buffer Overflows Security risks based on buffer overflows is explained previous paragraphs. Preventing methods described below can not prevent all of these attacks. owever, can make it more difficult to access buffer overflows and, hence, destroy the consistency of stacks. 4.1 Writing secure code: Buffer overflows are the result of running over the buffer with long code. C library functions such as strcpy (), strcat (), sprintf () and vsprintf ()and perform no bounds checking. They operate on null terminated strings. gets () is another function that reads user input (into a buffer) from stdin until a terminating newline or EOF is found.scanf() also can cause buffer overflow. Therefore while using functions, programnmers must be careful, so if they are educated about C Libraries lacknesses and less utilization of these vulnerabilities functions, buffer overflows problems can decrease. 4.2 Stack execute invalidation: Spiteful and malicious code resides in the stack.If the stack is disqualified to execute any instructions, these harmful codes cant be successful. This is possible in Linux, however other compilers which use trampoline functions for taking the address of a function working stack. When the address of a nested function is taken, a trampoline(small piece of code) is a created at run-time 4.3 Compiler tools: Several compiler tools are available which present warnings on the use of unsafe constructs such as gets (), strcpy () and the like. For instance, this code

6 int main () { char *str = (char *)malloc(10);// allocate 10 bytes for str gets (str); // reads input from stdin and store into str } when compiled with GCC, returns the following warning: /tmp/cc203ViF.o: In function "main": /tmp/cc203ViF.o(.text+0x1f): the "gets" function is dangerous and should not be used. These compilers return warning messages, dont change the source code. However rebuilt source code tools are developed. But the application is not open source, a problem can be occurred. These tools affect executable size and execution time in bad direction, they are generally increased. These compilers generate the code with built-in safeguards that try to prevent the use of illegal addresses. Any code that tries to access an illegal address is not allowed to execute. This class of solutions inserts code for runtime detection at compile time. StackGuard and StackShield are the instances of compiler extensions for runtime detection. 4.3.1 StackGuard: StackGuard-enhanced programs define the behavior of writing to the return address of a function while it is still active. StackGuard prevents changes to active return addresses by either detecting the change of the return address before the function returns, or by completely preventing the write to the return address. Detecting changes to the return address is a more efficient and portable technique, while preventing the change is more secure. StackGuard supports both techniques, as well as adaptively switching from one mode to the other. StackGuard works by inserting a guard value (called a canary) in front of the return address; if a buffer overflow overwrites the return address, the canary's value changes and the system detects this before using it. This is quite valuable, but note that this does not protect against buffer overflows overwriting other values (which they may still be able to use to attack a system). 4.3.2 StackShield: StackShield is also a GNU C compiler extension that protects the return address. When a function is called StackShield copies away the return address to a nonoverflowable area, and restores the return address upon returning from a function. Even if the return address on the stack is altered, it has no effect since the original return address is remembered. As with Stack-Guard, programs needs to be recompiled. 4.4 Dynamic run-time checks: This method primarily relies on the safety code being preloaded before an application is executed. This preloaded component can either provide safer versions of the standard unsafe functions, or it can ensure that return addresses are not overwritten. One example of such a tool is libsafe. The libsafe library provides a way to secure calls to these functions, even if the function is not available. It makes use of the fact that stack frames are linked together by frame pointers. When a buffer is passed as an argument to any of the unsafe functions, libsafe follows the frame pointers to the correct stack frame. It then checks the distance to the nearest return address, and when the function executes, it makes sure that address is not overwritten. 4.4.1 Libsafe: Libsafe is an implementation of vulnerable copy functions in C library such as strcpy(). In addition to the original functionality of those functions, it imposes a limit on the involved copy operations such that they do not overwrite the return address. The limit is determined based on the notion that the buffer cannot extend beyond its stack frame. Thus the maximum size of a buffer is the distance between the address of the buffer and the

7 corresponding frame pointer. Libsafe is implemented as a shared library that is preloaded to intercept C library function calls. Programs are protected without recompilation unless they are statically linked with the C library. Libsafe protects only those C library functions whereas StackGuard and StackShield protect all functions. The name is derived from the coal mining practice of taking a canary down with the workers. The canary was more sensitive to poisonous gas than humans, so examining the state of the canary could reveal a dangerous buildup of poison gas. 4.5 Runtime array bounds checking: The array bounds and pointer checking technique by Jones and Kelly is an extension to the GNU C compiler that imposes the access check on C pointers and arrays. Instead of changing the pointer representation, it maintains a table of all the valid storage objects that holds such informations as the base address and size etc. The heap variables are entered into the table via a modified malloc() function and deleted from the table via a modified free() function. Stack variables are entered into / deleted from the table by the constructor / destructor function, which is inserted inside a function definition at the point a stack variable enters / goes out of the scope. The access check is done by substituting the pointer and array operations with the functions that perform bounds check using the table in addition to the original operation. Since native C pointers are used, this technique is compatible with existing programs. The obvious advantage of array bounds checking approaches are that they completely eliminate buffer over- flow vulnerabilities. However, these are also the most expensive solution, particularly for pointer- and arrayintensive programs since every pointer and array operation must be checked. This may not be suitable for a production system. 4.6. Static analysis of array bounds checking: This is a technique that detects possible buffer overflow in the vulnerable C library functions. A string buffer is modeled as a pair of integer ranges (lower bound, upper bound) for its allocated size and its current length. A set of integer constraints is predefined for a set of string operations (e.g. character array declaration, vulnerable C library functions and assignment statements involving them). Using those integer constraint, the technique analyzes the source code by checking each string buffer to see whether its inferred allocated size is at least as large as its inferred maximum length. This tecnique based on annotations which are provided by programmer s to detect possible buffer overflow. For example, annotations for strcpy() contain a claim that the destination buffer has been allocated to hold at least as many characters as are readable in the source buffer. This technique protects any annotated functions whereas the integer range analysis only protects C library functions. Generally, a pure compile-time analysis like the above can produce many false alarms due to the lack of run time information. For example, gets() reads its input string from stdin so the size of the string is not known at compile time. For such a case a warning is issued as a possible buffer overflow. In fact, all the legitimate copy operations that accept their strings from unknown sources (such as a command line argument or an I/O channel) are flagged as possible buffer overflows (since they are indeed vulnerable). Without further action, those vulnerabilities are identified but still open to attack. 4.7 Using Library Patches: 4.7.1 C Library Patches. Since many buffer overflow vulnerabilities are caused by unsafe C library functions such as gets and strcpy, this class of solutions patch the standard C library for detections. Libsafe intercepts function calls to shared C Library and conducts framepointer based boundary checking. These approaches can fail in two cases: if a program chooses not to enable frame pointer at compile time (many programs do so for performance optimization) if the buffer overflow vulnerability is caused by application internal function calls.

As a result development stage is important, if programmer must develop a program which defends itself opposite buffer overflows. After this one, techniques which are explained before, can be used as a additional precaution . 4.7.2 Operating System Kernel Patches. By changing part of the operating system kernel, it is possible to prevent or detect certain types of buffer overflow attacks. The Non-executable Stack Linux Patch prevents execution of malicious code on the stack by making the stack nonexecutable. This approach is transparent to application programs and offers zero performance overhead. However, the return-into-libc [19] attack completely defeats this scheme by overwriting the return address and transfers execution control to the heap or shared libraries. in Linux the Linux kernel could be modified so that the stack segment is not executable; such a patch to Linux does exist. But this is not built into the Linux kernel. Part of the rationale is that this is less protection than it seems; attackers can simply force the system to call other interesting locations already in the program (e.g., in its library, the heap, or static data segments). Also, sometimes Linux does require executable code in the stack, e.g., to implement signals and to implement GCC trampolines. 4.7.3 Solar Designers non-executable stack patch: The stack smashing attack injects an attack code in the stack, which is executed when the function returns. One of the core features of the Solar Designers Linux kernel patch is to make the stack segment non-executable.This patch does not impose any performance penalty nor does it require program recompilation (except for the operating system kernel). 5. Conclusion Buffer overflows as shown are a very critical security problem. In this paper buffer overflows definition, security risks, techniques for prevention these risks and how these can be done are explained. Although preventive measures are available, while using C librariries care and knowledge is important for obtaining effective programs.

6. References Jun Xu, Zbigniew Kalbarczyk, Sanjay Patel and Ravishankar K. Iyer Architecture Support for Defending Against Buffer Overflow Attacks

A Practical Dynamic Buffer Overflow Detector

Type-Assisted Dynamic Buffer Overflow Detection

Bu_er overflow bounds for multiplexed regulated

tra_c streams

F. M. Guillemin1, N. Likhanov2, R. R. Mazumdar, C. Rosenberg and Yu Ying3 BUFFER OVERFLOW Eswar Balasubramanian Spring 2002 ECE578 Computer and Network Security
SecurityFocus. http://online.securityfocus.com/bid. StackShield. http://www.angelfire.com/sk/stackshield. I. Simon. A comparative analysis of methods of defense against buffer overflow attacks http://www.mcs.csuhayward.edu/simon/security/boflo.html.