Using Valgrind Part 2

Using Memcheck tool of valgrind to detect memory leaks and other memory errors

Here is the demo program. You can try that. I have also attached it.

#include<stdio.h>
#include<stdlib.h>
void func()
{
int *f=(int *)malloc(sizeof(int)*10); ////Memory Leak
}
int main()
{
int *k;
int c;
int *p=(int *)malloc(sizeof(int));
free(p);
func();
c=*k;//use of uninitialized valirable
free(p);//double free
k=NULL;
printf("%d\n",*k);//accessing a null pointer
return 0;
}

Below is the valgrind memcheck tool output. I have added command also to use valgrind memcheck tool.
root@bt:~/Desktop# valgrind --tool=memcheck -v --leak-check=full --track-origins=yes ./a.out
==2230== Memcheck, a memory error detector
==2230== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==2230== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==2230== Command: ./a.out
==2230==
--2230-- Valgrind options:
--2230--    --tool=memcheck
--2230--    -v
--2230--    --leak-check=full
--2230--    --track-origins=yes
--2230-- Contents of /proc/version:
--2230--   Linux version 3.2.6 (root@bt) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #1 SMP Fri Feb 17 10:40:05 EST 2012
--2230-- Arch and hwcaps: X86, x86-sse1-sse2
--2230-- Page sizes: currently 4096, max supported 4096
--2230-- Valgrind library directory: /usr/local/lib/valgrind
--2230-- Reading syms from /lib/ld-2.11.1.so (0x4000000)
--2230--   Considering /lib/ld-2.11.1.so ..
--2230--   .. CRC mismatch (computed 45f50ae1 wanted 137bc614)
--2230--    object doesn't have a symbol table
--2230-- Reading syms from /root/Desktop/a.out (0x8048000)
--2230-- Reading syms from /usr/local/lib/valgrind/memcheck-x86-linux (0x38000000)
--2230--    object doesn't have a dynamic symbol table
--2230-- Reading suppressions file: /usr/local/lib/valgrind/default.supp
==2230== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-2230-by-root-on-???
==2230== embedded gdbserver: writing to   /tmp/vgdb-pipe-to-vgdb-from-2230-by-root-on-???
==2230== embedded gdbserver: shared mem   /tmp/vgdb-pipe-shared-mem-vgdb-2230-by-root-on-???
==2230==
==2230== TO CONTROL THIS PROCESS USING vgdb (which you probably
==2230== don't want to do, unless you know exactly what you're doing,
==2230== or are doing some strange experiment):
==2230==   /usr/local/lib/valgrind/../../bin/vgdb --pid=2230 ...command...
==2230==
==2230== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==2230==   /path/to/gdb ./a.out
==2230== and then give GDB the following command
==2230==   target remote | /usr/local/lib/valgrind/../../bin/vgdb --pid=2230
==2230== --pid is optional if only one valgrind process is running
==2230==
--2230-- Reading syms from /usr/local/lib/valgrind/vgpreload_core-x86-linux.so (0x4020000)
--2230-- Reading syms from /usr/local/lib/valgrind/vgpreload_memcheck-x86-linux.so (0x4023000)
--2230-- Reading syms from /lib/tls/i686/cmov/libc-2.11.1.so (0x4042000)
--2230--   Considering /lib/tls/i686/cmov/libc-2.11.1.so ..
--2230--   .. CRC mismatch (computed 2236eb0a wanted a071c0c3)
--2230--    object doesn't have a symbol table
--2230-- REDIR: 0x40b5b10 (rindex) redirected to 0x4027050 (rindex)
--2230-- REDIR: 0x40b1f40 (malloc) redirected to 0x40263bf (malloc)
--2230-- REDIR: 0x40b1e60 (free) redirected to 0x4025fd9 (free)
==2230== Use of uninitialised value of size 4
==2230==    at 0x8048499: main (hell1.c:14)
==2230==  Uninitialised value was created by a stack allocation
==2230==    at 0x8048471: main (hell1.c:8)
==2230==
==2230== Invalid free() / delete / delete[] / realloc()
==2230==    at 0x402605E: free (vg_replace_malloc.c:427)
==2230==    by 0x80484AA: main (hell1.c:15)
==2230==  Address 0x419d028 is 0 bytes inside a block of size 4 free'd
==2230==    at 0x402605E: free (vg_replace_malloc.c:427)
==2230==    by 0x804848F: main (hell1.c:12)
==2230==
==2230== Invalid read of size 4
==2230==    at 0x80484B7: main (hell1.c:17)
==2230==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==2230==
==2230==
==2230== Process terminating with default action of signal 11 (SIGSEGV)
==2230==  Access not within mapped region at address 0x0
==2230==    at 0x80484B7: main (hell1.c:17)
==2230==  If you believe this happened as a result of a stack
==2230==  overflow in your program's main thread (unlikely but
==2230==  possible), you can try to increase the size of the
==2230==  main thread stack using the --main-stacksize= flag.
==2230==  The main thread stack size used in this run was 8388608.
==2230==
==2230== HEAP SUMMARY:
==2230==     in use at exit: 40 bytes in 1 blocks
==2230==   total heap usage: 2 allocs, 2 frees, 44 bytes allocated
==2230==
==2230== Searching for pointers to 1 not-freed blocks
==2230== Checked 56,076 bytes
==2230==
==2230== 40 bytes in 1 blocks are definitely lost in loss record 1 of 1
==2230==    at 0x4026444: malloc (vg_replace_malloc.c:263)
==2230==    by 0x8048465: func (hell1.c:5)
==2230==    by 0x8048494: main (hell1.c:13)
==2230==
==2230== LEAK SUMMARY:
==2230==    definitely lost: 40 bytes in 1 blocks
==2230==    indirectly lost: 0 bytes in 0 blocks
==2230==      possibly lost: 0 bytes in 0 blocks
==2230==    still reachable: 0 bytes in 0 blocks
==2230==         suppressed: 0 bytes in 0 blocks
==2230==
==2230== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 11 from 6)
==2230==
==2230== 1 errors in context 1 of 4:
==2230== Invalid read of size 4
==2230==    at 0x80484B7: main (hell1.c:17)
==2230==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==2230==
==2230==
==2230== 1 errors in context 2 of 4:
==2230== Invalid free() / delete / delete[] / realloc()
==2230==    at 0x402605E: free (vg_replace_malloc.c:427)
==2230==    by 0x80484AA: main (hell1.c:15)
==2230==  Address 0x419d028 is 0 bytes inside a block of size 4 free'd
==2230==    at 0x402605E: free (vg_replace_malloc.c:427)
==2230==    by 0x804848F: main (hell1.c:12)
==2230==
==2230==
==2230== 1 errors in context 3 of 4:
==2230== Use of uninitialised value of size 4
==2230==    at 0x8048499: main (hell1.c:14)
==2230==  Uninitialised value was created by a stack allocation
==2230==    at 0x8048471: main (hell1.c:8)
==2230==
--2230--
--2230-- used_suppression:     11 dl-hack3-cond-1
==2230==
==2230== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 11 from 6)
Segmentation fault
root@bt:~/Desktop#

You can clearly see that valgrind has pointed out commented errors and also display the detail description about that error.
Valgrind options     -     uses
--tool=memcheck        - using valgrind memcheck tool
--leak-check=full     - to see details of leaked memory
--track-origins=yes    - to see where uninitialised values come from
-v             - For counts of detected and suppressed errors

Memory Pools: describing and working with custom allocators

There are many different sorts of custom allocator, so Memcheck attempts to reason about them using a loose, abstract model. We use the following terminology when describing custom allocation systems:
• Custom allocation involves a set of independent "memory pools".
• Memcheck’s notion of a memory pool consists of a single "anchor address" and a set of non-overlapping "chunks" associated with the anchor address.
• Typically a pool’s anchor address is the address of a book-keeping "header" structure.
• Typically the pool’s chunks are drawn from a contiguous "superblock" acquired through the system malloc or mmap.

The Valgrind mempool client request API is intentionally vague about the exact structure of a mempool. There is no specific mention made of headers or superblocks.

Typically, before making client requests related to mempools, a client program will have allocated such a header and superblock for their mempool, and marked the superblock NOACCESS using the VALGRIND_MAKE_MEM_NOACCESS client request.
When dealing with mempools, the goal is to maintain a particular invariant condition: that Memcheck believes the unallocated portions of the pool’s superblock (including redzones) are NOACCESS. To maintain this invariant, the client program must ensure that the superblock starts out in that state; Memcheck cannot make it so, since Memcheck never explicitly learns about the superblock of a pool, only the allocated chunks within the pool.

Once the header and superblock for a pool are established and properly marked, there are a number of client requests programs can use to inform Memcheck about changes to the state of a mempool. e.g - VALGRIND_CREATE_MEMPOOL,VALGRIND_DESTROY_MEMPOOL,VALGRIND_MEMPOOL_ALLOC,VALGRIND_MEMPOOL_FREE, etc.

For details refer Valgrind Manual Page.

Debugging MPI Parallel Programs with Valgrind

Memcheck supports debugging of distributed-memory applications which use the MPI message passing standard. This support consists of a library of wrapper functions for the PMPI_* interface. When incorporated into the application’s address space, either by direct linking or by LD_PRELOAD, the wrappers intercept calls to PMPI_Send, PMPI_Recv, etc. They then use client requests to inform Memcheck of memory state changes caused by the function being wrapped.

For details refer valgrind Manual Page.

Using Valgrind Part 1

What Valgrind does with your program

You can invoke valgrind  using command

valgrind [valgrind-options] your-prog [your-prog-options]

The most important option is --tool which dictates which Valgrind tool to run. The default tool is memcheck. 

Regardless of which tool is in use, Valgrind takes control of your program before it starts. Debugging information is read from the executable and associated libraries, so that error messages and other outputs can be phrased in terms of source code locations, when appropriate.

Your program is then run on a synthetic CPU provided by the Valgrind core. As new code is executed for the first time, the core hands the code to the selected tool. The tool adds its own instrumentation code to this and hands the result back to the core, which coordinates the continued execution of this instrumented code.

First off, consider whether it might be beneficial to recompile your application and supporting libraries with debugging info enabled (the -g option). Without debugging info, the best Valgrind tools will be able to do is guess which function a particular piece of code belongs to, which makes both error messages and profiling output nearly useless. With -g, you'll get messages which point directly to the relevant source code lines.

 Another option you might like to consider, if you are working with C++, is -fno-inline. That makes it easier to see the function-call chain, which can help reduce confusion when navigating around large C++ apps.

If you are planning to use Memcheck: On rare occasions, compiler optimisations (at -O2 and above, and sometimes -O1) have been observed to generate code which fools Memcheck into wrongly reporting uninitialised value errors, or missing uninitialised value errors. So the best solution is to turn off optimisation altogether

Memcheck: a memory error detector

To use this tool, you may specify --tool=memcheck on the Valgrind command line.

It can detect the following problems that are common in C and C++ programs.

• Accessing memory you shouldn't, e.g. overrunning and underrunning heap blocks, overrunning the top of the stack, and accessing memory after it has been freed.
• Using undefined values, i.e. values that have not been initialised, or that have been derived from other undefined values.
• Incorrect freeing of heap memory, such as double-freeing heap blocks, or mismatched use of malloc/new/new[] versus free/delete/delete[] 
• Overlapping src and dst pointers in memcpy and related functions.
• Memory leaks.

Gdb(GNU Debugger)

A debugger for several languages, including C and C++. It allows you to inspect what the program is doing at a certain point during execution.Errors like segmentation faults may be easier to find with the help of gdb.

You need to build program with -g option to enable built-in debugging support.

gcc [other flags] -g <source files> -o <output file>

  

gdb commands

• getting help - you can get help of gdb commands
  (gdb)help CommandName
  (gdb)help
  will list classes of commands.
• Invoking gdb -Once your executable has been generated by the compiler, it is run under the control of gdb by typing.

  >gdb program 

  if you have not specified filename with gdb you can specify later using command file.
  (gdb) file program
  where program is the name of the executable. it loads the debugging information of this file.
  You can also start with both an executable program and a core file
  

  >gdb program core

  You can, instead, specify a process ID as a second argument, if you want to debug a running process:
  

  >gdb program 1234 

• Quiting gdb - you can exit gdb by using quit command.
  (gdb)quit
• Running the program - When gdb starts, your program is not actually running. It won't run until you tell gdb how to run it.To run the program, just use:
  (gdb) run
  if there is no issue it runs normally, else you (should) get some useful information like the line number where it crashed, and parameters to the function that caused the error.
• Setting breakpoints - Breakpoints can be used to stop the program run in the middle, at a some point. you can use “break” command.
  (gdb) break prog1.c:6
  sets a breakpoint at line 6, of prog1.c.
  You can also tell gdb to break at a particular function.Suppose you have a function my func:
  int func(int a);
  you can apply breakpoint to this function using command
  (gdb) break func
  Once you’ve set a breakpoint, you can try using the run command again. This time, it should stop where you placed breakpoint.
• You can proceed onto the next breakpoint by typing “continue”. because typing run will restart the program from beginnning.
  (gdb) continue
• You can single-step (execute just the next line of code) by typing “step.”
  (gdb) step
• Similar to “step,” the “next” command single-steps as well, except this one doesn’t execute each line of a sub-routine, it just treats it as one instruction.
  (gdb) next
• Removing breakpoints - you can use delete command to remove breakpoint.
  (gdb) delete N
  deletes Nth breakpoint. you can get that using command
  (gdb)info break.
• Passing arguments - you can specify arguments using set args command.
• Examining variables - The print command prints the value of the specified variables.
  (gdb) print var
• backtrace - produces a stack trace of the function calls that lead to a seg fault.
  (gdb)backtrace

For quick reference of gdb commands refer link http://users.ece.utexas.edu/~adnan/gdb-refcard.pdf

Memory Profiling

Here we will see about memory profiling, i.e. debugging programs that consume too much memory. Excessive memory consumption can be due to either inefficient data structures, missing memory deallocation, or simply because of an incorrect estimate on how much memory the program will need. Excessive memory use can also increase the runtime of a program, by forcing the program to access main memory instead of the faster cache, and by overflowing the available main memory (paging).

Memory profilers

Memory profilers are tools that do detailed book keeping of memory usage. Because most of these tool only watch dynamic memory allocated on the heap with malloc()/new, they are also called heap profilers. A memory profiler keeps records when a piece of dynamic memory is allocated, by whom (call stack) it was allocated, its size and when and by whom it was deallocated. After the program ends, the memory profiler outputs graphs and log files which reveal details about the memory usage and make it easy to locate the largest memory users.

Some of the memory profilers are:

AQtime

AQtime is a commercial tool sold by AutomatedQA. It is a runtime and memory profiler that works on Windows with the Microsoft, Borland, Intel, Compaq, and GNU compilers. AQtime is integrated into Microsoft Visual Studio and Borland Developer Studio.

Detailed Information can be found at: http://www.automatedqa.com/products/aqtime

mpatrol

mpatrol is an Open Source software memory debugger which also has memory profiling abilities. It is a library that is linked into the executable and intercepts calls to malloc(), free(), and similar functions. The use model is similar to gprof. 

Detailed Information can be found at: http://sourceforge.net/projects/mpatrol

Massif

Massif is a heap profiler. It measures how much heap memory your program uses. This includes both the useful space, and the extra bytes allocated for book-keeping and alignment purposes. It can also measure the size of your program's stack(s), although it does not do so by default. 

It is part of valgrind tool.

Detailed information can be found at: http://valgrind.org/

Memory Management Issues

The C/C++ language is able to manage memory resources, and can access memory directly through pointers.The allocation of dynamic memory (also known as heap memory) in C/C++ is under the control of the programmer. New memory is allocated with functions such as malloc() and various forms of the operator new. Unused memory is returned with free() or delete.

The memory handling in C/C++ gives a large degree of freedom, control, and performance, but comes at a high price: the memory access is a frequent source of bugs. The most frequent sources of memory access bugs are memory leaks, incorrect use of memory management, buffer overruns, and reading uninitialized memory.

Memory Leaks

Memory leaks are data structures that are allocated at runtime, but not deallocated once they are no longer needed in the program. If the leaks are frequent or large, eventually all available main memory in your computer will be consumed. The program will first slow down, as the computer starts swapping pages to virtual memory, and then fail with an out-of-memory error.

Incorrect Use of Memory Management

A whole class of bugs is associated with incorrect calls to memory management: freeing a block of memory more than once, accessing memory after freeing it, or freeing a block that was never allocated.

Buffer Overruns

Buffer overruns are bugs where memory outside of the allocated boundaries is overwritten, or corrupted. Buffer overruns can occur for global variables, local variables on the stack, and dynamic variables that were allocated on the heap with memory management.

One nasty artifact of memory corruption is that the bug may not become visible at the statement where the memory is overwritten. Only later, another statement in the program will access this memory location. Because the memory location has an illegal value, the program can behave incorrectly in a number of ways: the program may compute a wrong result, or, if the illegal value is in a pointer, the program will try to access protected memory and crash. If a function pointer variable is overwritten, the program will do a jump and try to execute data as program code.

Uninitialized Memory Bugs

Reading uninitialized memory can occur because C/C++ allows creation of variables without an initial value.The memory allocation function malloc() and operator new also do not initialize or zero out the allocated memory blocks.

Memory Debuggers

Memory debuggers do detailed bookkeeping of all allocated/deallocated dynamic memory. They also intercept and check access to dynamic memory. Some memory debuggers can check access to local variables on the stack and statically allocated memory.

Purify

Purify is a commercial memory debugging tool, available on Linux, Windows, and Solaris. Purify works by instrumenting the object code of a program during the link phase. No source code is required, and no special compiler flags or a recompilation of the object files are needed.

More information is available here:

http://en.wikipedia.org/wiki/IBM_Rational_Purify

http://www-306.ibm.com/software/awdtools/purifyplus

Valgrind

Valgrind is Open Source software. It currently is available on Linux for x86 and PowerPC processors. The use model of Valgrind is simple. Valgrind interprets the object code, so it does not need to modify object files or executable, and therefore does not require special compiler flags, recompiling, or relinking the program. The

valgrind command is simply added at the shell command line, in front of the program to be executed. A further advantage of Valgrind is that no program source code is required, so Valgrind can be used to analyze black-box software modules from third parties where the source code is confidential and unavailable.

Valgrind comes as a collection of tools for the following purposes:

• Memcheck: a memory checker

• Callgrind: a runtime profiler

• Cachegrind: a cache profiler

• Helgrind: find race conditions

• Massif: a memory profiler

Documentation and download instructions for Valgrind are available here:

http://valgrind.org

Insure++

Insure++ is a commercial tool for detecting runtime memory errors. Insure++ uses source code instrumentation: it modifies the source on the fly just before it is given to the compiler. This use model requires recompilation of the source files. There is some provision to support object code libraries where source code is not available.

More information is available here:

http://www.parasoft.com