Strace
Strace is a common basic debugging tool. It displays system calls and abbreviated results of these when it is run with a program and its arguments as the argument to strace. For instance:
$ strace ls -l
will execute ls -l and print its system calls as it does so.
Strace also proves useful when an application is giving trouble such as "File not found" but fails to report which file cannot be found. Often, when one runs the application under strace, there will be an fopen() which will return NULL [error] or open() call which will return -1 [error]. Armed with this knowledge, the file in question or one of its directories can be re-spelled, located or created, allowing execution to continue as usual.
A very similar tool to strace is ltrace. For more information, see the strace man page.
Analyzing strace's output
Analyzing strace's output is tedious if you do not know how it goes. You must know that the first keyword in a line of output from strace is always a syscall like open, read, gettimeofday and so on. The meaning its parameters and results can be found with the command
man 2 syscall
output
Here is an example output from strace that we are going to analyze:
open("/lib64/libexpat.so.1", O_RDONLY) = 6
read(6, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@>\0\0\0\0\0\0"..., 832) = 832
fstat(6, {st_mode=S_IFREG|0755, st_size=170240, ...}) = 0
mmap(NULL, 2265264, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 6, 0) = 0x7f5d5ced6000
analysis
The above example consists of 4 lines that can be analyzed like this:
open("/lib64/libexpat.so.1", O_RDONLY) = 6
This first line performs the syscall open on the file /lib64/libexpat.so.1. The file shall be opened read-only (O_RDONLY). This call returns the file descriptor 6. /lib64/libexpat.so.1 is now file number 6. You can get this information from man 2 open.
read(6, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@>\0\0\0\0\0\0"..., 832) = 832
The above line reads from the file number 6. The file content is character 177, then the string ELF, then character 2, 1, 1, 0 and so on.
fstat(6, {st_mode=S_IFREG|0755, st_size=170240, ...}) = 0
Here, fstat tells that file number 6 (/lib64/libexpat.so.1) is a regular file.
mmap(NULL, 2265264, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 6, 0) = 0x7f5d5ced6000
The above line performs the syscall mmap. See man 2 mmap about the parameters - if you do you will find out that the file with descriptor number 6 is mapped to memory at address 0x7f5d5ced6000.
Related Commands
These all relate to running commands in an altered context.
- chroot - Confine the program to "jail".
- env - Change variables.
- nice - Change priority.
- nohup - Protect from hangups (modem) or network outages.
- stdbuf - Change buffering of standard I/O filestreams.
- su - Change user
- timeout - Limit the time.
- valgrind - Validate program behavior