Implementing a System Call on Linux 2.6 for i386

Amit Choudhary

http://amit-choudhary.blogspot.com

amit2030 AT gmail DOT com

Version 1.0, 27 October 2006

This HOWTO describes information about implementing system calls in Linux operating system for i386 architecture with 2.6 kernels. General information about the used and modified files is available as well as description of the source and makefiles and test user space C program too.

1. Introduction

This document explains how to implement a new system call on linux 2.6. A system call is used by application (user) programs to request service from the operating system.

2. System call

A system call is used by application (user) programs to request service from the operating system. The following statements illustrate why system calls are needed. An operating system can access a system's hardware directly, but a user program is not given direct access to the hardware. This is done so that the kernel can keep the system safe and secure from malicious user programs. But often, a user program requires some information from the hardware (e.g., from a web camera to show you the picture), but it cannot get the information directly. So, it requests the operating system to supply it the information. This request is made by using an appropriate system call.

A system call executes in the kernel mode. Every system call has a number associated with it. This number is passed to the kernel and that's how the kernel knows which system call was made. When a user program issues a system call, it is actually calling a library routine. The library routine issues a trap to the Linux operating system by executing INT 0x80 assembly instruction. It also passes the system call number to the kernel using the EAX register. The arguments of the system call are also passed to the kernel using other registers (EBX, ECX, etc.). The kernel executes the system call and returns the result to the user program using a register. If the system call needs to supply the user program with large amounts of data, it will use another mechanism (e.g., copy_to_user call).

3. List of files to be modified/created

Assume that your linux source base directory is /usr/src/linux.

Kernel files to be modified are listed below:

1. /usr/src/linux/arch/i386/kernel/syscall_table.S

2. /usr/src/linux/include/asm-i386/unistd.h

3. /usr/src/linux/include/linux/syscalls.h

4. /usr/src/linux/Makefile

New kernel files/directories to be created are listed below:

1. /usr/src/linux/mycall - Directory that will contain the source file, header file and the Makefile for our system call (You can also implement your system call in an existing file).

2. /usr/src/linux/mycall/mycall.c - Source file containing our system call code.

3. /usr/src/linux/mycall/Makefile - Makefile

New user space files, to be created, to test our system call are listed below:

1. testmycall.c – Source file that will call our system call.

2. testmycall.h - Header file.

4. Kernel files to be modified

4.1 syscall_table.S

Full path of the file - /usr/src/linux/arch/i386/kernel/syscall_table.S

This file contains system call names.

Add a line to the end of this file (Let's assume that the name of our system call is mycall).

Add ".long sys_mycall" at the end of the list.

4.2 unistd.h

Full path of the file - /usr/src/linux/include/asm-i386/unistd.h

This file contains the system call number that is passed to the kernel through the register (EAX) when a system call is invoked.

Add "#define __NR_mycall <Last_System_Call_Num + 1>" at the end of the list.

If the last system call defined here is:

"#define __NR_vmsplice316", then add:

"#define __NR_mycall317" at the end of the list.

Increment the "NR_syscalls" by 1. So, if NR_syscalls is defined as:

"#define NR_syscalls 317", then change it to:

"#define NR_syscalls 318"

4.3 syscalls.h

Full path of the file - /usr/src/linux/include/linux/syscalls.h

This file contain the declarations for system calls.

Add the following line at the end of the file:

"asmlinkage long sys_mycall(int i);"

4.4 Makefile

Full path of the file - /usr/src/linux/Makefile

Add mycall/ to core-y (Search for regex: core-y.*+=). You will be creating this directory. This directory will contain the source file, header file and the Makefile for our system call.

5. New kernel files/directories to be created

5.1 mycall

Full path of the file - /usr/src/linux/mycall

Create a new directory in /usr/src/linux and name it "mycall".

5.2 mycall.c

Full path of the file - /usr/src/linux/mycall/mycall.c

Create a source file named "mycall.c" in dir "mycall". mycall.c will have the code for our system call. The definition of the system call in the source file would be asmlinkage long sys_mycall(...){...} . It should include the file linux/linkage.h So, the file "mycall.c" will look like:

/*---Start of mycall.c----*/

#include<linux/linkage.h>

asmlinkage long sys_mycall(int i)

{

return i+10;

}

/*---End of mycall.c------*/

What is asmlinkage?

Asmlinkage is used to look for the arguments on the kernel stack.

5.3 Makefile

Full path of the file - /usr/src/linux/mycall/Makefile

The Makefile in dir "mycall" will have only one line:

#####Makefile Start#####

obj-y := mycall.o

#####Makefile End#######

6. New user space files, to be created, to test our system call

6.1 testmycall.h (new user space header file to be created)

testmycall.h

Create a header file called testmycall.h. This header file should be included by any program calling our system call.

Add three lines to it

Line 1: This is needed because we need the definition of _syscall1.

#include<linux/unistd.h>

Line 2: This is needed because we need the number of our system call.

#define __NR_mycall 317

Line 3: This is needed for system calls with 1 argument. It is explained in detail below.

_syscall1(long, mycall, int, i)

So, our user header file looks like:

/*---Start of header file------*/

#include<linux/unistd.h>

#define __NR_mycall 317

_syscall1(long, mycall, int, i)

/*---End of header file--------*/

6.2 testmycall.c (new user space source file to be created)

testmycall.c

Create a C file called testmycall.c in the same directory as testmycall.h. The C file will look like:

/*---Start of C file------*/

#include<stdio.h>

#include "testmycall.h"

int main(void)

{

printf("%d\n", mycall(15));

}

/*---End of C file------*/

7. _syscallN macro

_syscall0(int,mycall) indicates that:

The name of the system call is mycall.

It takes zero arguments.

It returns an int.

_syscall1(int,mycall,int,number) indicates that:

The name of the system call is mycall.

It takes one argument.

The argument is an int named number.

It returns an int.

When you expand _syscall1(long,mycall,int,i), you get the following code:

long mycall(int i)

{

return syscall(__NR_mycall, i);

}

But the definition of _syscallN macros are different in the kernel. You can look at /usr/src/linux/include/asm-i386/unistd.h for the definition.

8. Testing our new system call

Step 1: Recompile and install the new kernel so that our system call becomes available to the operating system.

Step 2: Compile and execute the user space C file (testmycall.c) that we created above.

RESULT: You should see the output as 25. This has been tested on kernel 2.6.17.13.

9. Questions And Suggestions

Please send a mail to Amit Choudhary (amit2030 AT gmail DOT com).

10. Copyright.

Permission is granted to copy, distribute and/or modify this document

under the terms of the GNU Free Documentation License, Version 1.2

or any later version published by the Free Software Foundation;

with no Invariant Sections, no Front-Cover Texts, and no Back-Cover

Texts. A copy of the license is included in the section entitled "GNU

Free Documentation License".

GFDL:Implement a system call on Linux 2.6 for i386

Contents