Article :: Using the Linux Syscalls in Assembly

Date: 2018-12-28

Last Modified: 2026-02-01

Reading Time: 6 mins

Categories: articles

Tags: assembly nasm dev glibc linux userland exploitation

Introduction

This hands-on tutorial teaches you how to write assembly programs that invoke Linux system calls directly, bypassing the C library and interacting with the kernel at the lowest level.

Overview

When a program needs to interact with the operating system; to write to a file, allocate memory, or exit gracefully; it makes a system call (syscall). Most programmers use these indirectly through C library wrappers, but understanding how to invoke syscalls directly from assembly gives you deep insight into how programs actually work.

In this tutorial, we’ll build a simple x64 Linux program that prints “Hack The Planet” to the screen using raw syscalls, with no C library dependency.

Prerequisites

Before starting, you should have:

Basic knowledge of assembly language (NASM syntax)
Understanding of x64 registers (RAX, RDI, RSI, etc.)
NASM assembler and ld linker installed
Familiarity with C programming helps but isn’t required

Install NASM

On Debian/Ubuntu: sudo apt install nasm On Fedora/RHEL: sudo yum install nasm

Objective

We’ll build a Linux ELF64 binary that uses basic syscalls to print a message on the screen. Along the way, we’ll learn:

How to find syscall numbers
How to set up registers for syscall invocation
Why programs segfault without proper exit handling

Finding Syscall Numbers

Every syscall in Linux has a unique number. For x64, these are defined in a header file. Let’s find the write syscall:

1
2
3
4
5
6
7
cat /usr/include/x86_64-linux-gnu/asm/unistd_64.h | grep write
#define __NR_write 1
#define __NR_pwrite64 18
#define __NR_writev 20
#define __NR_pwritev 296
#define __NR_process_vm_writev 311
#define __NR_pwritev2 328

The basic write syscall is number 1. Now let’s check the manual to understand its arguments:

1
man 2 write

As we can see, the write syscall has three arguments:

ssize_t write(int fd, const void *buf, size_t count);

fd - File descriptor (0=stdin, 1=stdout, 2=stderr)
buf - Pointer to the data to write
count - Number of bytes to write

Syscall Register Conventions

Reading the syscall manual, we learn how to set up x64 registers to invoke syscalls:

Register mapping for x64 syscalls:

RAX - Syscall number
RDI - First argument
RSI - Second argument
RDX - Third argument
R10 - Fourth argument
R8 - Fifth argument
R9 - Sixth argument

Different from Function Calls

These register conventions are specific to syscalls. Regular x64 function calls use a different ABI (Application Binary Interface). Don’t confuse the two!

Writing the Assembly Code

For our program, we’ll:

Use file descriptor 1 (stdout) to print to the screen
Point RSI to our message string “Hack The Planet”
Set RDX to 16 (the length of our message including newline)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
global _start

section .text

_start:
;
; Setting up the registers to print the message
; using the "write" syscall
;
  mov rax, 1          ; syscall number for write
    mov rdi, 1          ; file descriptor 1 = stdout
    mov rsi, msg        ; pointer to message
    mov rdx, length     ; message length
    syscall             ; invoke kernel

section .data
    msg: db 'Hack The Planet',0xa    ; message with newline
    length: equ $-msg                 ; calculate length

Understanding length calculation

The equ $-msg directive calculates the length by subtracting the starting address of msg from the current position $. This automatically gives us the string length.

Compiling the Code

Let’s compile and link this assembly code:

1
2
nasm -felf64 syscall-001.nasm -o syscall-001.o
ld syscall-001.o -o syscall-001.bin

Flags explained:

-felf64 - Output format is 64-bit ELF (Executable and Linkable Format)
ld - Links the object file into an executable binary

Once we run our program with ./syscall-001.bin, we can see that our message “Hack The Planet” is printed, but we get a Segmentation Fault error:

Why the Segfault?

After printing the message, our program doesn’t know what to do next. It keeps executing whatever bytes follow in memory, eventually hitting invalid instructions or memory. We need to exit properly!

Adding the Exit Syscall

Let’s find the exit syscall number:

1
2
3
cat /usr/include/x86_64-linux-gnu/asm/unistd_64.h | grep exit
#define __NR_exit 60
#define __NR_exit_group 231

The exit syscall is number 60. Let’s check its manual page:

1
man 2 exit

There’s only one argument: int status (the exit code, like 0 for success or non-zero for errors).

1
void _exit(int status);

Let’s add the exit syscall to our program. We’ll use exit status 1 for testing:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
global _start

section .text

_start:
;
; Setting up the registers to print the message
; using the "write" syscall
;
  mov rax, 1          ; syscall number for write
    mov rdi, 1          ; file descriptor 1 = stdout
    mov rsi, msg        ; pointer to message
    mov rdx, length     ; message length
    syscall             ; invoke kernel

;
; Setting up the registers to call the "exit" syscall
; and exit normally with return 1
; Note: if we comment this the process will finish with
; Segmentation Fault error
;
  mov rax, 60         ; syscall number for exit
    mov rdi, 1          ; exit status code
    syscall             ; invoke kernel

section .data
    msg: db 'Hack The Planet',0xa
    length: equ $-msg

Now compile again:

1
2
nasm -felf64 syscall-001.nasm -o syscall-001.o
ld syscall-001.o -o syscall-001.bin

After running ./syscall-001.bin, we can verify the exit status is 1 as we designed:

Perfect! No segfault. The program exits cleanly.

Analyzing with strace

If we use strace to analyze this ELF64 binary, we see interesting details. When we run the binary, the shell uses execve with our binary as an argument along with environment variables:

Understanding strace output

Strace shows us the exact syscalls our program makes, including their arguments and return values. This is invaluable for debugging and understanding program behavior. Read our Debugging with strace article to learn more.

Syscall Execution Flow

Here’s how syscalls work under the hood:

The process:

User program sets up registers (RAX for syscall number, RDI/RSI/RDX for arguments)
syscall instruction triggers a context switch to kernel mode
Kernel validates arguments and executes the requested operation
Kernel returns control to userland with result in RAX
Program continues execution

Key Takeaways

Quick Summary

Syscalls are the interface between userland programs and the kernel
x64 syscall numbers are defined in /usr/include/x86_64-linux-gnu/asm/unistd_64.h
Register convention: RAX=syscall number, RDI/RSI/RDX/R10/R8/R9 for arguments
Always call exit to terminate programs gracefully (syscall 60 on x64)
strace is essential for understanding and debugging syscall behavior
No C library needed - you can interact directly with the kernel

Practice Exercises

Try It Yourself

Modify the program to write to stderr (file descriptor 2) instead of stdout
Create a program that uses the read syscall (number 0) to get user input
Write a program that opens a file using the open syscall (number 2)
Add error checking by examining the return value in RAX after each syscall
Use strace on system utilities like ls and identify the syscalls they make