Home.

Practical guide to POSIX signals in TypeScript

Finn Christiansen
Finn Christiansen

created at:

Why you should understand POSIX signals

POSIX signals are a fundamental aspect of process management in Unix-based operating systems. These signals allow the operating system to communicate with processes, instructing them to take specific actions, such as termination, pausing, or resuming operations. Understanding POSIX signals is essential for developers building robust, resilient applications, particularly in environments where graceful shutdowns and efficient resource management are critical. For example, in containerized environments like Docker or Kubernetes, proper handling of termination signals (e.g., SIGTERM) ensures that applications can clean up resources and shut down gracefully.

Failing to handle signals properly can lead to data corruption, open file descriptors, or improper shutdown sequences. Furthermore, some signals, like SIGKILL, cannot be caught or ignored, enforcing an immediate termination of a process. This means that while certain signals offer the opportunity to clean up before exiting, others do not, requiring developers to handle different signals appropriately. Mastering POSIX signals is crucial for building resilient systems that can handle unexpected interruptions and termination in a controlled manner.

Relevant signals for ending processes

  • SIGINT (2): Terminal interrupt signal
  • SIGTERM (15): Termination signal
  • SIGKILL (9): Kill (cannot be caught or ignored)

there are many other signals. See here for more signals.

Handling POSIX signals in Typescript

SIGINT

process.on('SIGINT', () => {
	console.log('Received SIGINT signal');
	process.exit(0);
});

SIGTERM

process.on('SIGTERM', () => {
	console.log('Received SIGTERM signal');
	process.exit(0);
});

speciality of SIGKILL

One cannot handle sigkills. Thus, the following code

process.on('SIGKILL', () => {
	console.log('Received SIGKILL signal');
	process.exit(0);
});

Will produce the error

Error: uv_signal_start EINVAL

Program to see SIGINT, SIGTERM and SIGKILL in action

const run = async () => {
	console.log('waiting for 30 seconds');
	await new Promise((resolve) => setTimeout(resolve, 30000));
	console.log('done');
}

const handleSigInt = () => {
	console.log('Received SIGINT signal');
	process.exit(0);
}
process.on('SIGINT', handleSigInt);

const handleSigTerm = () => {
	console.log('Received SIGTERM signal');
	process.exit(0);
}
process.on('SIGTERM', handleSigTerm);

console.log("process.listeners('SIGINT'): ", process.listeners('SIGINT'));
console.log("process.listeners('SIGTERM'): ", process.listeners('SIGTERM'));

console.log('PID: ', process.pid);

run();

Sending SIGINT, SIGTERM and SIGKILL to the program

Sending with the command line, while the program is running in the foreground:

  • SIGINT: Ctrl + C
  • SIGTERM: no shortcut
  • SIGKILL: no shortcut

You can use the kill utility programm to send arbitary codes to processes.

Find out the PID (process id) of the programm with

ps aux | grep <program_name>

send SIGTERM

kill <PID>

because sigterm is the standard signal there is no need to state it explicitly. However, this can be done explicitly with

kill -9 <PID>

send SIGKILL

kill -9 <PID>

Output of the Signals

SIGINT

npx ts-node index.ts
process.listeners('SIGINT'):  [ [Function: handleSigInt] ]
process.listeners('SIGTERM'):  []
PID:  87731
waiting for 30 seconds
^CReceived SIGINT signal

SIGTERM

npx ts-node index.ts
process.listeners('SIGINT'):  [ [Function: handleSigInt] ]
process.listeners('SIGTERM'):  [ [Function: handleSigTerm] ]
PID:  87878
waiting for 30 seconds
Received SIGTERM signal

SIGKILL

npx ts-node index.ts
process.listeners('SIGINT'):  [ [Function: handleSigInt] ]
process.listeners('SIGTERM'):  [ [Function: handleSigTerm] ]
PID:  87951
waiting for 30 seconds
[1]    87936 killed     npx ts-node index.ts

POSIX-Signals send by Orchistrators like Kubernetes

Docker

In plain Docker (without Docker Compose), POSIX signals are also used to manage container processes during lifecycle events, such as stopping or restarting containers. Here's how Docker handles signals:

  • SIGTERM (Signal 15): When you stop a container with docker stop , Docker sends the SIGTERM signal to the main process running inside the container (typically PID 1). This signal is used to request a graceful shutdown, allowing the process time to clean up resources, close connections, or save data before termination.
  • SIGKILL (Signal 9): If the container's main process does not stop within a configurable grace period, Docker sends the SIGKILL signal to forcefully terminate the process. SIGKILL does not allow any cleanup; it immediately kills the process.

Graceful Shutdown in Docker: The grace period between the SIGTERM and SIGKILL signals can be configured when using the docker stop command by providing an optional timeout argument:

docker stop --time=30 <container>

This example waits 30 seconds after sending SIGTERM before Docker sends SIGKILL to forcefully stop the container. If no timeout is provided, Docker uses the default timeout of 10 seconds.

Process for Stopping Containers:

  • SIGTERM: Docker sends SIGTERM to the container’s main process (PID 1). Docker waits for the specified grace period (default 10 seconds, configurable via the --time option).
  • SIGKILL: If the container has not stopped after the grace period, Docker sends SIGKILL to forcefully stop the container.

Other POSIX Signals in Docker:

  • SIGINT (Signal 2): When running a container interactively (docker run -it), Docker will forward the SIGINT signal (e.g., from pressing Ctrl+C in the terminal) to the container’s main process. This allows you to manually interrupt the process inside the container.
  • SIGQUIT (Signal 3): Docker also forwards this signal, which is used to quit a process and generate a core dump.

Custom Signal Handling: The process running inside the container can also be set up to handle other signals. However, by default, Docker only sends SIGTERM and SIGKILL during container stop events.

Example: When you stop a container with docker stop:

docker stop <container>

The process inside the container receives SIGTERM, giving it time to shut down gracefully. If it does not stop within the grace period, Docker sends SIGKILL to forcefully terminate it.

To summarize:

  • SIGTERM is sent for graceful shutdown.
  • SIGKILL is sent after the grace period to force a stop.

You can adjust the grace period with the--time option in docker stop.

default grace period The default grace period in Docker for the time between sending the SIGTERM signal and SIGKILL is 10 seconds. This means that when you stop a container using the docker stop command, Docker sends a SIGTERM signal to the main process inside the container and waits 10 seconds for the container to shut down gracefully. If the container is still running after this grace period, Docker sends a SIGKILL signal to forcibly terminate the container.

You can modify this default timeout using the --time (or -t) option in the docker stop command:

docker stop --time=30 <container>

This example would extend the grace period to 30 seconds before SIGKILL is sent.

Docker-Compose

In Docker Compose, similar to Kubernetes, POSIX signals are used to manage containers, particularly when stopping or restarting them. Here are the primary signals Docker Compose sends:

  • SIGTERM (Signal 15): When stopping a container with docker-compose down or docker-compose stop, Docker Compose sends the SIGTERM signal to allow the container's processes to terminate gracefully. This gives the container a chance to clean up resources, save data, or perform other shutdown routines.
  • SIGKILL (Signal 9): If the container doesn't stop after a certain period (by default 10 seconds in Docker), Docker Compose sends the SIGKILL signal, which forces the immediate termination of all running processes within the container. This does not allow any cleanup operations, and the processes are killed instantly.

Process:

  • SIGTERM: Docker Compose first sends SIGTERM to allow graceful shutdown. This gives the application inside the container time to close connections, save state, and release resources.
  • SIGKILL: If the container does not stop within the grace period (configurable via the stop_grace_period option), Docker Compose sends SIGKILL to forcefully terminate the container.

Configuring Graceful Shutdown in Docker Compose: You can configure the grace period between SIGTERM and SIGKILL in your docker-compose.yml file using the stop_grace_period option:

services:
  my-service:
    image: my-image
    stop_grace_period: 30s

Kubernetes

In Kubernetes, various POSIX signals are sent to manage containers, especially when containers are stopped or restarted. The most important signals that Kubernetes sends are:

  • SIGTERM (signal 15): This is the primary signal that Kubernetes sends when a container is to be stopped. Kubernetes first sends a SIGTERM to give the container time to cleanly terminate all running processes and release resources. This gives the container the opportunity to perform a “graceful shutdown” phase.
  • SIGKILL (signal 9): If the container has not shut down within a specified “grace period” (default is 30 seconds, but can be customized) after sending SIGTERM, Kubernetes sends a SIGKILL. This signal forces the container to stop immediately, regardless of any processes still running.

Procedure:

  • SIGTERM: Kubernetes sends SIGTERM to all processes in the container and waits for the “grace period”. During this time, processes can clean up properly.
  • SIGKILL: If the container is still running after the grace period, a SIGKILL is sent to terminate the container immediately.

Settings in Kubernetes

You can define the terminationGracePeriodSeconds in the pod specification to control the time Kubernetes waits between SIGTERM and SIGKILL.

spec:
  terminationGracePeriodSeconds: 60

ECS (AWS Elastic Container Service)

In Amazon ECS (Elastic Container Service), POSIX signals are used to manage containers similarly to Kubernetes and Docker Compose. Here’s how ECS handles signals during container lifecycle events, particularly when stopping or terminating a container:

  • SIGTERM (Signal 15): When you stop or terminate a task in ECS (either manually or via scaling events, updates, etc.), ECS first sends a SIGTERM signal to the running container. This allows the containerized processes to perform a graceful shutdown, where they can clean up resources, save state, or close open connections.
  • SIGKILL (Signal 9): If the container does not stop within a grace period (which is configurable via ECS), ECS sends a SIGKILL signal to forcefully terminate the container. This stops all processes within the container immediately, without allowing any cleanup operations.

Grace Period in ECS: The grace period between the SIGTERM and SIGKILL signals is controlled by the stopTimeout parameter in ECS. You can configure this value in your ECS task definition under the containerDefinitions section:

{
  "containerDefinitions": [
    {
      "name": "my-container",
      "image": "my-image",
      "essential": true,
      "stopTimeout": 60
    }
  ]
}

Task Termination Process in ECS:

ECS sends SIGTERM to the container, allowing the processes to exit gracefully. ECS waits for the duration specified in the stopTimeout parameter (default is 30 seconds). If the container is still running after the timeout, ECS sends SIGKILL, forcibly stopping the container.

Other Signals in ECS: SIGINT (Signal 2): ECS does not explicitly send SIGINT, but this signal could be handled by the application inside the container if necessary. SIGINT is typically used to interrupt processes locally (e.g., via Ctrl+C). ECS primarily uses SIGTERM for graceful shutdown and SIGKILL to enforce termination if the container does not stop within the specified grace period. You can control the length of this grace period using the stopTimeout parameter in your task definition.

Resources