Home.

How to gracefully shutdown express.js with background processes

Finn Christiansen
Finn Christiansen

Problems if not gracefully shutdown

There are a lot of problems if not gracefully shutdown. E.g.

  • When a server or application fails to shut down gracefully, it can lead to corrupted data as active database transactions are abruptly interrupted.
  • Unfinished processes may leave behind temporary files or cache data, which can accumulate and consume valuable disk space, eventually degrading system performance.
  • Active connections to external services or databases may not close properly, resulting in resource leaks that affect both the server and external systems.
  • Inconsistent application states may occur, leaving background jobs or scheduled tasks stuck in limbo and potentially causing conflicts or duplications when the system restarts.
  • Logging and monitoring systems may not record crucial information about the shutdown, making it difficult to diagnose issues or understand what went wrong during the termination process.

Additional challenge: Async processes like queues

Async processes, such as queues and background jobs, pose an additional challenge during shutdown because they may still be processing tasks even when the main application is ready to terminate. If these tasks are not handled properly, they can be abruptly cut off, leading to incomplete operations, such as unprocessed messages or partially updated records in the database.

Gracefully stopping async processes often requires implementing custom logic to drain the queue, finish any in-progress tasks, and ensure no new tasks are accepted before the shutdown completes. Monitoring the status of these async tasks during shutdown is crucial, as it helps confirm that all tasks have been completed or safely rescheduled for processing after restart.

Coordinating the shutdown of async processes with the main application is complex, especially when dealing with distributed systems, where queues and workers may run on separate servers.

Official documentation of express.js

The official documentation for health checks and graceful shutdown provided the following example:

const server = app.listen(port)

process.on('SIGTERM', () => {
  debug('SIGTERM signal received: closing HTTP server')
  server.close(() => {
    debug('HTTP server closed')
  })
})

This example is not complete. The missing parts are:

  • setting the healthcheck to unhealth
  • stop the asynchronous jobs

Resources