Effects of Concurrent Connection Density on Node.js Proxy Throughput Under Sustained Load

1. Introduction

Modern web proxies routinely handle tens of thousands of concurrent TCP connections. When the proxy is implemented in Node.js, the single-threaded event loop becomes both a strength (efficient I/O multiplexing) and a potential bottleneck (CPU-bound work or excessive callback depth). Prior literature has examined Node.js performance primarily in the context of application servers; comparatively little quantitative data exists for pure proxy workloads under extreme connection density.

This study isolates the effects of concurrent connection count on three architectural variants while holding hardware, network conditions, and payload characteristics constant. Our goals are (1) to identify the practical upper bounds of each architecture, (2) to measure the contribution of common optimizations (HTTP keep-alive, TLS session tickets, and worker isolation), and (3) to document the failure modes that appear beyond those bounds.

2. Related Work

Early evaluations of Node.js concurrency focused on web-server benchmarks such as TechEmpower and the official Node.js HTTP benchmarks. More recent work has examined the cluster module and worker_threads for CPU-bound tasks. Studies of high-connection-count proxies have largely concentrated on C-based servers (nginx, HAProxy, Envoy). The present work bridges these lines by applying rigorous connection-density sweeps to Node.js proxy implementations.

3. Experimental Methodology

3.1 Hardware and Software

All tests ran on an OVH VPS instance equipped with 8 vCPU cores and 24 GB RAM, running Ubuntu 24.04 LTS. The Node.js versions under test were 18.20.4, 20.15.1, and 22.5.1. The proxy under test terminated TLS (Let's Encrypt certificates) and forwarded HTTP/1.1 traffic to a backend that returned a fixed 1.2 KB response.

3.2 Workload Generation

Concurrent clients were generated with wrk2 and vegeta, configured for a constant arrival rate and a 30-second measurement window after a 10-second warm-up. Connection density was increased in steps of 2 000 from 500 to 25 000. Each configuration was repeated five times; medians and 95th-percentile latencies are reported.

3.3 Architectures Compared

• Single-process: one Node.js process handling all connections.
• Cluster: primary process + N workers equal to the number of vCPUs, using the built-in cluster module.
• Worker-thread pool: single process with a fixed pool of 8 worker threads for request handling.

4. Results

Table 1 summarizes peak sustained throughput (requests per second) and corresponding 95th-percentile latency at selected connection densities for the cluster configuration running Node.js 22.x.

Table 1. Cluster-mode performance (Node.js 22.5.1, 8 workers).

Enabling HTTP keep-alive and TLS session resumption improved effective throughput by 11–14 % at high densities by reducing handshake overhead. Upgrading from Node.js 18 to 22 yielded a consistent 6–8 % gain, attributable largely to improvements in the underlying libuv and V8 versions.

5. Discussion

The sharp performance cliff beyond 18 000 concurrent connections in cluster mode coincides with the default Linux somaxconn and net.core.somaxconn limits, as well as increased garbage-collection frequency under memory pressure from large numbers of socket objects. Raising the accept queue and enabling TCP_FASTOPEN delayed the onset of the cliff by roughly 2 500 connections.

Worker-thread isolation reduced event-loop blocking from cryptographic operations but introduced additional serialization overhead that became measurable above 12 000 connections. For pure proxy workloads that perform little CPU work beyond TLS and header parsing, the classic cluster model remains the more efficient choice on the tested hardware.

6. Conclusions

Under the conditions examined, a Node.js cluster-mode reverse proxy can sustain high throughput up to approximately 18 000 concurrent connections on an 8-core system before kernel and event-loop limits dominate. Practical recommendations include: (1) prefer the cluster module over a pure worker-thread design for I/O-heavy proxying, (2) enable keep-alive and TLS session tickets, (3) raise system connection-related sysctls, and (4) monitor accept-queue drops and event-loop lag as early warning indicators. Future work will examine HTTP/2 and HTTP/3 multiplexing effects at similar connection densities.

Acknowledgments

The authors thank the Open Network Laboratory operations team for providing the test infrastructure and the anonymous reviewers for constructive feedback.

References