Linux optimization for processing 1.2 million JSON queries per second

Posted Detailed Linux Access Tuning Guide To achieve maximum HTTP processing performance -Crums. The proposed methods allowed to raise the JSON handler performance based on the libreactor environment surrounded by Amazon EC2 (4 VCPU) C 224 thousand API requests per second at regular Amazon Linux 2 settings with a kernel 4.14 to 1.2 million requests per second after optimization (increase of 436%), and also led to a reduction in delays in the processing of queries by 79%. The proposed methods are not specific to libreactor and work when using other HTTP servers, including NGINX, ACTIX, NETTY and NODE.JS (LibreActor used in tests, since the solution based on it showed better performance).

Basic optimization:

• Libreactor Optimization. As the basis, the option R18 from the Techempower set, which was Repair By Removal code to limit the number of CPU kernels used (optimization made it possible to speed up the work by 25-27%), assembling in GCC with options” -O3 “(increase 5-10%) and” -March- Native “(5-10%), replacing READ / WRITE calls on RecV / Send (5-10%) and reduce overhead with PTHREADS (2-3%). The overall performance increase after the code optimization was 55%, and the bandwidth increased from 224K REQ / S to 347K REQ / S.
• Disable protection against vulnerabilities caused by speculative instructions. Using the parameters when loading the kernel “NOSPECTRE_V1 NOSPECTRE_V2 PTI = OFF MDS = OFF TSX_ASYNC_ABORT = OFF” made it possible to raise performance by 28%, and the bandwidth increased from 347K REQ / S to 446K REQ / S. Separately the increase from the parameter “nospectre_v1” (Protection against Spectre V1 + SwAPGS) was 1-2%, “nospectre_v2” (Protection against Spectre V2) – 15-20%, “PTI = OFF” (Spectre V3 / Meltdown) – 6 %, “MDS = OFF TSX_ASYNC_ABORT = OFF” (MDS / Zombied and TSX ASYNCHRONOUS ABORT) – 6%. Left without changing settings to protect against L1TF / Foreshadow attacks (L1TF = Flush), ITLB Multihit, Speculative Store BYPASS and SRBDS, which did not affect performance, because they did not cross the configuration test (for example, specific for KVM, nested virtualization and other CPU models).