Problem

The API Gateway fails to process traffic and errors like the following appear in the trace file:

ERROR 01/Jan/2022:00:00:00.000 [0000:000000000000000000000000] error handling connection: Too many open files. accept failed

Resolution

The API Gateway start-up checks for the presence of the file /proc/sys/fs/nr_open which it uses to set to the max number of open files (ulimit -n). If this file doesn't exist, unlimited is used but this doesn't work with 2.6 kernels so the default (usually 1024) is used. This is likely to produce "too many open file" errors with a high workload because Java uses 1,000+ file descriptors just for JARs, leaving few available for network connections and causing exhaustion under relatively low loads.

Some Red Hat/CentOS/OEL distributions do not have the file /proc/sys/fs/nr_open so a numerically high ulimit -n needs to be set before running the API Gateway to avoid being limited to 1024 file descriptors, e.g. ulimit -n 1048576

Check your OS documentation on how to set ulimit values that persist across reboots if there are any doubts. This error is often confused with the error too many open connections, which has a different solution that is discussed in KB 178360. That error is not related to ulimit settings.