Problem

Your trace files are filled with "peer disconnected unexpectedly" errors like this and you want to know the reason:

ERROR 01/Nov/2016:10:00:04.000 [0000:000000000000000000000000] error handling connection: peer disconnected unexpectedly
ERROR 01/Nov/2016:10:00:06.000 [0000:000000000000000000000000] error handling connection: peer disconnected unexpectedly
ERROR 01/Nov/2016:10:00:08.000 [0000:000000000000000000000000] error handling connection: peer disconnected unexpectedly

Resolution

This essentially means that something connected to your gateway and then just hung up. While any system could, in theory, do this, a regular pattern of evenly spaced errors normally indicates a load balancer ping and is completely harmless. Note how the timestamps of the sample are all two seconds apart.

Best practice is to point your load balancer at a healthcheck policy instead of simply checking that the port is open. For more information on that, refer to the section on Zero Downtime Settings in the Administrator's Guide relevant to your version.

Finding out which system is responsible for these pings may require performing a packet capture to see what IP is responsible for a large number of reset (RST) packets to the API Gateway port reporting the problem. Load balancer connections where they're only testing if a port is open normally show a pattern where following the TCP stream shows the remote system performing a TCP handshake, then sending an RST without any packets containing application data.