Problem

A quick restart of an API Gateway instance with an embedded Cassandra server and a JMX listener (-DenableJMX) is failing to restart.

e.g.

# /etc/init.d/vshell-Group1-Server1 restart

API Gateway service is running

Stopping the API Gateway service [ OK ]

Starting the API Gateway service [FAILED]

After 2nd run the service is again started ok.

The same issue can also be seen with startinstance which shows the Cassandra JMX port 7199 is the problem.

./startinstance -n Server1 -g Group1 -k;./startinstance -n Server1 -g Group1 -d -DenableJMX

waiting for process 4232 to finish....done

Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 7199; nested exception is:

java.net.BindException: Address already in use

service failed to start

Anyone scripting with startinstance will hit this and have to put in some check for port 7199.

Resolution

For startinstance -k and init.d scripts wait for the port to become free by executing the following is shell scripts

TIMEOUT=90

i=0

while [ $i -le $TIMEOUT ]; do

netstat -na | awk '{print $4}' | egrep 7199

if [ $? -ne 0 ]; then

break

fi

sleep 5

i=$[$i+5]

done

Modify the port numbers if necessary "(7000|7199|9160)" to include any extra which may be required depending on configuration.