Problem

• High load on embedded ActiveMQ system causing some messages to be moved to the DLQ (Dead Letter Queue).
• Message in the DQL has dlqDeliveryFailureCause of "java.lang.Throwable: duplicate paged in from cursor for queue:[queue name]"
• Instance trace shows error message like:

ERROR Failed to page in more queue messages :
java.util.concurrent.CancellationExceptionat 
java.util.concurrent.FutureTask.report(FutureTask.java:121)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.gotToTheStore(AbstractStoreCursor.java:142)
...

Resolution

• Messages can end up in the DLQ for a variety of reasons, this KB only aplies to the error message noted.
• This may be fixed when an updated version of ActiveMQ is included in a future API Gateway version.
• To avoid the issue for now, and only if you are having the error noted above, please disable concurrentStoreAndDispatchQueues.

For each instance in a group that hosts embedded ActiveMQ, edit your groups/group-x/instance-y/conf/envSettings.props file. Add a line like the following and restart:

env.BROKER.persistence.concurrentStoreAndDispatchQueues=false

Note, this issue may only apply to API Gateway version using ActiveMQ version 5.15.x. Check jar versions in your apigateway/system/lib/embeddedAMQ folder.