KB Article #181755

Memory leak due to buildup of javax.crypto.JceSecurity$IdentityWrapper objects

Problem


You have a memory leak and jmap -histo:live <pid> output from an affected instance shows a buildup of the following objects:


java.util.concurrent.ConcurrentHashMap$Node
java.lang.ref.WeakReference
javax.crypto.JceSecurity$IdentityWrapper


Resolution

This leak has been observed as the result of various 3rd party JAR files that are not shipped as part of the API Gateway. The underlying issue is believed to be a result of this JDK bug. There is, however, a method to work-around this problem via a script that patches the affected Java class itself. To use the work-around script:


  • Download the JCE_leak_fix_groovy.txt file attached to this KB from the sidebar.
  • Create a new policy in Policy Studio.
  • Create a Scripting Language filter set to Groovy & paste in the contents of the text file.
  • Right click the filter and "Set as Start"
  • Attach the newly created policy to a path any listener.
  • Edit the path and disable the policy. The fix happens during Init(), so the script only needs to be loaded, not executed.
  • (Optional) Set the trace level to INFO in the FED so you can confirm this has been properly applied.
  • Deploy the new config.


When this has been done, you should see a trace like the following in your gateway at startup, assuming the trace level is set to INFO or higher:

 INFO    01/Jan/2020:00:00:00.000 [...]             Patching java crypto JCE provider


Note that you do not want to actually execute the script, merely having it get loaded from any listener is enough. Actually running the script will cause it to create a bunch of objects to attempt to replicate the leak. The Init() function of the Groovy script is what actually patches the Java JCE object responsible for the leak, so if you want to use it to replicate the leak, you could comment out the Init() function and invoke the script as part of a policy. Because that is not something most people will wish to do, it is recommended that you leave the path this script is loaded from disabled so that no one can invoke the script at all.