KB Article #176785
splitting a message into many messages causes memory / performance issues
Problem
-- a map / MBC create a several thousands of messages-- getting memory errors ater some time :
FATAL :20130813:05.47.17.72:procengine(procengine(hierchmsgenv)):memory allocation error
WARNING:20130813:05.47.17.74:procengine:A fatal error has ocurred in the interpreter, a dump of active coroutines has been written to the file procengine_fatal.dmp.
WARNING:20130813:05.47.18.63:procengine:A dump of interpreter variables has been written to the file procengine_fatal_vars.dmp.
or
FATAL :20141028:16.48.37.55:procengine(procengine(hierchmsgenv)):memory allocation error file "l:\\/src/4edi/interpreter/interp.c", line 30148
WARNING:20141028:16.48.43.16:procengine:A fatal error has ocurred in the interpreter, a dump of active coroutines has been written to the file procengine_fatal.dmp.
WARNING:20141028:16.49.33.53:procengine:A dump of interpreter variables has been written to the file procengine_fatal_vars.dmp
or
ERROR :20131210:04.09.30.24:procengine(procengine(transgateway)):../../transferlogger.s4:448:failed to write entry to logger server: the log entry is too big
-- if there is enough memory, there is no error, but the performance is very low (several hours to process message with several thousands of messages in output)
Resolution
* the problem is due to the number of simultaneous log commits done from HME.Our solution is to use CORE_MAX_LOG_COMMITS environment variable in order to limit the number of simultaneous log commits
1- set the environment variable CORE_MAX_LOG_COMMITS in $CORE_LOCAL/config/environment.dat
set it first to 100 (if the error still occurs, you may have to increase the value, depending on the number and size of the ouput messages)
2- restart Integrator
3- do a force a rebuild of runtime data
This will lower Hierarchical Messaging Task memory usage, allowing a large
number of output messages to be created.