Problem

FTP  vs. PeSIT performance differences (speed, time, consistency)

Resolution

First, please note that this, per definition, isn’t actually a Support request but an Audit / tuning intervention. Which Axway can do, of course, but actually isn’t Support.

[FTP]
- Is a streaming protocol (no flow control what so ever)
- Flow control, per definition, does what it says; it controls the flow between sender/receiver, which creates often a stop-start-stop-start situation and, while in « stop », the bandwidth isn’t used.

[PeSIT]
- Not a streaming protocol, therefore no flow control
- By default, uses synchronization points; amounts of data at which the protocol will stop sending, sends an SYN packet. Depending on the window size (= how many of such blocks of data may be « outstanding » without being Acked), the sender will stop sending data all together until the server Acked either one of the outstanding, or the last outstanding, block of data (where if the server ACK.SYN the last, it implicitly acks them all)
- NOTE that PeSIT can only «ack» such data, guaranteeing data integrity, if this received data has been flushed to disk as where FTP doesn't keep track of this and continues on sending data.

By finding a balance in the «start-stop-start-stop» working of PeSIT and system resources that remain «acceptable» PeSIT should be able to get close to 85% of the FTP performance.

To enhance to this performance, please do the following :
- Raising the interval (move 5Kb to 4096Kb for instance) and re-test
- Raising the data buffer size (from 4000 to 8000 to 16000 to 32000 and test each)
- Consider the transfer method ; binary, variable, fixed, variable ? (Limits PeSIT on-the-fly data manipulation)
- Consider transcoding ; if transco is to be made, consider doing this on the fastest machine
- Trace it again, and observe where the slowing factor lies (if it didn’t improve)

Mind, again, that PeSIT negotiates ; it always falls back to the lowest value between the partners. If nothing changed at all, check if both client and server made the suggested modifications (anyway, you can see that from the trace, too)
So, PeSIT will not be and cannot be as fast as FTP, but it wasn’t written for being that neither. On the other hand, it does give a great deal of flexibility and guarantees the integrity of the data transfer (as was requested by banks that wrote the specs) and written data, offers restart/retry mechanisms, backup paths, multi network (X.25, TCP/IP (PHSE)), …

[Special Notice]
Raising the interval between blocks and the number of outstanding blocks to huge values, to minimize such flushing isn't helping because unlike FTP, PeSIT negotiates a transfer; meaning that before the actual data transfer starts, the receiver exactly knows how much data, and how, it will receive.
By «delaying» flushing this received data to disk, how PeSIT works, you force the receiver to keep all this data in memory!
If you send large files, say that a Gateway is made for handling lots of transfers simultaneously; you’ll simply explode your memory …. (as said, PeSIT negotiates, it knows before the transfer starts how much space it needs to reserve on forehand)
So in theory, in such case, say you have 50 parallel transfers of 2 GB files each, the server, even before data transfer started, would in such case reserve 50 * 2 GB in memory, roughly 100 GB …