As I posted last week, I've found a severe problem in the Indy socket code in case you have a two characters separator, as the two characters might end up in two subsequent buffers and not be found by the MemoryPos call over the buffer. Before I get to the big fix, let me comment on the comments. First, at times you don't know the size of the data you are streaming in advanced. At times, rather than building an in memory stream and than copying it to the socket, we built the response right onto the output stream of the socket. Second, parsing the entire data looking for an extremely rare characters and than escaping it (which implies doing the same also on the other hand) is not my favorite approach, unless if an alternative is available.
A quick-and-dirty fix I applied last week was to let it search over the entire buffer received so fast, but this slows down the program considerably (about 20% in some real world tests). I though about checking the last character of the buffer to see it it might be a candidate for a hit (that is the first of the two-characters separator). However, in the end, I decided to leave the extra character in the section of the buffer to be evaluated at the next round. This is the original code:
LTermPos := MemoryPos(ATerminator, PChar(InputBuffer.Memory) + LSize, LInputBufferSize - LSize);
LSize := LInputBufferSize;
where LTermPos is the position of the terminator (if found), LSize is the position of the buffer we have received and checked so far), and LInputBufferSize is the total buffer received. And this is my fix:
LTermPos := MemoryPos(ATerminator, PChar(InputBuffer.Memory) + LSize, LInputBufferSize - LSize);
LSize := LInputBufferSize - 1; // keep the last character, maybe the first of the terminator
It seems to work fine!