My POST request on TIdHttpServer contains strange characters, json string - json

Question

I have an issue with the JSON string I get when receiving a POST request. Currently this is the way I'm reading it:
procedure TForm1.IdHTTPServer1CommandGet(AContext: TIdContext;
ARequestInfo: TIdHTTPRequestInfo; AResponseInfo: TIdHTTPResponseInfo);
var
Stream : TStream;
S : string;
begin
If ARequestInfo.Command = 'POST' then
begin
Stream := ARequestInfo.PostStream;
if assigned(Stream) then
begin
Stream.Position := 0;
S := UTF8ToAnsi(ReadStringFromStream(Stream));
end;
end;
end;
I tried ReadStringFromStream() alone and with UTF8ToAnsi()and AnsiToUTF8(), but I keep getting a string that looks like this:
'['#$A#9'{'#$A#9#9'"test":"bb",'#$A#9#9'"test":"aa"'#$A#9'}'#$A']'
I know it has something to do with encoding, but I don't know how to fix it.

Answer 1

You do know that the hash (#) sign denotes a character value and that the dollar ($) sign denotes hexadecimal values, do you. Thus #$A means character decimal 10, which happens to mean NewLine and #9 means character 9 which is the TAB character. There is nothing unexpected in the return string. If you feed it into something that understands a NewLine without a preceding CarriageReturn it will probably look as you expected.
The debugger for exmple, uses the #-syntax for characters that cant be otherwise visually represented.

Answer 2

The data you showed in your example is perfectly fine, as explained by Tom B. You are looking at the string data in the debugger, where #A is a line break and #9 is a tab character, so the actual string looks like this:
[
{
"test":"bb",
"test":"aa"
}
]
Which is valid JSON.
However, the way you are reading the data is not OK, especially if you are using a Unicode version of Delphi (2009+). You are not passing any value to the AByteEncoding parameter of ReadStringFromString(), so it will decode the stream bytes using Indy's default encoding, which is 7bit US-ASCII by default (see the GIdDefaultTextEncoding variable in the IdGlobal unit). JSON uses UTF-8 by default, so you will corrupt the JSON if it contains any non-ASCII characters. Using UTF8ToAnsi() after the fact won't fix that.
Your code should look like this instead:
procedure TForm1.IdHTTPServer1CommandGet(AContext: TIdContext; ARequestInfo: TIdHTTPRequestInfo; AResponseInfo: TIdHTTPResponseInfo);
var
Stream : TStream;
S : string;
begin
If ARequestInfo.CommandType = hcPOST then
begin
Stream := ARequestInfo.PostStream;
if Assigned(Stream) then
begin
S := ReadStringFromStream(Stream, -1, IndyTextEncoding_UTF8);
end
end
end;
That tells Indy to decode the stream bytes as UTF-8 to UTF-16, then return the decoded string (if you are using a non-Unicode version of Delphi, the UTF-16 data will be converted to ANSI upon exit, subject to the optional ADestEncoding parameter of ReadStringFromStream()).