Does SuperObject have UTF-8 support - json

Question

I have been using superobject for all my json parsing needs and today I ran into a bit of a problem that I cannot seem to fix. I downloaded a json file that had an entry in it that looked like this: "place" : "café"and when I tried to parse the file and show it in a messagebox the word café turned out like this: café which tells me that the there is some kind of conversion failure going on when the file was parsed using superobject so before I invest any more time in this library, I would like to know if it supports UTF-8 and if so, how would I go about enabling it.
BTW, The pseudo code I am using to parse the file looks something like this:
uses
SuperObject
...
const
jsonstr = '{ "Place" : "café" }';
...
var
SupOB : ISuperObject;
begin
SupOB := SO(jsonstr);
ShowMessage(SupOB['Place'].AsString);
end;
Is the conversion failing because I am casting the object as a string? I tried also using AsJsonto see if that would have any effect, but it did not so I am not sure what is needed to make objects like these display as they are intended and would appreciate some help. Finally, I have checked and verified that the original file that is being parsed is indeed encoded as UTF-8.

Answer 1

You say you are parsing a file, but your example is parsing a string. That makes a big difference, because if you are reading file data into a string first, you are likely not reading the file data correctly. Remember that Delphi strings use UTF-16 in Delphi 2009 and later, but use ANSI in earlier versions. Either way, not UTF-8. So if your input file is UTF-8 encoded, you must decode its data to the proper string encoding before you can then parse it. café is the UTF-8 encoded form of café being mis-interpreted as ANSI.

Answer 2

Reading and writing files json encoded utf8. Tested on Delphi 2007.
function ReadSO(const aFileName: string): ISuperObject;
var
input: TFileStream;
output: TStringStream;
begin
input := TFileStream.Create(aFileName, fmOpenRead, fmShareDenyWrite);
try
output := TStringStream.Create('');
try
output.CopyFrom(input, input.Size);
Result := TSuperObject.ParseString(PWideChar(UTF8ToUTF16(output.DataString)), true, true);
finally
output.Free;
end;
finally
input.Free;
end;
end;
procedure WriteSO(const aFileName: string; o: ISuperObject);
var
output: TFileStream;
input: TStringStream;
begin
input := TStringStream.Create(UTF16ToUTF8(o.AsJSon(true)));
try
output := TFileStream.Create(aFileName, fmOpenWrite or fmCreate, fmShareDenyWrite);
try
output.CopyFrom(input, input.Size);
finally
output.Free;
end;
finally
input.Free;
end;
end;
Functions UTF8ToUTF16 and UTF16ToUTF8 from unit JclConversions http://sourceforge.net/projects/jcl/.