This question already has answers here:
Additional text encountered after finished reading JSON content:
(2 answers)
Closed 4 years ago.
I am trying to use Json.NET to parse a file of comma-separated JSON objects:
{
JSON ...
},
{
JSON ...
},
{
JSON ...
}
The code below works fine if the stream contains no separators (i.e., commas above removed). However, commas produce an infinite loop where Json.NET keeps reading an "Undefined" token even after end of file was reached:
using (StreamReader fReader = File.OpenText("filename.json"))
using (JsonTextReader jReader = new JsonTextReader(fReader))
{
jReader.SupportMultipleContent = true;
while (jReader.Read())
{
var jToken = JToken.ReadFrom(jReader);
if (jToken is JObject)
Console.WriteLine("JSON object: " + ((JObject)jToken).ToString());
}
}
I tried skipping the comma by reading ahead and using JsonTextReader's Skip() method, but this doesn't work: JsonTextReader apparently buffers ahead, eating up the comma, which gives it indigestion.
It's hard to believe that I'd be the first to run into this problem, but despite searching here for a good bit, I haven't found any relevant posts (at least for C# and Json.NET). Is it really necessary to hack this up from scratch?
ETA: As per Brian Rogers' comment below, Json.NET 11.0.1 and above handle comma-separated JSON, so the above works fine now, commas or not.
One way to approach this would be make it a list. For example, you can add "[" "]" to either end of "comma separated Json" to convert it to a list and then deserialize it.
For example, consider the following code which has a Json Objects separated by Comma (for sake of simplicity, have created a simple json, but could work otherwise too)
var jsonString = #"{User:'anu viswan', Location:'India'},{User:'User2', Location:'India'}";
If you add "[" "]" to either end, and then serialize it, you could get a collection of RootObjects.
var result = JsonConvert.DeserializeObject<RootObject[]>($"[{jsonString}]");
In this particular case, RootObject is defined as
public class RootObject
{
public string User { get; set; }
public string Location { get; set; }
}
You can similarly convert the Json to a collection based on your Object definition.
Related
I´m making an application with dart / flutter and I need to keep the escape keys from the json server api response, how can I do that?
Example:
JSON reponse:
{"\"\"example\"\"": "value\'"}
Goal (after .fromJson and json.decode):
Class.example = "\"\"example\"\"";
Class.value = "value\'";
If you do jsonDecode(r'{"\"\"example\"\"": "value\'"}'), then you get a map with the key ""example"" and the value value' (the quotes are part of the string content here, not delimiters).
The backslashes are removed, which is to be expected because that's what the JSON specification says should happen. The escaped quotes are retained. The key does not become just example.
If you want to retain the backslashes too, then you don't actually want JSON decoding, because JSON decoding treats \" inside a string literal as introducing the character ".
If you wanted a string containing \"\"example\"\" from JSON decoding, then the original source needed to be "\\\"\\\"example\\\"\\\"".
So, if the sender of the JSON wanted you to see backslashes, they should have included and escaped those backslashes too.
If you JSON encode the string back, you get the backslashes again (at least for "), since jsonEncode('""example""') is "\"\"example\"\"".
If you really want access to the source code of the JSON input, I'd use a non-standard JSON parser, one which allows you to access the source instead of only the parsed value.
Example:
import "package:jsontool/jsontool.dart";
/// Parses [source] as JSON, but retains source escapes in strings.
Object? readJsonSourceStrings(String source) {
return _readJson(JsonReader.fromString(source));
}
Object? _readJson(JsonReader<StringSlice> reader) {
if (reader.tryObject()) {
var map = <String, Object?>{};
while (reader.hasNextKey()) {
var keySlice = reader.nextKeySource()!;
var key = keySlice.substring(1, keySlice.length - 1);
map[key] = _readJson(reader);
}
return map;
}
if (reader.tryArray()) {
var list = <Object?>[];
while (reader.hasNext()) {
list.add(_readJson(reader));
}
return list;
}
if (reader.checkString()) {
var slice = reader.expectAnyValueSource();
return slice.substring(1, slice.length - 1);
}
return reader.tryNum() ?? reader.tryBool() ??
(reader.tryNull() ? null : (throw "Unreachable"));
}
void main() {
var m = readJsonSourceStrings(r'''{"\"\"example\"\"":"value\'"}''');
print(m); // {\"\"example\"\": value\'}
}
(Disclaimer: I wrote the jsontool package.)
I would like to parse json string using JObject.Parse() of NewtonSoft.Json. Assume that the json string is like this:
{"json":"{\"count\":\"123\"}"}
The result of jObject.First.ToString() is "json": "{\"count\":\"123\"}".
The result of jObject["json"].ToString() is {"count":"123"}. Enumerating gets the same result as this.
The testing code I used is like this.
[TestMethod()]
public void JsonParseTest()
{
var json = "{\"json\":\"{\\\"count\\\":\\\"123\\\"}\"}";
var jObject = JObject.Parse(json);
Console.WriteLine($"json : {json}");
Console.WriteLine($"jObject.First.ToString() : {jObject.First}");
Console.WriteLine($"jObject[\"json\"].ToString() : {jObject["json"]}");
}
We can see that enumerating of jObject will lose the character '\'. What is the problem? I would be appreciated for any suggestion :)
EDIT 1
The version of NewtonSoft is 12.0.3 released in 2019.11.09.
The parser isn't loosing anything. There is no literal \ in your example. The backslashes are purely part of the JSON syntax to escape the " inside the string vlue. The value of the key json is {"count":"123"}.
If you want to have backslashes in that value (however I don't see why you would want that), then you need add them, just like you added them in your C# string (C# and JSON happen to have the same escaping mechanism):
{"json":"{\\\"count\\\":\\\"123\\\"}"}
with leads to the C# code:
var json = "{\"json\":\"{\\\\\\\"count\\\\\\\":\\\\\\\"123\\\\\\\"}\"}";
I'm using Flink to process the data coming from some data source (such as Kafka, Pravega etc).
In my case, the data source is Pravega, which provided me a flink connector.
My data source is sending me some JSON data as below:
{"key": "value"}
{"key": "value2"}
{"key": "value3"}
...
...
Here is my piece of code:
PravegaDeserializationSchema<ObjectNode> adapter = new PravegaDeserializationSchema<>(ObjectNode.class, new JavaSerializer<>());
FlinkPravegaReader<ObjectNode> source = FlinkPravegaReader.<ObjectNode>builder()
.withPravegaConfig(pravegaConfig)
.forStream(stream)
.withDeserializationSchema(adapter)
.build();
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<ObjectNode> dataStream = env.addSource(source).name("Pravega Stream");
dataStream.map(new MapFunction<ObjectNode, String>() {
#Override
public String map(ObjectNode node) throws Exception {
return node.toString();
}
})
.keyBy("word") // ERROR
.timeWindow(Time.seconds(10))
.sum("count");
As you see, I used the FlinkPravegaReader and a proper deserializer to get the JSON stream coming from Pravega.
Then I try to transform the JSON data into a String, KeyBy them and count them.
However, I get an error:
The program finished with the following exception:
Field expression must be equal to '*' or '_' for non-composite types.
org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:342)
org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:340)
myflink.StreamingJob.main(StreamingJob.java:114)
It seems that KeyBy threw this exception.
Well, I'm not a Flink expert so I don't know why. I've read the source code of the official example WordCount. In that example, there is a custtom splitter, which is used to split the String data into words.
So I'm thinking if I need to use some kind of splitter in this case too? If so, what kind of splitter should I use? Can you show me an example? If not, why did I get such an error and how to solve it?
I guess you have read the document about how to specify keys
Specify keys
The example codes use keyby("word") because word is a field of POJO type WC.
// some ordinary POJO (Plain old Java Object)
public class WC {
public String word;
public int count;
}
DataStream<WC> words = // [...]
DataStream<WC> wordCounts = words.keyBy("word").window(/*window specification*/);
In your case, you put a map operator before keyBy, and the output of this map operator is a string. So there is obviously no word field in your case. If you actually want to group this string stream, you need to write it like this .keyBy(String::toString)
Or you can even implement a customized keySelector to generate your own key.
Customized Key Selector
Please pardon me if this is a repeat question. I have been through some of the questions/answers with a similar requirement but somehow got a bit overwhelmed and confused at the same time. My requirement is:
I get a JSON string/object as a request parameter. ( eg: params.timesheetJSON )
I then have to parse/iterate through it.
Here is the JSON that my grails controller will be receiving:
{
"loginName":"user1",
"timesheetList":
[
{
"periodBegin":"2014/10/12",
"periodEnd":"2014/10/18",
"timesheetRows":[
{
"task":"Cleaning",
"description":"cleaning description",
"paycode":"payCode1"
},
{
"task":"painting",
"activityDescription":"painting description",
"paycode":"payCode2"
}
]
}
],
"overallStatus":"SUCCESS"
}
Questions:
How can I retrieve the whole JSON string from the request? Does request.JSON be fine here? If so, will request.JSON.timesheetJSON yield me the actual JSON that I want as a JSONObject?
What is the best way to parse through the JSON object that I got from the request? Is it grails.converters.JSON? Or is there any other easy way of parsing through? Like some API which will return the JSON as a collection of objects by automatically taking care of parsing. Or is programatically parsing through the JSON object the only way?
Like I said, please pardon me if the question is sounding vague. Any good references JSON parsing with grails might also be helpful here.
Edit: There's a change in the way I get the JSON string now. I get the JSON string as a request paramter.
String saveJSON // This holds the above JSON string.
def jsonObject = grails.converters.JSON.parse(saveJSON) // No problem here. Returns a JSONObject. I checked the class type.
def jsonArray = jsonArray.timesheetList // No problem here. Returns a JSONArray. I checked the class type.
println "*** Size of jsonArray1: " + jsonArray1.size() // Returns size 1. It seemed fine as the above JSON string had only one timesheet in timesheetList
def object1 = jsonArray[1] // This throws the JSONException, JSONArray[1] not found. I tried jsonArray.getJSONObject(1) and that throws the same exception.
Basically, I am looking to seamlessly iterate through the JSON string now.
I have wrote some code that explains how this can be done, that you can see below, but to be clear, first the answers to your questions:
Your JSON String as you wrote above will be the contents of your POST payload to the rest controller. Grails will use its data binding mechanism to bind the incomming data to a Command object that your should prepare. It has to have fields corresponding to the parameters in your JSON String (see below). After you bind your command object to your actual domain object, you can get all the data you want, by simply operating on fields and lists
The way to parse thru the JSON object is shown in my example below. The incomming request is esentially a nested map, with can be simply accessed with a dot
Now some code that illustrates how to do it.
In your controller create a method that accepts "YourCommand" object as input parameter:
def yourRestServiceMethod (YourCommand comm){
YourClass yourClass = new YourClass()
comm.bindTo(yourClass)
// do something with yourClass
// println yourClass.timeSheetList
}
The command looks like this:
class YourCommand {
String loginName
List<Map> timesheetList = []
String overallStatus
void bindTo(YourClass yourClass){
yourClass.loginName=loginName
yourClass.overallStatus=overallStatus
timesheetList.each { sheet ->
TimeSheet timeSheet = new TimeSheet()
timeSheet.periodBegin = sheet.periodBegin
timeSheet.periodEnd = sheet.periodEnd
sheet.timesheetRows.each { row ->
TimeSheetRow timeSheetRow = new TimeSheetRow()
timeSheetRow.task = row.task
timeSheetRow.description = row.description
timeSheetRow.paycode = row.paycode
timeSheet.timesheetRows.add(timeSheetRow)
}
yourClass.timeSheetList.add(timeSheet)
}
}
}
Its "bindTo" method is the key piece of logic that understands how to get parameters from the incomming request and map it to a regular object. That object is of type "YourClass" and it looks like this:
class YourClass {
String loginName
Collection<TimeSheet> timeSheetList = []
String overallStatus
}
all other classes that are part of that class:
class TimeSheet {
String periodBegin
String periodEnd
Collection<TimeSheetRow> timesheetRows = []
}
and the last one:
class TimeSheetRow {
String task
String description
String paycode
}
Hope this example is clear enough for you and answers your question
Edit: Extending the answer according to the new requirements
Looking at your new code, I see that you probably did some typos when writting that post
def jsonArray = jsonArray.timesheetList
should be:
def jsonArray = jsonObject.timesheetList
but you obviously have it properly in your code since otherwise it would not work, then the same with that line with "println":
jsonArray1.size()
shuold be:
jsonArray.size()
and the essential fix:
def object1 = jsonArray[1]
shuold be
def object1 = jsonArray[0]
your array is of size==1, the indexing starts with 0. // Can it be that easy? ;)
Then "object1" is again a JSONObject, so you can access the fields with a "." or as a map, for example like this:
object1.get('periodEnd')
I see your example contains errors, which lead you to implement more complex JSON parsing solutions.
I rewrite your sample to the working version. (At least now for Grails 3.x)
String saveJSON // This holds the above JSON string.
def jsonObject = grails.converters.JSON.parse(saveJSON)
println jsonObject.timesheetList // output timesheetList structure
println jsonObject.timesheetList[0].timesheetRows[1] // output second element of timesheetRows array: [paycode:payCode2, task:painting, activityDescription:painting description]
I am trying to parse some JSON objects which is made just of (string,string) pairs, in order to emulate Resjson behaviour. The file I am parsing contains this.
{
"greeting":"Hello world",
"_greeting.comment":"Hello comment.",
"_greeting.source":"Original Hello",
}
Please note the last comma is incorrect, and I also used http://jsonlint.com/ to test JSON syntax. It tells me it is incorrect, as I expected. My - slightly modified - code is :
string path = #"d:\resjson\example.resjson";
string jsonText = File.ReadAllText(path);
IDictionary<string, string> dict;
try
{
dict = JsonConvert.DeserializeObject<IDictionary<string, string>>(jsonText);
}
catch(Exception ex)
{
// code never reaches here
}
My above code returns the IDictionary with the 3 keys as if the formatting was correct. If I serialize back, the string obtained is without the last comma.
My questions are :
Is Newtonsoft.Json so permissive that it allows users slight errors ?
If so, can I set the permissiveness so that it is more strict ?
Is there a way to check if a string is valid JSON format, using
Newtonsoft.Json with and/or without the permissiveness?