Create JSON.NET structure with JTokenWriter - json

Hey all I have the following json output that I would like to create:
{
"scheduleName": "",
"firstName": "",
"lastName": "",
"theRole": "",
"linker": "",
"Schedule": {
"ID": "",
"totalHrs": "",
"Mon": "",
"Tue": "",
"Wed": "",
"Thu": "",
"Fri": "",
"Sat": ""
},
"empInfo": {
"ID": "",
"Email": "",
"Phone": "",
"Active": "",
"Img": "",
"Badge": ""
},
"availability": {
"ID": "",
"Mon": "",
"Tue": "",
"Wed": "",
"Thu": "",
"Fri": "",
"Sat": ""
},
"training": {
"name": "",
"id": ""
}
}
Using the newtonsoft Create JSON with JTokenWriter I am wondering how to create the "Schedule", "empInfo", etc in my json output since there are no examples on the page of those types.
The only example it shows is structured like so:
{
"name1": "value1",
"name2": [
1,
2
]
}
The first few values are easy to create:
Dim jsonWriter As New JTokenWriter()
jsonWriter.WriteStartObject()
jsonWriter.WritePropertyName("scheduleName")
jsonWriter.WriteValue("value1")
jsonWriter.WritePropertyName("firstName")
jsonWriter.WriteValue("value2")
jsonWriter.WritePropertyName("lastName")
jsonWriter.WriteValue("value3")
jsonWriter.WritePropertyName("theRole")
jsonWriter.WriteValue("value4")
jsonWriter.WritePropertyName("linker")
jsonWriter.WriteValue("value5")
'"?": {
' "?": "?",
' "?": "?",
' etc....
'?
jsonWriter.WriteEndObject()
But that's where I have to stop since I do not know how to go about making the other structure.

To write a nested object as the value of a property, write the property name, then do a nested WriteStartObject(), followed by the properties to be written, and finally a nested WriteEndObject(). E.g.:
Dim jsonWriter As New JTokenWriter()
jsonWriter.WriteStartObject() 'Start the root object
jsonWriter.WritePropertyName("scheduleName")
jsonWriter.WriteValue("value1")
jsonWriter.WritePropertyName("Schedule") 'Write the "Schedule" property name
jsonWriter.WriteStartObject() 'Start the nested "Schedule" object
jsonWriter.WritePropertyName("ID")
jsonWriter.WriteValue("ID Value")
jsonWriter.WriteEndObject() 'End the Schedule object
jsonWriter.WriteEndObject() 'End the root object
Sample fiddle.

Related

JQ edit file in place after using JQ Select

to edit a json file inplace with JQ like -i in sed I have found many solutions like
jq ... input.json > tmp.json && mv tmp.json input.json
This works, but I need to filter on my file, add some data, and place it back in the original. (ie. Update a specific object in a file)
My file contains over 1000 objects with different Element. I will always be using a filter on Element, I shortened for question. I have a sample file original.json ...
{
"Element": "acton",
"objectName": "contacts",
"Path": "/contacts",
"Scenario": "",
"serviceLevel": "",
"IBM": "",
"Gap": "",
"clientPhase": "",
"parentPhase": "",
"existsToday": ""
}
{
"Element": "acton",
"objectName": "optouts",
"Path": "/optouts",
"Scenario": "",
"serviceLevel": "",
"IBM": "",
"Dependency": "",
"Gap": "",
"clientPhase": "",
"parentPhase": "",
"existsToday": ""
}
{
"Element": "acton",
"objectName": "subscriptionTypes",
"Path": "/subscription-types",
"Scenario": "",
"serviceLevel": "",
"IBM": "",
"Dependency": "",
"Gap": "",
"clientPhase": "",
"parentPhase": "",
"existsToday": ""
}
I want to filter by objectName for contacts add some data to empty IBM field and save to the file
Applying the same logic to inplace edit the IBM field to "Y" on object containing "objectName": "contacts"
jq 'select(.objectName == "contacts") | .IBM = "Y"' original.json > tmpjson.json && mv tmpjson.json original.json
Now my file original.json shows
{
"Element": "acton",
"objectName": "contacts",
"Path": "/contacts",
"Scenario": "",
"serviceLevel": "",
"IBM": "Y",
"Dependency": "",
"Gap": "",
"clientPhase": "",
"parentPhase": "",
"existsToday": ""
}
Expected Result
{
"Element": "acton",
"objectName": "contacts",
"Path": "/contacts",
"Scenario": "",
"serviceLevel": "",
"IBM": "Y",
"Gap": "",
"clientPhase": "",
"parentPhase": "",
"existsToday": ""
}
{
"Element": "acton",
"objectName": "optouts",
"Path": "/optouts",
"Scenario": "",
"serviceLevel": "",
"IBM": "",
"Dependency": "",
"Gap": "",
"clientPhase": "",
"parentPhase": "",
"existsToday": ""
}
{
"Element": "acton",
"objectName": "subscriptionTypes",
"Path": "/subscription-types",
"Scenario": "",
"serviceLevel": "",
"IBM": "",
"Dependency": "",
"Gap": "",
"clientPhase": "",
"parentPhase": "",
"existsToday": ""
}
It seems after using select I can no longer use the provided solution https://github.com/stedolan/jq/wiki/FAQ#general-questions
Your jq filter is not the one you need because select selects. That is, it filters out the objects that don't match the selection criterion.
This jq filter should do what you want:
if (.objectName == "contacts") then .IBM = "Y" else . end
As for overwriting the file, many people who have access to sponge (part of the moreutils collection of CLI utilities) use it, e.g.
jq 'if (.objectName == "contacts") then .IBM = "Y" else . end' input.json |
sponge input.json

How to format json output in pyspark?

I am having a trouble to preserve the order of my json and pretty printing it in pyspark.
Below is sample code:
json_out = sqlContext.jsonRDD(sc.parallelize([json.dumps(info)]))
# here info is my ordered dictionary
json_out.toJSON().saveAsTextFile("file:///home//XXX//samplejson")
One more thing is that I want my output as single file and not as partitioned datasets.
Could anyone help in pretty printing and preserving the order of output json in my case?
info sample:
Note:TypeA,TypeB etc is a list meaning there can be more than one product in TypeA or TypeB.
{
"score": {
"right": ,
"wrong":
},
"articles": {
"TypeA": [{
"ID": 333,
"Name": "",
"S1": "",
"S2": "",
"S3": "",
"S4": ""
}],
"TypeB": [{
"ID": 123,
"Name": "",
"T1": "",
"T2": "",
"T3": "",
"T4": "",
"T5": "",
"T6": ""
}]
}
}
( I have tried using json.dumps(info,indent=2),but of no use.

SoapUI JSON Parsing - Get Parent Node using Child value

I have a JSON response like below from SoapUI.
{
"-1": {
"startDate": "",
"modifiedBy": "",
"endDate": "",
"projectId": 1,
"build": "",
"projectKey": "TEST",
"started": ""
},
"0": {
"startDate": "",
"modifiedBy": "",
"endDate": "",
"projectId": 2,
"build": "",
"projectKey": "BEST",
"started": ""
},
"2": {
"startDate": "",
"modifiedBy": "",
"endDate": "",
"projectId": 3,
"build": "",
"projectKey": "WORST",
"started": ""
}
}
My requirement is, I have to get the value/node "0" which has the projectkey="BEST" using JsonSurpler or Groovy Script Test Step. The projectkey is now under the node "0". Maybe it will be under "10" or "1000" or "-500".
How to get the parent node using a child node value?
There are different ways to achieve that. These are the simplest I can recollect now:
import groovy.json.JsonSlurper
String string = '''{
"-1": {
"startDate": "",
"modifiedBy": "",
"endDate": "",
"projectId": 1,
"build": "",
"projectKey": "TEST",
"started": ""
},
"0":{
"startDate": "",
"modifiedBy": "",
"endDate": "",
"projectId": 2,
"build": "",
"projectKey": "BEST",
"started": ""
},
"2": {
"startDate": "",
"modifiedBy": "",
"endDate": "",
"projectId": 3,
"build": "",
"projectKey": "WORST",
"started": ""
}
}'''
def json = new JsonSlurper().parseText(string)
assert json.find { it.value?.projectKey == 'BEST' }?.key == '0'
assert json.findResult { k, v -> v?.projectKey == 'BEST' ? k : null } == '0'
This will find the first item which satisfies the condition. If you need all the items which satisfy the condition then use findAll or findResults accordingly.

Issues parsing a 1GB json file using JSON.NET

I have gotten an application where the input has been scaled up from 50K location records to 1.1 Million location records.
This has caused serious issues as the entire file was previously de-serialized into a single object.
The size of the object is ~1GB for a production like file with 1.1 Million records.
Due to large object GC issues I want to keep the de-serialized object below the 85K mark.
I'm trying to parse out a single location object at a time and de-serialize it so I can control the number of objects
that get de-serialized and in turn control the size of the object. I'm using the Json.Net libraries to do this.
Below is a sample of the JSON file that I'm receiving as a stream into my application.
{
"Locations": [{
"LocationId": "",
"ParentLocationId": "",
"DisplayFlag": "Y",
"DisplayOptions": "",
"DisplayName": "",
"Address": "",
"SecondaryAddress": "",
"City": "",
"State": "",
"PostalCode": "",
"Country": "",
"Latitude": 40.59485,
"Longitude": -73.96174,
"LatLonQuality": 99,
"BusinessLogoUrl": "",
"BusinessUrl": "",
"DisplayText": "",
"PhoneNumber": "",
"VenueGroup": 7,
"VenueType": 0,
"SubVenue": 0,
"IndoorFlag": "",
"OperatorDefined": "",
"AccessPoints": [{
"AccessPointId": "",
"MACAddress": "",
"DisplayFlag": "",
"DisplayOptions": "",
"Latitude": 40.59485,
"Longitude": -73.96174,
"Status": "Up",
"OperatorDefined": "",
"RoamingGroups": [{
"GroupName": ""
},
{
"GroupName": ""
}],
"Radios": [{
"RadioId": "",
"RadioFrequency": "",
"RadioProtocols": [{
"Protocol": ""
}],
"WifiConnections": [{
"BSSID": "",
"ServiceSets": [{
"SSID": "",
"SSID_Broadcasted": ""
}]
}]
}]
}]
},
{
"LocationId": "",
"ParentLocationId": "",
"DisplayFlag": "Y",
"DisplayOptions": "",
"DisplayName": "",
"Address": "",
"SecondaryAddress": "",
"City": "",
"State": "",
"PostalCode": "",
"Country": "",
"Latitude": 40.59485,
"Longitude": -73.96174,
"LatLonQuality": 99,
"BusinessLogoUrl": "",
"BusinessUrl": "",
"DisplayText": "",
"PhoneNumber": "",
"VenueGroup": 7,
"VenueType": 0,
"SubVenue": 0,
"IndoorFlag": "",
"OperatorDefined": "",
"AccessPoints": [{
"AccessPointId": "",
"MACAddress": "",
"DisplayFlag": "",
"DisplayOptions": "",
"Latitude": 40.59485,
"Longitude": -73.96174,
"Status": "Up",
"OperatorDefined": "",
"RoamingGroups": [{
"GroupName": ""
},
{
"GroupName": ""
}],
"Radios": [{
"RadioId": "",
"RadioFrequency": "",
"RadioProtocols": [{
"Protocol": ""
}],
"WifiConnections": [{
"BSSID": "",
"ServiceSets": [{
"SSID": "",
"SSID_Broadcasted": ""
}]
}]
}]
}]
}]
}
I need to be able to pull out the individual Location objects, so that I would be looking at the following
{
"LocationId": "",
"ParentLocationId": "",
"DisplayFlag": "Y",
"DisplayOptions": "",
"DisplayName": "",
"Address": "",
"SecondaryAddress": "",
"City": "",
"State": "",
"PostalCode": "",
"Country": "",
"Latitude": 40.59485,
"Longitude": -73.96174,
"LatLonQuality": 99,
"BusinessLogoUrl": "",
"BusinessUrl": "",
"DisplayText": "",
"PhoneNumber": "",
"VenueGroup": 7,
"VenueType": 0,
"SubVenue": 0,
"IndoorFlag": "",
"OperatorDefined": "",
"AccessPoints": [{
"AccessPointId": "",
"MACAddress": "",
"DisplayFlag": "",
"DisplayOptions": "",
"Latitude": 40.59485,
"Longitude": -73.96174,
"Status": "Up",
"OperatorDefined": "",
"RoamingGroups": [{
"GroupName": ""
},
{
"GroupName": ""
}],
"Radios": [{
"RadioId": "",
"RadioFrequency": "",
"RadioProtocols": [{
"Protocol": ""
}],
"WifiConnections": [{
"BSSID": "",
"ServiceSets": [{
"SSID": "",
"SSID_Broadcasted": ""
}]
}]
}]
}]
}
I'm trying to use the Json.NET JsonTextReader to accomplish this, however I cannot get the reader to contain an entire location in its buffer, due to the size of the records in the stream the reader initially will have down as far as "RadioProtocols", which is mid way through the object, by the time the stream reaches the end of the object, the reader has discarded the start of the object.
The code I'm using to try to get this functionality to work is
var ser = new JsonSerializer();
using (var reader = new JsonTextReader(new StreamReader(stream)))
{
reader.SupportMultipleContent = true;
while (reader.Read())
{
if (reader.TokenType == JsonToken.StartObject && reader.Depth == 2)
{
do
{
reader.Read();
} while (reader.TokenType != JsonToken.EndObject && reader.Depth == 2);
var singleLocation = ser.Deserialize<Locations>(reader);
}
}
}
Any information on this or an alternative to doing it would be greatly appreciated. As a side note, the way our customers send the information cannot change at this time.
When the reader is positioned at the beginning of the object you want to deserialize (an entry in the Locations array in your case), you can just call ser.Deserialize<T>(reader) and it will work, advancing to the end of the object at that level, and no further. Thus the following should iterate through the Location objects in your file, loading each one separately:
public static IEnumerable<T> DeserializeNestedItems<T>(TextReader textReader)
{
var ser = new JsonSerializer();
using (var reader = new JsonTextReader(textReader))
{
reader.SupportMultipleContent = true;
while (reader.Read())
{
if (reader.TokenType == JsonToken.StartObject && reader.Depth == 2)
{
var item = ser.Deserialize<T>(reader);
yield return item;
}
}
}
}
And an example of use using your test string:
Debug.Assert(DeserializeNestedItems<Location>(new StringReader(json)).Count() == 2); // No assert.
var list = DeserializeNestedItems<Location>(new StringReader(json)).SelectMany(l => l.AccessPoints).Select(a => new { a.Latitude, a.Longitude }).ToList();
Debug.WriteLine(JsonConvert.SerializeObject(list, Formatting.Indented));
Which outputs:
[
{
"Latitude": 40.59485,
"Longitude": -73.96174
},
{
"Latitude": 40.59485,
"Longitude": -73.96174
}
]
Note - the Location class comes from posting your JSON to http://json2csharp.com/.
Thanks for all the help, I've managed to get it doing what I want which is de-serializing individual location objects.
If the item is converted to a JObject it will read in the full object and de-serialize it, this can be looped to get the solution.
This is the code that was settled on
while (reader.Read())
{
if (reader.TokenType == JsonToken.StartObject && reader.Depth == 2)
{
location = JObject.Load(reader).ToObject<Location>();
var lv = new LocationValidator(location, FootprintInfo.OperatorId, FootprintInfo.RoamingGroups, true);
var vr = lv.IsValid();
if (vr.Successful)
{
yield return location;
}
else
{
errors.Add(new Error(elNumber, location.LocationId, vr.Error.Field, vr.Error.Detail));
if (errors.Count >= maxErrors)
{
yield break;
}
}
++elNumber;
}
}

How to Parse this Json using Gson and get the field I want?

{
"ws_result":
[
{
"token": "",
"norm_token": "",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child":
[
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child":
[
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
},
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
}
]
},
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child":
[
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
}
]
}
]
},
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "2",
"child": [ ]
},
{
"token": "",
"norm_token": "",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
}
]
}
Such that some children are empty some is not, and some children contain more children. How do I actually parse this thing and get what I want. I am totally new with Json, and I am trying to use Gson. What I want is to get a value of a token with specific type in the nested Json. Thanks a lot for any help and directions.
I tried use com.google.gson.stream.JsonReader, but ist not working
JsonReader jsonReader = new JsonReader(new StringReader(result));
jsonReader.beginObject();
while(jsonReader.hasNext()){
String field = jsonReader.nextName();
if (field.equals("type")){
System.out.println(jsonReader.nextString());
} else if (field.equals("token")){
System.out.println(jsonReader.nextString());
} else {
jsonReader.skipValue();
}
}
jsonReader.endObject();
Parse your json recursively like this:
http://snipplr.com/view/71742/java-reflection-and-recursive-json-deserializer-using-gson/
private void parse(JsonObject o, PackagingResponse r){
Iterator<Entry<String, JsonElement>> i = o.entrySet().iterator();
while(i.hasNext()){
Entry<String, JsonElement> e = i.next();
JsonElement el = e.getValue();
if(el.isJsonObject())
parse(el.getAsJsonObject(), r);
//......
}
}