I am consuming an API who's response for a particular field is sometimes and object and sometimes and array of object.
I created a struct to unmarshall the json response and it works great. However, in the instances where the json response has an array of objects, obviously the unmarshalling fails. How can I deal with this situation in Go?
Single Response:
{
"net": {
"comment": {
"line": {
"$": "This space is statically assigned",
"#number": "0"
}
}
}
}
Array Response:
{
"net": {
"comment": {
"line": [
{
"$": "All abuse issues will only be responded to by the Abuse",
"#number": "0"
},
{
"$": "Team through the contact info found on handle ABUSE223-ARIN",
"#number": "1"
}
]
}
}
}
I thought about creating 2 versions of the struct and then somehow determining which instance I got back, but this feels quite wasteful. I have also tried unmarshalling into map[string]instance{} but I got a bit lost and wasn't sure if I was headed down the right path.
Any advice would be appreciated.
Have you tried unmarshall into map[string]interface{}?
type Net struct{
Comment map[string]interface{} `json:"comment"`
}
Then Comment["line"] value is possible array or object.
Related
supposing the json body returned from a call contains some dynamic keys ie
{
"message": "search results matching criteria",
"permission": {
"261ef70e-0a95-4967-b078-81e657e32699": {
"device": {
"read:own": [
"*"
]
},
"account": {
"read:own": [
"*"
]
},
"user": {
"read:own": [
"*"
]
}
}
}
I can validate the json as follows easily enough although I am having a lot of trouble working out how to validate the objects BELOW the dynamic guid level of the response.
pm.test("response body to have correct items", function () {
pm.expect(jsonData.message).to.eq("search results matching criteria");
pm.expect(jsonData).to.have.property('permission');
pm.expect(jsonData.permission).to.have.property(pm.variables.get("otherUserId"));
});
Would ideally like to verify the device and account and user levels of the object.
Anyone with some tips?
I've tried a few ways to try and reference the otherUserId variable but nothing is working. It is either not resolving the variable therefore failing the test as its looking for a level in the json called otherUserId or it fails to run the test due to a syntax error.
This works:
pm.expect(jsonData.permission[pm.variables.get("otherUserId")]).to.have.property('device');
I have following flow in NIFI , JSON has (1000+) objects in it.
invokeHTTP->SPLIT JSON->putMongo
Flow works fine, till I receive some keys in json with "." in the name. e.g. "spark.databricks.acl.dfAclsEnabled".
my current solution is not optimal, I have jotted down bad keys, and using multiple replace text processor to replace "." with "_". I am not using REGEX, I am using string literal find/replace. So each time I am getting failure in putMongo processor, I am inserting new replaceText processor.
This is not maintainable. I am wondering if I can use JOLT for this? couple of info regarding input JSON.
1) no set structure, only thing that is confirmed is. everything will be in events array. But event object itself is free form.
2) maximum list size = 1000.
3) 3rd party JSON, so I cant ask for change in format.
Also, key with ".", can appear anywhere. So I am looking for JOLT spec that can cleanse at all level and then rename it.
{
"events": [
{
"cluster_id": "0717-035521-puny598",
"timestamp": 1531896847915,
"type": "EDITED",
"details": {
"previous_attributes": {
"cluster_name": "Kylo",
"spark_version": "4.1.x-scala2.11",
"spark_conf": {
"spark.databricks.acl.dfAclsEnabled": "true",
"spark.databricks.repl.allowedLanguages": "python,sql"
},
"node_type_id": "Standard_DS3_v2",
"driver_node_type_id": "Standard_DS3_v2",
"autotermination_minutes": 10,
"enable_elastic_disk": true,
"cluster_source": "UI"
},
"attributes": {
"cluster_name": "Kylo",
"spark_version": "4.1.x-scala2.11",
"node_type_id": "Standard_DS3_v2",
"driver_node_type_id": "Standard_DS3_v2",
"autotermination_minutes": 10,
"enable_elastic_disk": true,
"cluster_source": "UI"
},
"previous_cluster_size": {
"autoscale": {
"min_workers": 1,
"max_workers": 8
}
},
"cluster_size": {
"autoscale": {
"min_workers": 1,
"max_workers": 8
}
},
"user": ""
}
},
{
"cluster_id": "0717-035521-puny598",
"timestamp": 1535540053785,
"type": "TERMINATING",
"details": {
"reason": {
"code": "INACTIVITY",
"parameters": {
"inactivity_duration_min": "15"
}
}
}
},
{
"cluster_id": "0717-035521-puny598",
"timestamp": 1535537117300,
"type": "EXPANDED_DISK",
"details": {
"previous_disk_size": 29454626816,
"disk_size": 136828809216,
"free_space": 17151311872,
"instance_id": "6cea5c332af94d7f85aff23e5d8cea37"
}
}
]
}
I created a template using ReplaceText and RouteOnContent to perform this task. The loop is required because the regex only replaces the first . in the JSON key on each pass. You might be able to refine this to perform all substitutions in a single pass, but after fuzzing the regex with the look-ahead and look-behind groups for a few minutes, re-routing was faster. I verified this works with the JSON you provided, and also JSON with the keys and values on different lines (: on either):
...
"spark_conf": {
"spark.databricks.acl.dfAclsEnabled":
"true",
"spark.databricks.repl.allowedLanguages"
: "python,sql"
},
...
You could also use an ExecuteScript processor with Groovy to ingest the JSON, quickly filter all JSON keys that contain ., perform a collect operation to do the replacement, and re-insert the keys in the JSON data if you want a single processor to do this in a single pass.
I have an object definition in common.json file that I need to use in number of other JSON files in terms of reusability. Is there any way to include my common.json file into other JSON files?
Edit:
I came across JSON Pointer while searching which made me thought JSON alone can handle it. To be more clear:
common.json
{
"common":
{
"course":
{
"type": "object",
"properties":
{
"course_name": { "type": "string" },
"course_id": { "type": "integer" },
"course_room": { "type": "integer" }
}
}
}
}
other.json
{
"weekly_schedule":
{
"mathematics": { "$ref": "common.json#/course" },
"history": { "$ref": "common.json#/course" }
}
}
What I understand from here is I can refer to a common JSON object from elsewhere using its path and the $ref keyword. Is that correct or am I missing some point?
JSON is a very simple metaformat. If you take a look at its specification, you will find how simple it is. In particular, it doesn't define any means of aggregation, namespaces, schemata like they are available in XML.
If you want to manipulate JSON or compose different JSON-files, you either treat them as a whole (i.e. as text) and then apply text tools or you decode them, manipulate the received data and then encode the results again.
No, JSON is just text. It doesn't do anything on it's own.
I'm trying to assemble some complex JSON like the following dynamically:
{
"data": {
"attributes": {
"duration_sec": 200
},
"relationships": {
"address": {
"data": {
"id": 1,
"type": "addresses"
}
}
}
},
"included": [
{
"type": "addresses",
"id": null,
"attributes": {
"zip_code": "90210"
}
}
]
}
That is, I have an array of Address structs that I need to loop through to assemble the relationships and included. I would prefer if this were a dictionary, but I could also use SwiftyJSON to just assemble JSON object instead.
When I tried doing this myself, I kept having issues with ambiguity from the compiler, so I honestly don't know how to tackle this problem. I come from dynamic programming languages so am feeling just generally flummoxed by this.
Hard to answer that one without exact compiler error, but usually errors about ambiguity come from the fact that you're trying to create a heterogeneous dictionary and compiler is not able to infer it's type.
In case you want to create a json object like the one above and have use JSON(dictionary:) initializer you should declare a type of dictionary manually as [String: AnyObject]
example:
let exampleDict: [String: AnyObject] = ["one" : 1, "two": "two", "three": [1, 2, 3]]
I have the following JSON;
{
"b2c": {
"languages": {
"de": {
"models": {
"t300": {
"name": "Aveo",
"bodyTypes": {
"t300-4d-my13": {
"trimLevels": {
"lt": {
"name": "LT",
"variants": {
"1.2_16V_86_Gas_MT": {
"name": "1.2 MT",
"price": {
"EUR": {
"value": 13990,
"formatted": "13.990,00 €"
}
},
"infoFeatures": {
"fuel_consumption_extra_urban#consumption": {
"name": "Kraftstoffverbrauch außerorts ",
"value": "4.6",
"formatted": "4,6"
},
"top_speed#kilometer_per_hour": {
"name": "Höchstgeschwindigkeit",
"value": "171",
"formatted": "171"
}
},
"images": null,
"documents": null
}
}
}
}
}
}
}
}
}
}
}
}
The values of b2c, de, t300, t300-4d-my13, It etc.. are dynamic but languages, models, bodyTypes, trimLevels, variants, inforFeatures, images and documents would remain same.
I need to extract all to access values like languages.["de"], models.["t300"].name, timeLevels.["It"], Variants and infoFeatures, as these keys [""] are dynamics so I am not sure what to refer.
I have tried,
var jsonSerializer = new JsonSerializer();
dynamic dynamicObject = jsonSerializer.Deserialize(new JsonTextReader(new StringReader(jsonString)));
//var level1 = dynamicObject.b2c
I have looked this as well
Deserialize JSON into C# dynamic object?
and tried
var dynamicObject = Json.Decode(jsonString);
but receiving following error;
Attempt by method 'System.Web.Helpers.Json.Decode(System.String)' to access field 'System.Web.Helpers.Json._serializer' failed.
For us it helped to uncheck "Enable the Visual Studio hosting process" in the Project properties > Debug tab, from the top answer to Attempt by method 'System.Web.Helpers.Json..cctor()' to access method 'System.Web.Helpers.Json.CreateSerializer()' failed
A general solution would be to use something like Json.net and serialize to C# Object - this is very flexible, and does not conflict with the dynamic nature of the json object coming from the client.
This error seems to occur when you have multiple projects with different versions of assemblies; eg, if you have JSON.NET 4.5.1 in one project and 5.0.6 in another. Things seem to get sorted if you make sure the same versions exist everywhere in the solution.