Extract value from nested json in terraform - json

In terraform I have an external data source
data "external" "example" {
program = [
"some_program_generating_json"
]
}
some_program_generating_json produces the following output:
{
"dict1": {
"key1": "value1"
},
"dict2": {
"key1": "value2"
}
}
How can I extract the value of [dict1][key1] from that data source and assign it to some local?
lets say:
locals {
extracted_value = ???
}
Thanks.

I tested it, and had no problems using [dict1][key1] notation. This is the example I used.
script file (test.sh)
#!/usr/bin/bash
# from https://github.com/hashicorp/terraform/issues/13991#issuecomment-526869879
printf '{"base64_encoded":"%s"}\n' $(echo '{"dict1": {"key1": "value1"}, "dict2": {"key1": "value2"}}' | base64 -w 0)
main.tf
data "external" "example" {
program = [
"${path.module}/scripts/test.sh"
]
}
locals {
json_value = jsondecode(base64decode(data.external.example.result["base64_encoded"]))
dict1_key1 = local.json_value["dict1"]["key1"]
}
output "result" {
value = local.dict1_key1
}
The output was:
result = value1

Seems like I figured it out. I had to convert values of dict1 and dict2 to strings:
{
"dict1": "{\"key1\": \"value1\"}",
"dict2": "{\"key1\": \"value2\"}"
}
and then use jsondecode on them. I.e.
locals {
key = "dict1"
extracted_dict = jsondecode("${data.external.example.result[local.key]}")
extracted_value = local.extracted_dict["key1"]
}

Related

Add item to JSON array using JsonPatchDocument

I am trying to add an element to a JSON array using Microsoft's JsonPatch implementation in .NET 6:
JSON input:
{ "foo": [ 1 ] }
Expected JSON output:
{ "foo": [ 1, 2 ] }
Following their documentation, I ended up with the following code:
string input = #"{ ""foo"": [ 1 ] }";
dynamic obj = JsonSerializer.Deserialize<ExpandoObject>(input);
var patch = new JsonPatchDocument();
patch.Add("/foo/-", 2);
string output = JsonSerializer.Serialize(obj);
Console.WriteLine(output); // throws JsonPatchException, expected { "foo": [ 1, 2 ] }
I expect the foo property of my object to contain an array equal to [1, 2], but instead it fails with the following error:
Microsoft.AspNetCore.JsonPatch.Exceptions.JsonPatchException: The target location specified by path segment '-' was not found.
A Replace operation on the foo property successfully updates the ExpandoObject, but the Add operation fails. Am I missing something obvious?
I also tried using JsonNode instead of ExpandoObject to no avail (JsonNode obj = JsonSerializer.Deserialize<JsonNode>(input);). The code throws the same error.
In the meantime, as a workaround, I am using JsonPatch.Net. The code looks similar:
string input = #"{ ""foo"": [ 1 ] }";
JsonNode obj = JsonSerializer.Deserialize<JsonNode>(input);
var patch = new JsonPatch(PatchOperation.Add(JsonPointer.Parse("/foo/-"), 2));
PatchResult patchResult = patch.Apply(obj);
string output = JsonSerializer.Serialize(patchResult.Result);
Console.WriteLine(output); // { "foo": [ 1, 2 ] }

How to provide keys with special char in karate.filterKeys?

Response:
[
{
"key1":78,
"key2":"value2",
"key3": {
"obj.key1":"objValue1",
"obj.key2":1236527454,
}
},
{
"key1":89,
"key2":"value2",
"key3": {
"obj.key1":"objValue1",
"obj.key2":9885546755,
}
}
]
Expected Response:
[
{
"key1":32,
"key2":"value2",
"key3": {
"obj.key1":"objValue1",
"obj.key2":8985436,
}
},
{
"key1":36,
"key2":"value2",
"key3": {
"obj.key1":"objValue1",
"obj.key2":655431,
}
}
]
key1, obj.key2 are dynamic values that need to be ignored while comparing the rest of the JSON.
* match karate.filterKeys(response,['key1']) == karate.filterKeys(expectedResponse,['key1'])
It works as expected and ignores key1 while comparing results.
* match karate.filterKeys(response,['key1','key3["obj.key2"]') == karate.filterKeys(expectedResponse,['key1','key3["obj.key2"]'])
The above throws error. I also tried using:
* match karate.filterKeys(response,['key1',key3['obj.key2']) == karate.filterKeys(expectedResponse,['key1',key3['obj.key2']])
As that's normally how special char keys are used in karate, but again getting error.
Can someone please provide a way around for this problem?
I recommend a different approach since you have a complicated nesting and JSON that has horrible keys. Use JSON transforms to get the data in the "shape" you want, and then things become easy.
* def fun =
"""
function(x) {
var y = {};
y.key2 = x.key2;
y.key3 = {};
y.key3['obj.key1'] = x.key3['obj.key1'];
return y;
}
"""
* def temp = karate.map(response, fun)
* match each temp == { key2: 'value2', key3: { 'obj.key1': 'objValue1' } }

How to convert a json into base64 string in jsonnet

I have a json file. I want to encode into base64 string and add it in my main json using jsonnet.
datasources.json:
{
"datasources": [{
"id": 1,
"orgId": 1,
"name": "prometheus"
}]
}
grafana.jsonnet:
local getDataSources() = {
'datasources': (import 'datasources.json').datasources,
};
local kp = {
dashboardDatasources+: {
data: std.base64(getDataSources().datasources),
}
}
kp
Please suggest how to get this done. Struggling to convert json into a string in jsonnet.
This can be done using std.manifestJsonEx
std.manifestJsonEx(value, indent) Convert the given object to a JSON
form. indent is a string containing one or more whitespaces that are
used for indentation:
Here is the solution:
local getDataSources() = {
'datasources': (import 'datasources.json').datasources,
};
local dataSources = getDataSources().datasources;
local kp = {
dashboardDatasources+: {
data: std.base64(std.manifestJsonEx(dataSources, " ")),
}
}
kp

Scala JSON to DenseMatrix

I am trying to convert this JSON response:
[
{
"metric":"energy.salted",
"tags":{
"sensor":"0",
"unit":"101"
},
"aggregateTags":[
],
"dps":{
"1477958400":1.9519165754318237,
"1477958401":-0.4030894637107849,
"1477958402":0.6892277598381042,
"1477958403":3.0232467651367188,
"1477958404":0.07003471255302429,
"1477958405":2.305912971496582
}
},
{
"metric":"energy.salted",
"tags":{
"sensor":"1",
"unit":"101"
},
"aggregateTags":[
],
"dps":{
"1477958400":4.979776382446289,
"1477958401":1.8036608695983887,
"1477958402":3.0569913387298584,
"1477958403":0.8318896889686584,
"1477958404":-0.3151852488517761,
"1477958405":2.563884735107422
}
}
]
Into a DenseMatrix of the values associated with the "dps" keys. The final DenseMatrix should look like this:
[
[1.9519165754318237, -0.4030894637107849, 0.6892277598381042, 3.0232467651367188, 0.07003471255302429, 2.305912971496582],
[4.979776382446289, 1.8036608695983887, 3.0569913387298584, 0.8318896889686584, -0.3151852488517761, 2.563884735107422]
]
I have parsed the JSON using play.api.libs.json.JSON, and I can even extract the "dps" values using:
jsonResponse = Json.parse(response.body)
for (i <- 0 until numSensors) {
dataMap = (jsonResponse.apply(i) \ "dps")
}
But how do I convert it to a dense matrix with the above format?
Assume you have same keys for all dps, you can try this, create an empty DenseMatrix and bind each array to the matrix with DenseMatrix.vertcat:
val keys = List("1477958400", "1477958401", "1477958402", "1477958403", "1477958404", "1477958405")
var mat = new DenseMatrix[Double](0, 6, Array.empty[Double])
for (i <- 0 until numSensors) {
val dataMap = (jsonResponse.apply(i) \ "dps")
val array = keys.map(key => (dataMap \ key).as[Double]).toArray
mat = DenseMatrix.vertcat(mat, new DenseMatrix[Double](1, 6, array))
}
mat
//breeze.linalg.DenseMatrix[Double] = 1.9519165754318237 -0.4030894637107849 0.6892277598381042 ... (6 total)
// 4.979776382446289 1.8036608695983887 3.0569913387298584 ...

Parsing nested arrays in JSON data swift

I'm currently working with trying to extract bits of information from a complicated json based database. After NSJSONSerialization.JSONObjectWithData I get output like follows (some returns added for clarity)
[
"title": Recorder Suite In A Minor - Viola Concerto - Tafelmusik,
"estimated_weight": 85,
"year": 0,
"thumb": ,
"identifiers": <__NSArrayI 0x600000089970>(
{
description = Text;
type = Barcode;
value = 4891030501560;
},
{
description = Printed;
type = Barcode;
value = "4 891030 501560";
},
{
type = ASIN;
value = B0000013L9;
},
{
type = "Mould SID Code";
value = "ifpi 8412";
},
{
type = "Matrix / Runout";
value = "CD PLANT AB 8550156 CDM01";
},
{
description = "SPARS Code";
type = Other;
value = DDD;
},
{
type = "Label Code";
value = "LC 9158";
}
),
"id": 885370,
"date_changed": 2014-06-17T03:53:03-07:00,
"master_url": https://api.discogs.com/masters/495830,
etc … ]
In particular, I need to know how to get the information out of the nested array. Note that the array is not (obviously) a nested dictionary - given the equal signs and the repeated keys. Any help with how to parse this would be appreciated.
I would use a Pod like SwiftyJSON.
First, you need to install CocoaPods, and then go for SwiftyJSON.
I would parse nested arrays in the following manner:
let json = JSON(data: dataFromNetworking)
if let items = json["items"].array {
for item in items {
if let title = item["title"].string {
println(title)
}
}
}
Check out the documentation and Usage section of SwiftyJSON for more info.
Cheers...