convert a json 2 dimensional array into a table - json

I am having hard time to find how to convert a 2D array (acuired through JSON format) into a table...
My JSON:
[
["index", 1, 2, 3, 4, 5, 6, 7, 8, 9],
["type", "years", "years", "years", "quarters", "quarters", "quarters", "months", "months", "months"],
["period", "2016", "2017", "2018", "Q2", "Q3", "Q4", "Dec", "Jan", "Feb"],
["nb_month", 12, 12, 12, 3, 3, 3, 1, 1, 1],
["MY_scrap", 0, 54, 1529, 325, 445, 532, 120, 193, 131],
["MY_completed", 6, 89, 2895, 394, 722, 1622, 437, 805, 542],
["SV_total", 6, 1806, 36520, 6143, 11772, 15318, 3579, 6407, 5216],
["SV_scrap", 0, 54, 1529, 325, 445, 532, 120, 193, 131],
["SV_other", 6, 1752, 34809, 5808, 11198, 14773, 3449, 6214, 5081],
["WIP", 0, 971, 3214, 1493, 2395, 3214, 3214, 3832, 4445],
["WLS", 0, 178, 650, 390, 475, 650, 650, 709, 902]
]
Each line on the JSON is a column in the table and the 1st "row" in the JSON is the table header.
I start to play around with power query and finally get a beginning of code:
let
Source = Json.Document(Web.Contents("http://localhost/api/test")),
trans1 = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
#"Transposed Table" = Table.Transpose(trans1)
in
#"Transposed Table"
The result is here:
But from here I do not understand how to manage a proper list expand...
Thank you for your help!

There's a M Function that is going to be very useful to you, I belive. Table.FromColumns(lists as list)
let
Source = Json.Document(Web.Contents("http://localhost/api/test")),
TableResult = Table.FromColumns(Source),
#"Promoted Headers" = Table.PromoteHeaders(TableResult, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"index", Int64.Type}, {"type", type text}, {"period", type text}, {"nb_month", Int64.Type}, {"MY_scrap", Int64.Type}, {"MY_completed", Int64.Type}, {"SV_total", Int64.Type}, {"SV_scrap", Int64.Type}, {"SV_other", Int64.Type}, {"WIP", Int64.Type}, {"WLS", Int64.Type}})
in
#"Changed Type"
I think this does exactly what you want.

Related

How do I determine the column type name from the columnType integer value in mysql2?

When querying with mysql2, the third argument to the callback function fields has the following documentation:
console.log(fields); // fields contains extra meta data about results, if available
Ok, great. But when I look at the values in the fields array, I see the following:
[
{
"_buf":{},
"_clientEncoding":"utf8",
"_catalogLength":3,
"_catalogStart":10,
"_schemaLength":0,
"_schemaStart":14,
"_tableLength":11,
"_tableStart":15,
"_orgTableLength":0,
"_orgTableStart":27,
"_orgNameLength":2,
"_orgNameStart":31,
"characterSet":63,
"encoding":"binary",
"name":"id",
"columnLength":11,
"columnType":3, // <-- column type 3? This is an INTEGER field
"flags":1,
"decimals":0
},
{
...
"columnType":253, // <-- column type 253? This is a VARCHAR field
...
}
]
How do I determine what the actual column type is from these integer values?
After doing some more digging, I was able to find the answer by looking at the source. mysql2 exposes a Types field that has the column type names as keys with the column id as values.
At the time of posting, that list looks like this:
{
"DECIMAL": 0,
"TINY": 1,
"SHORT": 2,
"LONG": 3,
"FLOAT": 4,
"DOUBLE": 5,
"NULL": 6,
"TIMESTAMP": 7,
"LONGLONG": 8,
"INT24": 9,
"DATE": 10,
"TIME": 11,
"DATETIME": 12,
"YEAR": 13,
"NEWDATE": 14,
"VARCHAR": 15,
"BIT": 16,
"JSON": 245,
"NEWDECIMAL": 246,
"ENUM": 247,
"SET": 248,
"TINY_BLOB": 249,
"MEDIUM_BLOB": 250,
"LONG_BLOB": 251,
"BLOB": 252,
"VAR_STRING": 253,
"STRING": 254,
"GEOMETRY": 255
}

How to do arithmetic substraction of same identical key values between 2 json files and print it in output json file

I've 2 json files of with identical key fields on both the file. I would like to get your assistance to do arithmetic subtraction of values of same key field between 2 files and present in 3rd output file [ delta of two json files to output json file].
Also 1st and 2nd json files has n of array of indicies so we need to do this in loop to map the difference values in 3rd json output file.
Example:
1st Json file: file1.json
[
{
"name": "Rock",
"pri": 21,
"size": 502173,
"gets": 0,
"searches": 40,
"search_time_ms": 25,
"fetches": 2,
"cache_mem_size": 0,
"cache_size": 0,
"cache_total": 2,
"hits": 0,
"misses": 2,
"index_total": 0,
"index_current": 0,
"merges_total": 0,
"refresh_total": 0
},
{
"name": "Rock:config",
"pri": 21,
"size": 512173,
"gets": 0,
"searches": 40,
"search_time_ms": 25,
"fetches": 2,
"cache_mem_size": 0,
"cache_size": 0,
"cache_total": 2,
"hits": 0,
"misses": 2,
"index_total": 0,
"index_curr": 0,
"merges_tot": 0,
"refresh_tot": 0
}
]
2nd Json file: file1.json:
[
{
"name": "Rock",
"pri": 22,
"size": 602173,
"gets": 0,
"searches": 40,
"search_time_ms": 25,
"fetches": 2,
"cache_mem_size": 0,
"cache_size": 0,
"cache_total": 2,
"hits": 0,
"misses": 2,
"index_total": 0,
"index_current": 0,
"merges_total": 0,
"refresh_total": 0
},
{
"name": "Rock:config",
"pri": 31,
"size": 602173,
"gets": 0,
"searches": 40,
"search_time_ms": 25,
"fetches": 2,
"cache_mem_size": 0,
"cache_size": 0,
"cache_total": 2,
"hits": 0,
"misses": 2,
"index_total": 0,
"index_curr": 0,
"merges_tot": 0,
"refresh_tot": 0
}
]
Output json file: file3.json should look like below.
[
{
"name": "Rock",
"pri": 1,
"size": 100000,
"gets": 0,
"searches": 40,
"search_time_ms": 25,
"fetches": 2,
"cache_mem_size": 0,
"cache_size": 0,
"cache_total": 2,
"hits": 0,
"misses": 2,
"index_total": 0,
"index_current": 0,
"merges_total": 0,
"refresh_total": 0
},
{
"name": "Rock:config",
"pri": 10,
"size": 90000,
"gets": 0,
"searches": 40,
"search_time_ms": 25,
"fetches": 2,
"cache_mem_size": 0,
"cache_size": 0,
"cache_total": 2,
"hits": 0,
"misses": 2,
"index_total": 0,
"index_curr": 0,
"merges_tot": 0,
"refresh_tot": 0
}
]
please help me with the logic
This can be done as simple as u follow the following steps
Read the json object from 2 json files and save them as list of dictionaries
Loop through the list of dicts and perform your required manipulations
Save calculated output dicts to a new list
Write the resultant list to a new json file
the code is as follows,
import json
with open('file1.json') as f:
data1 = json.load(f)
with open('file2.json') as f:
data2 = json.load(f)
data = []
for i,j in zip(data1, data2):
x = {}
for m,n in zip(i.items(), j.items()):
# if m[0] != "name":
if m[0] == "pri" or m[0] == "size":
x[m[0]] = int(j[m[0]]) - int(i[m[0]])
else:
x[m[0]] = i[m[0]]
data.append(x)
with open('file3.json', 'w') as o:
json.dump(data, o)

Use Jest to Test Local JSON File?

I have a analytics application using React + Gatsby (Uses local JSON as data) and I have jest setup, but I can't figure out how to test for the following things:
JSON file exists
JSON file is JSON
Last object in JSON property date_updated
Below is my test that I've been doing trial and error + My JSON file I'm trying to test.
Needs Clarification:
This is my first time doing testing, so the whole mock thing is new to me. Does the JSON file need to be mocked? If so, why?
data.test.js
// Imports: Dependencies
import React from 'react';
import renderer from 'react-test-renderer';
// Imports: JSON File
import orangeCountyCovid19Data from '../data/orangeCountyCovid19Data.json';
// Test: Orange County Data
describe('Data: Orange County', () => {
// Last Day
const lastDay = orangeCountyCovid19Data[orangeCountyCovid19Data.length - 1];
console.log(lastDay)
// Data Should Be JSON
// expect(orangeCountyCovid19Data).toBe(JSON);
// Data Property: date_updated
// expect(lastDay).toHaveProperty('date_updated');
// expect(orangeCountyCovid19Data).toBe(true);
// jest.mock('../data/orangeCountyCovid19Data.json', ()=>({
// settings: 'someSetting'
// }), { virtual: true })
});
orangeCountyCovid19Data.json:
[
{
"date_updated": "06/23/2020",
"number_of_people_tested": null,
"new_tests": null,
"number_of_people_kits_can_test": null,
"cases": {
"new_cases": 147,
"total_cases": 10737,
"total_cases_male": 5383,
"total_cases_female": 5331,
"total_cases_0_to_17": 533,
"total_cases_18_to_24": 1147,
"total_cases_25_to_34": 2074,
"total_cases_35_to_44": 1710,
"total_cases_45_to_54": 1859,
"total_cases_55_to_64": 1505,
"total_cases_65_to_74": 854,
"total_cases_75_to_84": 588,
"total_cases_over_84": 465
},
"cases_by_city": {
"aliso_viejo": 50,
"anaheim": 2060,
"brea": 76,
"buena_park": 334,
"costa_mesa": 231,
"coto_de_caza": 8,
"cypress": 99,
"dana_point": 41,
"fountain_valley": 91,
"fullerton": 416,
"garden_grove": 619,
"huntington_beach": 495,
"irvine": 283,
"la_habra": 214,
"la_palma": 32,
"ladera_ranch": 23,
"laguna_beach": 54,
"laguna_hills": 55,
"laguna_niguel": 60,
"laguna_woods": 15,
"lake_forest": 110,
"los_alamitos": 88,
"midway_city": 31,
"mission_viejo": 119,
"newport_beach": 204,
"orange": 471,
"placentia": 195,
"rancho_mission_viejo": 12,
"rancho_santa_margarita": 43,
"rossmoor": 14,
"san_clemente": 82,
"san_juan_capistrano": 75,
"santa_ana": 2260,
"seal_beach": 92,
"stanton": 140,
"trabuco_canyon": 22,
"tustin": 199,
"villa_park": 13,
"westminster": 227,
"yorba_linda": 126,
"other": 394,
"unknown": 564
},
"deaths": {
"total_deaths": 273,
"total_deaths_male": 155,
"total_deaths_female": 118,
"total_deaths_0_to_17": 0,
"total_deaths_18_to_24": 0,
"total_deaths_25_to_34": 3,
"total_deaths_35_to_44": 7,
"total_deaths_45_to_54": 24,
"total_deaths_55_to_64": 37,
"total_deaths_65_to_74": 48,
"total_deaths_75_to_84": 70,
"total_deaths_over_84": 84
},
"hospitalized": {
"hospitals_reporting": null,
"total_hospitalized": null,
"total_icu": null
},
"travel_related": {
"travel_related": null
},
"person_to_person": {
"person_to_person": null
},
"community_acquired": {
"community_acquired": null
},
"under_investigation": {
"under_investigation": null
}
}
]

ignore the value of IntSequenceGenerator in JSON response

I have used #JsonIdentityInfo on my class to avoid Infinite Recursion as follows
#JsonIdentityInfo(generator = ObjectIdGenerators.IntSequenceGenerator.class, property = "id", scope = IseidaPerfile.class)
public class IseidaPerfile implements Serializable {}
It is fixed the issue and my JSON response looks goods
{
"id": 22,
"id": 3,
"literal": "Responsible",
"permissions": [
3,
4,
{
"id": 23,
"id": 6,
"literal": "Show menu"
},
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19
]
}
But I want to ignore the unused ids from Permissions as 3, 4,7, ..... 18, 19? which generated by IntSequenceGenerator

How can I get only the JSON keys without using tilde sign?

I am trying to get (via JSON Path) only key values from the JSON below but with no success.
I cannot use the tilde sign (~) because JMeter's JSON Path Extractor works under JSON PATH 4.0 and ~ is not recognized.
{
"facetCount": {
"designer": {
"4856430": 2,
"7313551": 14,
"7018102": 8,
"306838": 1,
"85146": 146,
"2654979": 11,
"221111": 4,
"180510": 40,
"3344622": 59,
"472718": 73,
"107993": 19,
"166170": 58,
"6908": 2,
"426629": 1,
"1358858": 9,
"9879178": 6,
"55006": 43,
"285396": 2,
"3355": 9,
"215501": 8,
"4968477": 4,
"11349629": 7,
"11229643": 27,
"11355128": 9,
"7093068": 3,
"11098281": 2,
"5833751": 1,
"4741301": 1,
"9198104": 21,
"991324": 4
},
"attributes": {
"135979:77": 290,
"135979:83": 27,
"136227:20": 141,
"136227:78": 670,
"135985:44": 123,
"135985:43": 669,
"135979:62": 700,
"135979:61": 1188,
"136644:176": 2,
"136331:7": 1,
"136331:8": 3,
"136641:190": 13,
"136641:191": 12,
"136061:144": 3
},
"category": {
"136103": 208,
"136105": 147,
"137322": 2,
"136389": 120,
"136215": 236,
"136214": 954,
"136216": 217,
"136217": 352,
"136218": 452,
"136219": 40,
"136480": 4,
"136220": 111,
"136221": 288,
"136222": 58,
"136223": 369,
"136224": 163,
"136986": 3,
"136307": 1125,
"136059": 10,
"136308": 956,
"136315": 984,
"136003": 574,
"136045": 267,
"136035": 1501,
"135985": 1380,
"137134": 27,
"136309": 60,
"137323": 9,
"136390": 1,
"136021": 16,
"136322": 1951,
"137166": 16,
"137317": 7,
"136005": 4,
"135983": 4019,
"136033": 1513,
"136310": 1224,
"136392": 18,
"135981": 2430,
"136031": 16,
"136326": 1312,
"136061": 79
},
"colour": {
"1": 41686,
"7": 14593,
"5": 9596,
"18": 1,
"13": 5185,
"6": 5259,
"3": 6391,
"11": 5715,
"12": 1537,
"4": 8767,
"16": 1466,
"9": 8590,
"15": 1730,
"8": 8333,
"14": 3208,
"2": 13269,
"10": 2730
},
"ninetyminutes": {
"3": 309
},
"sameday": {
"3": 1714,
"42": 254
},
"size": {
"135972:1620": 523,
"136657:2650": 1,
"136657:2850": 1
},
"location": {
"3": 2674,
"4": 7671,
"5": 35808,
"6": 2761,
"7": 11948
},
"labels": {
"1300": 2969
}
}
}
I would like to get the keys that are under facetCount element (designers, attributes, colour, etc.) and also (another JSON Path expression) get the keys that are inside these keys, such as 4856430 from designers, 135979:77 from attributes, and so on.
Could you help me, please?
Thanks in advance!
You can use JSR223 PostProcessor to get key in object JSON
new groovy.json.JsonSlurper().parse(prev.getResponseData()).facetCount.eachWithIndex{ def node, int idx ->
log.info('Key ' + idx + '=' + node.getKey())
vars.put('key_' + idx, node.getKey())
}
And you can get key with a variable like that:
${key_0}, ${key_1}, ...
More detail check here How to extract values from json in jmeter when the keys are unkown?
JMeter's JSON test elements rely on Jayway Jsonpath which doesn't has this tilde operator for querying the keys, you will have to go for:
JSR223 PostProcessor
Groovy language
JsonSlurper
Add JSR223 PostProcessor as a child of the request which returns the above JSON and use the following code:
For direct keys (designer, attributes, etc)
def counter = 1
new groovy.json.JsonSlurper().parse(prev.getResponseData()).facetCount.each { facet ->
vars.put('key_' + counter, facet.key)
counter++
}
vars.put('key_matchNr', counter - 1 as String)
For child keys (4856430, 135979:77, etc)
def counter = 1
new groovy.json.JsonSlurper().parse(prev.getResponseData()).facetCount.each { child ->
child.value.keySet().each { key ->
vars.put('childKey_' + counter, key)
counter++
}
}
vars.put('childKey_matchNr', counter -1 as String)
More information: Groovy - Parsing and producing JSON