How to copy the objects of one array of a JSON file to an array of another JSON file using PowerShell? For Example I have one JSON file like:
"type": "Employee",
"Properties": [
{
"Name": "Raj",
"Id": "18111",
"email": "emp1#company.com",
"Position": "Manager",
"DateOfJoining": "16.10.14",
}
],
"Description": "Employee details"
and another JSON file as:
"type": "Employee",
"Properties": [
{
"Name": "Ram",
"Id": "44000",
"email": "emp2#company.com",
"Position": "Admin",
"DateOfJoining": "10.12.14",
},
{
"Name": "Paul",
"Id": "44002",
"email": "emp3#company.com",
"Position": "Programmer",
"DateOfJoining": "10.9.14",
},
],
"Description": "Employee details"
I want to copy the arrays from 1st JSON file to the 2nd JSON file.
You can try something like this:
$c1 = Convert-FromJson (gc file1.json -raw)
$c2 = Convert-FromJson (gc file2.json -raw)
$c3 = $c1.Properties + $c2.Properties
$c3 | ConvertTo-Json
Related
I have several CSV files of item data for a game I'm messing around with that I need to convert to JSON for consumption. The data can be quite irregular with several empty fields per record, which makes for sort of ugly JSON output.
Example with dummy values:
Id,Name,Value,Type,Properties/1,Properties/2,Properties/3,Properties/4
01:Foo:13,Foo,13,ACME,CanExplode,IsRocket,,
02:Bar:42,Bar,42,,IsRocket,,,
03:Baz:37,Baz,37,BlackMesa,CanExplode,IsAlive,IsHungry,
Converted output:
[
{
"Id": "01:Foo:13",
"Name": "Foo",
"Value": 13,
"Type": "ACME",
"Properties": ["CanExplode", "IsRocket", ""]
},
{
"Id": "02:Bar:42",
"Name": "Bar",
"Value": 42,
"Type": "",
"Properties": ["IsRocket", "", ""]
},
{
"Id": "03:Baz:37",
"Name": "Baz",
"Value": 37,
"Type": "BlackMesa",
"Properties": ["CanExplode", "IsAlive", "IsHungry"]
}
]
So far I've been quite successful with using Miller. I've managed to remove completely empty columns from the CSV as well as aggregate the Properties/X columns into a single array.
But now I'd like to do two more things to improve the output format to make consuming the JSON easier:
remove empty strings "" from the Properties array
replace the other empty strings "" (e.g. Type of the second record) with null
Desired output:
[
{
"Id": "01:Foo:13",
"Name": "Foo",
"Value": 13,
"Type": "ACME",
"Properties": ["CanExplode", "IsRocket"]
},
{
"Id": "02:Bar:42",
"Name": "Bar",
"Value": 42,
"Type": null,
"Properties": ["IsRocket"]
},
{
"Id": "03:Baz:37",
"Name": "Baz",
"Value": 37,
"Type": "BlackMesa",
"Properties": ["CanExplode", "IsAlive", "IsHungry"]
}
]
Is there a way to achieve that with Miller?
My current commands are:
mlr -I --csv remove-empty-columns file.csv to clean up the columns
mlr --icsv --ojson --jflatsep '/' --jlistwrap cat file.csv > file.json for the conversion
It's not probably the way you want to do it. I use also jq.
Running
mlr --c2j --jflatsep '/' --jlistwrap remove-empty-columns then cat input.csv | \
jq '.[].Properties|=map(select(length > 0))' | \
jq '.[].Type|=(if . == "" then null else . end)'
you will have
[
{
"Id": "01:Foo:13",
"Name": "Foo",
"Value": 13,
"Type": "ACME",
"Properties": [
"CanExplode",
"IsRocket"
]
},
{
"Id": "02:Bar:42",
"Name": "Bar",
"Value": 42,
"Type": null,
"Properties": [
"IsRocket"
]
},
{
"Id": "03:Baz:37",
"Name": "Baz",
"Value": 37,
"Type": "BlackMesa",
"Properties": [
"CanExplode",
"IsAlive",
"IsHungry"
]
}
]
Using Miller, you can "filter out" the empty fields from each record with:
mlr --c2j --jflatsep '/' --jlistwrap put '
$* = select($*, func(k,v) {return v != ""})
' file.csv
remark: actually, we're building a new record containing the non-empty fields instead of deleting the empty fields from the record; the final result is equivalent though:
[
{
"Id": "01:Foo:13",
"Name": "Foo",
"Value": 13,
"Type": "ACME",
"Properties": ["CanExplode", "IsRocket"]
},
{
"Id": "02:Bar:42",
"Name": "Bar",
"Value": 42,
"Properties": ["IsRocket"]
},
{
"Id": "03:Baz:37",
"Name": "Baz",
"Value": 37,
"Type": "BlackMesa",
"Properties": ["CanExplode", "IsAlive", "IsHungry"]
}
]
I wish to parse individual elements of inner JSON object to build / load in the database.
The following is the JSON object. How can I parse elements like id, name queue etc? I will iterate it in loop and work and build the insert query.
{
"apps": {
"app": [
{
"id": "application_1540378900448_18838",
"user": "hive",
"name": "insert overwrite tabl...summary_view_stg_etl(Stage-2)",
"queue": "Data_Ingestion",
"state": "FINISHED",
"finalStatus": "SUCCEEDED",
"progress": 100
},
{
"id": "application_1540378900448_18833",
"user": "hive",
"name": "insert into SNOW_WORK...metric_definitions')(Stage-13)",
"queue": "Data_Ingestion",
"state": "FINISHED",
"finalStatus": "SUCCEEDED",
"progress": 100
}
]
}
}
You're better off converting the data to a format easily consumed by a database processor, like csv, then do something about it.
$ jq -r '(.apps.app[0] | keys_unsorted) as $k
| $k, (.apps.app[] | [.[$k[]]])
| #csv
' input.json
its pretty simple just fetch elment which is having an array of values.
var JSONOBJ={
"apps": {
"app": [
{
"id": "application_1540378900448_18838",
"user": "hive",
"name": "insert overwrite tabl...summary_view_stg_etl(Stage-2)",
"queue": "Data_Ingestion",
"state": "FINISHED",
"finalStatus": "SUCCEEDED",
"progress": 100
},
{
"id": "application_1540378900448_18833",
"user": "hive",
"name": "insert into SNOW_WORK...metric_definitions')(Stage-13)",
"queue": "Data_Ingestion",
"state": "FINISHED",
"finalStatus": "SUCCEEDED",
"progress": 100
}
]
}
}
JSONOBJ.apps.app.forEach(function(o){console.log(o.id);console.log(o.user);console.log(o.name);})
I have a json file to store my data and I convert it to CSV to edit my data. But when i convert it to json again it all goes unconstructed. How can i convert my csv to same structure as my old json.
JSON
{
"product": [
{
"id": "item0001",
"category": "12",
"name": "Name1",
"tag": "tag1",
"more": [
{
"id": "1",
"name": "AL"
},
{
"id": "1",
"name": "BS"
}
],
"active": true
},
{
"id": "item0002",
"categoryId": "13",
"name": "Name2",
"tag": "tag2",
"size": "2",
"more": [
{
"id": "2",
"name": "DL"
},
{
"id": "2",
"name": "AS"
}
],
"active": true
}
]
}
CSV
id,categoryId,name,shortcut,more/0/optionId,more/0/price,more/1/optionId,more/1/price,active,more/2/optionId,more/2/price,spanSize
item0001,ab92d2c6-010e-4182-844d-65050e746617,Name1,Shortcut1,1,60,1,70,TRUE,,,
item0002,ab92d2c6-010e-4182-844d-65050e746617,Name2,Shortcut2,2,60,2,70,TRUE,2,2,4
You can use Miller (mlr) to convert you file both ways
https://miller.readthedocs.io/en/latest/flatten-unflatten/
first from JSON to CSV
mlr --ijson --ocsv cat test.json > test.csv
then edit CSV (Visidata is a very nice command line tool for the job)
then convert it back to CSV
mlr --icsv --ojson cat test.csv > test_v2.json
If you want to have some JSON lines structure instead, use --ojonl
I am looking for help/guidance to combine multiple JSON into 1 JSON based on a node in PERL. For example here are the two JSON content that I want to combine.
JSON #1:
{
"title": "All",
"lastModified": "2017-04-11T00:00:00.000+0000",
"users": [{
"name": "Alpha One",
"title": "Security Chief",
"locations": [{"id": "730WLCS"}, {"id": "943MCS"}]
},
{
"name": "Alpha Two",
"title": "Security Manager"
}
]
}
JSON #2:
{
"title": "All",
"lastModified": "2017-04-11T00:00:00.000+0000",
"users": [{
"name": "Beta One",
"title": "Architect",
"locations": [{"id": "730WLCS"}]
}
]
}
RESULT JSON :
{
"title": "All",
"lastModified": "2017-04-11T00:00:00.000+0000",
"users": [{
"name": "Alpha One",
"title": "Security Chief",
"locations": [{"id": "730WLCS"}, {"id": "943MCS"}]
},
{
"name": "Alpha Two",
"title": "Security Manager"
},
{
"name": "Beta One",
"title": "Architect",
"locations": [{"id": "730WLCS"}]
}
]
}
Basically, I want to combine only the "users" node.
I tried to get the node using from_json and tried to push into an array, but it is not working.
Here is the code that I tried:
my $json_obj1 = from_json($json_txt1, {utf8 => 1});
my $json_obj2 = from_json($json_txt2, {utf8 => 1});
push(#myJSON, #{$json_obj1->{'users'}});
push(#myJSON, #{$json_obj2->{'users'}});
Any help is much appreciated.
Thank you.
my $json_obj1 = decode_json($json_txt1); # Same as from_json($json_txt1, {utf8 => 1})
my $json_obj2 = decode_json($json_txt2);
push #{ $json_obj2->{users} }, #{ $json_obj1->{users} };
If you want to remove duplicates , keeping the newer records (assuming $json_obj1 is the older state), you can use the following:
my %seen;
#{ $json_obj2->{users} } =
grep !$seen{$_->{name}}++,
#{ $json_obj2->{users} },
#{ $json_obj1->{users} };
I am getting this result when I am using graph api . it is in array format
{
"id": "216805086",
"name": "raj sharma",
"first_name": "raj ",
"last_name": "sharma",
"link": "https://www.facebook.com/raj.sharma.5",
"username": "raj .sharma.5",
"favorite_teams": [
{
"id": "198358615428",
"name": "Mumbai Indians"
},
{
"id": "190313434323691",
"name": "Indian Cricket Team"
}
],
"favorite_athletes": [
{
"id": "100787839962234",
"name": "Saina Nehwal"
}
],
"gender": "male",
"email": "raj.discoverme#gmail.com",
"timezone": 5.5,
"locale": "en_GB",
"verified": true,
"updated_time": "2013-08-13T06:01:17+0000"
}
I am working in a php language and phpmyadmin database . Now i want to insert the array into my database . Should i make a column for id , name , first_name ,last_name,link,favorite_teams etc or should i make a one column for all of this ........
how toinsert tha array into the database
Actually this is not an array. This is JSON. In JSON there are two formats,
JSONArray [ ]
JSONObject { }
You are getting the JSONObject as your output. There is a function in PHP callerd JSONDecode.
Go through this you will get idea.
Storing facebook app data in a database is against Facebook policy http://developers.facebook.com/policy/
$data = '{
"id": "216805086",
"name": "raj sharma",
"first_name": "raj ",
"last_name": "sharma",
"link": "https://www.facebook.com/raj.sharma.5",
"username": "raj .sharma.5",
"favorite_teams": [
{
"id": "198358615428",
"name": "Mumbai Indians"
},
{
"id": "190313434323691",
"name": "Indian Cricket Team"
}
],
"favorite_athletes": [
{
"id": "100787839962234",
"name": "Saina Nehwal"
}
],
"gender": "male",
"email": "raj.discoverme#gmail.com",
"timezone": 5.5,
"locale": "en_GB",
"verified": true,
"updated_time": "2013-08-13T06:01:17+0000"
}';
//decode to get as php variable
$values = json_decode($data,true); //true to decode as a array not an object
$sql = "INSERT INTO TableName (id,name,first_name,last_name,link,username)
VALUES ('".$values['id']."','".$values['name']."','".$values['first_name']."','".$values['last_name']."','".$values['link']."','".$values['username']."')";
mysql_query($sql);
Json_decode() takes a JSON encoded string and converts it into a PHP variable.