PowerShell - Iterate through JSON - json

I have looked at many examples, but I cannot get my code to work. The structure of this JSON seems to be different than any others I've seen. I need to iterate through these results and get "id" and "legalName".
[
"URL: https://myurl.com/rest/api",
{
"value": [
{
"id": 271,
"client": "#{id=245; clientCode=; clientName=}",
"dbaName": null,
"federalEIN": "",
"legalCode": "01",
"legalName": "Test1",
"naicsCode": null,
"status": "Active",
"links": " "
},
{
"id": 272,
"client": "#{id=245; clientCode=; clientName=}",
"dbaName": null,
"federalEIN": "",
"legalCode": "02",
"legalName": "Test2",
"naicsCode": null,
"status": "Terminated",
"links": " "
}
],
"Count": 6
}
]
I've tried lots of different code examples, but none of them have worked. This returns nothing. I can't seem to get down to the properties I need.
$var = Get-RestData("https://myurl.com/rest/api/") | ConvertTo-Json
$var["value"] | select id

You can use the ConvertFrom-Json cmdlet to convert from a JSON formatted string to a PSCustomObject with which you can iterate over.
$json = #"
[
"URL: https://myurl.com/rest/api",
{
"value": [
{
"id": 271,
"client": "#{id=245; clientCode=; clientName=}",
"dbaName": null,
"federalEIN": "",
"legalCode": "01",
"legalName": "Test1",
"naicsCode": null,
"status": "Active",
"links": " "
},
{
"id": 272,
"client": "#{id=245; clientCode=; clientName=}",
"dbaName": null,
"federalEIN": "",
"legalCode": "02",
"legalName": "Test2",
"naicsCode": null,
"status": "Terminated",
"links": " "
}
],
"Count": 6
}
]
"#
$json | ConvertFrom-Json | ForEach-Object { $_.value } | Select id, legalName
which returns:
id legalName
-- ---------
271 Test1
272 Test2

Related

Compare 2 JSON and retrieve subset from one of them based on condition in Powershell

I have two JSON files abc.json and xyz.json.
Content in abc.json is:
[{"id": "121",
"name": "John",
"location": "europe"
},
{"id": "100",
"name": "Jane",
"location": "asia"
},
{"id": "202",
"name": "Doe",
"location": "america"
}
]
Updated -> Content in xyz.json is:
{
"value": [
{
"id": "111",
"city": "sydney",
"profession": "painter"
},
{
"id": "200",
"city": "istanbul",
"profession": "actor"
},
{
"id": "202",
"city": "seattle",
"profession": "doctor"
}
],
"count": {
"type": "Total",
"value": 3
}
}
I want to get those records of abc.json in when the id in both objects are equal.In this case:
{"id": "202",
"name": "Doe",
"location": "america"
}
I need to do this in Powershell and the version I am using is 5.1.This is what I have tried:
$OutputList = #{}
$abcHash = Get-Content 'path\to\abc.json' | Out-String | ConvertFrom-Json
$xyzHash = Get-Content 'path\to\xyz.json' | Out-String | ConvertFrom-Json
$xyzResp = $xyzHash.value
foreach($item in $xyzResp){
foreach ($record in $abcHash){
if ($item.id -eq $record.id){
$OutputList.Add($record, $null)
}
}
}
Write-Output $OutputList
But on printing the OutputList , I get like this:
Key: #{"id": "202",
"name": "Doe",
"location": "america"
}
Value:
Name:#{"id": "202",
"name": "Doe",
"location": "america"
}
What I require is more of a PSObject like:
id: 202
name:Doe
location:america
I tried using Get-Member cmdlet but could not quite reach there.
Is there any suggestion I could use?
I have corrected your example xyz.json because there was an extra comma in there that should not be there. Also, the example did not have an iten with id 202, so there would be no match at all..
xyz.json
{
"value": [
{
"id": "111",
"city": "sydney",
"profession": "painter"
},
{
"id": "202",
"city": "denver",
"profession": "painter"
},
{
"id": "111",
"city": "sydney",
"profession": "painter"
}
],
"count": {
"type": "Total",
"value": 3
}
}
That said, you can use a simple Where-Object{...} to get the item(s) with matching id's like this:
$abc = Get-Content 'path\to\abc.json' -Raw | ConvertFrom-Json
$xyz = Get-Content 'path\to\xyz.json' -Raw | ConvertFrom-Json
# get the items with matching id's as object(s)
$abc | Where-Object { $xyz.value.id -contains $_.id}
Output:
id name location
-- ---- --------
202 Doe america
Of course you can capture the output first and display as list and/or save to csv, convert back to json and save that.

How to exclude data from json file to convert into csv file powershell

I want to convert my json file into csv -
{
"count": 28,
"value": [
{
"commitId": "65bb6a911872c314a9225815007d74a",
"author": {
"name": "john doe",
"email": "john.doe#gmail.com",
"date": "2020-06-09T17:03:33Z"
},
"committer": {
"name": "john doe",
"email": "john.doe#gmail.com",
"date": "2020-06-09T17:03:33Z"
},
"comment": "Merge pull request 3 from dev into master",
"changeCounts": {
"Add": 6,
"Edit": 0,
"Delete": 0
},
"url": "https://dev.azure.com/",
"remoteUrl": "https://dev.azure.com/"
},
{
"commitId": "bdcb4a1e1e671c15333eacc31aa9795fe32",
"author": {
"name": "john doe",
"email": "john.doe#gmail.com",
"date": "2020-06-09T17:03:33Z"
},
"committer": {
"name": "john doe",
"email": "john.doe#gmail.com",
"date": "2020-06-09T17:03:33Z"
},
"comment": "Updated FGDH",
"changeCounts": {
"Add": 0,
"Edit": 1,
"Delete": 0
},
"url": "https://dev.azure.com/",
"remoteUrl": "https://dev.azure.com/"
}
]
}
I don't want all the fields in my CSV file. I want commitid,commiter.name, commiter.date and comment only.
Get-Content "\commitinfo.json" -Raw | ConvertFrom-Json | Select -Expand value |Export-Csv
"\commitinfo.csv"
with this I get all data , How can I get selected data ?
Another json file -
{
"value": [
{
"id": "5c264dd2-bbcf-4537-8a63-19a0a4d2dc",
"name": "Develop",
"url": "https://dev.azure.com/",
"project": {
"id": "0042dc5c-fd13-4e3c-bfd7-7feb52e287",
"name": "test",
"url": "https://dev.azure.com/",
"state": "wellFormed",
"revision": 11,
"visibility": "private",
"lastUpdateTime": "2020-04-15T04:04:30.01Z"
},
"defaultBranch": "refs/heads/master",
"size": 55438,
"remoteUrl": "https://dev.azure.com",
"sshUrl": "git#ssh.dev.azure.com:v3",
"webUrl": "https://dev.azure.com/"
},
{
"id": "3219e8e2-281d-40ad-81c8-1cecf8",
"name": "automation",
"url": "https://dev.azure.com/",
"project": {
"id": "0e2df786-94a0-42c8-b068-c2656b",
"name": "automation",
"url": "https://dev.azure.com/",
"state": "wellFormed",
"revision": 19,
"visibility": "private",
"lastUpdateTime": "2020-05-02T03:32:50.937Z"
},
"size": 0,
"remoteUrl": "https://dev.azure.com/",
"sshUrl": "git#ssh.dev.azure.com:v3",
"webUrl": "https://dev.azure.com/"
}
]
}
from this file I only want to select projects with size - 0 , CSV should look like this
name size
a 0
b 0
Note: Piping the commands below to Export-Csv is omitted for brevity.
You need to use Select-Object with calculated properties:
(Get-Content commitinfo.json -Raw | ConvertFrom-Json).value |
Select-Object commitid,
#{n='committerName'; e={ $_.committer.name } },
#{n='committerDate'; e={ $_.committer.date } },
comment
In the second case, add Where-Object to filter in the objects of interest:
(Get-Content other.json -Raw | ConvertFrom-Json).value |
Where-Object size -eq 0 |
Select-Object name, size

How to avoid generating all combinations of selected data while constructing an object?

My original JSON is given below.
[
{
"id": "1",
"name": "AA_1",
"total": "100002",
"files": [
{
"filename": "8665b987ab48511eda9e458046fbc42e.csv",
"filename_original": "some.csv",
"status": "3",
"total": "100002",
"time": "2020-08-24 23:25:49"
}
],
"status": "3",
"created": "2020-08-24 23:25:49",
"filenames": "8665b987ab48511eda9e458046fbc42e.csv",
"is_append": "0",
"is_deleted": "0",
"comment": null
},
{
"id": "4",
"name": "AA_2",
"total": "43806503",
"files": [
{
"filename": "1b4812fe634938928953dd40db1f70b2.csv",
"filename_original": "other.csv",
"status": "3",
"total": "21903252",
"time": "2020-08-24 23:33:43"
},
{
"filename": "63ab85fef2412ce80ae8bd018497d8bf.csv",
"filename_original": "some.csv",
"status": "2",
"total": 0,
"time": "2020-08-24 23:29:30"
}
],
"status": "2",
"created": "2020-08-24 23:35:51",
"filenames": "1b4812fe634938928953dd40db1f70b2.csv&&63ab85fef2412ce80ae8bd018497d8bf.csv",
"is_append": "0",
"is_deleted": "0",
"comment": null
}
]
From this JSON I want to create new objects by combining fields from objects which have status: 2 and their files which also have the same pair, status: 2.
So, I am expecting a JSON array as below.
[
{
"id": "4",
"name": "AA_2",
"file_filename": "63ab85fef2412ce80ae8bd018497d8bf.csv",
"file_status": 2
}
]
So far I tried with this JQ filter:
.[]|select(.status=="2")|[{id:.id,file_filename:.files[].filename,file_status:.files[].status}]
But this produces some invalid data.
[
{
"id": "4", # want to remove this as file.status != 2
"file_filename": "1b4812fe634938928953dd40db1f70b2.csv",
"file_status": "3"
},
{
"id": "4",
"file_filename": "1b4812fe634938928953dd40db1f70b2.csv",
"file_status": "2"
},
{
"id": "4", # Repeat
"file_filename": "63ab85fef2412ce80ae8bd018497d8bf.csv",
"file_status": "3"
},
{
"id": "4", # Repeat
"file_filename": "63ab85fef2412ce80ae8bd018497d8bf.csv",
"file_status": "2"
}
]
How do I filter the new JSON using JQ and remove these duplicate objects?
By applying [] operator to files twice, you're running into a combinatorial explosion. That needs to be avoided, for example:
[ .[] | select(.status == "2") | {id, name} + (.files[] | select(.status == "2") | {file_filename: .filename, file_status: .status}) ]
Online demo

Transform a Map to List

I am trying to transform the map of the object in json to List using JQ ...
Source:
{
"title": "title",
"keyword": "keyword",
"desc": {
"user1": {
"name": "kumar",
"Duration": null,
"Time": null,
" Dominance": "Upper Field First"
},
"user2": {
"id": "user2",
"name": "user2",
"Duration": null,
"Time": null,
" Dominance": "Upper Field First"
}
}
}
Target:
[
{
"id": "user1",
"name": "kumar",
"Duration": null,
"Time": null,
"Dominance": "Upper Field First"
},
{
"id": "user2",
"name": "user2",
"Duration": null,
"Time": null,
"Dominance": "Upper Field First"
}
]
I tried various options like , but not able to get the extact thing i need.
.desc | . as $in| keys[]
to_entries
JQ Workspace ::: https://jqplay.org/s/MiJ9w1Sz5L
The following filter will do what you want, assuming you've fixed the input:
.desc
| to_entries
| map( {id: .key} + .value)
To understand this, simply read up on to_entries in the online manual (see jq).

Combining Nested Json using PowerShell

I have the following Json script:
{
"merchant_info": {
"email": "merchant#example.com",
"first_name": "David",
"last_name": "Larusso",
"business_name": "Mitchell & Murray",
"phone": {
"country_code": "001",
"national_number": "4085551234"
},
"address": {
"line1": "1234 First Street",
"city": "Anytown",
"state": "CA",
"postal_code": "98765",
"country_code": "US"
}
},
"billing_info": [{
"email": "bill-me#example.com",
"first_name": "Stephanie",
"last_name": "Meyers"
}
],
"shipping_info": {
"first_name": "Stephanie",
"last_name": "Meyers",
"address": {
"line1": "1234 Main Street",
"city": "Anytown",
"state": "CA",
"postal_code": "98765",
"country_code": "US"
}
},
"items": [{
"name": "Zoom System wireless headphones",
"quantity": 2,
"unit_price": {
"currency": "USD",
"value": "120"
},
"tax": {
"name": "Tax",
"percent": 8
}
}, {
"name": "Bluetooth speaker",
"quantity": 1,
"unit_price": {
"currency": "USD",
"value": "145"
},
"tax": {
"name": "Tax",
"percent": 8
}
}
],
"discount": {
"percent": 1
},
"shipping_cost": {
"amount": {
"currency": "USD",
"value": "10"
}
},
"note": "Thank you for your business.",
"terms": "No refunds after 30 days."
}
And I want to use PowerShell to get the following Record and export it to CSV:
So far I created the following Script:
$JsonFile = "C:\Users\me\Documents\myfile.json"
$OutputFile = "C:\Users\me\Documents\newtext.csv"
Get-Content -Path $OutputFile
$json = ConvertFrom-Json (Get-Content $JsonFile -Raw)
$json.merchant_info | Select "first_name","last_name",#{Label = "phone"; Expression = {$_.phone.national_number}} |
Export-Csv $OutputFile -NoTypeInformation
I am able to bring values from (Merchant_info, Shipping_info, item) separetely but how do I bring it all in combined like in my screen shot above.
but how do I bring it all in combined like in my screen shot above.
We can only guess; assuming this entire json block is one order, with one merchant and one customer, but multiple items, then each row is an item. So start with that as the input:
Create an output record (PSCustomObject) with the repeated data, and then the individual item data:
$json.items | ForEach-Object {
[PSCustomObject]#{
MerchantInfoFirstName = $json.merchant_info.first_name
MerchantInfoLastName = $json.merchant_info.last_name
MerchantInfoPhoneNumber = $json.merchant_info.phone.national_number
ShippingInfoFirstName = $json.shipping_info.first_name
ShippingInfoLastName = $json.shipping_info.last_name
ItemName = $_.name
ItemQuantity = $_.quantity
}
} | Export-Csv ... etc.