Prettify json in powershell 3 - json

Given a standard json string value:
$jsonString = '{ "baz": "quuz", "cow": [ "moo", "cud" ], "foo": "bar" }'
How can I get this to be all pretty with newlines, preferably without brute-force regex?
Simplest method I've found so far is:
$jsonString | ConvertFrom-Json | ConvertTo-Json
However, that seems kinda silly.

Works for me. Parentheses make sure get-content is done before piping. Default depth of convertto-json is 2, which is often too low.
function pjson ($jsonfile) {
(get-content $jsonfile) | convertfrom-json | convertto-json -depth 100 |
set-content $jsonfile
}

If you really don't want to go down the simplest route, which is to use inbuilt PowerShell functions | ConvertFrom-Json | ConvertTo-Json, here is another method, using JSON.net
# http://james.newtonking.com/projects/json-net.aspx
Add-Type -Path "DRIVE:\path\to\Newtonsoft.Json.dll"
$jsonString = '{ "baz": "quuz", "cow": [ "moo", "cud" ], "foo": "bar" }'
[Newtonsoft.Json.Linq.JObject]::Parse($jsonString).ToString()

I put this in my profile
function PrettyPrintJson {
param(
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
$json
)
$json | ConvertFrom-Json | ConvertTo-Json -Depth 100
}
Which works with pipes, and can be auto-completed, so it's at least somewhat less typing:
cat .\file.json | PrettyPrintJson
curl https://api.twitter.com/1.1/statuses/user_timeline.json | PrettyPrintJson

Adding to #JS2010's answer I added logic to escape out certain characters and clean up my output even further. The parenthesis seems key and -depth is a big one since you can lose details without it, from what I've seen, on depth that goes beyond the default of 5, I believe it is.
function Format-Json ($JSON)
{
$PrettifiedJSON = ($JSON) | convertfrom-json | convertto-json -depth 100 | ForEach-Object { [System.Text.RegularExpressions.Regex]::Unescape($_) }
$PrettifiedJSON
}

I think what you are looking for is this:
$jsonString = #{
'baz' = 'quuz'
'cow'= "moo, cud"
'foo'= "bar"
}
$jsonString|ConvertTo-Json
it produces this output
{
"baz": "quuz",
"cow": "moo, cud",
"foo": "bar"
}
Added note
You could also array your cow values to "prettify" it a bit more:
$jsonString = #{
'baz' = 'quuz'
'cow'= #("moo"; "cud")
'foo'= "bar"
}
output:
{
"baz": "quuz",
"cow": [
"moo",
"cud"
],
"foo": "bar"
}

Related

Populate collection of objects from one JSON file to the collection of another one with PowerShell

I have two JSON files and want to transfer collection of objects from one file to another. Suppose, the from.json file contains property which represents collection of clients:
"Clients":
[
{
"Name": "Name1",
"Age": "12"
},
{
"Name": "Name2",
"Age": "14"
}
]
to.json file contains an empty collection, "Objects: []" ,which must be filled with objects from from.json. Each objects in toJson variable must contain additional property - Id, so eventually, my "to.json" file should look like this:
"Objects":
[
{
"Id": "{new-id}",
"Name": "Name1",
"Age": "12"
},
{
"Id": "{new-id}",
"Name": "Name1",
"Age": "12"
}
]
I've converted two files into variables:
$fromJson = (Get-Content -Raw -Path {fromPath}) | ConvertFrom-Json
$toJson = (Get-Content -Raw -Path {toPath}) | ConvertFrom-Json
I know that objects from fromJson to toJson can be transferred in the following manner:
toJson.Objects += fromJson.Clients, but that's not enough in my case. I think that it could be done by iterating through fromJson.Clients array but have no idea how to create an object and add it into toJson.Objects collection.
Here's a more efficient solution, based on:
Use of a calculated property with Select-Object, which allows you to place the new property first in the output objects.
Instead of building the array one by one with += (which is inefficient, because a new array must technically be created behind the scenes in every iteration), the solution below lets PowerShell collect the output objects of the Select-Object call in an array automatically (the [array] type constraint is needed to ensure that an array is created even if only one object happens to be output.)
# Sample input.
$fromJson = ConvertFrom-Json '{"Clients":[{"Name":"Name1","Age":"12"},{"Name":"Name2","Age":"14"}]}'
$toJson = ConvertFrom-Json '{ "Objects": [] }'
[array] $toJson.Objects =
$fromJson.Clients |
Select-Object #{ Name='Id'; Expression = { [string] (New-Guid) } }, *
$toJson | ConvertTo-Json -Depth 3 # append | Set-Content as needed.
Kind of new to the PowerShell, but after a bit of investigation came up with the following solution:
fromJson.Clients | ForEach-Object {
$_ | Add-Member -MemberType NoteProperty -Name 'Id' -Value ([guid]::NewGuid().Guid.ToString())
$toJson += $_
}
...
$toJson | ConvertTo-Json | Out-File {to.json_path}
Frankly, don't know if that is a 'proper' way to do that, but generally it works for that particular case. For now, see no other solution.

Extracting fields to CSV from a JSON file with Powershell

i have a json extracted from a API exact like this:
{
"LPEONASVVAP0": {
"LPEONASVVAP0": {
"id": "urn:vcloud:vm:f526d27d-e0f9-4d4f-ae81-4824e397c027",
"name": "LPEONASVVAP0",
"description": "_vm_desc_",
"dateCreated": "2021-04-06T14:56:09.640+0000"
}
},
"WDEONDSVDIS6": {
"WDEONDSVDIS6": {
"id": "urn:vcloud:vm:7ed43492-a7ce-4963-b5bb-5ec2ca89477c",
"name": "WDEONDSVDIS6",
"description": "",
"dateCreated": "2021-04-13T13:44:29.973+0000"
}
},
"WDEONASVSTR0": {
"WDEONASVSTR0": {
"id": "urn:vcloud:vm:7afa34fe-b239-4abe-90df-3f270b44db1f",
"name": "WDEONASVSTR0",
"description": "",
"dateCreated": "2021-03-10T16:17:50.947+0000"
}
},
}
I need extract only fields id, name and description to create a csv with them. I test this but the output file is in blank:
$pathToJsonFile = x
$pathToOutputFile = x
$obj = Get-Content $pathToJsonFile -Raw | ConvertFrom-Json
print $obj
$obj | select id, name, description | Convertto-csv > $pathToOutputFile
You'll need to "discover" the parent property names (eg. 'LPEONASVVAP0') via the psobject hidden memberset. Since the outer and inner properties are named the same, we can re-use the name to get the inner property value:
$obj.psobject.Properties |ForEach-Object {
$_.Value.$($_.Name)
} |Select id,name,description |Export-Csv -NoTypeInformation -Path $pathToOutputFile
Edit: Mathias R. Jessens answer is better written than this, i would do it that way instead of how i posted.
Okay so i copied the json you posted, imported it. Since each array of information is stored like this
"WDEONDSVDIS6": {
"WDEONDSVDIS6": {
i used get-member to iterate each of the arrays and then select the info from that.
Also, you dont need to use Convertto-csv > $pathToOutputFile, use the export-csv command instead.
Below is my code how i would have done it, there is probably a better way but this works :)
$pathToOutputFile = x.csv
$obj = Get-Content example.json -Raw| ConvertFrom-Json
$obj2 = ($obj | Get-Member -MemberType noteproperty).Name
$result = foreach($item in $obj2){
$obj.$item.$item | select id,name,description
}
$result | Export-Csv -Path $pathToOutputFile -Encoding utf8 -NoTypeInformation

Merge two json objects

I have the following inputs - 2 json files one is the base one and the second contains the same properties but the different values, I'd like to merge that objects.
For example:
{
a:{
b:"asda"
}
c: "asdasd"
}
And the second file:
{
a:{
b:"d"
}
}
And the result should be like this:
{a:{b:"d"},c:"asdasd"}
Is that is possible with powershell?
Join (Join-Object) is not a built-in CmdLet
This is an extension of #Mark's answer which also recurses through child objects.
function merge ($target, $source) {
$source.psobject.Properties | % {
if ($_.TypeNameOfValue -eq 'System.Management.Automation.PSCustomObject' -and $target."$($_.Name)" ) {
merge $target."$($_.Name)" $_.Value
}
else {
$target | Add-Member -MemberType $_.MemberType -Name $_.Name -Value $_.Value -Force
}
}
}
merge $Json1 $Json2
$Json1
$Json1 | Join $Json2 -Merge {$Right.$_} | ConvertTo-Json (see update below)
Install-Module -Name JoinModule
($Json1 ConvertFrom-Json) | Merge ($Json2 ConvertFrom-Json) | ConvertTo-Json
Result:
{
"c": "asdasd",
"a": {
"b": "d"
}
}
You might consider not to overwrite the left value:
($Json1 ConvertFrom-Json) | Join ($Json2 ConvertFrom-Json) | ConvertTo-Json
In that case the result will be:
{
"c": "asdasd",
"a": [
{
"b": "asda"
},
{
"b": "d"
}
]
}
For details see: https://stackoverflow.com/a/45483110/1701026
Update 2019-11-16
The -Merge parameter has been depleted and divided over the -Discern and -Property parameters (sorry for the breaking change). The good news is that the default parameter settings for an object merge are accommodated in a proxy command named Merge-Object (alias Merge) which simplifies the concerned syntax to just: $Object1 | Merge $Object2. For details, see readme or embedded help.
If you know the names of the elements (per your example above), you could do it explicitly like this:
$Json1 ='{
a: {
b:"asda"
},
c: "asdasd"
}
' | ConvertFrom-Json
$Json2 = '{
a:{
b:"d"
}
}
' | ConvertFrom-Json
$Json1.a = $Json2.a
Result:
$Json1 | ConvertTo-Json
{
"a": {
"b": "d"
},
"c": "asdasd"
}
If you're looking for something that will merge the two without knowing the explicit key name, you could do something like the following. This will essentially overwrite any properties in the first Json with those from the second Json, where they are duplicated at the first level (it won't seek matches in the nested properties and again this is an overwrite not a merge):
$Json2.psobject.Properties | ForEach-Object {
$Json1 | Add-Member -MemberType $_.MemberType -Name $_.Name -Value $_.Value -Force
}

Convert JSON to CSV using PowerShell

I have a sample JSON-formatted here which converts fine if I use something like: https://konklone.io/json/
I've tried the following code in PowerShell:
(Get-Content -Path $pathToJsonFile | ConvertFrom-Json)
| ConvertTo-Csv -NoTypeInformation
| Set-Content $pathToOutputFile
But the only result I get is this:
{"totalCount":19,"resultCount":19,"hasMore":false,"results":
How do I go about converting this correctly in PowerShell?
By looking at just (Get-Content -Path $pathToJsonFile) | ConvertFrom-Json it looks like the rest of the JSON is going in to a results property so we can get the result I think you want by doing:
((Get-Content -Path $pathToJsonFile) | ConvertFrom-Json).results |
ConvertTo-Csv -NoTypeInformation |
Set-Content $pathToOutputFile
FYI you can do ConvertTo-Csv and Set-Content in one move with Export-CSV:
((Get-Content -Path $pathToJsonFile) | ConvertFrom-Json).results |
Export-CSV $pathToOutputFile -NoTypeInformation
You have to select the results property inside your CSV using the Select-Object cmdlet together with the -expand parameter:
Get-Content -Path $pathToJsonFile |
ConvertFrom-Json |
Select-Object -expand results |
ConvertTo-Csv -NoTypeInformation |
Set-Content $pathToOutputFile
I was getting my json from a REST web api and found that the following worked:
Invoke-WebRequest -method GET -uri $RemoteHost -Headers $headers
| ConvertFrom-Json
| Select-Object -ExpandProperty <Name of object in json>
| ConvertTo-Csv -NoTypeInformation
| Set-Content $pathToOutputFile
I end up with a perfectly formatted csv file
Trying to use Mark Wrang's answer failed for me. While Piemol's comment from Jan 30 '19 solved a basic problem with Mark Wrang's answer, it also didn't work for me.
JSON strings do not always represent rectangular data sets. They may contain ragged data. For example, the Power BI activities log outputs JSON that contains different members depending on variables like what activities occurred in the requested data or what features were available at the time.
Using Piemol's comment, I processed this JSON:
[
{
"a": "Value 1",
"b": 20,
"g": "Arizona"
},
{
"a": "Value 2",
"b": 40,
"c": "2022-01-01T11:00:00Z"
},
{
"a": "Value 3",
"d": "omicron",
"c": "2022-01-01T12:00:00Z"
},
{
"a": "Value 4",
"b": 60,
"d": "delta",
"e": 14,
"c": "2022-01-01T13:00:00Z"
}
]
The script produced this CSV:
"a","b","g"
"Value 1","20","Arizona"
"Value 2","40",
"Value 3",,
"Value 4","60",
Notice that columns c, d, and e are missing. It appears that Export-CSV uses the first object passed to determine the schema for the CSV to output.
To handle this, use the UnifyProperties function:
function UnifyProperties {
$Names = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase)
$InputCollected = #($Input)
$InputCollected.ForEach({
foreach ($Name in $_.psobject.Properties.Name) { $Null = $Names.Add($Name) }
})
$inputCollected | Select-Object #($Names)
}
$pathToInputFolder = (New-Object -ComObject Shell.Application).NameSpace('shell:Downloads').Self.Path + "\" + "PowerBIActivities\combined\"
$pathToInputFile = $pathToInputFolder + "Activities.json"
$pathToOutputFile = $pathToInputFolder + "Activities.csv"
$content = Get-Content -Path $pathToInputFile -Raw
$psObj = ConvertFrom-Json -InputObject $content
$psObj | UnifyProperties | Export-CSV $pathToOutputFile -NoTypeInformation

Nested arrays and ConvertTo-Json

To use a REST API, I must pass a JSON object that looks like this:
{ "series" :
[{
"metric": "custom.powershell.gauge",
"points":[[1434684739, 1000]]
}
]
}
Note the nested array here. I cannot get to reproduce this. Here is my code:
[int][double]$unixtime=get-date ( (get-date).ToUniversalTime() ) -UFormat %s
$obj=#{}
$series=#{}
$array=#()
$points=#()
$value=get-random -Minimum 0 -Maximum 100
$series.add("metric","custom.powershell.gauge")
$points=#(#($unixtime, $value))
$series.add("points",$points)
$obj.Add("series",#($series))
$json=$obj | ConvertTo-Json -Depth 30 -Compress
$json
And here is the output:
{"series":[{"points":[1434685292,95],"metric":"custom.powershell.gauge"}]}
I've tried many things, I cannot get the 2 arrays to be nested, it always end up looking like a single array.
On the same note, came someone explain this please:
> $a=(1,2)
> $a
1
2
> $a | ConvertTo-Json
[
1,
2
]
> $b=($a,$a)
> $b
1
2
1
2
> $b | ConvertTo-Json
[
{
"value": [
1,
2
],
"Count": 2
},
{
"value": [
1,
2
],
"Count": 2
}
]
Where are these value and Count coming from?
Thanks for your help.
The explanation is that (1,2),(3,4) is an array of array, but Powershell split the first level with the pipe |, and you don't give a name for these arrays so the serializer supplies it. First have a try to this :
# First build your array of array
$z = (1,2),(3,4)
# convert it to JSON using the ,
,$z | ConvertTo-Json -Depth 5 -Compress
[psobject]#{"points"=$z} | ConvertTo-Json -Depth 5 -Compress
It gives the first step:
{"value":[[1,2],[3,4]],"Count":2}
{"points":[[1,2],[3,4]]}
Now the solution I propose :
# First build your array of array
$z = (1,2),(3,4)
# Then build a PSCustom object
$a = [pscustomobject]#{"series" = ,#{"metric"="custom.powershell.gauge"; "points"=$z}}
# At the end convert it to JSON
# don't forget the **Depth** parameter (use **Compress** to retreive one line like above)
$a | ConvertTo-Json -Depth 5
For me it gives something close to what you need:
{
"series": [
{
"points": [
[
1,
2
],
[
3,
4
]
],
"metric": "custom.powershell.gauge"
}
]
}
Late to the party, but I'd like to propose a more visually intuitive solution that's easy to expand (I, like others, am a visual learner so code blocks like the below help me understand things more easily):
[int][double]$unixtime = Get-Date ((Get-Date).ToUniversalTime()) -UFormat %s
$value = Get-Random -Minimum 0 -Maximum 100
$body = #{
'series' = #(
[Ordered]#{
'metric'='custom.powershell.gauge'
'points' = #(
,#($unixtime,$value)
)
}
)
}
ConvertTo-Json -InputObject $body -Depth 4
Outputs:
{
"series": [
{
"metric": "custom.powershell.gauge",
"points": [
[
1473698742,
96
]
]
}
]
}
-Depth 4 gets you the additional set of square brackets around your point values, and [Ordered] ensures that the hashtable is ordered as originally specified. Don't forget -Compress before sending, like others have said.
As JPBlanc suggested, creating a custom object worked. Below is my code:
[long]$value=Get-Random -Minimum 0 -Maximum 100
$points=,#($unixtime, $value)
$metricname="custom.powershell.gauge"
$obj = [pscustomobject]#{"series" = ,#{"metric" = $metricname; "points"=$points}}
$json=$obj | ConvertTo-Json -Depth 5 -Compress
Which outputs:
{"series":[{"points":[[1434810163,53]],"metric":"custom.powershell.gauge"}]}
Don't forget to specify a depth >2.
Thanks!