Nested arrays and ConvertTo-Json - json

To use a REST API, I must pass a JSON object that looks like this:
{ "series" :
[{
"metric": "custom.powershell.gauge",
"points":[[1434684739, 1000]]
}
]
}
Note the nested array here. I cannot get to reproduce this. Here is my code:
[int][double]$unixtime=get-date ( (get-date).ToUniversalTime() ) -UFormat %s
$obj=#{}
$series=#{}
$array=#()
$points=#()
$value=get-random -Minimum 0 -Maximum 100
$series.add("metric","custom.powershell.gauge")
$points=#(#($unixtime, $value))
$series.add("points",$points)
$obj.Add("series",#($series))
$json=$obj | ConvertTo-Json -Depth 30 -Compress
$json
And here is the output:
{"series":[{"points":[1434685292,95],"metric":"custom.powershell.gauge"}]}
I've tried many things, I cannot get the 2 arrays to be nested, it always end up looking like a single array.
On the same note, came someone explain this please:
> $a=(1,2)
> $a
1
2
> $a | ConvertTo-Json
[
1,
2
]
> $b=($a,$a)
> $b
1
2
1
2
> $b | ConvertTo-Json
[
{
"value": [
1,
2
],
"Count": 2
},
{
"value": [
1,
2
],
"Count": 2
}
]
Where are these value and Count coming from?
Thanks for your help.

The explanation is that (1,2),(3,4) is an array of array, but Powershell split the first level with the pipe |, and you don't give a name for these arrays so the serializer supplies it. First have a try to this :
# First build your array of array
$z = (1,2),(3,4)
# convert it to JSON using the ,
,$z | ConvertTo-Json -Depth 5 -Compress
[psobject]#{"points"=$z} | ConvertTo-Json -Depth 5 -Compress
It gives the first step:
{"value":[[1,2],[3,4]],"Count":2}
{"points":[[1,2],[3,4]]}
Now the solution I propose :
# First build your array of array
$z = (1,2),(3,4)
# Then build a PSCustom object
$a = [pscustomobject]#{"series" = ,#{"metric"="custom.powershell.gauge"; "points"=$z}}
# At the end convert it to JSON
# don't forget the **Depth** parameter (use **Compress** to retreive one line like above)
$a | ConvertTo-Json -Depth 5
For me it gives something close to what you need:
{
"series": [
{
"points": [
[
1,
2
],
[
3,
4
]
],
"metric": "custom.powershell.gauge"
}
]
}

Late to the party, but I'd like to propose a more visually intuitive solution that's easy to expand (I, like others, am a visual learner so code blocks like the below help me understand things more easily):
[int][double]$unixtime = Get-Date ((Get-Date).ToUniversalTime()) -UFormat %s
$value = Get-Random -Minimum 0 -Maximum 100
$body = #{
'series' = #(
[Ordered]#{
'metric'='custom.powershell.gauge'
'points' = #(
,#($unixtime,$value)
)
}
)
}
ConvertTo-Json -InputObject $body -Depth 4
Outputs:
{
"series": [
{
"metric": "custom.powershell.gauge",
"points": [
[
1473698742,
96
]
]
}
]
}
-Depth 4 gets you the additional set of square brackets around your point values, and [Ordered] ensures that the hashtable is ordered as originally specified. Don't forget -Compress before sending, like others have said.

As JPBlanc suggested, creating a custom object worked. Below is my code:
[long]$value=Get-Random -Minimum 0 -Maximum 100
$points=,#($unixtime, $value)
$metricname="custom.powershell.gauge"
$obj = [pscustomobject]#{"series" = ,#{"metric" = $metricname; "points"=$points}}
$json=$obj | ConvertTo-Json -Depth 5 -Compress
Which outputs:
{"series":[{"points":[[1434810163,53]],"metric":"custom.powershell.gauge"}]}
Don't forget to specify a depth >2.
Thanks!

Related

Populate collection of objects from one JSON file to the collection of another one with PowerShell

I have two JSON files and want to transfer collection of objects from one file to another. Suppose, the from.json file contains property which represents collection of clients:
"Clients":
[
{
"Name": "Name1",
"Age": "12"
},
{
"Name": "Name2",
"Age": "14"
}
]
to.json file contains an empty collection, "Objects: []" ,which must be filled with objects from from.json. Each objects in toJson variable must contain additional property - Id, so eventually, my "to.json" file should look like this:
"Objects":
[
{
"Id": "{new-id}",
"Name": "Name1",
"Age": "12"
},
{
"Id": "{new-id}",
"Name": "Name1",
"Age": "12"
}
]
I've converted two files into variables:
$fromJson = (Get-Content -Raw -Path {fromPath}) | ConvertFrom-Json
$toJson = (Get-Content -Raw -Path {toPath}) | ConvertFrom-Json
I know that objects from fromJson to toJson can be transferred in the following manner:
toJson.Objects += fromJson.Clients, but that's not enough in my case. I think that it could be done by iterating through fromJson.Clients array but have no idea how to create an object and add it into toJson.Objects collection.
Here's a more efficient solution, based on:
Use of a calculated property with Select-Object, which allows you to place the new property first in the output objects.
Instead of building the array one by one with += (which is inefficient, because a new array must technically be created behind the scenes in every iteration), the solution below lets PowerShell collect the output objects of the Select-Object call in an array automatically (the [array] type constraint is needed to ensure that an array is created even if only one object happens to be output.)
# Sample input.
$fromJson = ConvertFrom-Json '{"Clients":[{"Name":"Name1","Age":"12"},{"Name":"Name2","Age":"14"}]}'
$toJson = ConvertFrom-Json '{ "Objects": [] }'
[array] $toJson.Objects =
$fromJson.Clients |
Select-Object #{ Name='Id'; Expression = { [string] (New-Guid) } }, *
$toJson | ConvertTo-Json -Depth 3 # append | Set-Content as needed.
Kind of new to the PowerShell, but after a bit of investigation came up with the following solution:
fromJson.Clients | ForEach-Object {
$_ | Add-Member -MemberType NoteProperty -Name 'Id' -Value ([guid]::NewGuid().Guid.ToString())
$toJson += $_
}
...
$toJson | ConvertTo-Json | Out-File {to.json_path}
Frankly, don't know if that is a 'proper' way to do that, but generally it works for that particular case. For now, see no other solution.

Reading a json file in key value pair in the same order that's given in input

I am writing a PowerShell Script, which will read a json file having different sections, like job1, job2 and so on.. Now my objective is to read each section separately and to loop through it as a key value pair. I also need to maintain the order of the input file, because the jobs are scheduled in sequence. and these jobs run taking the values from the json file as input.
I tried using Powershell version 5.1, in which I created PSCustomObject but the order is getting sorted alphabetically, which I DON'T want.
Json File :
{ "Job1": [
{
"Ram" : "India",
"Anthony" : "London",
"Elena" : "Zurich"
}],
"Job2": [
{
"Build" : "fail",
"Anthony" : "right",
"Sam" : "left"
}]}
$json = Get-Content -Path C:\PowershellScripts\config_File.json |
ConvertFrom-Json
$obj = $json.Job1
$json.Job1 | Get-Member -MemberType NoteProperty | ForEach-Object {
$key = $_.Name
$values = [PSCustomObject][ordered]#{Key = $key; Value = $obj."$key"}
$values
}
I am expecting to loop through each section separately and in the same order that's provided in the json file. For example looping through Job1 section and to fetch only the Values in the same order that's in the json file.
I will guarantee that this is not the best way to do this, but it works.
$json = Get-Content -Path C:\PowershellScripts\config_File.json |
ConvertFrom-Json
$out = ($json.Job1 | Format-List | Out-String).Trim() -replace "\s+(?=:)|(?<=:)\s+"
$out -split "\r?\n" | ForEach-Object {
[PSCustomObject]#{Key = $_.Split(":")[0]; Value = $_.Split(":")[1]}
}
Explanation:
The JSON object is first output using Format-List to produce the Property : Value format, which is piped to Out-String to make that output a single string. Trim() is used to remove surrounding white space.
The -replace removes all white space before and after : characters.
The -split \r?\n splits the single string into an array of lines. Each of those lines is then split by the : character (.Split(":")). The [0] index selects the string on the left side of the :. The [1] selects the string on the right side of the :.
Can you change the json schema?
I would probably make changes to the json schema before i tried to parse this (if possible of course).
Like this (changed only Job1):
$json = #"
{ "Job1": [
{
"Name": "Ram",
"Location" : "India"
},
{
"Name": "Anthony",
"Location": "London"
},
{
"Name": "Elena" ,
"Location": "Zurich"
}
],
"Job2": [
{
"Build" : "fail",
"Anthony" : "right",
"Sam" : "left"
}]}
"# | convertfrom-json
foreach ($obj in $json.Job1) {
$key = $obj.Name
$values = [PSCustomObject][ordered]#{Key = $key; Value = $obj."$key" }
$values
}

Convert powershell table to json with column 1 as key and column 2 as value

I'm trying to produce a JSON dictionary from a powershell object with formatting that the ConvertTo-Json cmdlet doesn't provide by default.
For example the ConvertTo-Json will take the output of
(get-counter '\Process(*)\% Processor Time').CounterSamples | select InstanceName, CookedValue
which looks like:
InstanceName CookedValue
------------ -----------
idle 92.02923730929
process1 3
process2 1
process3 0
process4 2
process5 0
process6 2
process7 0
.... ....
and produce a JSON object that looks like this:
[
{
"InstanceName": "idle",
"CookedValue": 92.02923730929
},
{
"InstanceName": "process1",
"CookedValue": 3
},
{
"InstanceName": "process2",
"CookedValue": 1
},
...
]
Would it be possible to instead format the JSON object like this?:
{
idle: 92.02923730929,
process1: 3,
process2: 1,
process3: 0,
process4: 2,
process5: 0,
}
Thanks
Create a new object with properties made up of the sample property values:
$properties = #{}
(Get-Counter '\Process(*)\% Processor Time').CounterSamples |ForEach-Object {
$properties[$_.InstanceName] = $_.CookedValue
}
New-Object psobject -Properties $properties |ConvertTo-Json
Since you could have multiple instances with the same process name, you might want to extract the process name and instance number from the counter Path instead:
$properties = #{}
(Get-Counter '\Process(*)\% Processor Time').CounterSamples |ForEach-Object {
$InstanceName = if($_.Path -match 'process\((.*#\d+)\)'){
$Matches[1]
}
else {
$_.InstanceName
}
$properties[$InstanceName] = $_.CookedValue
}
New-Object psobject -Property $properties |ConvertTo-Json
just pipe it to ConvertTo-Json
(get-counter '\Process(*)\% Processor Time').CounterSamples | select InstanceName, CookedValue | ConvertTo-Json
I cannot answer your second question of reformatting the JSON output

Powershell JSON pipeline expand multiple values into one column csv

I'm trying to automate some data pipelines with Powershell, but I'm kinda stuck with converting a JSON list to a single cell per row in a CSV file. Hope some of you can help me out.
The JSON I get looks like the following:
{"result": [
{
"uid": "1",
"EducationHistory": []
},
{
"uid": "2",
"EducationHistory": []
},
{
"uid": "3",
"EducationHistory": []
},
{
"uid": "4",
"EducationHistory": {
"10466632": {
"euid": 10466632,
"degree": "Highschool",
"educationLevel": null
},
"10466634": {
"euid": 10466634,
"degree": "Law",
"educationLevel": "batchelor"
},
"10466635": {
"euid": 10466635,
"degree": "Law",
"educationLevel": "master"
}
}
},
{
"uid": "5",
"EducationHistory": {
"10482462": {
"euid": 10482462,
"degree": "IT",
"educationLevel": "master"
}
}
}
]
}
What I want to do is collect the educationLevels per uid in one column. So something like this:
uid | educationLevel
----+------------------
1 |
2 |
3 |
4 | barchelor, master
5 | master
Normally I would like Expandproperty to get down to a lower level, but this doesn't work for this case because every EducationHistory entry is behind a euid for that specific entry. Expanding every single one of them like in the example below isn't workable because of the number of records.
So I think I need something of a loop, but I don't know how. Hope you can help me. First post here and a Powershell newbie, so I hope my question is clear. Please let me know if you need more info.
The code for one entry, as example:
$json = Get-content -raw -path C:\TEMP\File.json
(ConvertFrom-Json -InputObject $json).result |
Select-Object uid,
#Expand one of the entries:
#{Name = "Edu.Level";E={$_.EducationHistory | Select-Object -
expandproperty 10466632 |Select-Object -expandpropert degree }} |
Format-Table
$content = Get-Content .\test.json
$result = ($content | ConvertFrom-Json).result
$totalResult = #()
foreach($res in $result) {
$tempArray = #()
if($res.EducationHistory -ne $null) {
$properties = $res.EducationHistory | Get-Member -MemberType NoteProperty
foreach($property in $properties) {
$eduLevel = $res.EducationHistory.$($property.Name).educationLevel
if(![String]::IsNullOrEmpty($eduLevel)) {
$tempArray += $eduLevel
}
}
}
$totalResult += [PSCustomObject]#{
uid = $res.uid
educationLevel = $tempArray -join ", "
}
}
$totalResult
This will output desired result for the input you have provided.
The trickiest part is the value of EducationHistory property. You have to use Get-Member cmdlet (see Get-Help Get-Member) to get the properties of the current object in loop. Then using the name of the property to access the educationLevel.
Your first question, my first answer I believe :) Similar to the last answer. You need to jump through the hoop of finding the object names in EducationalHistory to reference them.
$json = (Get-content C:\TEMP\File.json | ConvertFrom-Json).result
$results = #()
foreach ( $u in $json)
{
foreach ( $h in $u.EducationHistory)
{
$results += $h.PSObject.properties.Name | ForEach-Object{new-object PSObject -property #{ uid=$u.uid; degree=$h.$_.degree}}
}
}
$results | ConvertTo-Csv | select -skip 1

Prettify json in powershell 3

Given a standard json string value:
$jsonString = '{ "baz": "quuz", "cow": [ "moo", "cud" ], "foo": "bar" }'
How can I get this to be all pretty with newlines, preferably without brute-force regex?
Simplest method I've found so far is:
$jsonString | ConvertFrom-Json | ConvertTo-Json
However, that seems kinda silly.
Works for me. Parentheses make sure get-content is done before piping. Default depth of convertto-json is 2, which is often too low.
function pjson ($jsonfile) {
(get-content $jsonfile) | convertfrom-json | convertto-json -depth 100 |
set-content $jsonfile
}
If you really don't want to go down the simplest route, which is to use inbuilt PowerShell functions | ConvertFrom-Json | ConvertTo-Json, here is another method, using JSON.net
# http://james.newtonking.com/projects/json-net.aspx
Add-Type -Path "DRIVE:\path\to\Newtonsoft.Json.dll"
$jsonString = '{ "baz": "quuz", "cow": [ "moo", "cud" ], "foo": "bar" }'
[Newtonsoft.Json.Linq.JObject]::Parse($jsonString).ToString()
I put this in my profile
function PrettyPrintJson {
param(
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
$json
)
$json | ConvertFrom-Json | ConvertTo-Json -Depth 100
}
Which works with pipes, and can be auto-completed, so it's at least somewhat less typing:
cat .\file.json | PrettyPrintJson
curl https://api.twitter.com/1.1/statuses/user_timeline.json | PrettyPrintJson
Adding to #JS2010's answer I added logic to escape out certain characters and clean up my output even further. The parenthesis seems key and -depth is a big one since you can lose details without it, from what I've seen, on depth that goes beyond the default of 5, I believe it is.
function Format-Json ($JSON)
{
$PrettifiedJSON = ($JSON) | convertfrom-json | convertto-json -depth 100 | ForEach-Object { [System.Text.RegularExpressions.Regex]::Unescape($_) }
$PrettifiedJSON
}
I think what you are looking for is this:
$jsonString = #{
'baz' = 'quuz'
'cow'= "moo, cud"
'foo'= "bar"
}
$jsonString|ConvertTo-Json
it produces this output
{
"baz": "quuz",
"cow": "moo, cud",
"foo": "bar"
}
Added note
You could also array your cow values to "prettify" it a bit more:
$jsonString = #{
'baz' = 'quuz'
'cow'= #("moo"; "cud")
'foo'= "bar"
}
output:
{
"baz": "quuz",
"cow": [
"moo",
"cud"
],
"foo": "bar"
}