i have a json extracted from a API exact like this:
{
"LPEONASVVAP0": {
"LPEONASVVAP0": {
"id": "urn:vcloud:vm:f526d27d-e0f9-4d4f-ae81-4824e397c027",
"name": "LPEONASVVAP0",
"description": "_vm_desc_",
"dateCreated": "2021-04-06T14:56:09.640+0000"
}
},
"WDEONDSVDIS6": {
"WDEONDSVDIS6": {
"id": "urn:vcloud:vm:7ed43492-a7ce-4963-b5bb-5ec2ca89477c",
"name": "WDEONDSVDIS6",
"description": "",
"dateCreated": "2021-04-13T13:44:29.973+0000"
}
},
"WDEONASVSTR0": {
"WDEONASVSTR0": {
"id": "urn:vcloud:vm:7afa34fe-b239-4abe-90df-3f270b44db1f",
"name": "WDEONASVSTR0",
"description": "",
"dateCreated": "2021-03-10T16:17:50.947+0000"
}
},
}
I need extract only fields id, name and description to create a csv with them. I test this but the output file is in blank:
$pathToJsonFile = x
$pathToOutputFile = x
$obj = Get-Content $pathToJsonFile -Raw | ConvertFrom-Json
print $obj
$obj | select id, name, description | Convertto-csv > $pathToOutputFile
You'll need to "discover" the parent property names (eg. 'LPEONASVVAP0') via the psobject hidden memberset. Since the outer and inner properties are named the same, we can re-use the name to get the inner property value:
$obj.psobject.Properties |ForEach-Object {
$_.Value.$($_.Name)
} |Select id,name,description |Export-Csv -NoTypeInformation -Path $pathToOutputFile
Edit: Mathias R. Jessens answer is better written than this, i would do it that way instead of how i posted.
Okay so i copied the json you posted, imported it. Since each array of information is stored like this
"WDEONDSVDIS6": {
"WDEONDSVDIS6": {
i used get-member to iterate each of the arrays and then select the info from that.
Also, you dont need to use Convertto-csv > $pathToOutputFile, use the export-csv command instead.
Below is my code how i would have done it, there is probably a better way but this works :)
$pathToOutputFile = x.csv
$obj = Get-Content example.json -Raw| ConvertFrom-Json
$obj2 = ($obj | Get-Member -MemberType noteproperty).Name
$result = foreach($item in $obj2){
$obj.$item.$item | select id,name,description
}
$result | Export-Csv -Path $pathToOutputFile -Encoding utf8 -NoTypeInformation
Related
I have two JSON files and want to transfer collection of objects from one file to another. Suppose, the from.json file contains property which represents collection of clients:
"Clients":
[
{
"Name": "Name1",
"Age": "12"
},
{
"Name": "Name2",
"Age": "14"
}
]
to.json file contains an empty collection, "Objects: []" ,which must be filled with objects from from.json. Each objects in toJson variable must contain additional property - Id, so eventually, my "to.json" file should look like this:
"Objects":
[
{
"Id": "{new-id}",
"Name": "Name1",
"Age": "12"
},
{
"Id": "{new-id}",
"Name": "Name1",
"Age": "12"
}
]
I've converted two files into variables:
$fromJson = (Get-Content -Raw -Path {fromPath}) | ConvertFrom-Json
$toJson = (Get-Content -Raw -Path {toPath}) | ConvertFrom-Json
I know that objects from fromJson to toJson can be transferred in the following manner:
toJson.Objects += fromJson.Clients, but that's not enough in my case. I think that it could be done by iterating through fromJson.Clients array but have no idea how to create an object and add it into toJson.Objects collection.
Here's a more efficient solution, based on:
Use of a calculated property with Select-Object, which allows you to place the new property first in the output objects.
Instead of building the array one by one with += (which is inefficient, because a new array must technically be created behind the scenes in every iteration), the solution below lets PowerShell collect the output objects of the Select-Object call in an array automatically (the [array] type constraint is needed to ensure that an array is created even if only one object happens to be output.)
# Sample input.
$fromJson = ConvertFrom-Json '{"Clients":[{"Name":"Name1","Age":"12"},{"Name":"Name2","Age":"14"}]}'
$toJson = ConvertFrom-Json '{ "Objects": [] }'
[array] $toJson.Objects =
$fromJson.Clients |
Select-Object #{ Name='Id'; Expression = { [string] (New-Guid) } }, *
$toJson | ConvertTo-Json -Depth 3 # append | Set-Content as needed.
Kind of new to the PowerShell, but after a bit of investigation came up with the following solution:
fromJson.Clients | ForEach-Object {
$_ | Add-Member -MemberType NoteProperty -Name 'Id' -Value ([guid]::NewGuid().Guid.ToString())
$toJson += $_
}
...
$toJson | ConvertTo-Json | Out-File {to.json_path}
Frankly, don't know if that is a 'proper' way to do that, but generally it works for that particular case. For now, see no other solution.
Guys this is my JSON file and I want to create a PowerShell script which will give me result like
I have used method like Get-Content and other but there are some issues with the JSON parsing. Please find what is my requirement I have explained in details below.
MyLocalMachineHome
LocalMachine = Sahil_LocalMachine
Second_MyLocalMachine = Sahil_MylocalMachine
Second_MyLocalMachine = ""
Staging
Second_Staging = Sahil;_Secconf
Staging = Sahil_Staging
third_staging = stsajiii
There is also one functionality which I would like to have if I want to get only variables of "staging".
I was using this function Get-Content -Raw -Path E:\shell\Powershell\1ReleasePipelines.json | ConvertFrom-Json | select -ExpandProperty variables on my original JSON file but somehow there is some kind of limit in storing string which I was getting from this method.
{
"environments": [
{
"id": 3,
"name": "MyLocalMachineHome",
"variableGroups": [],
"variables": {
"LocalMachine": {
"value": "Sahil_LocalMachine"
},
"Second_MyLocalMachine": {
"value": "Sahil_MylocalMachine"
},
"thirf_mylocal": {
"value": ""
}
}
},
{
"id": 7,
"name": "Staging",
"variableGroups": [],
"variables": {
"Second_Staging": {
"value": "Sahil;_Secconf"
},
"Staging": {
"value": "Sahil_Staging"
},
"third_staging": {
"value": "stsajiii"
}
}
}
]
}
If we assume that $json contains your JSON content, you can do the following ugly code:
$environment = 'staging'
$j = $json | ConvertFrom-Json
($j.environments | where name -eq $environment).variables | Foreach-Object {
$CurrentObject = $_
$CurrentObject | Get-Member -MemberType NoteProperty |
Select-Object -Expand Name | Foreach-Object {
$CurrentObject.$_.value
}
}
It appears your issue is that you don't know what variables are going to be contained within your JSON. So you can't easily use Select-Object variable or $object.variable. You need a dynamic approach.
If you know your variables ahead of time, things become simpler. You can store your variable names in an array and loop over them.
$variables = 'Second_Staging','Staging','third_staging'
$environment = 'staging'
$j = $json | ConvertFrom-Json
$jsonVars = ($j.environments | where name -eq $environment).variables
$variables | Foreach-Object {
$jsonVars.$_.value
}
View all the sub-properties of variables with format-list instead of format-table. Since the properties vary, format-table won't show all of them. There's a lot of sloppy object construction in json.
$a = get-content file.json
$a.environments.variables | format-table
LocalMachine Second_MyLocalMachine thirf_mylocal
------------ --------------------- -------------
#{value=Sahil_LocalMachine} #{value=Sahil_MylocalMachine} #{value=}
$a.environments.variables | format-list
LocalMachine : #{value=Sahil_LocalMachine}
Second_MyLocalMachine : #{value=Sahil_MylocalMachine}
thirf_mylocal : #{value=}
Second_Staging : #{value=Sahil;_Secconf}
Staging : #{value=Sahil_Staging}
third_staging : #{value=stsajiii}
Get the staging variables?
$a.environments | where name -eq staging | foreach variables
Second_Staging Staging third_staging
-------------- ------- -------------
#{value=Sahil;_Secconf} #{value=Sahil_Staging} #{value=stsajiii}
cls
start-transcript -path 'C:\E\Devops\PowerShell_Chapters\ABC.txt'
write-output "**********Variables of Release************"
get-content -raw -path 'C:\E\Devops\PowerShell_Chapters\Release.json'| Convertfrom-Json | Select -ExpandProperty variables
$json = get-content -raw -path 'C:\E\Devops\PowerShell_Chapters\Release.json'| Convertfrom-Json | Select -ExpandProperty environments
$EnvirnomentsVariables = get-content -raw -path 'C:\E\Devops\PowerShell_Chapters\Release.json'| Convertfrom-Json | Select -ExpandProperty environments |Select -ExpandProperty name
$ReleaseVariable = get-content -raw -path 'C:\E\Devops\PowerShell_Chapters\Release.json'| Convertfrom-Json | Select -ExpandProperty environments |Select -ExpandProperty variables
$i = 0
foreach($a in $EnvirnomentsVariables)
{
$ABC_Staging = $EnvirnomentsVariables[$i]
#write-output $ABC_Staging
if( $ABC_Staging -match "ABC Staging")
{
write-output "****************Variables of " $EnvirnomentsVariables[$i]*************"
#add-content 'C:\E\Devops\PowerShell_Chapters\ABC.txt' $EnvirnomentsVariables[$i]
# Set-content -path 'C:\E\Devops\PowerShell_Chapters\Sahil.json'| ConvertTo-Json | select $EnvirnomentsVariables[$i]
write-output $ReleaseVariable[$i]
# add-content 'C:\E\Devops\PowerShell_Chapters\ABC.txt' $ReleaseVariable[$i]
# Set-content -path 'C:\E\Devops\PowerShell_Chapters\Sahil.json'| ConvertTo-Json | select $ReleaseVariable[$i]
}
$i = $i + 1
}
stop-transcript
Kind of similar to Convert nested JSON array into separate columns in CSV file but instead of flattened csv (ie discipline_01, discicpline_01) exporting to multiple lines of the csv:
{
"data": [{
"attributes": {
"id": 10011,
"title": "Test procedure",
"slug": "slug",
"url": "http://test.test",
"disciplines": [
"discipline_a",
"discipline_b",
"discipline_c"
]
}
}]
}
export to
"id","title","slug","url","discipline"
"10011","Test procedure","slug","http://test.test","discipline_a"
"10011","Test procedure","slug","http://test.test","discipline_b"
"10011","Test procedure","slug","http://test.test","discipline_c"
Thanks to Export fields with nested values from JSON to CSV I've gotten this far:
$foo = invoke-restmethod $restquery -headers $headers
$foo |
select -ExpandProperty data |
select -ExpandProperty attributes |
select id, title, slug, url, disciplines |
foreach {
$_.disciplines = $_disciplines -join ' '
$_ |
export-csv c:\outfile.csv -notypeinformation
This gives me
"10011","Test procedure","slug","http://test.test","discipline_a discipline_b discipline_c"
But no clue how to get it to:
"id","title","slug","url","discipline"
"10011","Test procedure","slug","http://test.test","discipline_a"
"10011","Test procedure","slug","http://test.test","discipline_b"
"10011","Test procedure","slug","http://test.test","discipline_c"
Any help is appreciated.
While your posted Json is invalid,
this might do:
## Q:\Test\2019\05\31\SO_56401395.ps1
$foo = invoke-restmethod $restquery -headers $headers
$Data = $foo.data.attributes | ForEach-Object {
foreach ($discipline in $_.disciplines}(
[PSCustomObject]#{
id = $_.id
title = $_.title
slug = $_.slug
url = $_.url
discipline = $discipline
}
}
}
$Data | Export-Csv c:\outfile.csv -NoTypeInformation
I'm trying to automate some data pipelines with Powershell, but I'm kinda stuck with converting a JSON list to a single cell per row in a CSV file. Hope some of you can help me out.
The JSON I get looks like the following:
{"result": [
{
"uid": "1",
"EducationHistory": []
},
{
"uid": "2",
"EducationHistory": []
},
{
"uid": "3",
"EducationHistory": []
},
{
"uid": "4",
"EducationHistory": {
"10466632": {
"euid": 10466632,
"degree": "Highschool",
"educationLevel": null
},
"10466634": {
"euid": 10466634,
"degree": "Law",
"educationLevel": "batchelor"
},
"10466635": {
"euid": 10466635,
"degree": "Law",
"educationLevel": "master"
}
}
},
{
"uid": "5",
"EducationHistory": {
"10482462": {
"euid": 10482462,
"degree": "IT",
"educationLevel": "master"
}
}
}
]
}
What I want to do is collect the educationLevels per uid in one column. So something like this:
uid | educationLevel
----+------------------
1 |
2 |
3 |
4 | barchelor, master
5 | master
Normally I would like Expandproperty to get down to a lower level, but this doesn't work for this case because every EducationHistory entry is behind a euid for that specific entry. Expanding every single one of them like in the example below isn't workable because of the number of records.
So I think I need something of a loop, but I don't know how. Hope you can help me. First post here and a Powershell newbie, so I hope my question is clear. Please let me know if you need more info.
The code for one entry, as example:
$json = Get-content -raw -path C:\TEMP\File.json
(ConvertFrom-Json -InputObject $json).result |
Select-Object uid,
#Expand one of the entries:
#{Name = "Edu.Level";E={$_.EducationHistory | Select-Object -
expandproperty 10466632 |Select-Object -expandpropert degree }} |
Format-Table
$content = Get-Content .\test.json
$result = ($content | ConvertFrom-Json).result
$totalResult = #()
foreach($res in $result) {
$tempArray = #()
if($res.EducationHistory -ne $null) {
$properties = $res.EducationHistory | Get-Member -MemberType NoteProperty
foreach($property in $properties) {
$eduLevel = $res.EducationHistory.$($property.Name).educationLevel
if(![String]::IsNullOrEmpty($eduLevel)) {
$tempArray += $eduLevel
}
}
}
$totalResult += [PSCustomObject]#{
uid = $res.uid
educationLevel = $tempArray -join ", "
}
}
$totalResult
This will output desired result for the input you have provided.
The trickiest part is the value of EducationHistory property. You have to use Get-Member cmdlet (see Get-Help Get-Member) to get the properties of the current object in loop. Then using the name of the property to access the educationLevel.
Your first question, my first answer I believe :) Similar to the last answer. You need to jump through the hoop of finding the object names in EducationalHistory to reference them.
$json = (Get-content C:\TEMP\File.json | ConvertFrom-Json).result
$results = #()
foreach ( $u in $json)
{
foreach ( $h in $u.EducationHistory)
{
$results += $h.PSObject.properties.Name | ForEach-Object{new-object PSObject -property #{ uid=$u.uid; degree=$h.$_.degree}}
}
}
$results | ConvertTo-Csv | select -skip 1
I have a sample JSON-formatted here which converts fine if I use something like: https://konklone.io/json/
I've tried the following code in PowerShell:
(Get-Content -Path $pathToJsonFile | ConvertFrom-Json)
| ConvertTo-Csv -NoTypeInformation
| Set-Content $pathToOutputFile
But the only result I get is this:
{"totalCount":19,"resultCount":19,"hasMore":false,"results":
How do I go about converting this correctly in PowerShell?
By looking at just (Get-Content -Path $pathToJsonFile) | ConvertFrom-Json it looks like the rest of the JSON is going in to a results property so we can get the result I think you want by doing:
((Get-Content -Path $pathToJsonFile) | ConvertFrom-Json).results |
ConvertTo-Csv -NoTypeInformation |
Set-Content $pathToOutputFile
FYI you can do ConvertTo-Csv and Set-Content in one move with Export-CSV:
((Get-Content -Path $pathToJsonFile) | ConvertFrom-Json).results |
Export-CSV $pathToOutputFile -NoTypeInformation
You have to select the results property inside your CSV using the Select-Object cmdlet together with the -expand parameter:
Get-Content -Path $pathToJsonFile |
ConvertFrom-Json |
Select-Object -expand results |
ConvertTo-Csv -NoTypeInformation |
Set-Content $pathToOutputFile
I was getting my json from a REST web api and found that the following worked:
Invoke-WebRequest -method GET -uri $RemoteHost -Headers $headers
| ConvertFrom-Json
| Select-Object -ExpandProperty <Name of object in json>
| ConvertTo-Csv -NoTypeInformation
| Set-Content $pathToOutputFile
I end up with a perfectly formatted csv file
Trying to use Mark Wrang's answer failed for me. While Piemol's comment from Jan 30 '19 solved a basic problem with Mark Wrang's answer, it also didn't work for me.
JSON strings do not always represent rectangular data sets. They may contain ragged data. For example, the Power BI activities log outputs JSON that contains different members depending on variables like what activities occurred in the requested data or what features were available at the time.
Using Piemol's comment, I processed this JSON:
[
{
"a": "Value 1",
"b": 20,
"g": "Arizona"
},
{
"a": "Value 2",
"b": 40,
"c": "2022-01-01T11:00:00Z"
},
{
"a": "Value 3",
"d": "omicron",
"c": "2022-01-01T12:00:00Z"
},
{
"a": "Value 4",
"b": 60,
"d": "delta",
"e": 14,
"c": "2022-01-01T13:00:00Z"
}
]
The script produced this CSV:
"a","b","g"
"Value 1","20","Arizona"
"Value 2","40",
"Value 3",,
"Value 4","60",
Notice that columns c, d, and e are missing. It appears that Export-CSV uses the first object passed to determine the schema for the CSV to output.
To handle this, use the UnifyProperties function:
function UnifyProperties {
$Names = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase)
$InputCollected = #($Input)
$InputCollected.ForEach({
foreach ($Name in $_.psobject.Properties.Name) { $Null = $Names.Add($Name) }
})
$inputCollected | Select-Object #($Names)
}
$pathToInputFolder = (New-Object -ComObject Shell.Application).NameSpace('shell:Downloads').Self.Path + "\" + "PowerBIActivities\combined\"
$pathToInputFile = $pathToInputFolder + "Activities.json"
$pathToOutputFile = $pathToInputFolder + "Activities.csv"
$content = Get-Content -Path $pathToInputFile -Raw
$psObj = ConvertFrom-Json -InputObject $content
$psObj | UnifyProperties | Export-CSV $pathToOutputFile -NoTypeInformation