I am trying to convert an xml file to json for further processing.
This is actually first time I use powershell so I need some help.
I get an xml file with this structure:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="https://www.something..."?>
<tabela>
<naslov>Robno knjigovodstvo - pregled automatskog stanja zaliha artikala</naslov>
<dataset>qu1RobaZalihe</dataset>
<datum_kreiranja>12.02.2021</datum_kreiranja>
<zaglavlje><sifra>Šifra</sifra><naziv>Naziv</naziv><katbroj>Kataloški broj</katbroj><cfJedmj>JedMj</cfJedmj><stanje>Stanje</stanje><cijena>Nabavna cijena</cijena><datum>Datum dokumenta</datum></zaglavlje>
<redovi>
<red>
<sifra>1</sifra>
<naziv>CORE HIT</naziv>
<katbroj>A4B0000006606</katbroj>
<cfJedmj>KOM</cfJedmj>
<stanje>1</stanje>
<cijena>100,00</cijena>
<datum>01.01.2021</datum>
</red>
<red>
<sifra>2</sifra>
<naziv>CORE HIT</naziv>
<katbroj>A4B0000008340</katbroj>
<cfJedmj>KOM</cfJedmj>
<stanje>8</stanje>
<cijena>100,00</cijena>
<datum>01.01.2021</datum>
</red>
...
</redovi>
</tabela>
I need to ger an array of objets separated with comma like this:
[
{
"sifra": "1",
"naziv": "CORE HIT 7050",
"jed_mj": "KOM",
"kolicina": "1",
"skladiste": "1",
"pn": "TNX:A4B0000006606"
},
{
"sifra": "2",
"naziv": "CORE HIT 7020",
"jed_mj": "KOM",
"kolicina": "8",
"skladiste": "1",
"pn": "TNX:A4B0000008340"
},
{
"sifra": "3",
"naziv": "SYSTEM SW LICENCE AND CD-ROM",
"jed_mj": "KOM",
"kolicina": "1",
"skladiste": "1",
"pn": "TNX:A4B0000007760"
},
...
]
... but instead I get only objects like this:
{
"sifra": "1",
"naziv": "CORE HIT 7050",
"pn": "TNX:A4B0000006606",
"jed_mj": "KOM",
"kolicina": "1",
"skladiste": "1"
}
{
"sifra": "2",
"naziv": "CORE HIT 7020",
"pn": "TNX:A4B0000008340",
"jed_mj": "KOM",
"kolicina": "8",
"skladiste": "1"
}
{
"sifra": "3",
"naziv": "SYSTEM SW LICENCE AND CD-ROM",
"pn": "TNX:A4B0000007760",
"jed_mj": "KOM",
"kolicina": "1",
"skladiste": "1"
}
So far I managed to figure out some things but not all :)
Code I use to get this is following:
[xml]$xml = Get-Content xml-skladiste.XML
foreach ($atr in $xml.tabela.redovi.red)
{
$prop = [ordered]#{
'sifra' = $atr.sifra
'naziv' = $atr.naziv
'pn' = $atr.katbroj
'jed_mj' = $atr.cfJedmj
'kolicina' = $atr.stanje
'skladiste' = "1"
}
New-Object -Type PSCustomObject -Property $prop | ConvertTo-Json | Add-Content -Path "output.json"
}
What I have to add in order to get desirable structure?
You are currently creating multiple JSON documents, one for each element of the array $xml.tabela.redovi.red, joined together in a single file.
Move ConvertTo-Json out of the loop to fix the problem. This way you separate the creation of the data from conversion to JSON so ConvertTo-JSON can operate on the complete data structure.
[xml]$xml = Get-Content xml-skladiste.XML
$array = foreach ($atr in $xml.tabela.redovi.red)
{
$prop = [ordered]#{
'sifra' = $atr.sifra
'naziv' = $atr.naziv
'pn' = $atr.katbroj
'jed_mj' = $atr.cfJedmj
'kolicina' = $atr.stanje
'skladiste' = "1"
}
New-Object -Type PSCustomObject -Property $prop
}
$array | ConvertTo-Json | Set-Content -Path "output.json"
Explanation:
$array = foreach ... captures the output of the foreach loop in the $array variable, automatically creating an array.
New-Object ... produces the output that is captured. Note that we didn't have to use Write-Output because of PowerShell's implicit output behaviour (read more about it).
$array | ConvertTo-Json ... passes the whole array to a single invocation of ConvertTo-Json and finally writes it to the output file.
Related
I am trying to write a powershell script that creates registry keys and their values from a csv or json file containing the list of the registries .
did any one code such thing can help ? :)
Using ConvertFrom-Json, convert your JSON to a custom PSCustomObject object. Then iterate through the object properties and call New-ItemProperty to create the new registry entries with the appropriate values.
PowerShell
$jsonFile = "<path to JSON file>"
$customObject = Get-Content $jsonFile | ConvertFrom-Json
$customObject.PSObject.Properties | ForEach-Object {
[void](New-ItemProperty -LiteralPath $_.Value.path -Name $_.Value.name -Value $_.Value.value -PropertyType $_.Value.type -Force)
}
JSON
{
"reg1": {
"path": "path1",
"name": "name1",
"value": "value1",
"type": "type1"
},
"reg2": {
"path": "path2",
"name": "name2",
"value": "value2",
"type": "type2"
},
"reg3": {
"path": "path3",
"name": "name3",
"value": "value3",
"type": "type3"
}
}
Links
ConvertFrom-Json (learn.microsoft.com)
New-ItemProperty (learn.microsoft.com)
How can I make my code a lot more performant when I wish to make an easily/fast parseable object/PSCustomObject from a JSON payload $JSON?
An example of the structure of the PAYLOAD I receive is:
[
{
"name": "system.enablenetflow",
"value": "false"
},
{
"name": "system.deviceGroupId",
"value": "186,3060"
},
{
"name": "system.prefcollectorid",
"value": "144"
},
{
"name": "system.collectorplatform",
"value": "windows"
}
]
As you can see its in a very annoying format.
Note that the payloads I attempt to parse are much larger and variable in count from 500 of these Name/Value objects to 50000, rather than just the 4 listed above.
###########################################################################
MY GOAL
To have this turn into a key:value pair scenario for easier parsing later
NOT This:
With the JSON I have to do $JSON.where({$_.name -eq "system.enablenetflow"}).value
YES THIS:
I want the end state to be that the new variable $obj I create will let me get the value with $obj."system.enablenetflow"
###########################################################################
MY CURRENT ATTEMPT THAT IS SUPER SLOW
I did the following:
Create an Empty PSCustomObject and saved it as variable $obj
Did a foreach method on the $JSON variable which iterated through the JSON Array
Add-Member to $obj with setting the 'name' as PropertyName and 'value' as PropertyValue
Heres a sample of my code:
$obj = [PSCustomObject] #{}
$json.foreach({
$thisitem = $_
$obj | Add-member -NotePropertyName $($thisitem.name) -NotePropertyValue $($thisitem.name)
})
HOW CAN I MAKE THIS FASTER?
# Sample input JSON.
$json = #'
[
{
"name": "system.enablenetflow",
"value": "false"
},
{
"name": "system.deviceGroupId",
"value": "186,3060"
},
{
"name": "system.prefcollectorid",
"value": "144"
},
{
"name": "system.collectorplatform",
"value": "windows"
}
]
'#
# Initialize the (ordered) result hashtable.
$result = [ordered] #{}
# Note: In PowerShell (Core) 7+, add -AsHashTable to the ConvertFrom-Json
# call for additional performance gain, combined with -AsArray,
# in which case you don't need the `(...)` around the call anymore.
foreach ($element in (ConvertFrom-Json $json)) {
$result[$element.name] = $element.value
}
The above creates an (ordered) hashtable instead of a [pscustomobject] instance - especially if the latter are iteratively constructed via Add-Member calls.
Hashtables are lighter-weight and faster to construct than [pscustomobject] instances.
Using a foreach loop rather than processing the ConvertFrom-Json output in a pipeline via ForEach-Object also speeds up processing.
PowerShell allows you to use the familiar dot notation also with hashtables; so, for instance, after running the above, you'll get:
PS> $result.'system.collectorplatform'
windows
If you do need $result to be a [pscustomobject] instance, you can simply cast the fully populated hashtable to that type:
PS> $obj = [pscustomobject] $result; $obj.'system.collectorplatform'
windows
Here is how my json file looks like :
{
"count": 12,
"name": "Daily Ticket",
"columnNames": [
"User",
"Channel",
"Date",
"# of Closed Incidents",
"Open",
"Response",
"Remark",
"Closed"
],
"rows": [
[
"abc",
"Service Web",
"\u00272020-06-13 00:00:00\u0027",
"1",
"0",
"0",
"this is a text,please replace with null",
"1"
],
[
"xyz",
"Email",
"\u00272020-06-13 00:00:00\u0027",
"21",
"1",
"0",
"this is a text,please replace with null",
"7"
]
]
}
I want to replace all the values in columns of Remark with null and convert into a csv file using powershell. Please help to achieve this.
I want column names as header and rows as rows separated with comma in csv.
My output csv file should look like below one:
User,Channel,Date,# of Closed Incidents,Open,Response,Remark,Closed
abc,Service Web,\u00272020-06-13 00:00:00\u0027,1,0,0,,1
xyz,Email,\u00272020-06-13 00:00:00\u0027,1,0,0,,1
To convert this json into a CSV file is not that difficult.
Just load the JSON, convert it into an object and loop through the properties building an array of new objects you can save as CSV:
$json = Get-Content -Path 'D:\Test\DailyTicket.json' -Raw | ConvertFrom-Json
$headers = $json.columnNames
$result = foreach ($row in $json.rows) {
# just a precaution to not run into index errors when there are
# more items in the array than there are headers or vice-versa
$items = [math]::Min($row.Count, $headers.Count)
# create a new empty (ordered) hashtable
$hash = [ordered]#{}
for ($i = 0; $i -lt $items; $i++) {
# fill the hashtable, except for iten 'Remark'
$hash[$headers[$i]] = if ($headers[$i] -ne 'Remark') { $row[$i] } else { $null }
}
# If you insist on keeping the apostrophe characters in the date field in unicode format `\u0027`
# $hash['Date'] = $hash['Date'] -replace "'", '\u0027'
# output a PSObject to be collected in array $result
[PsCustomObject]$hash
}
# output on screen
$result | Format-Table -AutoSize
# output to CSV file
$result | Export-Csv -Path 'D:\Test\DailyTicket.csv' -NoTypeInformation
Resulting CSV file:
"User","Channel","Date","# of Closed Incidents","Open","Response","Remark","Closed"
"abc","Service Web","'2020-06-13 00:00:00'","1","0","0",,"1"
"xyz","Email","'2020-06-13 00:00:00'","21","1","0",,"7"
I have a folder with hundreds of json files in it & need to read them & create an output file with the various fields & values in it.
{
"id": "02002010",
"booktitle": "",
"pagetitle": "Demo Page",
"parent": "02002000",
"img": [
{
"imgfile": "02A.png",
"imgname": "02A.png"
}
],
"fmt": "",
"entries": [
{
"itemid": "1",
"partnumber": "1234567",
"partdescription": "Washer",
"partqty": "2",
"Manufacturer": "ACME",
"partdescriptionlocal": "Washer"
},
{
"itemid": "2",
"partnumber": "98765-B",
"partdescription": "Screw",
"partqty": "8",
"Vendor": "Widget Inc",
"TYPE": "Galv",
"partdescriptionlocal": "Screw"
}]
}
The json files will have generally the same structure, except that the "entries" may contain various fields in it that may not be the same from one entry to the next, or one json file to the next. Some may have fields within entry that I do not know the name of. There will be a few common fields in each "entries" section, but they could vary, and could be in a different order than what is shown.
I would like to write the output to a text/csv file that would be delimited that could then be imported into Excel. One column header with all fields listed. As new "entries" fields are found, tack them on to the end of each row & add to the header also.
you mean to do something like this?
$json = gc C:\temp\file.json | ConvertFrom-Json
$props = $json.entries | % {$_ | gm -MemberType NoteProperty} | select -exp name -Unique
$results = #()
foreach ($entry in $json.entries) {
$obj = $json | select *
foreach ($prop in $props) {
$obj | Add-Member -MemberType NoteProperty -Name $prop -Value $($entry | select -exp $prop -ea 0)
}
$results += $obj
}
$results | epcsv C:\temp\file.csv -NoTypeInformation -Encoding ASCII
$results
Consider JSON in this format :
"Stuffs": [
{
"Name": "Darts",
"Type": "Fun Stuff"
},
{
"Name": "Clean Toilet",
"Type": "Boring Stuff"
}
]
In PowerShell 3, we can obtain a list of Stuffs :
$JSON = Get-Content $jsonConfigFile | Out-String | ConvertFrom-Json
Assuming we don't know the exact contents of the list, including the ordering of the objects, how can we retrieve the object(s) with a specific value for the Name field ?
Brute force, we could iterate through the list :
foreach( $Stuff in $JSON.Stuffs ) {
But I am hopeful there exists a more direct mechanism ( similar to Lync or Lambda expressions in C# ).
$json = #"
{
"Stuffs":
[
{
"Name": "Darts",
"Type": "Fun Stuff"
},
{
"Name": "Clean Toilet",
"Type": "Boring Stuff"
}
]
}
"#
$x = $json | ConvertFrom-Json
$x.Stuffs[0] # access to Darts
$x.Stuffs[1] # access to Clean Toilet
$darts = $x.Stuffs | where { $_.Name -eq "Darts" } #Darts
I just asked the same question here: https://stackoverflow.com/a/23062370/3532136
It has a good solution. I hope it helps ^^.
In resume, you can use this:
The Json file in my case was called jsonfile.json:
{
"CARD_MODEL_TITLE": "OWNER'S MANUAL",
"CARD_MODEL_SUBTITLE": "Configure your download",
"CARD_MODEL_SELECT": "Select Model",
"CARD_LANG_TITLE": "Select Language",
"CARD_LANG_DEVICE_LANG": "Your device",
"CARD_YEAR_TITLE": "Select Model Year",
"CARD_YEAR_LATEST": "(Latest)",
"STEPS_MODEL": "Model",
"STEPS_LANGUAGE": "Language",
"STEPS_YEAR": "Model Year",
"BUTTON_BACK": "Back",
"BUTTON_NEXT": "Next",
"BUTTON_CLOSE": "Close"
}
Code:
$json = (Get-Content "jsonfile.json" -Raw) | ConvertFrom-Json
$json.psobject.properties.name
Output:
CARD_MODEL_TITLE
CARD_MODEL_SUBTITLE
CARD_MODEL_SELECT
CARD_LANG_TITLE
CARD_LANG_DEVICE_LANG
CARD_YEAR_TITLE
CARD_YEAR_LATEST
STEPS_MODEL
STEPS_LANGUAGE
STEPS_YEAR
BUTTON_BACK
BUTTON_NEXT
BUTTON_CLOSE
Thanks to mjolinor.
David Brabant's answer led me to what I needed, with this addition:
x.Stuffs | where { $_.Name -eq "Darts" } | Select -ExpandProperty Type
Hows about this:
$json=Get-Content -Raw -Path 'my.json' | Out-String | ConvertFrom-Json
$foo="TheVariableYourUsingToSelectSomething"
$json.SomePathYouKnow.psobject.properties.Where({$_.name -eq $foo}).value
which would select from json structured
{"SomePathYouKnow":{"TheVariableYourUsingToSelectSomething": "Tada!"}
This is based on this accessing values in powershell SO question
. Isn't powershell fabulous!
In regards to PowerShell 5.1 ...
Operating off the assumption that we have a file named jsonConfigFile.json with the following content from your post:
{
"Stuffs": [
{
"Name": "Darts",
"Type": "Fun Stuff"
},
{
"Name": "Clean Toilet",
"Type": "Boring Stuff"
}
]
}
This will create an ordered hashtable from a JSON file to help make retrieval easier:
$json = [ordered]#{}
(Get-Content "jsonConfigFile.json" -Raw | ConvertFrom-Json).PSObject.Properties |
ForEach-Object { $json[$_.Name] = $_.Value }
$json.Stuffs will list a nice hashtable, but it gets a little more complicated from here. Say you want the Type key's value associated with the Clean Toilet key, you would retrieve it like this:
$json.Stuffs.Where({$_.Name -eq "Clean Toilet"}).Type
It's a pain in the ass, but if your goal is to use JSON on a barebones Windows 10 installation, this is the best way to do it as far as I've found.
This is my json data:
[
{
"name":"Test",
"value":"TestValue"
},
{
"name":"Test",
"value":"TestValue"
}
]
Powershell script:
$data = Get-Content "Path to json file" | Out-String | ConvertFrom-Json
foreach ($line in $data) {
$line.name
}