Replace column value with null in json file using Powershell - json

Here is how my json file looks like :
{
"count": 12,
"name": "Daily Ticket",
"columnNames": [
"User",
"Channel",
"Date",
"# of Closed Incidents",
"Open",
"Response",
"Remark",
"Closed"
],
"rows": [
[
"abc",
"Service Web",
"\u00272020-06-13 00:00:00\u0027",
"1",
"0",
"0",
"this is a text,please replace with null",
"1"
],
[
"xyz",
"Email",
"\u00272020-06-13 00:00:00\u0027",
"21",
"1",
"0",
"this is a text,please replace with null",
"7"
]
]
}
I want to replace all the values in columns of Remark with null and convert into a csv file using powershell. Please help to achieve this.
I want column names as header and rows as rows separated with comma in csv.
My output csv file should look like below one:
User,Channel,Date,# of Closed Incidents,Open,Response,Remark,Closed
abc,Service Web,\u00272020-06-13 00:00:00\u0027,1,0,0,,1
xyz,Email,\u00272020-06-13 00:00:00\u0027,1,0,0,,1

To convert this json into a CSV file is not that difficult.
Just load the JSON, convert it into an object and loop through the properties building an array of new objects you can save as CSV:
$json = Get-Content -Path 'D:\Test\DailyTicket.json' -Raw | ConvertFrom-Json
$headers = $json.columnNames
$result = foreach ($row in $json.rows) {
# just a precaution to not run into index errors when there are
# more items in the array than there are headers or vice-versa
$items = [math]::Min($row.Count, $headers.Count)
# create a new empty (ordered) hashtable
$hash = [ordered]#{}
for ($i = 0; $i -lt $items; $i++) {
# fill the hashtable, except for iten 'Remark'
$hash[$headers[$i]] = if ($headers[$i] -ne 'Remark') { $row[$i] } else { $null }
}
# If you insist on keeping the apostrophe characters in the date field in unicode format `\u0027`
# $hash['Date'] = $hash['Date'] -replace "'", '\u0027'
# output a PSObject to be collected in array $result
[PsCustomObject]$hash
}
# output on screen
$result | Format-Table -AutoSize
# output to CSV file
$result | Export-Csv -Path 'D:\Test\DailyTicket.csv' -NoTypeInformation
Resulting CSV file:
"User","Channel","Date","# of Closed Incidents","Open","Response","Remark","Closed"
"abc","Service Web","'2020-06-13 00:00:00'","1","0","0",,"1"
"xyz","Email","'2020-06-13 00:00:00'","21","1","0",,"7"

Related

How to remove nested JSON object members

I'm trying to remove the length value pair from the following JSON file:
{
"uuid": "6f74b1ba-0d7c-4c85-955b-2a4309f0e8df",
"records": {
"record1": [
{
"locale": "en_US",
"category": "alpha",
"contents": "My hovercraft is full of eels",
"length": 29
}
],
"record2": [
{
"locale": "cs_CZ",
"category": "alpha",
"contents": "Moje vznášedlo je plné úhořů",
"length": 28
}
]
}
}
Even though the length property is apparently found, it's not deleted, because the output file is identical to the input file.
I'm using the following code:
$infile = "C:\Temp\input.json"
$outfile = "C:\Temp\output.json"
$json = Get-Content $infile -Encoding UTF8 | ConvertFrom-Json
$records = $json.records
$records.PSObject.Properties | ForEach-Object {
if (($_.Value | Get-Member -Name "length")) {
Write-Host "length property found."
$_.Value.PSObject.Properties.Remove("length")
}
}
$json | ConvertTo-Json -Depth 3 | Out-File $outfile -Encoding UTF8
What am I doing wrong?
The record* properties are arrays, so you need a nested loop to process them:
foreach( $property in $records.PSObject.Properties ) {
foreach( $recordItem in $property.Value ) {
if( $recordItem | Get-Member -Name 'length' ) {
$recordItem.PSObject.Properties.Remove( 'length' )
}
}
}
For code clarity and performance I've replaced the ForEach-Object command by the foreach statement. Especially in nested loops, foreach helps to improve clarity as we no longer have to think about the context of the automatic $_ variable. Also the foreach statement is faster as it doesn't involve pipeline overhead.

Remove Object from Array of Objects in Json/powershell

I am importing JSON into Powershell to replace some values, but I also need to remove some objects from there.
Import:
$fileJson = Get-Content -path/Template.json -Encoding UTF8
I have an array of object which looks like this:
{
"resources": [
{
"name": "name1"
"type": "type1"
"....": "....."
},
{
"name": "name2"
"type": "type2"
"....": "....."
},
{
"name": "name3"
"type": "type1"
"....": "....."
}
]
}
and I want to remove a specific object from this array of objects. For example I want to remove Object where "type" equals "type2".
I have already tried to replace values with .Replace, however I can only replace single values and not the complete object.
Is it possible to delete or skip entire object with condition?
Convert the JSON to a custom object:
$fileJson = Get-Content -path/Template.json -Encoding UTF8
$data = $fileJson |ConvertFrom-Json
Use Where-Object to filter the resources array:
$data.resources = #($data.resources |Where-Object type -ne type2)
Convert the now modified object back to JSON and write to disk:
$data |ConvertTo-Json |Set-Content ./path/to/updatedTemplate.json -Encoding UTF8
I suggest to convert json into PowerShell objects, then make desired changes. then convert back to json if needed.
Example
$root = Get-Content -Path "C:\source.json" | ConvertFrom-Json
$root.resources = $root.resources | where type -eq 'type2'
$root | ConvertTo-Json -Depth 5 | Out-File "C:\destination.json"

Converting XML to JSON with Powershell

I am trying to convert an xml file to json for further processing.
This is actually first time I use powershell so I need some help.
I get an xml file with this structure:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="https://www.something..."?>
<tabela>
<naslov>Robno knjigovodstvo - pregled automatskog stanja zaliha artikala</naslov>
<dataset>qu1RobaZalihe</dataset>
<datum_kreiranja>12.02.2021</datum_kreiranja>
<zaglavlje><sifra>Šifra</sifra><naziv>Naziv</naziv><katbroj>Kataloški broj</katbroj><cfJedmj>JedMj</cfJedmj><stanje>Stanje</stanje><cijena>Nabavna cijena</cijena><datum>Datum dokumenta</datum></zaglavlje>
<redovi>
<red>
<sifra>1</sifra>
<naziv>CORE HIT</naziv>
<katbroj>A4B0000006606</katbroj>
<cfJedmj>KOM</cfJedmj>
<stanje>1</stanje>
<cijena>100,00</cijena>
<datum>01.01.2021</datum>
</red>
<red>
<sifra>2</sifra>
<naziv>CORE HIT</naziv>
<katbroj>A4B0000008340</katbroj>
<cfJedmj>KOM</cfJedmj>
<stanje>8</stanje>
<cijena>100,00</cijena>
<datum>01.01.2021</datum>
</red>
...
</redovi>
</tabela>
I need to ger an array of objets separated with comma like this:
[
{
"sifra": "1",
"naziv": "CORE HIT 7050",
"jed_mj": "KOM",
"kolicina": "1",
"skladiste": "1",
"pn": "TNX:A4B0000006606"
},
{
"sifra": "2",
"naziv": "CORE HIT 7020",
"jed_mj": "KOM",
"kolicina": "8",
"skladiste": "1",
"pn": "TNX:A4B0000008340"
},
{
"sifra": "3",
"naziv": "SYSTEM SW LICENCE AND CD-ROM",
"jed_mj": "KOM",
"kolicina": "1",
"skladiste": "1",
"pn": "TNX:A4B0000007760"
},
...
]
... but instead I get only objects like this:
{
"sifra": "1",
"naziv": "CORE HIT 7050",
"pn": "TNX:A4B0000006606",
"jed_mj": "KOM",
"kolicina": "1",
"skladiste": "1"
}
{
"sifra": "2",
"naziv": "CORE HIT 7020",
"pn": "TNX:A4B0000008340",
"jed_mj": "KOM",
"kolicina": "8",
"skladiste": "1"
}
{
"sifra": "3",
"naziv": "SYSTEM SW LICENCE AND CD-ROM",
"pn": "TNX:A4B0000007760",
"jed_mj": "KOM",
"kolicina": "1",
"skladiste": "1"
}
So far I managed to figure out some things but not all :)
Code I use to get this is following:
[xml]$xml = Get-Content xml-skladiste.XML
foreach ($atr in $xml.tabela.redovi.red)
{
$prop = [ordered]#{
'sifra' = $atr.sifra
'naziv' = $atr.naziv
'pn' = $atr.katbroj
'jed_mj' = $atr.cfJedmj
'kolicina' = $atr.stanje
'skladiste' = "1"
}
New-Object -Type PSCustomObject -Property $prop | ConvertTo-Json | Add-Content -Path "output.json"
}
What I have to add in order to get desirable structure?
You are currently creating multiple JSON documents, one for each element of the array $xml.tabela.redovi.red, joined together in a single file.
Move ConvertTo-Json out of the loop to fix the problem. This way you separate the creation of the data from conversion to JSON so ConvertTo-JSON can operate on the complete data structure.
[xml]$xml = Get-Content xml-skladiste.XML
$array = foreach ($atr in $xml.tabela.redovi.red)
{
$prop = [ordered]#{
'sifra' = $atr.sifra
'naziv' = $atr.naziv
'pn' = $atr.katbroj
'jed_mj' = $atr.cfJedmj
'kolicina' = $atr.stanje
'skladiste' = "1"
}
New-Object -Type PSCustomObject -Property $prop
}
$array | ConvertTo-Json | Set-Content -Path "output.json"
Explanation:
$array = foreach ... captures the output of the foreach loop in the $array variable, automatically creating an array.
New-Object ... produces the output that is captured. Note that we didn't have to use Write-Output because of PowerShell's implicit output behaviour (read more about it).
$array | ConvertTo-Json ... passes the whole array to a single invocation of ConvertTo-Json and finally writes it to the output file.

Filter JSON based on the user input powershell

I have user input which is separated by commas and I am using split function to get different values. I have an API that returns some data in JSON. I want to filter data from API Json based on the user input
Powershell code
#Get Input data
$GetIds = Read-Host -Prompt 'Enter Ids:'
#Example 1,2
#If they enter 1,2, I want results data of John and Mark
#API Call Data
$json = #'
{
"results": [
{
"id": "1",
"name": "John",
},
{
"id": "2",
"name": "Mark",
},
{
"id": "3",
"name": "Rachel",
}
]
}
'#
$Obj = ConvertFrom-Json $json
#Split by comma
$userInputData = -split $GetIds
#Filter json with $userInputData
$FilteredData = $json | Where-Object { $_.id -eq #loop through $userInputData }
I want the filtered data to return $json filtered by the userInput data. Thank you
First, use the binary form of the -split operator if you want to split by commas (,) - the unary form splits by whitespace only.
# Sample user input
$GetIds = '1, 2'
# Split by ",", remove surrounding whitespace, convert to integers.
# For brevity, there's no error handling her,
# so an empty / blank input wouldn't be interpreted as id 0,
# and input such as `'1 2'` (no comma) would break.
[int[]] $userInputData = ($GetIds -split ',').Trim()
Next, it is $Obj that you must filter with Where-Object, i.e. the custom-object graph that ConvertFrom-Json parsed your JSON text into, not the raw JSON:
$filteredData = $Obj.results | Where-Object id -in $userInputData
The -in operator allows you to test the LHS for being part of the RHS array.
To put it all together:
Note: Your sample JSON is technically invalid, due to trailing commas after the last property in the .results objects, which I've corrected below. In PowerShell [Core] v6+, ConvertFrom-Json would accept even the invalid JSON, but not in Windows PowerShell.
# Sample user input, in lieu of the Read-Host call.
$GetIds = '1, 2'
# Split by ",", remove surrounding whitespace, convert to integers.
# For brevity, there's no error handling her,
# so an empty / blank input wouldn't be interpreted as id 0,
# and input such as `'1 2'` (no comma) would break.
[int[]] $userInputData = ($GetIds -split ',').Trim()
$Obj = ConvertFrom-Json #'
{
"results": [
{
"id": "1",
"name": "John"
},
{
"id": "2",
"name": "Mark"
},
{
"id": "3",
"name": "Rachel"
}
]
}
'#
$filteredData = $Obj.results | Where-Object id -in $userInputData
# Output the matching objects
$filteredData
The above yields:
id name
-- ----
1 John
2 Mark
How about...
# You can split on the read
$GetIds = (Read-Host -Prompt 'Enter Ids') -split (',')
# Results
<#
1
2
#>
# Your JSON string was not valid
<#
Error: Parse error on line 4:
... "name": "John", }, { "id": "2",
----------------------^
Expecting 'STRING', got '}'
#>
# Corrected, notice the removed comma after the names
$json = #'
{
"results": [
{
"id": "1",
"name": "John"
},
{
"id": "2",
"name": "Mark"
},
{
"id": "3",
"name": "Rachel"
}
]
}
'#
$Obj = $json | ConvertFrom-Json
No need for this...
$userInputData = -split $GetIds
... since the split is on the Read
# Filter json with $userInputData
$Obj.results |
Where-Object id -in $GetIds
# Results
<#
id name
-- ----
1 John
2 Mark
#>

Powershell - json to text or csv

I have a folder with hundreds of json files in it & need to read them & create an output file with the various fields & values in it.
{
"id": "02002010",
"booktitle": "",
"pagetitle": "Demo Page",
"parent": "02002000",
"img": [
{
"imgfile": "02A.png",
"imgname": "02A.png"
}
],
"fmt": "",
"entries": [
{
"itemid": "1",
"partnumber": "1234567",
"partdescription": "Washer",
"partqty": "2",
"Manufacturer": "ACME",
"partdescriptionlocal": "Washer"
},
{
"itemid": "2",
"partnumber": "98765-B",
"partdescription": "Screw",
"partqty": "8",
"Vendor": "Widget Inc",
"TYPE": "Galv",
"partdescriptionlocal": "Screw"
}]
}
The json files will have generally the same structure, except that the "entries" may contain various fields in it that may not be the same from one entry to the next, or one json file to the next. Some may have fields within entry that I do not know the name of. There will be a few common fields in each "entries" section, but they could vary, and could be in a different order than what is shown.
I would like to write the output to a text/csv file that would be delimited that could then be imported into Excel. One column header with all fields listed. As new "entries" fields are found, tack them on to the end of each row & add to the header also.
you mean to do something like this?
$json = gc C:\temp\file.json | ConvertFrom-Json
$props = $json.entries | % {$_ | gm -MemberType NoteProperty} | select -exp name -Unique
$results = #()
foreach ($entry in $json.entries) {
$obj = $json | select *
foreach ($prop in $props) {
$obj | Add-Member -MemberType NoteProperty -Name $prop -Value $($entry | select -exp $prop -ea 0)
}
$results += $obj
}
$results | epcsv C:\temp\file.csv -NoTypeInformation -Encoding ASCII
$results