I need some help with converting a CSV File to Json using PowerShell. I have a script that gets the send message logs from Office365 using PowerShell and export them into a CSV file. I will then use the ConvertTo-Json cmdlet to Convert the CSV file into Json. The output Json file includes so many unnecessary texts that make it hard to read. For example, here is what I want:
{
"SenderAddress": "support#email.com",
"RecipientAddress": "zzzz#email.edu",
"Subject": "Microsoft PowerShell",
"Status": "Delivered",
}
Here is what I get after converting the CSV to Json:
{
"value": "\"support#email.com\",\"zzzz#email.edu\",\"Microsoft PowerShell\",\"Delivered\"",
"PSPath": "C:\\Logs\\Logs-1.csv",
"PSParentPath": "C:\\Logs",
"PSChildName": "Logs-1.csv",
"PSDrive": {
"CurrentLocation": "Windows\\system32",
"Name": "C",
"Provider": "Microsoft.PowerShell.Core\\FileSystem",
"Root": "C:\\",
"Description": "Windows",
"MaximumSize": null,
"Credential": "System.Management.Automation.PSCredential",
"DisplayRoot": null
},
"PSProvider": {
"ImplementingType": "Microsoft.PowerShell.Commands.FileSystemProvider",
"HelpFile": "System.Management.Automation.dll-Help.xml",
"Name": "FileSystem",
"PSSnapIn": "Microsoft.PowerShell.Core",
"ModuleName": "Microsoft.PowerShell.Core",
"Module": null,
"Description": "",
"Capabilities": 52,
"Home": "C:\\Users\\test",
"Drives": "C D E"
},
"ReadCount": 3
Here is the PowerShell cmdlet I run to convert the CSV file to Json.
Get-Content "C:\Logs\logs-1.csv" | ConvertTo-Json | Add-Content -Path "C:\Logs\logs-1.json
I also tried the following:
Get-Content "C:\Logs\logs-1.csv" |Select-Object -Property SenderAddress, RecipientAddress, Subject, Status ConvertTo-Json | Add-Content -Path "C:\Logs\logs-1.json
I go this result:
{
"SenderAddress": null,
"RecipientAddress": null,
"Subject": null,
"Status": null
}
Any idea what I can do to convert the CSV file to Json correctly without the extra info?
Thanks in advance for any help.
I'm guessing you need to use Import-Csv not Get-Content or Get-Item.
Try:
Import-Csv "C:\Logs\logs-1.csv" |
ConvertTo-Json |
Add-Content -Path "C:\Logs\logs-1.json"
Or:
Import-Csv "C:\Logs\logs-1.csv" |
Select-Object -Property SenderAddress, RecipientAddress, Subject, Status |
ConvertTo-Json |
Add-Content -Path "C:\Logs\logs-1.json"
If you intend to overwrite C:\Logs\logs-1.json, use Set-Content instead of Add-Content.
Also, you could define how many levels of contained objects are included in the JSON representation (default depth = 2).
import-csv .\INPUTFILE.csv | ConvertTo-Json -depth 100 | Out-File .\OUTPUTFILE.json
Related
I have a need to export the json results of a Invoke-WebRequest call to a .CSV file so that non-technical users can edit the data. This is what the json looks like when I save it to a .CSV file:
{
"Forecasts": [
{
"AccountNumber": "12345",
"Period": "2022-02-01T00:00:00",
"Field1": 3998.87968239,
"Field2": 2133.91206875
},
{
"AccountNumber": "12346",
"Period": "2022-03-01T00:00:00",
"Field1": 6741.483,
"Field2": 4007.857
}
]
}
I'm creating the .CSV file with this line of code:
$json.Forecasts | ForEach-Object { $_ | Export-Csv -path $fileName -NoTypeInformation -Append -Force }
And this is working good. After the user edits the data and saves and closes the .CSV file, I need to convert the .CSV data back into a JArray that looks like the original json structure (like above) so I can make another Invoke-WebRequest call (Patch), passing the JArray as the Body.
When I execute this line of code:
$editedJson = Get-Content -Raw -Path "$fileName" | ConvertFrom-CSV | Select-Object -Property AccountNumber,Period,Field1,Field2 | ConvertTo-json
The json looks like this:
[
{
"AccountNumber": "12345",
"Period": "2022-02-01T00:00:00",
"Field1": "3900.87968239",
"Field2": "2100.91206875"
},
{
"AccountNumber": "12346",
"Period": "2022-03-01T00:00:00",
"Field1": "6700.483",
"Field2": "4000.857"
}
]
As you can see, it's missing the "Forecasts" portion of the JArray and I don't know how to get the .CSV data back into this format. Can anyone help with this ?
You simply need to create an object with that property (Forecasts) and attach the imported CSV to said property:
# This line would be where you import the CSV:
# $csv = Import-Csv .....
$csv = #'
"AccountNumber","Period","Field1","Field2"
"12345","2/1/2022 12:00:00 AM","3998.87968239","2133.91206875"
"12346","3/1/2022 12:00:00 AM","6741.483","4007.857"
'# | ConvertFrom-Csv
#{
Forecasts = $csv
} | ConvertTo-Json
If you want to assure that the JSON representation of Forecasts is always an array (even when the CSV only has one row):
#{
Forecasts = [array]$csv
} | ConvertTo-Json
I'm trying to write a powershell script that could read a JSON, its extension is .pros, then change the project name from "test" to "test 12/12 13:24" or something similar. So far I was able to read it, but then I can't get to the project_name field.
my code:
$pros = Get-ChildItem -Path $house -Recurse -Include *.pros
$time = Get-Date -Format 'u'
$pog = Get-Content $pros -raw | ConvertFrom-Json
$pog.update | $_.project_name=$time
$pog | ConvertTo-Json -depth 32| set-content $pros
Write-Output $time
The .pros:
{
"py/object": "pros.conductor.project.Project",
"py/state": {
"project_name": "Marcos_Yuck_new",
"target": "v5",
"templates": {
"kernel": {
"location": "/home/dragon/.config/pros/templates/kernel#3.5.3",
"metadata": {
I have not used PowerShell much. I have an API request which returns JSON that I need to turn into a CSV or xlsx file.
$output = Get-SurveyParticipents `
-url "https://orxsurveys.limequery.com/admin/remotecontrol" `
-session $sessionKey `
-id "5133965" `
-start "0" `
-limit "2" `
-unused $False `
-attributes ["completed", "usesleft"]
Write-Host($output | ConvertTo-Json)
{
"id": 1,
"result": [
{
"tid": "6",
"token": "35ddmyQTlNpzLat",
"participant_info": "#{firstname=Hsdfng; lastname=Gsdfh; email=gosdfdsfz.com}"
},
{
"tid": "7",
"token": "nQ_S838LjYT4mR6",
"participant_info": "#{firstname=Ofdlga; lastname=Yadfdfa; email=olsdfdsfivska#axsdfdsfnce.com}"
}
],
"error": null
}
The participant_info doesn't look like a normal JSON structure.
Also I am unsure how to turn this JSON into a CSV. Something like:
$output | ConvertTo-Json | Export-Csv -Path "c:\Scripts"
You don't need to do both Export-Csv and ConvertTo-Json. Export-Csv will convert a stream of powershell objects into a file containing CSV records.
Assuming the objects in the "results" array in your json output are what you want, you might try something like:
$output.result | Export-Csv -NoTypeInformation foo.csv
You are correct to say that participant_info is not Json. You'll have to process this yourself. ConvertFrom-StringData might be helpful.
I have one json file mytest.json like below I want to update values using PowerShell script
update.json
{
"update": [
{
"Name": "test1",
"Version": "2.1"
},
{
"Name": "test2",
"Version": "2.1"
}
]
}
I want to write a PowerShell script where if Name=="test1" I want to update Version= "3"
How can i do it using parameters?
Here is a way :
$a = Get-Content 'D:\temp\mytest.json' -raw | ConvertFrom-Json
$a.update | % {if($_.name -eq 'test1'){$_.version=3.0}}
$a | ConvertTo-Json -depth 32| set-content 'D:\temp\mytestBis.json'
According to #FLGMwt and #mikemaccana I improve the ConvertTo-Json with -depth 32 because the default depth value is 2 and for object deeper than 2 you will receive class informations in spite of objects.
I have also faced the same kind of issue. I was looking to change the records of the below JSON file
{
"SQS_QUEUE_URL": "https://que-url.com/server1",
"SQS_EVENTS_QUEUE_URL": "https://events-server.com/server1/development_events",
"REGION": "region1",
"BUCKET": "test-bucket",
"AE_WORK_PATH": "C:\\workpath\\path1",
"ENV": "env"
}
Finally, I managed to find the easiest way to generate a JSON file from Powershell.
$json = Get-Content "c:\users\bharat.gadade\desktop\test.json" | ConvertFrom-Json
$json.SQS_QUEUE_URL = "https://que-url.com/server2"
$json.SQS_EVENTS_QUEUE_URL = "https://events-server.com/Server2/development_events"
$json.REGION = "region1 "
$json.BUCKET = "test-bucket"
$json.AE_WORK_PATH = "C:\workpath\path1"
$json.ENV = "env"
$json | ConvertTo-Json | Out-File "c:\users\bharat.gadade\desktop\test.json"
I have converted the following JSON file to powershell representation object.
{
"computer": [
{
"children": [
{
"children": [ {
"children": [ {
"path": "T:\Dropbox\kvaki.html",
"name": "kvaki",
"type": "url",
"url": "http://example.com"
} ],
"path": "T:\Dropbox\",
"name": "Njusha",
"type": "folder"
}, {
"path": "T:\Dropbox\Europa.html",
"name": "Europa",
"type": "url",
"url": "http://example.com"
}, {
"path": "T:\Dropbox\math.html",
"name": "math",
"type": "url",
"url": "http://example.com"
} ],
"path": "T:\Dropbox\",
"name": "Money",
"type": "folder"
}
],
"full_path_on_file_sys": "T:\Dropbox\"
}
]
}
After doing some computations with powershell representation I would like to save it to file as JSON.
But command $jsonRepresentation | ConvertTo-Json | Out-File "D:\dummy_path\file.json" saves it in this way
{
"computer": [
{
"children": " ",
"full_path_on_file_sys": "T:\Dropbox\"
}
]
}
Question: how to achieve correct saving of complex powershell JSON representation?
-depth argument for ConvertTo-Json solves the issue.
$jsonRepresentation | ConvertTo-Json -depth 100 | Out-File "D:\dummy_path\file.json"
Just pipe it to Set-Content, or Out-File:
Get-Process powershell |
ConvertTo-Json |
Set-Content json.txt
If you want to both view the output and save it to file, you can pipe the tee command.
Get-Process powershell | ConvertTo-Json | Tee-Object json.txt
$json.properties.metadata | ConvertTo-Json -Compress
if you are stuck with PowerShell Version 2, the JSON module of Joel Bennett from the 'PowerShell Code Repository' might help.
1) The below command can be used to convert a json to CSV
Example:
Get-Content package.json | Out-String | ConvertFrom-Json | Select parameter1, parameter2, parameter3 | ConvertTo-Csv -NoTypeInformation | Format-Table >> C:\JenkinsWorkspace\Result.csv
Get-Content: This is like "cat" command in linux which will get all data of file "package.json" and converts from Json (Using ConvertFrom-Json function) extracting the details of only required parameters and then converting them into CSV using "ConvertTo-Csv" function without any unwanted Type Headers and formatting them into Table.
2) The above result can also be formatted into proper csv to view it as Excel format without any duplicates and also having Text-To-Column conversion using below command:
Import-Csv "C:\Result.csv" -delimiter "," | Sort-Object _from -Unique | Export-csv "C:\FINAL_REPORT_$date.csv"