Convert csv to a table in Powershell - json

I have a csv in the format below that needs to be converted to json or a table so that I can parse it easily and use the values as arguments later in my code
Directory : \\foo\foofoo\nightly.188\share\name
Name : name.pst
Length : 271360
CreationTime : 6/4/2009 2:42:21 PM
LastWriteTime : 6/9/2011 8:58:50 AM
Directory : \\foo\foofoo\nightly.188\share\name
Name : name2.pst
Length : 71123968
CreationTime : 10/5/2010 2:41:56 PM
LastWriteTime : 8/1/2011 4:08:32 PM
I have tried using Format-List and Format-Table but does not help.
Any inputs would be appreciated.
I would like the output to be in a tabular structure as:
Directory Name Length CreationTime LastWriteTime
\foo\foofoo\nightly.188\share\name name.pst 271360 6/4/2009 2:42:21 PM 8/1/2011 4:08:32 PM
\foo\foofoo\nightly.188\share\name name2.pst 71123968 10/5/2010 2:41:56 PM 8/1/2011 4:08:32 PM

Format-List and Format-table convert your data into a form that's suitable for display on your console, or copying into a text file. This is useful for parsing, but not for passing the data on as arguments. If you just do an import like this:
Import-Csv myfile.csv
Powershell will do an implicit Format-Table or Format-List for you before putting it on your console.
If you want to use the contents as arguments, be aware that the contents of a csv file are transformed into an array of PSCustomObject. Each PSCustomObject has properties with the same name as a the field name provided in the header of the CSV file.
Import-Csv thefile.csv | % {
"The file $($_.name) contains:"
Get-Content $_.name
}

To view CSV data to the console as a table do the following
$sorceFile = "D:\temp\source.csv"
Import-Csv $sorceFile | Format-Table -AutoSize
You can also use the "Out-GridView" command if you want to view the data interactively
$sorceFile = "D:\temp\source.csv"
Import-Csv $sorceFile | Out-GridView
The following converts CSV data to Json data
$sorceFile = "D:\temp\source.csv"
$JsonData = Import-Csv $sorceFile | ConvertTo-Json
$JsonData
This however is just a string. If you are wanting to process the data further from the CSV file then do the following to save to a variable.
$sorceFile = "D:\temp\source.csv"
$CsvData= Import-Csv $sorceFile
# view formatted table in console
$CsvData | format-table
#example data query
$CsvData | Where-Object{$_.Name -eq 'name.pst'}
#example loop
foreach ($row in $CsvData) {
Write-Output "Name: $($row.name) CreationTime: $(get-date $row.CreationTime -Format 'dddd MM/dd/yyyy HH:mm K')"
}

Related

Adjust json file using ConvertFrom-Json

I have a json file with the following content
{
"name":"myApp",
"version":"1"
}
I would like to use Powershell to read this json file, adjust the version number, then output the new json to the same file.
I currently have this, but it doesn't work and only outputs an empty file
Get-Content file.json | ConvertFrom-Json | ForEach{$_.version = "2"} | ConvertTo-Json | Out-File file.json -Encoding ascii
I am getting stuck on how to update values on the Json object created by ConvertFrom-Json and pipe this back into ConvertTo-Json
I would like to do this all in one line if possible.
Once you assign the JSON to an object, you should be able to change it from there then write it to a file (if desired).
Create JSON object from file contents:
$jsonObject = Get-Content file.json -Raw | ConvertFrom-Json
Change the current object. In your example the "version" property:
#update object
$jsonObject| %{$_.version = "2"}
Write object to file:
$jsonObject | ConvertTo-Json | Out-File "file.json" -Encoding ascii

use where-object to find data, but want to add data to every row also the export to csv

Hi I have a script that reads a csv file, creates a json file, checks the users in the file against a service, then i get the result as a json file.
I take that result and finds the users i csv file and creates a new file.
I do that with a where-object
But i need to add some extra values on every user before i export it to csv
This is my 2 lines for finding users and then export
$matches = $users | where-object { $_.number -in $response.allowedItemIds } | Select-Object -Property Number,Surname,Forename,Emailaddress
$matches | Export-Csv -path $Saved$savefile -NoTypeInformation -Append
Is that possible or do i need to do a for each?
Cheers
Assuming I've interpretted your question correctly, you should be able to use PowerShell's Calculated Properties for this purpose.
For example, if you wanted to add a field called "Date" and set the current Date/Time to each user row, you could do the following:
$matches = $users | where-object { $_.number -in $response.allowedItemIds } | Select-Object -Property Number,Surname,Forename,Emailaddress, #{Name="Date";Expression={Get-Date}}
The Expression value can either be a static value such as "StaticValue", a variable such as $i (useful if used as part of a loop, for example) or more complex value that is returned from other cmdlets (as in my example above)

Parse/Extract JSON data in CSV file

I've hundreds of rows in CSV file while contains JSON data like below. Below is a sample of each row.
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"}
I tried to convert the same into CSV as below but no luck as of now.
Expected format of CSV file:
id RecordType CreationTime Operation
value value value value
$properties = #('Id', 'RecordType', 'CreationTime', 'Operation')
(Get-Content -Path-to_CSVfile -Raw | ConvertFrom-Json) |
Select-Object -Property $properties |
Export-Csv -NoTypeInformation -Path $path-to-new-csv-file
If someone has an idea about this please help me. I tried ConvertTo-Json but it's failing with error:
ConvertFrom-Json : Invalid JSON primitive: "id"
Here are the first two rows of CSV data.
{"Id":"ac325bc9-97f0-4b29-8fc4-90b80b945f6c","RecordType":20,"CreationTime":"2019-09-14T08:07:22","Operation":"AnalyzedByExternalApplication","OrganizationId":"f38a5ecd-2813-4862-b11b-ac1d563c806f","UserType":0,"UserKey":"3fee8456-6d20-4794-8219-5a7c381e965f","Workload":"PowerBI","UserId":"abcd#mail.com","ClientIP":"000.000.50.177","UserAgent":"MSOLAP 15.0 Client","Activity":"AnalyzedByExternalApplication","ItemName":"Other","DatasetName":"XYZ Driven Company","ObjectId":"Other","IsSuccess":true,"RequestId":"6836be8e-6e97-4bc9-a838-bf6e7b71e0c8","ActivityId":"7E92AE6A-F548-448D-93A8-6F5736DEA085"}
{"Id":"3a20c8a9-ef44-483a-b9c0-43e10deae9ae","RecordType":20,"CreationTime":"2019-09-14T08:07:20","Operation":"AnalyzedByExternalApplication","OrganizationId":"f38a5ecd-2813-4862-b11b-ac1d563c806f","UserType":0,"UserKey":"3fee8456-6d20-4794-8219-5a7c381e965f","Workload":"PowerBI","UserId":"abcd#mail.com","ClientIP":"000.000.50.177","UserAgent":"MSOLAP 15.0 Client","Activity":"AnalyzedByExternalApplication","ItemName":"Other","DatasetName":"XYZ Driven Company","ObjectId":"Other","IsSuccess":true,"RequestId":"02e5d772-057b-45b6-ae60-22b7fa610f98","ActivityId":"7E92AE6A-F548-448D-93A8-6F5736DEA085"}
I'm looking this data in another CSV file as below. Each value after ":" should insert into CSV as rows.
Id RecordType CreationTime Operation OrganizationId UserType UserKey Workload UserId ClientIP UserAgent Activity ItemName DatasetName ObjectId IsSuccess RequestId ActivityId
ac325bc9-97f0-4b29-8fc4-90b80b945f6c 20 2019-09-14T08:07:22 AnalyzedByExternalApplication f38a5ecd-2813-4862-b11b-ac1d563c806f 0 3fee8456-6d20-4794-8219-5a7c381e965f PowerBI abcd#mail.com 000.000.50.177 MSOLAP 15.0 Client AnalyzedByExternalApplication Other xyz Driven Company Other TRUE 6836be8e-6e97-4bc9-a838-bf6e7b71e0c8 7E92AE6A-F548-448D-93A8-6F5736DEA085
3a20c8a9-ef44-483a-b9c0-43e10deae9ae 20 2019-09-14T08:07:20 AnalyzedByExternalApplication f38a5ecd-2813-4862-b11b-ac1d563c806f 0 3fee8456-6d20-4794-8219-5a7c381e965f PowerBI abcd#mail.com 000.000.50.177 MSOLAP 15.0 Client AnalyzedByExternalApplication Other XYZ Driven Company Other TRUE 02e5d772-057b-45b6-ae60-22b7fa610f98 7E92AE6A-F548-448D-93A8-6F5736DEA085
Correct data from CSV when opened in text editor.
"{""Id"":""ac325bc9-97f0-4b29-8fc4-90b80b945f6c"",""RecordType"":20,""CreationTime"":""2019-09-14T08:07:22"",""Operation"":""AnalyzedByExternalApplication"",""OrganizationId"":""f38a5ecd-2813-4862-b11b-ac1d563abchrf"",""UserType"":0,""UserKey"":""3fee8456-6d20-4794-8219-5a7c38abcdfe"",""Workload"":""Pxyswer"",""UserId"":""abcd#mail.com"",""ClientIP"":""123.456.50.177"",""UserAgent"":""MSOLAP 15.0 Client"",""Activity"":""AnalyzedByExternalApplication"",""ItemName"":""Other"",""DatasetName"":""ABCD Driven Company"",""ObjectId"":""Other"",""IsSuccess"":true,""RequestId"":""6836be8e-6e97-4bc9-a838-bf6e7b71e0c8"",""ActivityId"":""7E92AE6A-F548-448D-93A8-6F5736DEA085""}"
If your input file contains just that single example data row the code you posted will work. If the input file contains multiple statements like that your code will not work, because it'd be invalid JSON data.
Valid JSON:
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"}
Valid JSON:
[
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"},
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"}
]
Invalid JSON:
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"}
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"}
To convert the latter kind of input data you need to convert each row as a separate JSON document:
$properties = 'Id', 'RecordType', 'CreationTime', 'Operation'
Get-Content 'C:\path\to\input.csv' |
ConvertFrom-Json |
Select-Object $properties |
Export-Csv 'C:\path\to\output.csv' -NoType
To export all input fields except particular ones you'd define the properties to exclude rather than the ones to include:
$exclude = 'foo', 'bar'
Get-Content 'C:\path\to\input.csv' |
ConvertFrom-Json |
Select-Object -Properties * -Exclude $exclude |
Export-Csv 'C:\path\to\output.csv' -NoType
Edit:
Apparently your input file is a CSV with only one column and no header, so you can import it via Import-Csv, but you need to specify the column header yourself. Expand the field to get the individual JSON values, then proceed as described above.
$properties = 'Id', 'RecordType', 'CreationTime', 'Operation'
Import-Csv 'C:\path\to\input.csv' -Header foo |
Select-Object -Expand foo |
ConvertFrom-Json |
Select-Object $properties |
Export-Csv 'C:\path\to\output.csv' -NoType
If you want all JSON values exported, simply omit the Select-Object $properties step.

Powershell: Reading in a column from a .CSV file and then adding a specific string to the beginning and exporting again

I'm attempting to write a script which will read in a CSV generated by querying AD for user information (that part is done) but then will allow me to add a string to the beginning of each value of a column in the CSV file and then export it.
For instance we have this CSV file:
"displayname","Office"
Bob,7142
Janet,8923
SantaClaus,0912
NicCage,0823
I want to take each entry for "Office", add the string "BUG" before it and then export it back out. The modified CSV should look like:
"displayname","Office"
Bob,BUG7142
Janet,BUG8923
SantaClaus,BUG0912
NicCage,BUG0823
At this point, I've been attempting to read in just the "Office" column and then displaying it with "Write-Host". The idea being that if I can do that then maybe I can create a new variable that would be something like:
$BUG = "BUG"
$NewVar = $BUG$Office
Which would hopefully look like the second CSV file. I am extremely new to powershell scripting.
The attempts I've made so far are these:
Attempt #1:
$UserList = Import-CSV C:\Users\username\CSV.csv
$UserList | ForEach-Object ($_.Office) { $UserList }
Attempt #2:
$projectName = import-csv C:\Users\username\CSV.csv | % {$_.Office}
$BUG = "BUG"
$projectName | ForEach-Object ($_) {$projectName}
Attempt #3:
$UserList = Import-CSV C:\Users\username\CSV.csv
#ForEach ($Office in $Userlist) {
#Write-Host $UserList.Office
#}
Attempt #4:
Import-Csv "C:\Users\username\CSV.csv" -Header ("displayname","Office","whenCreated","EmailAddress") | Select-Object Office | Export-CSV -Path C:\users\Username\test.csv
I have gotten it to read out just the Office numbers before using the ForEach-Object loop structure but then it never stops reading out the office numbers so that's unhelpful.
I think I'm going in the right direction, but I just can't figure out how to modify a column like this.
Instead of trying to extract the Office column, just pipe the full data set (all columns) to ForEach-Object, change the value of the Office property and pipe it back to Export-Csv:
$Prefix = "BUG"
Import-Csv .\file.csv | ForEach-Object {
$_.Office = $Prefix + $_.Office
$_
} | Export-Csv .\file_modified.csv -NoTypeInformation

Export Matches array object to CSV in Powershell

I have a powershell script to find particular instances and then export them to CSV. Here's an example of the way the code works
$items = "Hello Tim", "Hola Bob", "Hello Susan"
$filter = $items | Select-String -Pattern "Hello"
$filter | Select-Object Line, Matches | Export-Csv "C:\log.csv"
Invoke-Item "C:\log.csv"
When I run the Select-Object in PS, it's nicely formatted info like this:
However, when I export to CSV, it exports the whole object and writes it as the following string: System.Text.RegularExpressions.Match[]
How can I get it to export just the first match or a listing of all matches into a single field when writing to CSV?
Here is one way using a PSObject:
$items = "Hello Tim", "Hola Bob", "Hello Susan"
$filter = $items | Select-String -Pattern "Hello"
$filter | % {New-Object PSObject -property #{
Line = $_.Line
Matches = $_.Matches.Groups[0].Value}
} | Export-Csv "C:\log.csv" -NoTypeInformation
Quickly note that Matches is an array which may create issues exporting to a csv.
Try joining the array into a string with a chosen delimiter. I used "::" in my example.
$filter | Select Line, #{Expression={$_.Matches -join "::"}; Label="Matches"} | Export-Csv "C:\log.csv"