Export Matches array object to CSV in Powershell - csv

I have a powershell script to find particular instances and then export them to CSV. Here's an example of the way the code works
$items = "Hello Tim", "Hola Bob", "Hello Susan"
$filter = $items | Select-String -Pattern "Hello"
$filter | Select-Object Line, Matches | Export-Csv "C:\log.csv"
Invoke-Item "C:\log.csv"
When I run the Select-Object in PS, it's nicely formatted info like this:
However, when I export to CSV, it exports the whole object and writes it as the following string: System.Text.RegularExpressions.Match[]
How can I get it to export just the first match or a listing of all matches into a single field when writing to CSV?


Here is one way using a PSObject:
$items = "Hello Tim", "Hola Bob", "Hello Susan"
$filter = $items | Select-String -Pattern "Hello"
$filter | % {New-Object PSObject -property #{
Line = $_.Line
Matches = $_.Matches.Groups[0].Value}
} | Export-Csv "C:\log.csv" -NoTypeInformation


Quickly note that Matches is an array which may create issues exporting to a csv.
Try joining the array into a string with a chosen delimiter. I used "::" in my example.
$filter | Select Line, #{Expression={$_.Matches -join "::"}; Label="Matches"} | Export-Csv "C:\log.csv"

Related

Parse/Extract JSON data in CSV file

I've hundreds of rows in CSV file while contains JSON data like below. Below is a sample of each row.
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"}
I tried to convert the same into CSV as below but no luck as of now.
Expected format of CSV file:
id RecordType CreationTime Operation
value value value value
$properties = #('Id', 'RecordType', 'CreationTime', 'Operation')
(Get-Content -Path-to_CSVfile -Raw | ConvertFrom-Json) |
Select-Object -Property $properties |
Export-Csv -NoTypeInformation -Path $path-to-new-csv-file
If someone has an idea about this please help me. I tried ConvertTo-Json but it's failing with error:
ConvertFrom-Json : Invalid JSON primitive: "id"
Here are the first two rows of CSV data.
{"Id":"ac325bc9-97f0-4b29-8fc4-90b80b945f6c","RecordType":20,"CreationTime":"2019-09-14T08:07:22","Operation":"AnalyzedByExternalApplication","OrganizationId":"f38a5ecd-2813-4862-b11b-ac1d563c806f","UserType":0,"UserKey":"3fee8456-6d20-4794-8219-5a7c381e965f","Workload":"PowerBI","UserId":"abcd#mail.com","ClientIP":"000.000.50.177","UserAgent":"MSOLAP 15.0 Client","Activity":"AnalyzedByExternalApplication","ItemName":"Other","DatasetName":"XYZ Driven Company","ObjectId":"Other","IsSuccess":true,"RequestId":"6836be8e-6e97-4bc9-a838-bf6e7b71e0c8","ActivityId":"7E92AE6A-F548-448D-93A8-6F5736DEA085"}
{"Id":"3a20c8a9-ef44-483a-b9c0-43e10deae9ae","RecordType":20,"CreationTime":"2019-09-14T08:07:20","Operation":"AnalyzedByExternalApplication","OrganizationId":"f38a5ecd-2813-4862-b11b-ac1d563c806f","UserType":0,"UserKey":"3fee8456-6d20-4794-8219-5a7c381e965f","Workload":"PowerBI","UserId":"abcd#mail.com","ClientIP":"000.000.50.177","UserAgent":"MSOLAP 15.0 Client","Activity":"AnalyzedByExternalApplication","ItemName":"Other","DatasetName":"XYZ Driven Company","ObjectId":"Other","IsSuccess":true,"RequestId":"02e5d772-057b-45b6-ae60-22b7fa610f98","ActivityId":"7E92AE6A-F548-448D-93A8-6F5736DEA085"}
I'm looking this data in another CSV file as below. Each value after ":" should insert into CSV as rows.
Id RecordType CreationTime Operation OrganizationId UserType UserKey Workload UserId ClientIP UserAgent Activity ItemName DatasetName ObjectId IsSuccess RequestId ActivityId
ac325bc9-97f0-4b29-8fc4-90b80b945f6c 20 2019-09-14T08:07:22 AnalyzedByExternalApplication f38a5ecd-2813-4862-b11b-ac1d563c806f 0 3fee8456-6d20-4794-8219-5a7c381e965f PowerBI abcd#mail.com 000.000.50.177 MSOLAP 15.0 Client AnalyzedByExternalApplication Other xyz Driven Company Other TRUE 6836be8e-6e97-4bc9-a838-bf6e7b71e0c8 7E92AE6A-F548-448D-93A8-6F5736DEA085
3a20c8a9-ef44-483a-b9c0-43e10deae9ae 20 2019-09-14T08:07:20 AnalyzedByExternalApplication f38a5ecd-2813-4862-b11b-ac1d563c806f 0 3fee8456-6d20-4794-8219-5a7c381e965f PowerBI abcd#mail.com 000.000.50.177 MSOLAP 15.0 Client AnalyzedByExternalApplication Other XYZ Driven Company Other TRUE 02e5d772-057b-45b6-ae60-22b7fa610f98 7E92AE6A-F548-448D-93A8-6F5736DEA085
Correct data from CSV when opened in text editor.
"{""Id"":""ac325bc9-97f0-4b29-8fc4-90b80b945f6c"",""RecordType"":20,""CreationTime"":""2019-09-14T08:07:22"",""Operation"":""AnalyzedByExternalApplication"",""OrganizationId"":""f38a5ecd-2813-4862-b11b-ac1d563abchrf"",""UserType"":0,""UserKey"":""3fee8456-6d20-4794-8219-5a7c38abcdfe"",""Workload"":""Pxyswer"",""UserId"":""abcd#mail.com"",""ClientIP"":""123.456.50.177"",""UserAgent"":""MSOLAP 15.0 Client"",""Activity"":""AnalyzedByExternalApplication"",""ItemName"":""Other"",""DatasetName"":""ABCD Driven Company"",""ObjectId"":""Other"",""IsSuccess"":true,""RequestId"":""6836be8e-6e97-4bc9-a838-bf6e7b71e0c8"",""ActivityId"":""7E92AE6A-F548-448D-93A8-6F5736DEA085""}"

If your input file contains just that single example data row the code you posted will work. If the input file contains multiple statements like that your code will not work, because it'd be invalid JSON data.
Valid JSON:
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"}
Valid JSON:
[
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"},
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"}
]
Invalid JSON:
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"}
{"Id":"value","RecordType":"value","CreationTime":"value","Operation":"value"}
To convert the latter kind of input data you need to convert each row as a separate JSON document:
$properties = 'Id', 'RecordType', 'CreationTime', 'Operation'
Get-Content 'C:\path\to\input.csv' |
ConvertFrom-Json |
Select-Object $properties |
Export-Csv 'C:\path\to\output.csv' -NoType
To export all input fields except particular ones you'd define the properties to exclude rather than the ones to include:
$exclude = 'foo', 'bar'
Get-Content 'C:\path\to\input.csv' |
ConvertFrom-Json |
Select-Object -Properties * -Exclude $exclude |
Export-Csv 'C:\path\to\output.csv' -NoType
Edit:
Apparently your input file is a CSV with only one column and no header, so you can import it via Import-Csv, but you need to specify the column header yourself. Expand the field to get the individual JSON values, then proceed as described above.
$properties = 'Id', 'RecordType', 'CreationTime', 'Operation'
Import-Csv 'C:\path\to\input.csv' -Header foo |
Select-Object -Expand foo |
ConvertFrom-Json |
Select-Object $properties |
Export-Csv 'C:\path\to\output.csv' -NoType
If you want all JSON values exported, simply omit the Select-Object $properties step.

PowerShell: Get 2 strings into a hashtable and out to .csv

PowerShell newbie here,
I need to:
Get text files in recursive local directories that have a common string, students.txt in them.
Get another string, gc.student="name,name" in the resulting file set from #1 and get the name(s).
Put the filename from #1, and just the name,name from #2 (not gc.student="") into a hashtable where the filename is paired with its corresponding name,name.
Output the hashtable to an Excel spreadsheet with 2 columns: File and Name.
I've figured out, having searched and learned here and elsewhere, how to output #1 to the screen, but not how to put it into a hashtable with #2:
$documentsfolder = "C:\records\"
foreach ($file in Get-ChildItem $documentsfolder -recurse | Select String -pattern "students.txt" ) {$file}
I'm thinking to get name in #2 I'll need to use a RegEx since there might only be 1 name sometimes.
And for the output to Excel, this: | Export-Csv -NoType output.csv
Any help moving me on is appreciated.

I think this should get you started. The explanations are in the code comments.
# base directory
$documentsfolder = 'C:\records\'
# get files with names ending with students.txt
$files = Get-ChildItem $documentsfolder -recurse | Where-Object {$_.Name -like "*students.txt"}
# process each of the files
foreach ($file in $files)
{
$fileContents = Get-Content $file
$fileName = $file.Name
#series of matches to clean up different parts of the content
#first find the gc.... pattern
$fileContents = ($fileContents | Select-String -Pattern 'gc.student=".*"').Matches.Value
# then select the string with double quotes
$fileContents = ($fileContents | Select-String '".*"').Matches.Value
# then remove the leading and trailing double quotes
$fileContents = $fileContents -replace '^"','' -replace '"$',''
# drop the objects to the pipeline so that you can pipe it to export-csv
# I am creating custom objects so that your CSV headers will nave nice column names
Write-Output [pscustomobject]#{file=$fileName;name=$fileContents}
} | Export-Csv -NoType output.csv

Issue while saving collection into csv using powershell

I have below powershell code using which am saving results into csv file but I couldnt save them in csv file.
[Reflection.Assembly]::LoadWithPartialName("Microsoft.AnalysisServices")
$SSASServerName = "SSAS_ServerName"
$SSASDB = "TESTDB"
$SSASServer = New-Object Microsoft.AnalysisServices.Server
$SSASServer.Connect($SSASServerName)
$SSASDatabase = $SSASServer.Databases.Item($SSASDB)
$SSASDatabase.Roles | Select-Object Name, Members | Export-Csv C:\dev\psout\test.Csv
pause
This script extracts name of the role and members associated to that role. one role can have multiple members.
I tried above script, it exports role but in Members field, I see string "Microsoft.AnalysisServices.RoleMemberCollection" for all the roles.
If I do not export to csv, I can view the members in either ps window or text file.
what am i missing?

You can only export values that can be represented as a string to a csv-file. Members is a collection-object that may include multiple RoleMember-objects, so you need use a calculated property to access the Name-property inside each RoleMember. How to approach this depends on the desired output.
You can join the objects Name-property to a single string
$SSASDatabase.Roles |
Select-Object Name, #{n="Members";e={ ($_.Members | % { $_.Name }) -join '; '}} |
Export-Csv C:\dev\psout\test.Csv -NoTypeInformation
Role1,"User1; User2"
Role2,"User3; User4"
Or you could make one row in the csv-file "per row per member" which I usually prefer since it's easier to filter in Excel.
$SSASDatabase.Roles | ForEach-Object {
#Store role-reference so we can access it later inside the member-foreach
$r = $_
$r.Members | ForEach-Object {
#Store member-reference so it's available inside Selec-Object
$m = $_
$r | Select-Object Name, #{n="Member";e={ $m.Name }}
}
} | Export-Csv C:\dev\psout\test.Csv -NoTypeInformation
Role1,User1
Role1,User2
Role2,User3
...

Find and Replace many Items with Powershell from Data within a CSV, XLS or two txt documents

So I recently have found the need to do a find and replace of mutliple items within a XML document. Currently I have found the code below which will allow me to do multiple find and replaces but these are hard coded within the powershell.
(get-content c:\temp\report2.xml) | foreach-object {$_ -replace "192.168.1.1", "Server1"} | foreach-object {$_ -replace "192.168.1.20", "RandomServername"} | set-content c:\temp\report3.xml
Ideally instead of hard coding the value I would like to find and replace from a list, ideally in a CSV or and XLSX. Maybe two txt file would be easier.
If it was from a CSV it could grab the value to find from A1 and the value to replace it with from B1 and keep looping down until the values are empty.
I understand I would have to use the get-content and the for each command I was just wondering if this was possible and how to go about it/ if anybody could help me.
Thanks in advance.
SG

#next line is to clear output file
$null > c:\temp\report3.xml
$replacers = Import-Csv c:\temp\replaceSource.csv
gc c:\temp\aip.xml | ForEach-Object {
$output = $_
foreach ($r in $replacers) {
$output = $output -replace $r.ReplaceWhat, $r.ReplaceTo
}
#the output has to be appended, not to rewrite everything
return $output | Out-File c:\temp\report3.xml -Append
}
Content of replaceSource.csv looks like:
ReplaceWhat,ReplaceTo
192.168.1.1,server1
192.168.1.20,SERVER2
Note the headers

Import-Csv TrimEnd Column Header

I need import a CSV and run it through a foreach loop. I want to trim the end on the column header DeviceName to avoid any potential issues. I have tried the following but it is not working as expected.
$Import = Import-CSV $csv
foreach ($i in ($import.DeviceName).TrimEnd())
{do something}
Any help? Thank you!

If you need to change both the header and the content in the column for devicename which has spaces I have come up with this forgiving code.
$csvData = import-csv $csv
$properties = $csvData[0].psobject.Properties.name
$csvHeader = "`"$(($properties | ForEach-Object{$_.Trim()}) -join '","')`""
$deviceHeader = $properties -match "DeviceName"
$csvHeader
$csvHeader | Set-Content $file
$csvData | ForEach-Object{
$_.$deviceHeader = ($_.$deviceHeader).trim()
$_
} | ConvertTo-Csv -NoTypeInformation | Select-Object -Skip 1 | Add-Content $file
What this does is read in the CSV like normal. Parse the property names of the object in the order they appear. We find the one that has DeviceName no matter how many spaces (if there is more that one you could have a problem). Keep that so we can use it to call the correct property of each "row".
Export the new cleaned header to the file. Then we go through each "row" removing all the leading and trailing space from the DeviceName. Once that is done write back the CSV to the original file.

The best solution would be to tell the other team to fix their generation procedure. However, if for some reason that's not an option, I'd recommend pre-processing the file before you import it as a CSV.
$filename = 'C:\path\to\your.csv'
(Get-Content $filename -Raw) -replace '^(.*DeviceName)[ ]*(.*)', '$1$2' |
Set-Content $filename
Reading the file as a single string (-Raw) and anchoring the expression at the beginning of the string (^) ensures that only the column title is replaced.
For large input files you may want to consider a different approach, though, since the above reads the entire file into memory before replacing the first line.
$infile = 'C:\path\to\input.csv'
$outfile = 'C:\path\to\output.csv'
$firstLine = $true
Get-Content $infile | % {
if ($firstLine) {
$_ -replace '(DeviceName)[ ]*', '$1'
$firstLine = $false
} else {
$_
}
} | Set-Content $outfile
Thinking about it some more and taking inspiration from a comment to #Zeek's answer, you could also extract the headers first and then convert the rest of the file.
$infile = 'C:\path\to\input.csv'
$outfile = 'C:\path\to\output.csv'
$header = (Get-Content $infile -First 1) -split '\s*,\s*'
Get-Content $infile |
select -Skip 1 |
ConvertFrom-Csv -Header $header |
Export-Csv $outfile -NoType

Is this all you're trying to do? This will give you a collection of objects imported from your csv file but trim the end of the DeviceName property on each object.
$items = Import-CSV -Path $csv
$items.ForEach({ $_.DeviceName = $_.DeviceName.TrimEnd() })