Merge Two CSV Files Using Two Common Keys - csv

Hello I Used below code to Merge CSV files using one common Key for adding one column from Child.CSV to Parent.CSV.
{
Import-Csv "D:\CSV check\Sourcecount.csv" -Header Sourcecount, MONTH_YEAR | ForEach-Object -Begin {
$Employees = #{}
} -Process {
$Employees.Add($_.MONTH_YEAR,$_.Sourcecount)
}
Import-Csv "D:\CSV check\RenameFinal.csv" | ForEach-Object {
$_ | Add-Member -MemberType NoteProperty -Name Sourcecount -Value $Employees."$($_.MONTH_YEAR)" -PassThru
} | Export-Csv -NoTypeInformation "D:\CSV check\sourcefinal.csv"
}
Question:
Can you please help for the scenario joining for getting one column value from child.csv to Parent.csv for two common keys (by data and by userid).

Below is the code I used to solve the above Query:
$f2=Import-Csv "D:\Sourcecount.csv" -Header Sourcecount,MONTH_YEAR,UserID
$f1= Import-csv "D:\Final.csv" -Header MONTH_YEAR,DestinationCount,UserID,Result,SourceCount
$f1|
%{
$Month_YEAR=$_.MONTH_YEAR
$UserID=$_.UserID
$m=$f2|?{$_.MONTH_YEAR -eq $Month_YEAR}|?{$_.UserID -eq $UserID}
$_.SourceCount=$m.Sourcecount
}
$f1| Export-Csv "D:\SourceFinal.csv" -NoTypeInformation

Related

Powershell ForEach-Object column variables

Kind of have a strange problem. I have a large JSON file that needs to be processed. Based on another question I need to stream the file since it will otherwise gets me problems because of memory: JSON Powershell memory issue
What I have is this:
get-content -Path largefile.json | ForEach-Object {
$row = $_ = $_.TrimStart('[').TrimEnd(']')
if ($_) { $_ | Out-String | ConvertFrom-Json }
New-Item -Path $($Row.Id).txt
Set-Content -Path $($Row.Id).txt -Value ($row.Body)
}
I can easily do $row to publish the last processed row in the Largefile.json. I want to create a file with the name of the Id in the row that is currently processed and add the body column to the file. But when I want to show a specific column using $row.Id, unfortunately this shows up empty.
The structure of the Largefile.json is as followed:
[{"Id":"1","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"data1"}
{"Id":"2","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"data2"}
{"Id":"3","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"data3"}
{"Id":"4","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"data4"}
{"Id":"5","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"data5"}
]
The end result should be that I have 5 files:
1.txt - Value inside the file should be: data1
2.txt - Value inside the file should be: data2
3.txt - Value inside the file should be: data3
4.txt - Value inside the file should be: data4
5.txt - Value inside the file should be: data5
I use Powershell 7.1.3
Is there any way that I can use $row.Id and $row.ParentId just like a regular ForEach would do?
thanks for your help.
It seems to me that this is what you're looking for:
Get-Content largefile.json | ForEach-Object {
$row = $_.TrimStart('[').TrimEnd(']') | ConvertFrom-Json
if ($null -ne $row) {
Set-Content -Path ($row.Id) -Value ($row.Body)
}
}
I am still not sure what you expect as an outcome.
But I think you want to do this:
#'
[{"Id":"1","ParentId":"parent1","Name":"1.txt","OwnerId":"owner","CreatedDate":"date","Body":"Data1"}
{"Id":"2","ParentId":"parent2","Name":"2.txt","OwnerId":"owner","CreatedDate":"date","Body":"Data2"}
{"Id":"3","ParentId":"parent3","Name":"3.txt","OwnerId":"owner","CreatedDate":"date","Body":"Data3"}
{"Id":"4","ParentId":"parent4","Name":"4.txt","OwnerId":"owner","CreatedDate":"date","Body":"Data4"}
{"Id":"5","ParentId":"parent5","Name":"5.txt","OwnerId":"owner","CreatedDate":"date","Body":"Data5"}
]
'# | Set-Content .\largefile.json
Get-Content .\largefile.json | ForEach-Object {
$_ = $_.TrimStart('[').TrimEnd(']')
If ($_) {
$Row = ConvertFrom-Json $_
Set-Content -Path ".\$($Row.Name)" -Value $Row.Body
}
}
The question has many errors. Assuming the json has the missing commas in, I would do it this way, if I understand the question. This should work with the new updates to the question. I also have a more unusual solution involving streaming json with jq here: Iterate though huge JSON in powershell Json streaming support may be added later: ConvertFrom-JSON high memory consumption #7698
[{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"},
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"},
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"},
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"},
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"}
]
get-content -Path largefile.json | ForEach-Object {
$_ = $_.TrimStart('[').TrimEnd(']').TrimEnd(',')
if ($_) {
$row = $_ | ConvertFrom-Json
Set-Content -Path ($Row.Id + '.txt') -Value $row.Body
}
}
get-content ID.txt
*******
As others already explained, your json example is invalid.
However, since this is a huge file to process, you can use switch for this.
switch -Regex -File D:\Test\largefile.json {
'"Id":"(\d+)".*"Body":"(\w+)"' {
Set-Content -Path ('D:\Test\{0}.txt' -f $matches[1]) -Value $matches[2]
}
}
Results using your example would be 5 files called 1.txt .. 5.txt, each having a single line data1 .. data5

Modifying JSON file using data from CSV in PowerShell

I'm trying to modify some specific values in a .json file based on two columns in a .csv file. If the current value in the .json file is identical to the one in the left column, I want to change it to the one in the right column.
This is my first time with PowerShell though, so I'm struggling to figure out how to go about doing this. I feel like my solution is not only wrong, but is using a double for loop when it might not need to. Here's what I have so far.
$jsonData = Get-Content -Path $jsonFile | ConvertFrom-Json
$csvData = Get-Content -Path $csvFile | Select-Object -Skip 1 # Skipping the header
foreach ($jsonItem in $jsonData.'Placeable List') {
foreach ($csvRow in $csvData) {
$splitRow = $csvRow -split ","
$lCol = $splitRow[0]
$rCol = $splitRow[1]
$currentItem = $jsonItem.'value'.'Appearance'.'value'
if ($currentItem -eq $lCol) {
$currentItem -eq $rCol
}
}
}
I managed to figure it out.
$csvData = Get-Content -Path $csvFile | Select-Object -Skip 1 # Skipping the header
$jsonData = Get-Content -Path $jsonFile -raw | ConvertFrom-Json
foreach($csvRow in $csvData) {
$splitRow = $csvRow -split ","
$lCol = $splitRow[0]
$rCol = $splitRow[1]
foreach($item in $jsonData.'Placeable List'.value) {
$item.Appearance | % {
if ($_.value -eq $lCol) {
$_.value = $rCol
}
}
}
}
$jsonData | ConvertTo-Json -depth 32 | Set-Content $jsonFile

Expand PSobjects to export to csv

How can I convert the PSobjects returned by Get-Help to strings and export to csv?. I think this is an easy one, but I can't quite get the format I'm looking for. From reading previous answers, I was trying:
Get-Help Get-Content | Select-Object -First 1 -Property #{
Name="temp"
Expression = { $_.Name,$_.Synopsis,$_.syntax,$_.Description }
} | Select-Object -ExpandProperty temp
Which is almost what I want, except it looks like the fields are arrays not strings. I am trying to have just one string for each element, ie (Name, Synopsis, Syntax, etc).
If I try to export that to csv, I get the lengths of the objects instead of the objects themselves:
Get-Help Get-Content | Select-Object -First 1 -Property #{
Name="temp"
Expression = { $_.Name,$_.Synopsis,$_.syntax,$_.Description }
} | Select-Object -ExpandProperty temp | export-csv -NoType -Path $env:HOME\test.txt
I suppose you want to export each property in a seperate column (your script currently only exports one). To do this, you have to select each property. For Syntax and Description I used the Out-String cmdlet to convert it to a string and removed all \r\n to get a valid CSV:
Get-Help Get-Content | Select-Object -First 1 |
Select-Object Name, Synopsis,
#{l='Syntax'; e={($_.Syntax | out-string) -replace "`r?`n"}},
#{l='Description '; e={($_.Description | out-string)-replace "`r?`n"}} |
Export-Csv -Path $env:HOME\test.txt -NoTypeInformation

Include file owner to powershell pipe

I've got a script which works fine which lists all files modified since last 7 days and want to modify it to add file owner to the export csv file.
$dir_to_look="F:\"
$month_backdate=$(Get-Date).AddDays(-7)
Get-Childitem $dir_to_look -Recurse | ? { !($_.psiscontainer) -and $_.LastWriteTime -gt $month_backdate } | ForEach-Object {Get-Acl $_.FullName}.owner | Select-object LastWriteTime, Directory, FullName | export-csv -path \\sharename\report.csv -encoding "unicode"
But not sure how to correctly add get-acl to the pipe as currently it prints nothing to my report file
Your Foreach-Object command should be:
... | Foreach-Object {Add-Member -Input $_ -Type NoteProperty -Name Owner -Value (Get-Acl $_.Fullname).Owner -PassThru} | Select-object LastWriteTime, Directory, FullName, Owner | ...
Add-Member is handy for adding properties (and methods) to objects.
Use a hash table with Select-Object.
$dir_to_look="F:\"
$month_backdate=$(Get-Date).AddDays(-7)
Get-Childitem $dir_to_look -Recurse | ? { !($_.psiscontainer) -and $_.LastWriteTime -gt $month_backdate } |
Select-object LastWriteTime, Directory, FullName, #{n='Owner';e={(Get-ACL $_.FullName).owner}}
$dir_to_look="F:\"
$month_backdate=$(Get-Date).AddDays(-7)
Get-Childitem $dir_to_look -Recurse | ? { !($_.psiscontainer) -and $_.LastWriteTime -gt $month_backdate } |
Select-object LastWriteTime, Directory, FullName, #{n='Owner';e={(Get-ACL $_.FullName).owner}}

Convert CSV to JSON and JSON to CSV using PowerShell

I have a CSV file that I am trying to convert to JSON using PowerShell.
The CSV file contains the following data.
web_url.csv
wikipedia,https://en.wikipedia.org/wiki/%s
wolframalpha,http://www.wolframalpha.com/input/?i=%s
drive,http://www.drive.google.com/
I would like to convert to json in the following format. Similarly how do you convert this json back to original csv in the format shown above?
web_url.json
{
"wikipedia": "https://en.wikipedia.org/wiki/%s",
"wolframalpha": "http://www.wolframalpha.com/input/?i=%s",
"drive": "http://www.drive.google.com/"
}
When I run the command,
Get-Content -path web_url.csv | ConvertFrom-Csv -Delimiter ',' |
ConvertTo-Json
it returns the following output which is not what I want.
[
{
"wikipedia": "wolframalpha",
"https://en.wikipedia.org/wiki/%s": "http://www.wolframalpha.com/input/?i=%s"
},
{
"wikipedia": "drive",
"https://en.wikipedia.org/wiki/%s": "http://www.drive.google.com/"
}
]
# PowerShell script
import-csv "SampleInput.csv" | ConvertTo-Json | Add-Content -Path "output.json"
Your csv doen't look like a "proper" csv to me: columns are swapped with rows. If you have control over input file, you may fix it there already:
#'
wikipedia,wolframalpha,drive
https://en.wikipedia.org/wiki/%s,http://www.wolframalpha.com/input/?i=%s,http://www.drive.google.com/
'# | ConvertFrom-Csv | ConvertTo-Json
If that is not possible, you just have to perform some extra steps to get what you need:
$propertyList = #'
wikipedia,https://en.wikipedia.org/wiki/%s
wolframalpha,http://www.wolframalpha.com/input/?i=%s
drive,http://www.drive.google.com/
'# | ConvertFrom-Csv -Header Name, Value
$properties = [ordered]#{}
foreach ($property in $propertyList) {
$properties.Add($property.Name, $property.Value)
}
New-Object PSObject -Property $properties | ConvertTo-Json
And back, again - some extra work is required:
(#'
{
"wikipedia": "https://en.wikipedia.org/wiki/%s",
"wolframalpha": "http://www.wolframalpha.com/input/?i=%s",
"drive": "http://www.drive.google.com/"
}
'# | ConvertFrom-Json).PSObject.Properties |
Select-Object Name, Value |
ConvertTo-Csv -NoTypeInformation |
Select-Object -Skip 1
what about this ?
$csv=#"
wikipedia,https://en.wikipedia.org/wiki/%s
wolframalpha,http://www.wolframalpha.com/input/?i=%s
drive,http://www.drive.google.com/
"#
$obj= $csv |convertfrom-csv -delim ',' -Header "name","url"
#$obj
$json=$obj |convertto-json
#$json
$csv2 =$json |convertfrom-json |select -expand syncroot |convertto-csv -NoTypeInformation -Delimiter ','
#$csv2